KAP chart display offset and crashing

[TimThornton]

Hi,
I'm creating KAP charts for TeamSurv's bathymetry, so they can be used in OpenCPN.
As the images are computer generated, and the process will be automated, the route I'm taking is to write the KAP header instead of digitising the image, and then to use imgkap to create the final KAP file.
This is basically working, except for 2 things:
- the chart image is offset within the chart boundary
- sometimes if I zoom in and out the offset keeps increasing and then OpenCPN crashes

The attached KAP file shows the effect (just a small low res image for test purposes). Location is in the Aegean around 37.5N 27E. Remove the .xls extension from the file name before opening, obviously.

The header is as follows:
VER/3.0
CRR/2013, TeamSurv. All rights reserved.
BSB/NA=Kos-Aegean Sea,NU=36500_027000_05_05,RA=623,779
KNP/SC=45000,GD=WGS84,PR=MERCATOR,PP=36.75,SK=0.0,TA=9 0,UN=METRES,SD=MEAN LOWER LOW WATER
REF/1,0,779,36.5,27.0
REF/2,0,0,36.5,27.5
REF/3,623,0,37.0,27.5
REF/4,623,779,37.0,27.0
PLY/1,36.5,27.0
PLY/2,36.5,27.5
PLY/3,37.0,27.5
PLY/4,37.0,27.0
CED/ED=07/10/2013
DTM/0.0,0.0

Any thoughts on what is going wrong are much appreciated!

Also, in working on this I have put together some more info on the KAP header format. Is the best thing to just post it in the forum, or can I pass it to someone to put on the OpenCPN web site so it is more accessable to all?

Tim

[TimThornton]

PS The chart opens up fine in Fugawi, though it crashes Caris' EasyView!

[nohal]

Tim...
I currently don't have time to look at it in-depth, but such stuff happens when the calibration points are not in a correct order, for example if you switch the order of the corners while running imgkap, no warning is issued, but the resulting KAP is useless...

Pavel

[TimThornton]

Pavel,
Thanks for that.
Looking at a number of KAP files, the REF fields seem to be OK in any order, whereas the PLY fields are in sequential order, but again there is variation in both the start vertex and whether it goes round clockwise or anti clockwise.
But the chances are that the files I looked at where not generated by imgkap.

Can someone who has a successful file generated using imgkap tell me how these fields are presented? I can then regenerate my file in the same way, and see if that fixes things.
Tim

[nohal]

Tim...
Have a look at http://www.cruisersforum.com/forums/...ml#post1250645

Pavel

[TimThornton]

No, that didn't fix it.
FYI, that kap file had data ordered as follows:
REF starts at pixel (0,0) in NW corner and continues clockwise
PLY starts in NW corner and continues clockwise
It also had KNP/PI=UNKNOWN,SP=UNKNOWN which I had omitted from my file, and PP set to 0 (incorrectly - it should be the mid point latitude) and DX, DY set (which OpenCPN is documented as ignoring).
Tim

[nohal]

Tim...
This order worked for the thousands of charts of all shapes and sizes we did in the NGA chart calibration project. Will have a closer look at your chart tonight, now that link to a chart that works is all I can provide...

Pavel

[cagney]

Tim
If you enter the lat&long on the comand line, Pavel is right.
If you, like me, prefer to write your own header file, for more control, you have entered the wrong lat& long for the REF:s.
The convention, introduced by maptech, is to start with the REFs and PLYs in the SW corner, and proceed clock wise.
Have a look at the attached folder. I used this command to create the kap.

imgkap tes.png test.txt changed.kap

Folder.7z.pdf

Also, in working on this I have put together some more info on the KAP header format. Is the best thing to just post it in the forum, or can I pass it to someone to put on the OpenCPN web site so it is more accessable to all?

Have a look at imgkap | Official OpenCPN Homepage where we are trying to summarize our present knowledge of the kap format. If you have any corrections and/or additions, please just post it here and I'll add i to the knowledge base

Thomas

[TimThornton]

OK, thanks. No huge rush on this as it is just a small part of work on automating the final chart production in TeamSurv.

Here's the info I've put together - I used your web page as a starting point and then added so it is now as follows:

There is a .BSB file for the chart (not required by OpenCPN and many others), and a .KAP file for each panel within the chart (and an additional optional file for updates to charts, which we won’t use). A header file is required. Create this in the tile folder with file name chartname.bsb.txt and chartname.kap.txt. This has the following formats. In all cases, date is in the format mm/dd/yyyy. Data fields must not have commas in them where there are sub fields, and probably not /. In the NA fields (and possibly others) /A indicates a new line. Subfields can be separated with a “,” or a new line.
Note that the origin for pixels is the top left (NW) corner of the chart, whereas for positions it is the usual SW corner.
Tests in OpenCPN have produced a minimum data set requirement for the KAP files. Fields that are required for chart display or for operational reasons have been highlighted in italics in the description. In addition, Raytech requires the BSB file to be present.
BSB Header

! An example BSB text header
VER/3.0
CRR/2013, TeamSurv. All rights reserved.
CHT/NA=Australia 3000000, NU=123
CHF/Coastal
CED/SE=70,RE=01,ED=07/25/2012
NTM/NE=70.00,ND=07/25/2012, BF = on, BD=07/25/2012
CHK/1,123
ORG/TeamSurv
MFR/TeamSurv
CGD/5
RGN/4,6
K01/NA= Australia 3000000
NU=123
TY=Base
FN=123_1.KAP
N000005580027/RT=N,KN=12221_1,CA=CHART,DE=TIDE BOX,P1=3020,8412
P2=3020,8771,P3=4114,8771,P4=4114,8412
N000005580041/RT=L,KN=12221_1,LK=N000005580027,DE=TIDE BOX,
P1=8527, 707
The text header is terminated with a <Control-Z><NUL> sequence (ASCII characters 26 and 0).

• ! - Comment line
• VER - Version number of BSB format – we will use 3.0
• CRR – Copyright message. Free text
• CHT – General parameters
  • NA - Chart name given to the BSB chart (can represent more than one .KAP)
  • NU - Chart number.
• CHF Chart format (e.g. Overview, General, Coastal, Approach, River, Harbour or Berthing)
• CED – Chart edition parameters - optional
  • SE - Source edition / number of paper chart
  • RE - Raster edition / number
  • ED - Chart edition date/number
• NTM – Notices to mariners - optional
  • NE - NTM edition number
  • ND - NTM date
  • BF - Base flag on or off (I guess this allows a base edition of the chart to be specified?)
  • BD - Base date
• CHK – number of KAPs; KAP number(s). Note sure of format for > 1 KAP. Not sure if the separator should be , or ;
• CGD - ? coast guard district – optional or can be unknown
• RGN - ? - optional
• ORG – Producing agency identifier – TeamSurv for us
• MFR – Manufacturer of the RNC chart – TeamSurv for us
• Knn – Records 1 – n for each KAP file
  • NA - Chart name given to the pane
  • NU - Pane number e.g. 123_A
  • TY - Type. Base for the base chart, Inset for a plan contained within the base chart, or extension for a plan outside of the base area
  • FN – KAP file name e.g. 123_1.kap
• Naa – List of text boxes and hot spot links on the chart
  • RT – N for text box, L for hot spot link
  • KN - ?
  • CA – Category e.g. Chart, Caution, General etc
  • DE – Type e.g. Tide box, note etc
  • LK – Links this hot spot to the specified text box
  • P1, P2, P3, P4 – Hot spot location (just P1) or text box boundaries as 4 point polygon in x, y pixels

KAP Header

! An example KAP text header
VER/3.0
CRR/2013, TeamSurv. All rights reserved.
BSB/NA=Australia 3000000
NU=,RA=625,480,DU=50
KNP/SC=3000000,GD=,PR=LAMBERT CONFORMAL CONIC,PP=145.0
PI=0.0,SP=Unknown,0,SK=0.0
UN=METRES,SD=,DX=6000.0,DY=6000.0
KNQ/EC=RF,GD=NARC,VC=UNKNOWN,SC=MLLW,PC=MC,P1=UNKNOWN, P2=37.083
P3=NOT_APPLICABLE,P4=NOT_APPLICABLE,GC=NOT_APPLICA BLE,RM=POLYNOMIAL
CED/SE=70,RE=01,ED=07/25/2012
NTM/NE=70.00,ND=07/25/2012, BF = on, BD=07/25/2012
OST/1
IFM/3
RGB/1,199,231,252
RGB/2,174,234,84
RGB/3,255,254,206
RGB/4,226,65,6
CPH/0.0000000000
WPX/2,863264.4957,11420.23114,-85.46756208,1.913941167,-0.4081181078
0.7362163163
WPY/2,390032.0953,69.56409751,-6745.589267,0.4669253601,0.0367153316
-96.0547565
PWX/2,-76.48368342,8.999135076e-005,5.758392982e-009,-1.392859319e-012
-2.377189159e-013,-3.432372134e-013
PWY/2,37.44988807,-3.111799225e-009,-7.171936009e-005,2.694372983e-013
-1.725045227e-014,-3.594145418e-011
REF/1,374,8790,36.8166861111,-76.4500000000
REF/2,374,695,37.4000111111,-76.4500000000
REF/3,4505,695,37.4000111111,-76.0783222222
ERR/1,0.0395099814,0.1453734568,0.0000106128,0.0000035 393
ERR/2,0.2568631181,0.1909729033,0.0000135084,0.0000230 797
ERR/3,0.2741345061,0.0861261497,0.0000060346,0.0000246 567
PLY/1,36.8166666667,-76.4500000000
PLY/2,37.4000000000,-76.4500000000
PLY/3,37.4000000000,-76.0783333333
PLY/4,37.4083333333,-76.0783333333
DTM/0.0,0.0
The text header is terminated with a <Control-Z><NUL> sequence (ASCII characters 26 and 0).
Note that where the meaning of a field is the same as in the .BSB file above, it is not repeated here.

• VER - Version number of BSB format – we will use 3.0
• BSB (or NOS for older GEO/NOS or GEO/NO1 files) – General parameters
  • NA - Pane name
  • NU - Pane number. If chart is 123 and contains a plan A, the plan should be numbered 123_A
  • RA - width, height - width and height of raster image data in pixels
  • DU - Drawing Units in pixels/inch (same as DPI resolution) e.g. 50, 150, 175, 254, 300
• KNP – Detailed chart parameters
  • SC - Scale e.g. 25000 means 1:25000
  • GD - Geodetic Datum e.g. WGS84 for us
  • PR - Projection e.g. MERCATOR for us. Other known values are TRANSVERSE MERCATOR or LAMBERT CONFORMAL CONIC or POLYCONIC. This must be one of those listed, as the value determines how PP etc. are interpreted. Only MERCATOR and TRANSVERSE MERCATOR are supported by OpenCPN.
  • PP – Projection parameter. For Mercator charts this is where the scale is valid, i.e. +lat_ts – use average latitude of the chart. For transverse Mercator it is the +lon_0 value.
  • PI – Projection interval ? e.g. 0.0, 0.033333, 0.083333, 2.0
  • SP -? – Unknown is valid
  • SK - Skew angle e.g. 0.0 for us. Angle of rotation of the chart
  • TA - text angle e.g. 90 for us
  • UN – Depth units (for depths and heights) e.g. METRES, FATHOMS, FEET
  • SD - Sounding Datum e.g. MEAN LOWER LOW WATER, HHWLT or normally LAT
  • DX – X resolution, distance (meters) covered by one pixel in X direction. OpenCPN ignores this and DY
  • DY – Y resolution, distance covered by one pixel in Y direction
• CED – Chart edition parameters (see BSB section)
• NTM – Chart edition parameters (see BSB section)
• OST - Offset STrip image lines (number of image rows per entry in the index table) e.g. 1. Generated by imgkap.
• IFM - Depth of the colormap (bits per pixel). BSB supports 1 through 7 (2 through 127 max colors). Or compression type? Generated by imgkap.
• RGB – Default colour palette - entries in the raster colormap of the form index,red,green,blue (index 0 is not used in BSB). Generated by imgkap.
• DAY – Optional day colour palette
• DSK – Optional dusk colour palette
• NGT – Optional night colour palette
• NGR – Optional night red colour palette
• GRY – Optional grey scale palette
• PRC – Optional palette
• PRG – Optional grey palette
• REF – Reference point record, gives point n, then position x, y in pixels, then in longitude, latitude - optional
• CPH – Phase shift value - optional
• WPX – Polynomial longitude to x - optional
• PWX – Polynomial x to longitude - optional
• WPY – Polynomial latitude to y - optional
• PWY – Polynomial y to latitude - optional
• ERR – Error record – optional – looks like the error at each point in REF between corrected and uncorrected position? Same number of entries as REF
• PLY - Border polygon record – coordinates of the panel within the raster image, given in chart datum lat/long. Any shape polygon
• DTM – Datum shift parameters - Datum's northing and easting in floating point seconds to go between chart datum and WGS84 (omitted or 0,0 for WGS84 charts)
• KNQ – Additional parameters - Optional
  • P1=?,
  • P2 - the same as KNP PP for Mercator and Transverse Mercator projection.
  • P3= scale factor for Transverse Mercator; 1st standard parallel for lambert conic
  • P4= 0 for Transverse Mercator; 2nd standard parallel for lambert conic
  • P5= Central meridian for Transverse Mercator and lambert conic
  • P6= origin of latitude for Transverse Mercator and lambert conic
  • P7 = +x_0 for Transverse Mercator and lambert conic
  • P8 = +y_0 for Transverse Mercator and lambert conic
  • EC=?
  • GD=?
  • VC= ?
  • SC=?
  • PC=?. Set to TC for Transverse Mercator.

[TimThornton]

Great Thomas, that fixed it.
Update the above docs as follows:
Great, Thomas, that's fixed it.
Update the above docs as follows:

• REF – Reference point record, gives point n, then position x, y in pixels, then in longitude, latitude. Must start at SW corner and proceed clockwise.
  
• PLY - Border polygon record – coordinates of the panel within the raster image, given in chart datum lat/long. Any shape polygon. Must start at SW corner and proceed clockwise.

Tim

[cagney]

Tim

I added your extended documentation here The BSB/KAP file format. | Official OpenCPN Homepage.
Get yourself an account if you haven't got one already, and help keep the page updated. It's a wiki!
I'll look through my "papers", to see if I can fill in some of the blanks in the documentation.

My earlier comment about the convention for REFs and PLYs where based on what we did in the old NGA project. We almost exclusively dealt with rectangular chart pictures and only used the four corners as REFs.

If you have a perfect Mercator picture it's actually enough to use two diagonally opposite corners as REFs, at least this works fine for OpenCPN.
There is no absolute need to have the REFs in a particular order. If you need to use a lot of REFs it's actually not very practical to process them in clockwise order. One situation were you need more REFs is to calculate the polynomial coefficients . Search this forum for posts by GPSMarco for more details.

PolarNavy, for example, does not support Transverse Mercator charts directly. Instead it does an on the fly re-projection to Mercator charts. To do this, it's necessary with the polynomials in the header file.
Other chart-plotters may do the same thing. Be warned, if you are thinking of using Tmercs.


For PLYs I think that it's only necessary that they are in a clockwise order, but it makes sense to treat all charts in a consistent manner.

The convention to start both in the SW corner, makes it easy to process. With 4 REFs and 4 PLYs just use the last part of the REFs as value for the PLYs.

Thomas