Charts

[oem]

3 more free maps here from Norway:
http://www.iho.shom.fr/publicat/free/files/S-63_Testdata_v1.1.zip

My further investigation of S63 revealed a test dataset for the validation of the S63 data protection scheme. It contains 3 small maps. The dataset contains S63 data and the corresponding S57 data in order to verify the decryption algorithm. One of them attached.

[sinbad7]

Hi all.

Great maps of the Mediterranean incl. Greece and Turkey..

[Harryjak]

I downloaded a bunch of inland river charts (IENC's) from the Corps of engineers at

Inland Electronic Navigation Charts

They came in shapefile format whatever that is, supposed to adhere to the S 57 format. I unzipped them, put them in my Nav folder, pointed OpenCPN at them in tools and it didn't see them. Downloaded 1.3.6 and it saw the files when it went through the uploading part when I clicked OK but they don't show on the out line of the US. I like that feature by the way.

So are these files supported and I am doing something wrong or are they unsupported.

HJ

[dacust]

Originally Posted by Harryjak

I downloaded a bunch of inland river charts (IENC's) from the Corps of engineers at

Inland Electronic Navigation Charts

They came in shapefile format whatever that is, supposed to adhere to the S 57 format. I unzipped them, put them in my Nav folder, pointed OpenCPN at them in tools and it didn't see them. Downloaded 1.3.6 and it saw the files when it went through the uploading part when I clicked OK but they don't show on the out line of the US. I like that feature by the way.

So are these files supported and I am doing something wrong or are they unsupported.

HJ

Use the little DOWNLOAD button above the "Download charts in shapefile format" link.

-dan

[jonasaberg]

Referring to Marcos post #459 and a discussion on a Swedish site I would like to lift the issue of the georeferrencing used in OpenCPN.
I haven't read the code myself so bear with if I have misunderstood the inner workings of OpenCPN...

On the Swedish Cruising Federation (SXK) web site an OpenCPN user/tester reports very bad georeferencing of scewd/rotated charts. See an example below. The chart is of the river/harbour flowing out in Gothenburg in Sweden. It is a commercial professionally prepared/georeferenced chart.

As the user reports, AIS targets are located up on dry land (The ship is at the black X) but is placed by OpenCPN up on land as are the other AIS targets.

By over-zooming in wildly and placing the cursor on a LAT/LON crossing at N57.43 E11.57 on the chart it can be seen that OpenCPN gives the coordinate as N57 42.794 E11 56.605.
This is more than 0.2NM, more than 300 meters, wrong.

I emailed and have had a discussion with Olle, the developer of SeaClear who is silently cheering on the OpenCPN development. He let us know that Seaclear does not use any polynomial approximations at all due to that some charts comes out very poorly using polynomials. Instead he use the definitions of the projections - I think the same should be done in OpenCPN.
OpenCPN should return to the definitions of the projections and the formulas used for them. I asked for directions but he believes all the information is in the open and that the developing team has access to it already.

Maybe Dave when he returns could comment on whether the use of polynomials was a way to save development time or if there is some other decision behind it??

Polynomial approximations seem to work great in a lot of cases and really bad in some - and my view is that this is not really acceptable for navigations software.

I would like some support for lifting this issue a notch on the radar.

Maybe other community members can find other examples like this to support or dismerit it?

I am not sure whether this should be seen as an outright bug or a feature of the design of the software but for someone navigating the harbor in the example I am inclined to say its a bug.

Opinions and comments?

/Jonas

Below are four pictures:
1. An overview of the chart , the red OpenCPN fram is totally different from the chart.
2. The original observation with AIS targets on land.
3. A wildly zoomed portion to show that the chart and OpenCPN reports different positions.
4. The kap file opened in MapcalII to show that Seaclear gets it right using the mathematical formulas based on the projections instead of polynomials.

[GPS-Marco]

Hi Jonas,

I think OCPN will need 3 improvements in geref process:

1) a better algoritm to calculate the polys. Even if SeaClear does not use poly it should use some other approximation methods (splines, etc.). In fact the projection formulas shall be corrected with the ref point positions. I agree that the projection should be taken into account when calculating the poly, the spline, or whatever is used to approximate the projection to the ref points. Moreover the poly method is embedded in some BSB charts (poly coefficients are already there).

2) a better way to show the chart on screen for chart not beeing mercator (since OCPN always use a display that is in mercator projection). The suggestion here was to always center (optimize) the chart projection on the center of the screen lat/long and not on the centre of the whole map. Of course skewed charts have an even bigger problem here. I wander if it is feasible a real-time rotation to make the chart north-up (this would be useful to implement a course-up feature too)

3) a better way to pan raster charts (making pan independent by the chart georeferentiation). I know Dave is already working on this.

We will see when Dave will be back...

Ciao, Marco

[jonasaberg]

Hi Marco,

Originally Posted by GPS-Marco

I think OCPN will need 3 improvements in geref process:

1) a better algorithm to calculate the polys. Even if SeaClear does not use poly it should use some other approximation methods (splines, etc.).

I agree. It would be good to know details about how it is supposed to be done - but I doubt splines are used.

In fact the projection formulas shall be corrected with the ref point positions. I agree that the projection should be taken into account when calculating the poly, the spline, or whatever is used to approximate the projection to the ref points. Moreover the poly method is embedded in some BSB charts (poly coefficients are already there).

Yes, I heard about BSBs including polynomial coefficients too, but I do not think there are any Swedish charts with polys... so it does not explain this.

2) a better way to show the chart on screen for chart not being mercator (since OCPN always use a display that is in mercator projection). The suggestion here was to always center (optimize) the chart projection on the center of the screen lat/long and not on the centre of the whole map. Of course skewed charts have an even bigger problem here. I wander if it is feasible a real-time rotation to make the chart north-up (this would be useful to implement a course-up feature too)

The strange thing is that the SK parameter is not used, and this chart is really skewed. Wonder when it is supposed to be used?

The ref info for the chart:

BSB/NA=9312 gotaalv
NU=9312 gotaalv,RA=12906,6370,DU=300
KNP/SC=10000,GD=WGS84
PR=TRANSVERSE MERCATOR,PP=15.808278,PI=UNKNOWN
P1=0.000000,P2=1.000000,P3=0.000000,P4=0.000000
SP=UNKNOWN,SK=0,UN=Meter,SD=Medelvattenstånd,DX=0. 85,DY=0.85
CED/SE=1,RE=1,ED=03/22/08
VER/1.1
OST/1
REF/1,0,0,57.722796704,11.835047419
REF/2,6453,0,57.745110192,11.916645380
REF/3,12906,0,57.767372164,11.998345602
REF/4,0,3185,57.701258236,11.855626947
REF/5,6453,3185,57.723558078,11.937190176
REF/6,12906,3185,57.745806424,12.018855470
REF/7,0,6370,57.679716061,11.876181614
REF/8,6453,6370,57.702002276,11.957710125
REF/9,12906,6370,57.724237016,12.039340504
PLY/1,57.689162043,11.869306519
PLY/2,57.697036381,11.883363382
PLY/3,57.705088033,11.912868336
PLY/4,57.712516068,11.930070013
PLY/5,57.726818870,11.982445342
PLY/6,57.765265162,11.996427680
PLY/7,57.763559020,11.998051335
PLY/8,57.764327903,12.001003336
PLY/9,57.761941661,12.003385333
PLY/10,57.761047278,12.000473209
PLY/11,57.750213874,12.010777035
PLY/12,57.713184060,11.994564027
PLY/13,57.681180331,11.876887310
DTM/0.000000,0.000000
IFM/4

There are of course many PLY points but also 9 REF points.

Regarding rotation and course-up. One way would be to simply create x temporary (maybe 36) rotated images of a chart when it is opened and pick the one rotated to the direction the boat is going in, and the clean up the temps after leaving the chart. It would be possible to do this in a smart way and figure out which the next chart that is needed and do it before the boats reach the next chart. To rotate raster charts in real-time is probably a bit to system resource intensive, specially with the design objective on making it run smoothly on fairly modest hardware specs.

3) a better way to pan raster charts (making pan independent by the chart georeferentiation). I know Dave is already working on this.

I agree to all three of your suggestions.

As you say, will be interesting to hear Dave's views on this.

/Jonas

[cagney]

Jonas,
The colour of OpenCPNs chart bar indicates a geo ref error. What's the content of the relevant log-file lines?
Do you know where these charts originates from?

It is indeed a very interesting discussion!
Maybe OpenCPN needs some native support for the Transverse Mercator projection, which seems to be a standard for large scale charts, harbour plans etc.

Thomas

[jonasaberg]

Cagney,
Good observation, never got that about the color.

20:22:00: -------Starting opencpn-------
20:22:00: Version 1.3.6
8:22:01 PM: Using WVSChart datafile: C:\Program Files (x86)\OpenCPN\wvsdata\wvs43.dat
8:22:01 PM: Loading chart db version: V015
8:22:01 PM: Chart directory C:\Users\Jonas\Documents\ftp_down\Charts\9312 gotaalv
8:22:01 PM: GPS Watchdog Timeout is: 6 sec.
8:22:02 PM: Initializing Chart C:\Users\Jonas\Documents\ftp_down\Charts\9312 gotaalv\9312 gotaalv_1.KAP
8:22:02 PM: Georeference Chart_Error_Factor on chart C:\Users\Jonas\Documents\ftp_down\Charts\9312 gotaalv\9312 gotaalv_1.KAP is 0.254348
8:22:02 PM: Georeference Square_Factor on chart C:\Users\Jonas\Documents\ftp_down\Charts\9312 gotaalv\9312 gotaalv_1.KAP is 1.62484
8:22:27 PM: opencpn::MyFrame exiting cleanly...
8:22:27 PM: opencpn::MyApp exiting cleanly...

Would be interesting to know what the:
Chart_Error_Factor=0.254348
Square_Factor=1.62484
means and how OpenCPN evaluates the georef quality.

The chart is an georeffed by the company that has the publishing rights for BSB charts in Sweden.

I had the idea of redoing the georeferencing by bsb2tif... georeferencing it and converting it to kap again, but since it is a bit messy and I don't think it is the chart that is the problem I did not do it. Maybe I should...
Anyway - it is good that OpenCPN senses there is something wrong - but the warning is a bit to subtle for my taste.

/J

[Viking Sailor]

Most likely source of the ref errors is due to OpenCPN trying to apply a mercator poly to a transverse mercator projection. I believe Dave has already stated that OpenCPN does not handle transverse mercator projections correctly.

[cagney]

Most likely source of the ref errors is due to OpenCPN trying to apply a mercator poly to a transverse mercator projection. I believe Dave has already stated that OpenCPN does not handle transverse mercator projections correctly.

I think that is unlikely. I have converted a lot of Transverse Mercartor charts, among them many large scale NZ charts, without getting such an error. The accuracy suffer, yes, but with this kind of error it is more likely to be contradicting values involved in the geo referencing.
I have had these errors from time to time, in the end it almost always boils down to some (silly) mistake, that is easily corrected.
My observations so far, is that OpenCPN is good at catching these errors. I have not seen any false alarms! (Famous last words!!)

Thomas

[jonasaberg]

Viking Sailor, maybe, but this error is bigger than I have seen before.

I think most if not all Swedish charts are transverse mercator and if your are right it would be a bit of a drawback regarding OpenCPN.

Just to check the calibration I converted the chart to bmp, calibrated it with mapcalII and created a new kap.
The result is somewhat different but equally bad, the problem just moved a bit on the chart...

As can be seen OpenCPN still have the color coding indicating poor referencing.

The picture on the left shows the position that was way of in the "official" calibration, its ok now. But just east of it, it is not ok.

/Jonas

[bdbcat]

Hi Folks....
I am temporarily in Nassau harbor, so this is a "drive-by" posting....

Lots of good discussion regarding georef. Here are some points to ponder
1. Why polynomial georef?
a. Modern (BSB3.x) NOAA charts provide embedded georef co-efficients. These parameters are presumably vetted by the cartographer, and in my experience are very accurate.
b. A polynomial solution with cross-terms is the simplest way of supporting skewed Mercator charts. These things are quite common on the east coast of the US.
c. We have seen lots of cases where a scanned chart of supposedly Mercator projection simply is not useable if the standard projection equations are applied. If the scale value is determined by the linear-by-definition longitude (x) axis, and the latitude (y) axis uses Mercator rules with the same scale, poor georef results. Cagney and Marco will support this observation... The chart may be slightly skewed, or the KAP header may indicate the wrong Projection Lat, or.....The only answer here is polynomial georef with carefully selected points.

2. What to do?
Not sure what the optimium solution is in general, but we can say the following:

a. A KAP bitmap is either Mercator, or it is not. If it is Mercator, we need only two Ref points and a Projection Latitude to georef accurately. If it is not-quite-Mercator, we probably must resort to polynomial solutions, so the selection of REF points becomes critical. Latitude(y) points are very important, both in quantity and location. If the chart is unintentionally skewed in scanning, cross terms will be necessary.

b. Marco is spot-on regarding poly solution problems.. If there happens to be only three unique points in the latitude direction, then a 2nd order polynomial is the best that can be done. This will lead to poor georef if the chart is of small scale and covers a wide latitude range.

c. BSB 3.x charts with embedded coefficients should probably be used as intended.

d. Next Version of OpenCPN will properly detect charts which are declared to be other than Mercator, and produce warnings to that effect. We maybe ought to refuse to show them at all unless we can show them accurately?

e. One proposal: On declared Mercator charts, try to use a pure Mercator solution using some of the REF points, and then check all of the REF points for accuracy. If not accurate enough for navigation, try the best least-squares poly solution available. Check the REF points again for accuracy. If still not good enough, indicate to the user that this chart accuracy is poor, and errors of up to "xxx meters" may be expected.

3. Other issues:
a. Transverse Mercator: This is a whole different beast. 9312 is a case in point. TMerc projections look a little bit like Mercators only along the Central Meridian at small scale. Along this longitude, latitude(y) dimensions are linear, and longitude(x) dimensions are cylindrically projected. This is a bit odd if you are expecting Mercator values. As you move away from the Central Meridian at large scale, the lat/lon lines appear to be "skewed" with respect to the screen. This is normal. I think the Central Meridian for 9312 is 15.808278, from the KAP header.
So, for charts like 9312, we either have to use TMerc algorithms, tweaked to the REF points, or we use full poly solution with cross terms to handle the "apparent" skew, or declare non-support. I don't think a simple rotation will do the job....

Not sure why 9312 poly georef was so poor, though. Should have worked OK for large scale chart at least....Hmmmm

I worked a bit with 6144 (Sandhamn), also TMerc, and had some problems getting an acceptable match with standard TMerc algorithms. Maybe someone could send me copies of 9312 and some other TMerc charts with "official" KAP headers so that I can dig into this more fully?

Any clarifications to my understanding of TMerc will be appreciated. They are new to me in navigation applications, and the math is ugly....

Another fly-in-the-ointment: Quilting is coming. I have test code running now. How in the world are we going to use a TMerc projection on a Mercator quilt? Reprojection of the bitmap to Mercator is one solution, and will be slow with some loss of pixel quality.... Ideas?

Whew...Lots to think about here. If any of the above sounds like whining, please disregard. This whole topic is very interesting, and to me is very high in the list of things to fix or do better. There are no clear answers.

Comments solicited.

Thanks
Dave

[manimaul]

Originally Posted by bdbcat

How in the world are we going to use a TMerc projection on a Mercator quilt? Reprojection of the bitmap to Mercator is one solution, and will be slow with some loss of pixel quality.... Ideas?

Could we offer permanently reprojection? Pass TMerc projected charts to gdalwarp and libbsb when loading new charts. Opencpn will not use original TMerc KAP but newly generated Merc copy.

Where are some sample TMerc charts? I can play around with gdalwarp and see if this is feasible.

Originally Posted by bdbcat

Another fly-in-the-ointment: Quilting is coming. I have test code running now.

Is this test code available for testing? CVS branch?

Originally Posted by bdbcat

e. One proposal: On declared Mercator charts, try to use a pure Mercator solution using some of the REF points, and then check all of the REF points for accuracy. If not accurate enough for navigation, try the best least-squares poly solution available. Check the REF points again for accuracy. If still not good enough, indicate to the user that this chart accuracy is poor, and errors of up to "xxx meters" may be expected.

This seems to be the logical solution given a clear indication of chart error. Possible error presentation ideas:

1. simple dialog
2. embossed chart error message similar to depth unit
3. semi-transparent error range circle around ship

[Extemporaneous]

I have followed and read 99% (thousands of posts right from the beginning) of all the posts made on all of the OpenCPN threads, and I gota tell ya, you guys are amazing!
I watch in awe.
My hat goes off to you all.
Oh ya, and thanks for the hundreds, more likely thousands of hours you guys (Special thanks to you Dave) are putting in to make such a great program for the benefit of all.

My Best Regards,
Extemp.