mc2bsbh - MapCal Template to BSB Template Conversion

[sinbad7]

Hi Jonas..

All my resulting .hdr files have UN=UNKNOWN
Where does MapCal find the depth info? Or, does it have to be entered manually?
I have so far made corrections to the .kap file manually with a hex editor.

[jonasaberg]

Hi Sinbad,
I think you would do easier on yourself if you quit using the hex editor...
The hex editor is dangerous in that if you dont understand all the details even very small errors will make things not work.

I kind of described a pretty safe procedure in my #7 in this thread.

-Start by converting the image file to a *.png.
-Start MapCalII
-Open the png file by klicking "File->Open Image"
-Click "Edit->Chart information"
-Fill in the name of the chart, a number, a priority number, etc. There is also box for the depth units used and a text box to fill in extra information. Se my attached example.
-Close the "chart information" and create calibration points and border points. (which will be converted to REF and PLY points by ms2bsbh.exe)
-Save the calibration file:
File->Save calibration.
This will generate the CHARTCAL.DIR file including the DU=1 for "meters".

Question:
Does anyone understand the details of the Latitude 0 and Longitude 0 for the projection and the LAT and LONG shift for the chart datum used in MapCalII?

/Jonas

[GPS-Marco]

Originally Posted by jonasaberg

Hi Sinbad,
I think you would do easier on yourself if you quit using the hex editor...
The hex editor is dangerous in that if you dont understand all the details even very small errors will make things not work.

I kind of described a pretty safe procedure in my #7 in this thread.

-Start by converting the image file to a *.png.
-Start MapCalII
-Open the png file by klicking "File->Open Image"
-Click "Edit->Chart information"
-Fill in the name of the chart, a number, a priority number, etc. There is also box for the depth units used and a text box to fill in extra information. Se my attached example.
-Close the "chart information" and create calibration points and border points. (which will be converted to REF and PLY points by ms2bsbh.exe)
-Save the calibration file:
File->Save calibration.
This will generate the CHARTCAL.DIR file including the DU=1 for "meters".

Question:
Does anyone understand the details of the Latitude 0 and Longitude 0 for the projection and the LAT and LONG shift for the chart datum used in MapCalII?

/Jonas

Also this post is a good process explaination #14

Latitude 0 and Longitude 0 are related to projection.

I understood that if projection is MERCATOR, Latitude 0 should be the parallel to which the map scale refers (as you know scale changes with latiture in mercator projection; this parallel is reported on the map normally close to the scale). Maybe the longitude 0 is the central meridian of the TRANSVERSE MERCATOR projection. When you use other projections, other values will be asked but I don't know how to put those values in the BSB header...

LAT/LONG shift refers to map datum. When the chart uses a map datum different from WGS84, you will normally find written on the map two shift values which are the corrections to be used for points coming from WGS84 instruments (like GPS). These numbers goes in LAT/LONG shift of MapCal and will be handled by OCPN to correct map datum shifts.

About missing info in MapCal wrt BSB I've an idea: if we agree a standard "recognizable" metod in wrinting "other informations" like chart edition and corrections, then it will be possible to automatically extract those info to be put in the BSB header...

Ciao, Marco.

[sinbad7]

Thank you Jonas and Marco for the info.
I just had not noticed the depth input area in MapCal. and never used it.
I mostly use the Fugawi Marie ENC calibrating function.
Hex editors are fine when you want to do a small adjustment to an existing .kap
file and I am also familiar with the use thereof.

[jonasaberg]

Originally Posted by GPS-Marco

Also this post is a good process explaination #14


About missing info in MapCal wrt BSB I've an idea: if we agree a standard "recognizable" metod in wrinting "other informations" like chart edition and corrections, then it will be possible to automatically extract those info to be put in the BSB header...

Ciao, Marco.

Marco,
Somewhere down the line it would be good with a dedicated graphical rebuild of MapCal using lib2bsb to directly create BSBs... hope someone will pick up this eventually.
The MapCal interface is really quite good, but it needs minor adjustments.

Your idea is good though: Why not try to interpret the CR field? If it contains a line with
SE=2000,RE=4,ED=1/1/2008
Then it is simply added. Alternativelly, it should be possible to look for "SE=" and use the following characters up to the next "," as input for the "SE" parameter etc.

/Jonas

[GPS-Marco]

Originally Posted by jonasaberg

Marco,
Somewhere down the line it would be good with a dedicated graphical rebuild of MapCal using lib2bsb to directly create BSBs... hope someone will pick up this eventually.
The MapCal interface is really quite good, but it needs minor adjustments.

Your idea is good though: Why not try to interpret the CR field? If it contains a line with
SE=2000,RE=4,ED=1/1/2008
Then it is simply added. Alternativelly, it should be possible to look for "SE=" and use the following characters up to the next "," as input for the "SE" parameter etc.

/Jonas

Yes. Look at the following KNP header extract:

...
KNP/SC=25000,GD=LOCAL DATUM UNKNOWN,PR=MERCATOR,PP=41.24,PI=1.000,SP=
    SK=0.0000000,TA=90.0000000,UN=METERS,SD=MEAN LOW WATER SPRINGS
    DX=2.50,DY=2.50
CED/SE=2000,RE=1,ED=01/01/2001
...

if you want to add the SD=MEAN LOW WATER SPRINGS is a different stuff than adding a row.

I have to propose 2 solutions. One is very BSB syntax oriented and should look like:

1) if in the CR field is present the row

BSBHDR::CED/SE=2000,RE=4,ED=1/1/2008

then a new line "CED/SE=2000,RE=4,ED=1/1/2008" will be added to the BSB header exactly as it is

2) if a line in the CR field like the following is encoutered

BSBHDR:KNP/SD:MEAN LOW WATER SPRINGS

the parameter SD in the structure KNP is given the value "MEAN LOW WATER SPRINGS"

Maybe too complex... But might work. And it is very "future proof": if we discover the meaning of some new parameter next month we don't need to change the mc2bsbh code.

The second solution is less sintax oriented and then easier to read and write: we should agree a common wording to call things. Of course this is not future proof (if we discover something later we have to agree the wording and re-code accordingly). As an example we could put in the CR field

....
This is my chart of Bonifacio Channel
Soundings Datum: MEAN LOW WATER SPRINGS
Chart Edition: 2000
Revision: 4
Edition Date: 1/1/2008
....

Of course mc2bsbh shall be able to recognize all the strings and should know how to convert in BSB header parameter and syntax.

Let me say about a third "revolutionary" solution: A Graphical BSB Header Editor. A simple GUI where all the parameters of the header can be modified (or added if missing). I mean an editor only for parameters, not for graphical calibration. Of course this editor could load a CHARTCAL.DIR through mc2bsbh code or an already converted CHARTCAL.HDR. It might also load a CHART.WCI and extract/convert the calibration values or extract the header from a CHART.KAP. It should look like the "Chart Information" tab of MapCal but it should include the missing parameters.

Ciao, Marco.

[jonasaberg]

Marco,

Regarding your revolutionary idea:

Good idea, I like it.

However, it is getting closer and closer to a utility that might aswell include the graphical part and make it totally open. In the long run it is unwise to rely on MapCal allways being updated to new OS upgrades etc

Rearding the syntax:
I like the idea of a syntax oriented approach that does not need rewrites of mc2bsbh.exe whenever we find out something new. Dan might not have time to fix it directly and it also makes it possible to try and implement new knowledge more quickly

/J

[dacust]

Coding-wise, the first solution would be relatively simple. That is, it would take a fair amount of coding, but it's straight forward. And, as you say "Future-proof". If enough people want that, I'd be willing to do it.

The second one is not something I think I would be interested in doing. It sounds like ongoing work. You have to realize that this is not something I plan to work on forever. At some point I'll move on to something else. But don't worry, the source code is out there, so it won't have to stay stagnant if someone else wants to take it up. And we could put it on SourceForge so it has a permanent home. The reason for saying all this is that I'd like any solution to be a final one and not call for ongoing updates. And I said the rest so you'd know that even if I drop it at some point, it'll still be around.

The third one sounds good, except that is far over my head. I plan to learn how to code graphical apps, but it won't be very soon. Actually, I figure coding a stand alone geo-referencer would be a great project. Code it for various inputs (like the DIR files) and output various formats (like BSB). Built in calls to "convert" and create the BSBs directly. I'd love to code it, and maybe I will, but I have a huge learning curve to get to that point. At least months...

***

A question for you all, how many people would like the batch version? I can do it, but it'll also take a little work. I can easily add an error that there were multiples. But if enough people want it, I can code it to do batches.

If people DO want batch processing, which would you rather I work on first? Batch processing or CR=BSBHDR?

-dan

[dacust]

Originally Posted by jonasaberg

Marco,

Regarding your revolutionary idea:

Good idea, I like it.

However, it is getting closer and closer to a utility that might aswell include the graphical part and make it totally open. In the long run it is unwise to rely on MapCal allways being updated to new OS upgrades etc

Rearding the syntax:
I like the idea of a syntax oriented approach that does not need rewrites of mc2bsbh.exe whenever we find out something new. Dan might not have time to fix it directly and it also makes it possible to try and implement new knowledge more quickly

/J

You posted while I was composing and I missed it.

Sounds like we are on the same page.

-dan

[jonasaberg]

Dan,
I think most of us work that way. We dive in to things as long as they teach us something new or solve a need - and then we move on.

Regarding priorities:

BSBHDR:
I would put this first since I would rather process files one at a time than having to feed BSB headers manually.

Batch:
I think it would be really good. I am planning to scan a lot of charts and it would be nice not to have to split the chartcal.dir file into all its subparts, and then have to convert them one at a time.

All manual fiddling is a large error source when doing things repeatadly.

But I would really like to have both!

Regarding your C++ learning curve - I'd say your doing really well and I hope your still interested in this in Feb/March-2010 or so when your up for a graphics project.

/J

[dacust]

Originally Posted by jonasaberg

BSBHDR:
I would put this first since I would rather process files one at a time than having to feed BSB headers manually.

Batch:
I think it would be really good. I am planning to scan a lot of charts and it would be nice not to have to split the chartcal.dir file into all its subparts, and then have to convert them one at a time.

I'm probably going to have time for some coding this evening. So, unless I hear different I am going to work on the BSBHDR first. Both because I think it is probably the most beneficial, and because it will be easier!

For those of you that look at the source code, next time I post it, you will see the "-b" switch and some code for batch processing already in place. Don't let that throw you. It doesn't actually do anything, yet.

So, another question: Once I have the BSBHDR part working, should I remove the -ced switch? I think so.

-dan

[dacust]

For the BSBHDR I am adding a switch. If you don't specify it, it defaults to "1" which has the behavior I think you expect if there was no switch.

-o [0,1,2] - default is "1"

0 - ignore BSBHDR tags
1 - If data already exists for that tag, replace it with what is in the DIR file.
2 - Only use BSBHDR info if that data is not already in the DIR file.

I'm not sure the switch is necessary, but it's easier to code it in while I'm doing it than try to add it later. And if you never use the switch, no problem.

With no switch (or -o 1):

BSBHDR::BSB/NA=foo   >   replace the NA parm.  
BSBHDR::BSB/XX=foo   >   add an XX parm to the BSB string.
BSBHDR::KNP/YY=foo   >   add a  YY parm to the KNP string.
BSBHDR::DTM=foo      >   replace the DTM parm
BSBHDR::VER=foo      >   replace the VER parm
BSBHDR::CED=foo      >   add the CED parm
BSBHDR::XXX=foo      >   add a parm to the end of the file
BSBHDR::XXX/YY=foo   >   This would be ignored.  Only BSB or KNP can have a "/" subsetted parm.

For the XXX=foo, if it is a parm I know about, I'll add it the right place, like CED, OST, etc. And we can add these if we run into them.

Also, the coding will be simpler if I only allow a certain max number of BSBHDRs for the BSB and KNP sections. But I can set that max to 100, or whatever number sounds so outrageous that it'll never get reached. I normally don't like to code that way, but in this case I'm feeling lazy. As I get into it, I may change my mind and make it unlimited.

Hopefully none of you have gotten too attached to the -ced switch, because it will go away.

I'm not going to code too much until I get comments. I want to make sure this sounds right and see if any of you have better ideas.

Comments? Suggestions? Derisions?

-dan

[dacust]

For batch mode processing, this is what I am thinking.

Always process in batch mode. What this means is it will default to creating the file with the name of the [chart] in the DIR file. Same for one file or many.

no -b switch for batch processing.

Instead:
-h switch to override the output header file name - error if the DIR file contains more than one template.
-e switch to override the extension for the file name. default will be "hdr".

So, you can provide an -e or -h, but not both. And the -h is only valid if it is a single file.

mc2bsbh CHARTCAL.DIR
This would create one or more charts. The name would come from the [name.png] in the begining of the CHARTCAL section. Output file would be name.hdr.

mc2bsbh CHARTCAL.DIR -h mychart.tmp
This would give an error if more than one template existed in the CHARTCAL.DIR. Otherwise, would produce a BDB header named mychart.tmp.

mc2bsbh CHARTCAL.DIR -e tmp
This would create one or more charts like name.tmp

If I do this, be aware that any scripts you may have written using mc2bsbh CHARTCAL.DIR newchart.hdr would blow up.

Thoughts?

-dan

[dacust]

I made some changes and uploaded the new version.

WARNING - the usage has changed.

The output file name is now derived from the chart name in the MapCal DIR file. This is in preperation to doing the batch mode. Took me all morning to make it work, but it finally does.

If the first line in CHARTCAL.DIR has [NZ4314.PNG], then:

mc2bsbh CHARTCAL.DIR
> will create a file NZ4314.hdr

There are two new switches. You can only use one.

• -h headerfilename
• -e extension

mc2bsbh CHARTCAL.DIR -h mychart.tmp
> will create a file mychart.tmp
mc2bsbh CHARTCAL.DIR -e HDR
> will create a file NZ4314.HDR

Because of the changes required, looks like I will be doing the batch processing first, and add the BSBHDR part later.


-dan

[jonasaberg]

Dan,
I think this is excellent progress.
It makes for a nice workflow:
-Scan a bunch of charts. Convert to png using max 127 colors.
-Load them one at a time into MapCalII pressing save between each.
-Exit and batch convert all png to gif to enforce a color table format (using mogrify -colors 127 -format gif *.png in image magic)
-Batch convert to tif. (using mogrify -colors 127 -format tif *.gif in image magic)
-Use mc2bsbh to create *.hdr files.
-Use tif2bsb to create kap files.

Only thing left is a batch version of tif2bsb...
Would this be easy Dan?
A program that would do
tif2bsbb *.hdr

That would run through all *.hdr files, combine them with the corresponding *.tif files to create *.kap files.

What do you think?

/Jonas