Charts

[cagney]

Here is a utility program that fits in with Marco's latest posts.
Note that the file ending has to be changed from ".doc" to ".rar".
For now I call it DB.rb , because it is based on the old database library "dbm" and is written i ruby. As it stands for the moment it is a "nix" only program, but nothing prevent a win version, with a bit of tweaking.
It's so easy.

DB.rb 4111

This will do all the work for you and.....a chart file, nz4111.kap will be created for you in the $HOME/kap-charts directory, ready for use with OpenCPN.
To see which charts are available

DB.rb -h

will list them.
I hope some with a linux box available will help out with testing.
You need to have ,ruby including ruby gdbm, wget and of course
libbsb installed.
Keep producing those headers Marco, they can all be included
Anyone using ruby on a windows box?
Nothing prevent this code to be ported to any other scripting language........

Thomas

[GPS-Marco]

Thomas, well done! I do not have linux but I understand your work.

But since it is a really simple task, whay not just use a series of NZxx.hdr, bash in linux and cmd in win? I think it might be simpler to make a portable tool.

I can suggest:
1) we should put the MD5 of the tif file in the header since I think that if NZ map changes, the kap production process should stop (immagine a new correction made on the paper chart by NZ that cause a rescan of the map with a new tif file: the old calibration shall not be used).

2) the system should try to be more flexible and not to be limited to NZ maps.

I would put the MD5 and the LINK to the download page IN the header. So you read the header, find the download link, download the map, check the MD5 and only then you prepare the .kap.

Ciao, Marco.

[cagney]

One reason I'm using "ruby" is the excellent cross platform support, as a matter of fact I regularly taking a ruby program back and forth between work(xp) and home(linux) without problems.
The problem yesterday was mainly with the database, the problems disappeared as soon as I recreated the the database on win.
Ruby for windows is a very easy install, same with wget
http://users.ugent.be/~bpuype/wget/


I have attached a windows version, only difference is that the data structure is created on windows. The beauty of scripting languages as ruby is that the code mainly remains the same between different platforms. Don't forget that we, hopefully, will get a few Mac users
soon.

I have tested this version on windows except for the libbsb bit.
Change the file extension from .doc to .rar

Marco your suggestions are excellent.
Maybe I'll add an config file with multiple options, and some
popup dialogs to help win users not used to a command line

Thomas

[dacust]

Originally Posted by cagney

One reason I'm using "ruby" is the excellent cross platform support, as a matter of fact I regularly taking a ruby program back and forth between work(xp) and home(linux) without problems.
The problem yesterday was mainly with the database, the problems disappeared as soon as I recreated the the database on win.
Ruby for windows is a very easy install, same with wget
http://users.ugent.be/~bpuype/wget/


I have attached a windows version, only difference is that the data structure is created on windows. The beauty of scripting languages as ruby is that the code mainly remains the same between different platforms. Don't forget that we, hopefully, will get a few Mac users
soon.

I have tested this version on windows except for the libbsb bit.
Change the file extension from .doc to .rar

Marco your suggestions are excellent.
Maybe I'll add an config file with multiple options, and some
popup dialogs to help win users not used to a command line

Thomas

Cool!

Might add an option to the config file to get the .tifs from a directory rather than with wget. I already have the tifs.

-dan

[kiwiradical]

Seems there's a lot of work being done with the NZ chart tiffs...if someone has converted them all to bsb I'm happy to provide space to host them so that all us less technically able kiwis can download them to use with OpenCpN.

PM me if you have them and would like to share them.......

[dacust]

I found some maps of Indonesia at the University of Texas site. Not at all siutable for navigation, but nice for exploring. And maybe good to show some land detail if you are around there.

171 charts. I've geo-referenced a few. They are scanned crooked and warped so it takes a few reference points.

Full Size



Here's the detail shot.

Full Size




-dan

[sidetrack]

A wiki sounds like a pretty good place for a header repository. Maybe this one? Cruising and Sailing Wiki by CruiserLog - CruiserlogWiki

I like where cagney is going with that ruby script, and Marco's idea of a checksum in the header file is excellent.

[sinbad7]

Good to see the Panama canal on-line at last... hope they will stay..
Converted a couple of charts for Balboa and Christobal...

[dacust]

Originally Posted by sinbad7

Good to see the Panama canal on-line at last... hope they will stay..
Converted a couple of charts for Balboa and Christobal...

Oh, now THAT's cool. Have that up on one screen and one or more of their webcams up on another. This is almost enough to make me want to set up the AIS feed like you have done.

Where did you get the charts? If they are online, I need the URL. Haven't seen them before.

-dan

[Psyches]

Guys,

For actual chart metadata storage I'd propose something a bit more formal than ChartWiki - that is if anyone has time and space to donate; perhaps one of the cruiser forums would be a great place to host.

By formal, I mean a catalog that is reliably available and programmatically accessible. Otherwise finding the most recent, best chart covering a desired area will become hard to manage. I've spent a fair amount of time over the last few years hunting for both topo and marine charts, and I'm seeing trends: (a) free charts are becoming easier to create and to find, (b) it's getting hard to know where to go to get the best chart. Recent thread activity here highlights this. For topo, it's usually not such a big deal if the chart is a bit old. For marine, your boat and lives may depend on it. While the USCG makes LNTM available biweekly via email, the content is hard to consume electronically, so can take a fair amount of time.

Apparently others, like NOAA, are seeing this trend, so they're starting to standardize on aggregated catalog metadata and support automated management. Here is some doc on the NOAA chart automation infrastructure:

http://www.thsoa.org/hy09/0513P_03.pdf

The current catalogs are xml, but unfortunately don't reference schemas (dtd, xsd, ...); hopefully that will change, but the content is pretty straightforward. I see no reason why an open effort couldn't base catalog structure on the inferred schema, and extend as needed. For example, it could include extended header info, commentary, pointers to more cruise info, votes on accuracy, etc.

With catalogs such as these, it is possible for an app to easily highlight updates and related charts, as well as to automate management of locally cached maps.

Thoughts / comments?

Mark

[sinbad7]

Dan..

Had you puzzled there for a while huh??

The charts come from a MaxSea interpretation of c-map which is a little different from OpenCPN. (drop me an e-mail)

I am working as a marine consultant for 'Sxxxxec' and have a direct feed to their AIS network, to which I am also a contributor from my receiver in the Costa Verde,Brazil.

[sinbad7]

re Chart repository.

The only problem with a meta-data repository is that it pre-supposes that you have access to the original .tif chart. In many cases it is not immediately obvious where they come from and may even be edited scans of various sizes. It becomes difficult or impossible to re-create the exact chart the header was created for.

An alternate option would be to set up a special joint,synchronized storage space for converted -kap files using 'dropbox.com'. The beauty with dropbox is that all the users connected to the chain will automatically have the chart library updated each time a chart is added or updated. It does mean however that ALL the charts will be stored on all the participating computers.

[dacust]

Two unrelated thoughts:

I wonder if there is a manageable way to do this with torrent.

I wonder if someone like Active Captain would be interested in hosting.

-dan

[Psyches]

Originally Posted by sinbad7

re Chart repository.

The only problem with a meta-data repository is that it pre-supposes that you have access to the original .tif chart. In many cases it is not immediately obvious where they come from and may even be edited scans of various sizes. It becomes difficult or impossible to re-create the exact chart the header was created for.

An alternate option would be to set up a special joint,synchronized storage space for converted -kap files using 'dropbox.com'. The beauty with dropbox is that all the users connected to the chain will automatically have the chart library updated each time a chart is added or updated. It does mean however that ALL the charts will be stored on all the participating computers.

good points. If the .tif doesn't really match the header, then shouldn't the header id info be changed to indicate? Otherwise how could one distinguish between good and better versions of the charts over time?

I'm thinking of the marine equivalent of something between the NOAA infra and OpenStreetMap. So there might be thousands of charts available. I see no reason why we couldn't leverage todays formats for use in todays nav programs (notably OpenCPN), and focus on ensuring (a) a consistent place to store, version, and catalog charts, and (b) a means to not blindly sync, but identify which charts could be updated and what other charts nearby are available - all tailored for marine planning and navigation.

I took a look at dropbox.com - hadn't seen it before; it'd work for keeping things in sync, but doesn't offer the degree of flexibility or manageability I imagined. And for thousands of files, or for access from slow or remote locations (satellite, ...), I think you'd want something a bit different. But maybe this is all premature...

[Psyches]

Originally Posted by dacust

Two unrelated thoughts:

I wonder if there is a manageable way to do this with torrent.

I wonder if someone like Active Captain would be interested in hosting.

-dan

Dan, interesting.

I've only briefly looked at the internals of torrent format so I'm not sure - my gut says it's more tailored to larger streams, rather than the "pick-and-choose" style that would work for an automated system. That's because folks doing the updating, either via gui or automated service, would only be getting new replacements for their current maps, or maybe only new replacements as long as the size was < N, etc. I suspect that's why NOAA went with zip encapsulation rather than torrents. But I could be wrong.

It'd be interesting & cool if ActiveCaptain wanted to host. I'm a fan of theirs - I've talked with Jeff before, and he's been amenable to suggestions. But this would be quite a bit different than what they do now. If we had a prototype of this, we might shop it around to sites, including ActiveCaptain.