Downloading Only the Changed RNCs / ENCs

[allanpeda]

After downloading charts for OpenCPN, I noticed that the NOAA website offers two XML files (ENC and RNC) that they describe as being useful to fetch only the changed charts.

That sounded like a nifty idea: write a script that checks for updates to the charts that I have downloaded. But what of newly added charts? Should I care? This would be more of an intellectual exercise, as I just sail around the NYC area, but still it would be nice to always have fresh charts. I would release this under a GPLv2 license.

I started to look at these NOAA XML files, and honestly I think they offer more data than I need. I see scads of metadata about the charts, such as bounding polygons, "nm update agency", etc. What I would like to do is simply refresh the charts I already have, perhaps broken down by some sort of convenient grouping like coast guard district. Ideally I guess bounding longitude / latitude would be best but I'll bet that a tough nut.

I may simply skip all XML parsing and crawl the directory and save timestamps for the charts I have. I googled a bit, but I found no data dictionary for these XML files explaining each field. Does such a document exist?

It reads like this:
DS_Series
composedOf
DS_DataSet
has
MD_Metadata
parentIdentifier
parentIdentifier
gco:CharacterString>NOAA ENC Product Catalog</gco:CharacterString> *Finally*

I see some related data definitions here and here, but I think I am headed in the wrong direction.

Honestly with disk space so cheap it would seem simpler just to grab everything that is newer than the files I already have.

Thoughts?

[nohal]

allan,
I've partly working plugin for OpenCPN doing exactly this... If you can read (or better write ) C++ code, have a look at https://github.com/nohal/OpenCPN/tre...artdldr_pi/src
No time to finish it right now, but it's on the TODO list...

Pavel

[bdbcat]

pavel....

Been meaning to ping you about this. OCPN 2.5 has some new PlugIn API stuff that should make your PlugIn chart updater easier to create.
Take a look if you have the time.....

Dave

[nohal]

Dave...
I've seen it in the version announcement but didn't have a look yet. My biggest problem actually is to make the http client run in another thread reliably on all platforms. C is still a bit low level for me I will get back to it when we have that chart thing going...

Pavel

[Beausoleil]

Originally Posted by allanpeda

Honestly with disk space so cheap it would seem simpler just to grab everything that is newer than the files I already have.

Thoughts?

Sorry, but in today's cruising environment, disk space is not the limiting option - it's the bandwidth available. Physics is much more amenable to stuffing more and more data into a given space than it is to transmitting data remotely over a bandwidth-constrained channel. Broadband wireless requires a land-based network infrastructure, which doesn't exist at sea.

Seriously - I just got back yesterday from being part of the support crew for Diana Nyad's swim attempt across the Florida Straights from Havana to Key West. When in the middle of the straights, we were only 50 miles from land, but our only options for data communication were an Inmarsat BGAN terminal brought by an embedded CNN News crew, and a couple of Iridium phones with 9,600 bit/second modems. Imagine CNN's financial guys' response if we were to re-download the entire NOAA ENC/RNC library at $15/MB when all we needed was the latest chart for the Florida Straights...

Imagine if we were on Beausoleil and our only data connection was a 4800 b/second connection via HF e-mail... Or in the Bahamas where a typical Internet connection is a whole island sharing a single satcom link.

What you're asking for is fine for route planning from home, but for bandwidth constrained sailors, it's not feasible.

[StanL]

Perhaps a quick to implement version would be to grab all newer charts. A script such as this could save lots of people time.

If you still had the time & energy, adding a feature to limit by some region would be nice.

It seems to me the most important, and most commonly used, would be to update the charts we already have. I'm not sure how often new charts are made, but would think old charts getting updated is much more common.

In any event, it looks like a great idea.

[btrayfors]

It ain't really a disk storage problem because you overwrite the old files with the new ones :-)

And, as for the bandwidth problem, simply choose the chart or charts you need...no need to download a whole region or CG district.

Iridium at 9600 baud? Really??

Globalstar has 9600 baud, but I believe Iridium is limited to 2400.

Bill