VentureFarther Satellite Charts - vfkaps plugin

[petter5]

It seems that there is a problem with the KAP - see attached

Originally Posted by cagney

That is strange!

If you look at the beginning of a vfkap file, it looks like this for me:

! Created on www.VentureFarther.com - Use at your own risk!
VER/2.0
BSB/NA=VF39_8.086_N_76_26.213_W
    NU=VF39_8.086_N_76_26.213_W,RA=1280,1280,DU=0
KNP/SC=47728,GD=WGS84,PR=Mercator,PP=39.13476

Your error message indicates that the BSB key word on line 3 is missing.

I just created a new vfkap chart, without problem.
I suppose that this is a server-side error....?

Thomas

[jonturgeon]

Yes, it is a problem with the kap. First clue would be the file size, way too small for a valid kap. Second is the name. It should have a country name at the beginning. I added that recently to help users organize the files since the plugin saves all files to a single folder. It could be a server side error but maybe not. The kap engine takes a list of parameters to build the kap. If any of these are incorrect then there would be an issue. I need to make the error reporting for this better. First thing to help track down the issue would be for the plugin to log exactly what parameters are passed. Then we could see, if the parameters that are being passed are correct or if there is a server side problem.

[cagney]

Now on a good free wifi. I have problems replying on this forum using mobile hotspot through AT&T.

@rgleason

Thomas, are you suggesting that the VF server is discriminating against 64bit users?

No. VF works fine for me

@Petter5
Your latest attached file is an html file and not a kap file. Please try again.

@jonturgeon

Second is the name. It should have a country name at the beginning. I added that recently to help users organize the files since the plugin saves all files to a single folder.

A kap downloaded today, including a recent git update, still uses the "old" naming structure.

Thomas

[jonturgeon]

Hi Petter, I think I know what might be happening. I think you are trying to generate kaps in Norway? I suspect this because of the GPX you sent me which I am working on now, almost ready to release a version that allows it. Here is what I think is the issue. VF uses a spatial DB to locate a coordinate to a country/state (province, districts etc). The original source for this data had a problem with Norway. What I found is Norway's districts were "orphaned" not associated with a country. While underway today I was able to get enough cell coverage to correct this data. So, if you were trying to generate Norway area kaps, please try again. If all works right you should see the start of the kap file name be "Norway".

[Rasbats]

Jon/petter5,

With v0.4 the 'old naming' scheme is being used (No country name). When we move to zips (v0.5?) your country name will come through as part of the file name.

Mike

[rgleason]

@Thomas
The VF website via browser works fine for me too.

I am suggesting that when a 64 bit machine logs in with its user id, there is some 64bit to VF issue which is causing trouble with completing downloads that are good, such that scanning the chart db automatically is not possible.
MS win10 recent updates made python2.7.10 incompatible for example. Things are changing in the 64 bit area. The VF servers may not be aware of these changes and may be using some 32bit software or calls that are no longer compatible.

[rgleason]

PS chartdownloader uses the api to scan the chart db with no problem. And it works on my 64 bit machine. For example.

[rgleason]

Automatically.

[jonturgeon]

Hi all, one thing to note for people having problems using the VFKap plugin. If you get a minute could you try creating the same kap directly on the VF site? Please do it for as close to the area/zoom and sat provider as possible. You would of course need to manually "Scan and Update" the chart DB in OpenCPN but it could help narrow down the problem. If you do get a chance to do this could you also let us know which operating system you are using, 32 or 64 bit and the version of OpenCPN. If you do run into a problem going direct, please let me know. Thanks for the help.

[Rasbats]

Some observations:

#1 The ChartDownLoader plugin uses ForceChartDBUpdate() but because this is integrated into the OpenCPN core it only has the effect of ticking the ScanCheckBox. This means that when the user leaves the Toolbox->Charts->ChartDownloader page the scan takes place.

Using ForceChartDBUpdate from another plugin will have no effect unless the plugin has made an entry in the Toolbox->Charts page (like ChartDownLoader) (Is this possible?)

void ForceChartDBUpdate()
{
if( g_options )
{
g_options->pScanCheckBox->SetValue(true);
}
}

#2 'UpdateChartDBInplace' filters down to 'UpdateChartDatabaseInplace' in the core. It is found in 'MyFrame::ProcessOptionsDialog' in 'chart1.cpp'. Is this the Toolbox options page? If so 'UpdateChartDBInplace' is not going to be actioned without the Toolbox being open.

#3 'AddChartToDBInPlace' from a plugin leads to direct action in PluginManager. This refreshes the chart bar on 32bit but does not seem to refresh on 64bit. Could someone look at 'pluginmanager.cpp'

if 'g_boptionsactive' means that the Toolbox has to be open then the chart will not show on the chart bar. But it does on 32bit but not on 64bit. Can I set 'g_boptionsactive' to true from the plugin?

Mike

[rgleason]

Good sleuthing Mike. Perhaps TransmitterDan, Pavel or Bdbcat can comment. Would it be worthwhile to add an external plugin feature, to be able to update a particular chart dir that has been changed? Also to update chart db only for the highlighted dir to save time?

[Rasbats]

v0.5 has now been pushed to GitHub.

https://github.com/Rasbats/vfkaps_pi/releases/tag/v0.5

Some major changes:

#1. Uses zip format for downloads to improve speed.
#2. Ability to download surrounding charts as well as the selected area.
#3. 64bit Windows problem MAY be resolved.
#4. Option to delete the last file(s) downloaded. Allows changing satellite source for the charts.
#5. Area for selection better highlighted.
#6. Surrounding charts cannot be downloaded if zoom level less than 12 i.e. very small scale.
#7. Direction arrows to move to adjoining area for download.
NOTE: Press 'O' to release the box.
#8. Selection box changes in size to reflect size of satchart that will be downloaded. TODO: This is a major upgrade and the wiki instructions will need to be edited.
TODO: Extensive testing on Linux.
TODO: Resolve directory saving problem on Linux. Preferences does not retain the correct directory.



Please test and comment. I would like to move this to production shortly.


Mike

[rgleason]

Jon, I've tried to upload a cleaned up nmea file with just RMC, VBT, ZDA sentences. I have added date information to about 10 of the first RMC sentences and have added one ZDA sentence.

When I try to upload it, it just sits there, even though it is only 1.3mb. I believe the problem now is that I did not adjust the checksum of those RMC sentences or the ZDA sentence. At this point I don't know how to do this, but could find out how.

Nevertheless, the average user is not going to be able to do this kind of stuff, or bother to spend the time.
I think we need a plugin that will check a file, filter unneeded sentences, and check for anomalies like speeds >15 knots (user adjustable) which remove the offending sentences that create the anomalies, makes sure that date, time are ok, permits reduction of data by user setting of some timer (say between 0 seconds and 1 hour) and verifies all checksum then saves the file for upload.

[jonturgeon]

Hi Rick, as we have talked the file that you are trying to upload is lacking a date/time in your RMC sentences (might be because it is coming from an "Integrated Instrument"?). Not sure why this is happening but VF relies on this to calculate the current tide level so it is a bare minimum for processing.

As you also mentioned, since the RMC sentence should contain your position for a date/time (very important in a NMEA network), to insure the integrity there is also a checksum as the last attribute. So by manually changing the sentence it can not be processed since the checksum will not match. Since I was looking for the date/time from the RMC sentence I do not trap the ZDA (just date/time) sentence. I will add that but your file does not contain that either. Since this recording lasted 30 hours (what you told me earlier) the tide level will need to be calculated multiple times, so all of the RMC sentences need to be correct.

Now you mention that the file is only 1.3mb. You do realize that this file contains over 37,000 rows of data with half of them being depth recordings? I am not saying that this can not be handled, but these recordings happened over a much longer time frame than I had originally intended. I know that I should have planned for it, but my recordings generally last under 1 hour, pretty much to map out an anchorage (which was what I thought would be the most valuable to people and how it would be used). I insert into the DB each depth from the recording, so from this file there would be over 18,000 separate inserts into the db (fairly slow). I will change this to do some filtering before inserting and this will help performance. Less inserted rows, faster processing.

So given this, I am not sure if a plugin is actually needed at this point. What I need to do is generate a large nmea recording, and use that to do some performance testing and code modification. I will also add more feedback if the recording is lacking required info. Since VF is potentially where this info will be stored, it is where all the checks need to take place.

[jonturgeon]

Mike, nice work on the plugin!