OpenCPN 3.2 Crash / Invalid NMEA in Log

[kiltym]

Originally Posted by RhythmDoctor

I believe that the autopilot may have been causing your problem. The communications is totally different on 3.2, and all the data goes out to a common bus, which includes the autopilot. Under 3.0, since the output port was specifically limited to autopilot, the sentences were filtered by default. The new way on 3.2 is much more powerful, but more potential for problems, especially if you're using AIS.

It is related to the sending of the RMB sentence, but I am not sure if it is the autopilot, or the NMEA-Seatalk converter. When I send out the RMB onto Seatalk, Seatalk then produces some new sentence, presumably as there is an RMB equivalent on the Seatalk network, and now the converter is converting that back to NMEA. It is something here that causes the crash.

If I set O to send RMB to NavMonPC (which I now have in the middle between Seatalk converter and O), there are no issues. If I have NavMonPC forward the RMB sentence onto Seatalk, I have the crash.

Originally Posted by RhythmDoctor

The reason AIS is a problem is because it has so much more data so it's prone to overflowing the buffers. If you're taking in AIS at 38400 and open an output port at 4800 for the autopilot, you will definitely overflow the port unless you filter the AIS out. Whereas before the autopilot could simply ignore the irrelevant data, with AIS it gets deluged with more than it can handle.

I do not repeat/output AIS on any port in O, except UDP and TCP (although now I have these turned off). The only sentence outputted back to Seatalk and/or NavMonPC(depending on config), is ECRMB, nothing else goes "out".

Originally Posted by RhythmDoctor

I noticed this problem immediately when I upgraded from 3.0 to 3.2. I started getting crashes immediately in my testing (before I took my boat out). As soon as I turned off the autopilot output (which had been turned on because the settings came over from 3.0.2), everything worked fine.

This sounds as the same symptom I have, however I don't think it's related to AIS, or O's output buffers since when I send the RMB sentence only to NavMonPC, I have no crash, only when RMB is sent on to Seatalk.


I am working with Dave to get him a dump of the crash. For now, using NavMonPC, I can workaround the issue by filtering the sentence there, and only sending the 5-6 sentences to O that it requires.

[bdbcat]

Kiltym....

I am finally fully assimilating your system configuration, so can now comment intelligently.

It seems that if you send $ECRMB from OCPN out to Seatalk, he responds with a synthetic $IIRMB. This may be considered as a "loop-back" from the Seatalk<->NMEA bridge.

Here is an example sequence from your logs, I think:
From O:
$ECRMB,A,0.017,L,005,NM001,0046.766,S,09003.410,W, 11.486,283.607,5.700,V*23
Response via Seatalk:
$IIRMB,A,0.01,L,,NM001,,,,,11.48,283.5,,,A*3C

A few points and questions...
1. Could you confirm something like this in your NMEA logs?
2. Are you using Dashboard PlugIn at the time of the crash?
3. Please note the the $IIRMB message appears to be NMEA Version 2.3, whereas the O generated $ECRMB message is NMEA Version 2.2. Just an observation....
4. An any case, O does not process RMB, and strictly ignores RMB on input, regardless of O's input filter settings. The Dashboard reads and parses $xxRMB, but does nothing with the data.

I built a log file with nothing but the above $IIRMB in it, and played it back with VDR. OCPN ignores it, no crash, as expected.

So, I cannot see what is going on. Could be a timing exposure in the NMEA input thread, so would be very hard to reproduce except on your specific setup. It seems not to be a static problem, but subject to some dynamic variables....but what?

I would like to see NMEA logs for your minimal system crash setup, just to confirm our understanding of the effect of $ECRMB. Does Seatalk synthesize more than $IIRMB on autopilot output?

Head scratching, still
Dave

[kiltym]

Originally Posted by bdbcat

A few points and questions...
1. Could you confirm something like this in your NMEA logs?

Yes, this is what happens. When a RMB sentence is sent out, I will receive back from Seatalk:
IIRMB
IIAPB
IIXTE
IIZTG

These are only sent back when an RMB is "active" and being sent onto seatalk.

Originally Posted by bdbcat

2. Are you using Dashboard PlugIn at the time of the crash?

No. Do not use this, although the dll is present, but not active.

Originally Posted by bdbcat

3. Please note the the $IIRMB message appears to be NMEA Version 2.3, whereas the O generated $ECRMB message is NMEA Version 2.2. Just an observation....

OK

Originally Posted by bdbcat

4. An any case, O does not process RMB, and strictly ignores RMB on input, regardless of O's input filter settings. The Dashboard reads and parses $xxRMB, but does nothing with the data.

When I allow NavMonPC to send all messages to O, it causes the crash. When I allow NavMonPC to send the 4 "generated" messages above to O, it causes the crash. When I filter these sentences out in NavMonPC, O is OK. If I filter these sentences out inside of O, it still crashes.

Originally Posted by bdbcat

I would like to see NMEA logs for your minimal system crash setup, just to confirm our understanding of the effect of $ECRMB. Does Seatalk synthesize more than $IIRMB on autopilot output?

I have attached a log, and the crash report from the version of O (remove the .pdf extensions) you sent me. In the log, you can see, at 12:28:08, O is running, and getting Nav Data from Seatalk. At 12:28:16, I activated a route, and in turn, send ECRMB sentences onto seatalk, and receive the 4 II sentences as stated above. I then also get garbage sentences as I stated in the original post, but hard to say if they are a direct cause of the crash or not, as most are handled, and probably ignored. I get the garbage even if ECRMB is not getting sent to seatalk, so I don't think the garbage is causing the crash. Only when ECRMB is being sent does it seem to cause the crash.

Thanks for keeping on this, let me know what else I can provide.

[Hakan]

Hi
I've had two crashes to report as well.
It's on 3.2.0 and Win32-XP but not particularly connected to NMEA although I've a lot traffic trough the multiplexor. Both crashes appeared after a long time, days, running. One during sail so then I just restarted as quick as possible. The other while moored since a day and with the system running. I saw it now and collected some data and dump files. The activated plugins are WMM, Dashboard and the official Grib. The log says nothing, both occurrences are seen in the log file. The attached zip includes .ini, log and some windows dump files. (skip.pdf)
Any more I can do??
Thanks Håkan

[bdbcat]

Kiltym....

OK, thanks for the dump files, etc.

I think I found the smoking gun. I'm becoming convinced that the Seatalk-NMEA converter is sending bad sentences back to OCPN on receipt of $ECRMB.

Now, O should ignore these bad sentences, and usually will.
Unless (and here comes the scary part) the sentence contains unprintable or unconvertable (i.e. non-UTF8) characters. In which case a logic flaw in O causes him to crash horribly. Ouch....

I am working on a clean fix for this logic flaw.

Meanwhile, we can use the OCPN log to see the exact sentences received by the core logic, without interference from the VDR PlugIn.

Please add the following to your opencpn.ini

[Settings]
DebugNMEA=1100

What this will do is flood your logfile with lots of raw NEMA data (the first 1100 sentences received, at various stages of cooking). We should be able to see funny NMEA sentences coming in after you enable the $ECRMB sentence out.

Eventually, it will crash as before.

Post the logfile, and we will take a look at it.

We are getting somewhere, I think...

Thanks
Dave

[Hakan]

Again:
Another crash this morning after approx. one days run. I've Windows tracking set as of Dave's advice in http://www.cruisersforum.com/forums/...ml#post1211762 but couldn't find a proper log file. Any advices?
Log and ini -files attached.
Håkan

[kiltym]

Originally Posted by bdbcat;1218459
Please add the following to your opencpn.ini

[CODE

[Settings]
DebugNMEA=1100
[/CODE]

What this will do is flood your logfile with lots of raw NEMA data (the first 1100 sentences received, at various stages of cooking). We should be able to see funny NMEA sentences coming in after you enable the $ECRMB sentence out.

Will do this. Will use the version of O with the crash capturing also so you have everything I can get you.

[kiltym]

Here is the log and the crash dumps. I ran it twice as the first run crashed pretty quickly.

I don't seen anything of much help in the log file though, no bad sentences seem to be outputted.

I had to split the 2nd log file, so please put them back together for the entire log.

[kiltym]

Dave,

Good news. The version you sent me, 3.3.324, is not crashing so far. I have ran it twice, both for about 1+ hours, and no issues.

In between the two runs, I ran 3.2 again to make sure I was getting the crash, and did after about 15-20 minutes.

So, seems to be very invalid data from the Seatalk-NMEA converter.

So, my question is this, should I stick with this version to use across the Pacific and set up my PC to use it fully, or just stick with 3.2 and the NavMonPC workaround. Just not sure if there are potential other issues with using this test version you sent me, on a more full time basis.

Thanks for all the work on this, and glad we seem to have sorted this issue out.

I have no logs to post, as there was no crash.

[bdbcat]

Kiltym....

Good news indeed.

1. I would stick with your 3.2 and NavMonPC workaround for now. This will be lower risk than the 3.3.324, which has not even been Beta tested yet.
However, we expect to release a low risk Service Patch for 3.2 shortly, which will include this NMEA fix. When this becomes available (a few weeks, estimated) then migrate to the updated 3.2 .x Release, and drop NavMonPC.

2. Your logs would be interesting, even with no crash. If you include the DebugNMEA=1100 line in the opencpn.ini, you should see some indication of the bad NMEA sentences, as indicated by bad checksums.

Thanks
Dave

[kiltym]

Originally Posted by bdbcat

2. Your logs would be interesting, even with no crash. If you include the DebugNMEA=1100 line in the opencpn.ini, you should see some indication of the bad NMEA sentences, as indicated by bad checksums.

OK. I will post a log after a run. I think I need more than 1100 sentences however, but will break the log up to post.

[redog]

Can't find a thread about my problem so please steer me location if already an active discussion.

NMEA goes into an endless loop
1st computer connected to USB puck and outputting UDP 0.0.0.0 10110
2nd computer receiving this UDP, its USB puck disconnected.
(screenshots are 2nd computer)

I have removed all cables from 2nd comp and O is still showing green in the status.
NMEA debug window shows same data in endless loop.

Have I a setting wrong?
This potentially could be disastrous in a live situation.

thanks
redog

[Hakan]

redog..
I’m not sure I fully understand your problems but I’ve some experiences using UDP broadcast. First, the UDP can be rather surprisingly because the operating system will make everything on a single data port, e.g. 11010, available for all listeners, including the “sender”. So the 1st computer will receive an echo of all its outgoing data into itself. To avoid double OCPN handling I’ve a filter on the 1st filtering out all incoming messages. (Choose all in the “ignore commands”) Be sure to not allow computer 2 to send outgoing data, otherwise the echoing will be endless.
Then I think the green status on the 2nd comes from the xxRMC sentences received from the 1st. So yes you don’t need a GPS on the 2nd. It will receive all info from the 1st.

Attached screen shoot is from my 1st comp. (I think you can recognise it though translated to Swedish)

Håkan

[redog]

Haken
Thanks for the reply
I don't have a problem with filtering or priority settings.
I think I have them worked out enough to get a reliable set-up.

The problem is
I still have the second computer totally isolated from any NMEA, the only thing connected to it is the power supply.
This has been the case for some 12 hours now and the status is still showing green.
Even though no NMEA stream has been supplied in 12 hours to this computer, O still shows the endless loop of stagnant data.

The main concern to me is:
If I can achieve this at the kitchen table I'm sure someone could accidentally duplicate this in a live situation.

This could potentially cause someone a lot of grief if they didn't realize soon enough.

I'm sure I read something about this in a thread but I couldn't find it.

Thanks
redog

[Hakan]

redog..
So you've built a perpetual motion machine! The ultimate and cheep navigational gear? No, sorry - joking apart!
What if you take away the outputs on the 0.0.0.0 UDP connection? The UDP broadcast will continue echoing for ever with that set up. And the OCPN GPS watchdog is happy with data within every 6 seconds and don't recognise that's it all the same.
Håkan