Tactics Plugin

[bcn]

Originally Posted by bcn

People racing top notch prototype yachts may have some different requirements, but my feeling is that data coherent within 0,1 to 0,5 s should be sufficient.
Data recording in the 1 - 2hz range.

This should read in the 10 - 2 hz range of course....

[Canne]

Originally Posted by rgleason

I tried to connect O to an artisense outputting nmea2000, thinking it was nmea0183 (boatyard communication problem), and I was getting strange figures in the nmea window, but O was accepting very high speeds. I think it was 105 mbps.
So maybe it could run faster? Maybe that would help with latency?

Yes, increasing the overall throughput in O would be an implementation of my solution proposal (2) while waiting for NMEA 4 with timestamps (1) to hit in : being able read and sink faster than the upstream can send is the essence of the solution proposal (2): keeping the upstream buffers empty allows to retrieve all data the sensors can send without extra delays and this way leaves us with the communication and serialization delays only (which we would have anyway). More data allows also to do better statistical calculations both on-line and off-line (albeit the lack of timestamps makes the off-line calculations practically impossible).

My implementation proposal was to do the same outside of O only for some key parameters but if it can be done by O for all data nothing could be better! Is there any performance tests being conducted in typical installations with multiplexer(s) and/or multiple data sources like USB and WiFi talking at the same time? No breaks on at the reception, no data loss caused by this or by some scheduling issue and no extra tick-type of delay in the delivery of the sentences to the plugin - is there any test plan and report available for this? This can be a feature request for O v6, of course.

[Canne]

Originally Posted by RonSouthworth

...Wanting to know what a value is every 100ms is not a small demand, plenty of real time OS solutions (SCADA) with real time databases, lots of tools to very easily create dashboards...

Dear Ron,

I am not mixing my profession (SCADA and DAQ) with the low speed, non-industrial sensors in my boat. It is not rocket science to get a ten or so samples per second from a badly calibrated wind vane, heel sensor and electronic compass or so and to make sure that when I inspect, say the sample index [2] of each of them to make calculations I am sure that they are all taken at the same time, plus or minus some known jitter. Let's not overdo it, the the problem statement is simple and for sure one does not need a full blown SCADA system to implement the solution, I am the first to tell that.

But is it easy to resolve with the existing commercial boat instrumentation and O, that's the question where I have not yet found an answer.

[Canne]

Originally Posted by bcn

This should read in the 10 - 2 hz range of course....

Hi, I would be more than happy with a 10 Hz sampling rate on four or five major sensors! That would allow max. 5Hz signals to be analyzed, in a simple way at least (1/2B). I don't have a maxi yacht and I did not know that they are using O on those: but even in my modest 38 ft. racer I know that when it hits the bottom of a big wave, the wind vane is continuing its movement by its inertia and then oscillates, for example.

But never mind the sampling rate and frequency - the key question in the RFC was "how to associate the values coming from a known set of key sensors in time" (like at the bottom of the wave). Timestamps availability seems to be out, at least for my boat. Any ideas in the time domain?

[Canne]

Originally Posted by transmitterdan

O has multiple threads.

Sorry, I should have been more precise. I meant that instead of using wxThreads which shares the namespace and the main process with the GUI, I was thinking more like a modern browser type of solution where the the GUI process forks or requires the presence of a dedicated communication service with independent namespace and no dependencies to wxWidgets. the GUI and communication process would communicate through an IPC. The latter could run, for example, multiple threads or even other processes per each communication channel opened . At least in Linux it would be easy to define a dedicated CPU and IRQ affinity for it and another settings for the GUI part, so that both would be well served. )IMO, also each and every single plugin should run like this.) I know that would be a major major architectural decision and burden to implement so that's why I proposed to put it in the v6 feature wish list only.

But this is leading us away from the main subject of my RFC: how to assure the data integrity for some key instruments in the time domain in ov50?

[rgleason]

Jon Gough has had some great successes with FAI (Fully Automated Installation) as coined in this post. While creating additional packages, see ocpn_draw v1.6.4 he has also been editing/correcting the Developer Manual CI: Push build to Git Release. He spent extra hours on building and pushing .travis and I believe he added some notes on that.

I notice the instructions are quite skeletal with respect to building OSX. The note:

Plus, take the dashboard_tactics_pi.pkgproj.in where I modified the key NAME as instructed. Repeat the build process from zero on an empty build directory and let me know.

leads to the question where do the buildosx/installosx/ files come from and how do we create them? dashboard_tactics_pi.pkgproj.in and package_background.jpg ?

Some of the changes needed to the giant xml file plugin_name].pkgproj.in file for OpenCPN Version 5.0
https://github.com/jongough/ocpn_dra...2d219cede657f0


Perhaps we should have a new thread about this? Call it FAI Fully Automated Installation Building?

[transmitterdan]

If the instruments (wind, depth, speed, etc.) that create the NMEA message have no concept of time then I think you cannot achieve nirvana in respect of knowing precisely when a data point was taken.

But it may be possible to have the O5 multiplexer tag each message with an arrival time stamp. That should be pretty close to the actual sampling instant because NMEA instruments are pretty dumb typically. They don’t store and forward messages. Some WiFi multiplexers do that but the message does not stay in the store buffer longer than a few milliseconds.

It should be possible to add the time stamp to the data structure used to dispatch NMEA messages to plugins. Plugins that don’t care about time could ignore the time stamp data field.

[Canne]

Originally Posted by transmitterdan

But it may be possible to have the O5 multiplexer tag each message with an arrival time stamp. That should be pretty close to the actual sampling instant because NMEA instruments are pretty dumb typically. They don’t store and forward messages. Some WiFi multiplexers do that but the message does not stay in the store buffer longer than a few milliseconds.

It should be possible to add the time stamp to the data structure used to dispatch NMEA messages to plugins. Plugins that don’t care about time could ignore the time stamp data field.

Thanks for this! (I did not dare to suggest modifications to ov50 core.) Yes, this feature would be an asset and also a solution, at least in my installations with two multiplexers going pretty fast.

In my work related data handling we also timestamp the acquired data at the reception. Sometimes a data quality flag tells it is bad data because the time sync at remote did not work. But based at the reception time, the scientist can say later on "oh, about at that time I was producing good data" and he just flips the quality flag to position "inspected good"!

This solution is a good compromise to implement (1) in my RFC: even if there is no timestamps on NMEA 0183 we can assume that the reception flow is continuous and with a constant delay. We can also control the timestamps at reception and if we find inconsistencies we can warn the user to check the installation (for those who are interested in this).

VDR player would also profit from the reception time-stamps, making it better testing tool, IMO: it would really play back a navigation session as it was (for souvenir usage one probably still wants to have a FFW!).

[Canne]

Originally Posted by rgleason

Jon Gough has had some great successes with FAI (Fully Automated Installation)

I am glad that it worked for others, too!

Originally Posted by rgleason

I notice the instructions are quite skeletal with respect to building OSX.

Unfortunately I can help only in Linux and Windows part - to start with, nobody has reported a successful installation on a Mac from those deployed OSX packages! Better get that checked first and then revisit the instructions (like what OSX version to use, I just took the default).

Originally Posted by rgleason

Perhaps we should have a new thread about this? Call it FAI Fully Automated Installation Building?

+1 for that, thanks! I would call the thread "Continuous integration CIs" or something like that. The actual phases are building, testing and deployment. Now we address the building and deployment, hopefully one day some unit tests can be glided in. FAI (fully automated installation of OS + SW + config) is something else, in sysadm language.

[RonSouthworth]

Originally Posted by Canne

Dear Ron,




O, that's the question where I have not yet found an answer.

As you say it isn’t rocket science, also why re-invent the wheel equally applies.

The accuracy of the equipment is always a question, like all instrumentation, the equipment your wanting to use has a conversion process or two, to derive the root metric value and transmit it, usually that is dependent on at least one standard, time, if nothing else.

So each device that communicates on the network has a reference crystal / oscillator. Marine instruments are not all sync’d. It is a report on time based protocol. data drifts occur. Flow control often isn’t used on serial / multidrop. UDP is often used on TCP/IP and NAV on wireless networks isn’t taken into consideration.

Cheap crystals have a lot of inherent drift however that is managed by limiting refresh rates to 1hz, 2hz, 5 hz, or more so that the overall impact from crystal drift is for the most part negated.

Providing your comms network is well designed things will arrive as they are sent minus the small delay caused by the transmission channel overhead.

Most problems I have seen are caused due to poor comms layer design.

For marine automation applications i don’t see why time stamping cannot be when a valid message is received at some point ( when it arrives and is processed by O) if a sequence of events is important for your application ?


/Ron

[RonSouthworth]

Like the sound of timestamps on receipt.

[Canne]

Me too, when I hear the echo from the hollow caves of the upstream buffers! +1

[RonSouthworth]

Hi Petri,

I've done some tidying up hopefully all good now on my project repo per pr's and responded, just compiled the recent lot of changes on your efforts with the dashboard_tactics plugin. Still backing up everything for my studio equipment upgrade so no windows vm to play with wxFormsBuilder. The next commit is going to be for improving the windows aesthetic. As I am using form builder the graphic display for degree º sign makes it interesting to fix without breaking the form builder code, I am certain there is a way to sort it within form builder for cross platform support. For now I cheated and replaced the symbol with Deg.

to dashboard_tactics .....

On the surface, the only "item" I can see on macOS that isn't displaying as I would expect is the active route waypoint ID, it isn't displaying in either of the dials. When the tactics waypoint is selected its ID shows up in the dial with no issue.

Using my stand alone simulator I believe that it verify's what the plugin's both are calculating to be the respective values for current direction and speed.

I have pitch set upward at .1 and heel is 0.0. so a negligible value for leeway.

Re Identifying values are derived / calculated values and not direct from an "authoritative external NMEA source", there are a number of ways to handle it as I am certain you know, I like the notion of a simple flag representation on a display some easy reminder using a character like ≈ ~ looks like a minus sign on low font size and old eyes, if it is causing real issues for less tactically minded end users using some platform independent special format character might be the short term work around.

I know there is some interest reading the comms between the core developers of O to review the API and make instruments and the associated value more object based. Maybe in addition to the timestamp some quality flag/rating (eg as is the case with DNP3) may help in the long term redesign process to give some more real time database attributes may help a number of underlying data issues.

I do some work on audio plugin development as part of my studio activities and the notion of a patch bay springs to mind some times when I send different message types and different combinations and sequences to O with routing data elements about the place.... It isn't just an O thing and it isn't alone with expecting things to be a certain way.

Each vendor has their quirks with this. I guess It is one approach to address this type of variation when it happens is my idea. I can also see the argument for it being hard coded as well as it limits mis configuration issues.

And just maybe this sort of functionality is better suited to creating a more advanced raspberrypi 4 gateway device.

Thanks to you and to Rick (and to Tom) for all the work on this plugin. I for one really appreciate it. A feature I have been waiting for quite a while.


/Ron

[RonSouthworth]

Hi Petri,

Some more cross comparisons for you..

Found the current and direction instruments are in the list of items for the dashboard but not being assigned a window and the water temp isn't displaying

/Ron

[FrouFrou]

Hi, after installing O5 it looks like Tactics is calculating the wrong leeway direction. I use the NKE heelangle option in preference. With a heel to port the leeway is to starboard. If I change the radio button on fixed leeway the problem is over.