Weather Routing

[Gilletarom]

Hello All,

Good news. Collaboration progresses. Wouhaouuuuuuu !

Gilletarom.

[rgleason]

This morning after git fetch -all under origin/master I saw a message ..video looping similar to the grib_pi.dll git messages. So I 'git checkout origin/master' then 'git fetch --all' again then 'git branch -a' to make sure the head is on origin/master, then 'git pull origin master'

Then rather than try to download grib_pi.dll I assumed it was part of the origin/master (git). I then ran MSVC and Built Debug and Release. I ran Debug, enabled grib plugin and voila grib has looping and data sensitive popup with checkboxes!

So now people can test it some more. I don't know how to build an install package and perhaps Nohal will do the install package and post it on opencpn.org as an alternative, but I will update the files at this link in a few minutes.
Look above at Post 147 above with the link

Click here and you'll see the files. Then click on each file and download it.

This will update

opencpn.exe,
grib_pi.dll,
weather_routing_pi.dll,
New directory: weather_routing/data
New file under data: WeatherRoutingInformation.html

which are all necessary to run and test the new plugin weather_routing.

Look for Nohal's notice which will simplify installation, then you will only need the weather_routing_pi.dll and other Weather_routing files from this location.

[seandepagnier]

I'm trying to figure out the correct logic for using climatology wind data for weather routing.

Since the wind comes from different directions at various percentages of the time, but it needs to know a specific wind direction and speed for each geographic position at a time.... should it:

1. take the most likely wind direction and it's average strength
2. take the least favorable wind direction with it's average strength (pessemistic)
3. take most favorable (optimistic)
4. take min/max wind speeds to occur (to determine worst possible sea state or longest possible passage)
4. Sail with wind from each direction for the percentage of time it occurs on average in this step (general average)
5. Run many iterations with various possible scenarios and show all the different routings that might occur within the trend.
6. other?

Based on some of the above it would need an extended database to be generated from the raw data to support. This is getting complicated :-/

[rgleason]

I realize I had updated files two days ago. Sean/Jean let me know when you'd like it compiled again and updated for testers please.

I have updated File Thingie with some security so you will need to use 'opencpn' and 'betapass' with the link below.

OCPN Download -Weather_Routing Files, etc.

I have added subdirectorys with WxFax files (sound and wave) and some Grib files for currents (they take a lot of space).

If anyone has some really good wav files for the WxFax plugin, I can send user/pass combo so you can upload them to this link for all to try.

[rgleason]

1. take the most likely wind direction and it's average strength

averages sounds best to me, we know its just estimating.

2. take the least favorable wind direction with it's average strength (pessimistic)

too pessimistic. -Maybe there is a variability value that gets represented somehow? Colored route? Dashes? Fatter line? Something.

3. take most favorable (optimistic)

too optimistic. needs to be more realistic. -Variability value?

4. take min/max wind speeds to occur (to determine worst possible sea state or longest possible passage)

avoid areas with wind speeds > __knots for > __% of the time.
avoid areas with wind speeds < __knots for > __% of the time.

4. Sail with wind from each direction for the percentage of time it occurs on average in this step (general average)

Could do this for each step. So you sail during that step at the given wind angle and speed, and get the position. Then do it again and again, with lots of vectors starting at the current position. Then you can sum all the vectors up with weightings for duration of the wind from that angle, and get a single resultant position which represents the average direction and distance. Wouldn't that be computer intensive and talk a long time to run?

5. Run many iterations with various possible scenarios and show all the different routing that might occur within the trend.

This is interesting to me, because it creates a 'fat' variably intensive shaded route representing the true variation that is inherent with predictions. The denser area, brighter color or different color would represent the a 50%, 25% probability of the better routes being in this area or location...

To reduce computing time is it possible to throw out 30% or 50% of the data that has the lowest probability so there are fewer runs?

--How to string the probabilities together across the entire route....this could lead to all kinds of calcs... Sometimes you are sailing in the highest probability weather patterns and sometimes not. How do you determine when this happens, it is variable.

6. Other?

What are we trying to do here? Give some general routing guidance using averaged gribs or scat data? Averaged how?
Daily, Weekly, Monthly?

Give some sense in the amount of variability in the averaged route: timing, position, wind speeds (max-min)

Keep it simple enough to program and calculate. Keep the data storage requirements down.

[seandepagnier]

Originally Posted by rgleason

1. take the most likely wind direction and it's average strength

averages sounds best to me, we know its just estimating.

Yes this is sort of like a general idea, and should be pretty close in prevailing conditions

2. take the least favorable wind direction with it's average strength (pessimistic)

too pessimistic. -Maybe there is a variability value that gets represented somehow? Colored route? Dashes? Fatter line? Something.

3. take most favorable (optimistic)

too optimistic. needs to be more realistic. -Variability value?

How do I compute variability?
I thought with these two it would sort of give bounds to best/worst case scenarios, but considering how neither is realistic it may not be very useful

4. take min/max wind speeds to occur (to determine worst possible sea state or longest possible passage)

avoid areas with wind speeds > __knots for > __% of the time.
avoid areas with wind speeds < __knots for > __% of the time.

It already avoids areas for any amount of time, but it is not based on percentage. This is much harder as it would have to compute the route to the destination before knowing the percentage, then weed it out.. but this would possibly reveal previously eliminated less optimal routes that do not exceed the limits. I imagine this could take excessively long to compute and also be difficult to implement.

What I meant, was to try sailing with whatever the highest possible winds in the climatology data are for that position in that month, and also try with lowest winds, to give bounds (like above example) except for wind speed rather than direction. Also probably not practical.

4. Sail with wind from each direction for the percentage of time it occurs on average in this step (general average)

Could do this for each step. So you sail during that step at the given wind angle and speed, and get the position. Then do it again and again, with lots of vectors starting at the current position. Then you can sum all the vectors up with weightings for duration of the wind from that angle, and get a single resultant position which represents the average direction and distance. Wouldn't that be computer intensive and talk a long time to run?

While this method would take longer to calculate, not excessively so, because the algorithm complexity gets bad merging the routes... propagating is O(n) merging is O(n^2) so even though I reduced it to O(m^2) where m is about 1/8 of n it is still the computation bottleneck.

Also it could cache many computations initially in a table to make this method no slower. Again.. not sure how practical it would be.

5. Run many iterations with various possible scenarios and show all the different routing that might occur within the trend.

This is interesting to me, because it creates a 'fat' variably intensive shaded route representing the true variation that is inherent with predictions. The denser area, brighter color or different color would represent the a 50%, 25% probability of the better routes being in this area or location...

To reduce computing time is it possible to throw out 30% or 50% of the data that has the lowest probability so there are fewer runs?

--How to string the probabilities together across the entire route....this could lead to all kinds of calcs... Sometimes you are sailing in the highest probability weather patterns and sometimes not. How do you determine when this happens, it is variable.

Yes I think this is most interesting, but also would take the longest and is the most complicated to implement. Don't expect me to do this anytime in the next few weeks I have to slow a lot of developments.

6. Other?

What are we trying to do here? Give some general routing guidance using averaged gribs or scat data? Averaged how?
Daily, Weekly, Monthly?

Give some sense in the amount of variability in the averaged route: timing, position, wind speeds (max-min)

Keep it simple enough to program and calculate. Keep the data storage requirements down.

What am I trying to do exactly.. good question. I need to figure this one out.

I think one potential goal is being able to reproduce the world cruising routes using only algorithms and raw data. It would give much more detail, and visual feedback for passage planing, and may in-fact find some routes that were previously overlooked, as well as being able to describe infinite possible routes, not just specific ones. Also the routes would be specifically tailored to your vessel based on it's characteristics.

Another is being able to extend the range of grib predictions to complete weather routes for longer passages as it can switch from grib to climatology once the grib data has reached the end. It would be nice to use the grib as an initial condition and seed climatology calculations from this state rather than using a general trend, but this goal seems excessively complex. I would love to hear thoughts on this subject.

[Viking Sailor]

For planning use the current/predicted weather and current conditions (scat or grib data) to calibrate/select the climatology data set to be used. This method takes into account and extend the current weather pattern.

[rgleason]

Yes, I like what you're saying Sean. Current grib download (predicted-estimated also further out you go) becomes seamless by using the compressed scat data... Viking suggests that the plugin be told to go fetch appropriate data based on time and date from the resource. (Should probably be an option.)

I am wondering if it would be useful to have Scat average weeks data as an option, then make the months. Then if you want to fine grain it later we have that data? Or would it be easier to just do processing it again?

[rgleason]

1. take the most likely wind direction and it's average strength

averages sounds best to me, we know its just estimating.
Yes this is sort of like a general idea, and should be pretty close in prevailing conditions

2. take the least favorable wind direction with it's average strength (pessimistic)

too pessimistic. -Maybe there is a variability value that gets represented somehow? Colored route? Dashes? Fatter line? Something.

3. take most favorable (optimistic)

too optimistic. needs to be more realistic. -Variability value?

How do I compute variability?
I thought with these two it would sort of give bounds to best/worst case scenarios, but considering how neither is realistic it may not be very useful

-Take the two vectors
A. least favorable wind direction with it's average strength
B. most favorable wind direction with it's average? strength
Use the difference between them.... how...

-Variability Magnitude of the difference represents a range of variability. That is then plotted as a fat route, orthogonal to the average route with the fatness adjusted proportionally....?

-Variability Direction. Somehow we use the direction of the two vectors, drawing a line from the end of one vector to the other, to represent the variability of where the boat could be if it makes it to that current position following the averages.

-Afterthought, don't we need the base data for this? Which leads to the question what is in the base data Ah Ah now I realize its just the like pilot charts with probability of wind from various directions and average strength. So that's going to be in the compressed database.

4. take min/max wind speeds to occur (to determine worst possible sea state or longest possible passage)

avoid areas with wind speeds > __knots for > __% of the time.
avoid areas with wind speeds < __knots for > __% of the time.

It already avoids areas for any amount of time, but it is not based on percentage. This is much harder as it would have to compute the route to the destination before knowing the percentage, then weed it out.. but this would possibly reveal previously eliminated less optimal routes that do not exceed the limits. I imagine this could take excessively long to compute and also be difficult to implement.

--how about search the monthly data for windows of high and low wind conditions in the data and map them. The run the routing around those?

What I meant, was to try sailing with whatever the highest possible winds in the climatology data are for that position in that month, and also try with lowest winds, to give bounds (like above example) except for wind speed rather than direction. Also probably not practical.

I think these two ideas might be merging, see 3# above? How can we represent the worst case and the best case, with colored red and blue vector arrows? The the line between which represents a range of where the boat could be is a black line? This would not take much calculation time and it might draw pretty well I think. Maybe we have to simplify it with just drawing the line between and having 1/2 red for slow and the other 1/2 blue for fast.

Maybe we should be calculating variability data somehow from the 30 years base data and store it in the compressed monthly files with the averages? ---I''m still thinking about this, might be too much.

4. Sail with wind from each direction for the percentage of time it occurs on average in this step (general average)

Could do this for each step. So you sail during that step at the given wind angle and speed, and get the position. Then do it again and again, with lots of vectors starting at the current position. Then you can sum all the vectors up with weightings for duration of the wind from that angle, and get a single resultant position which represents the average direction and distance. Wouldn't that be computer intensive and talk a long time to run?

While this method would take longer to calculate, not excessively so, because the algorithm complexity gets bad merging the routes... propagating is O(n) merging is O(n^2) so even though I reduced it to O(m^2) where m is about 1/8 of n it is still the computation bottleneck.

Also it could cache many computations initially in a table to make this method no slower. Again.. not sure how practical it would be.

From what you say, this sounds like it might be a good way to do it get the best average route, weighted for variability, so the direction and magnetude from the current location is a properly weighted representation of the next point for the boat.

I suppose some of the negative wind directions are going to hold the routing back, but they will be weighted for duration and frequency during the month, so it will be realistic. Same goes for the most optimistic wind directions and magnitudes.

We then just need to represent the amount of variability to expect at each step, and there are some ideas above for a "fat" route and and a red-blue line drawn between the best vector and the worst vector. This might not want to be shown at every step, but every 10th step...or something.

5. Run many iterations with various possible scenarios and show all the different routing that might occur within the trend.

This is interesting to me, because it creates a 'fat' variably intensive shaded route representing the true variation that is inherent with predictions. The denser area, brighter color or different color would represent the a 50%, 25% probability of the better routes being in this area or location...

To reduce computing time is it possible to throw out 30% or 50% of the data that has the lowest probability so there are fewer runs?

--How to string the probabilities together across the entire route....this could lead to all kinds of calcs... Sometimes you are sailing in the highest probability weather patterns and sometimes not. How do you determine when this happens, it is variable.

Yes I think this is most interesting, but also would take the longest and is the most complicated to implement. Don't expect me to do this anytime in the next few weeks I have to slow a lot of developments.

Ok, Sean, understood. How about taking running two routes either side of the best average route, that selects a next step that is 10-20% of the optimal to give a range? Would that be meaningful?

6. Other?

What are we trying to do here? Give some general routing guidance using averaged gribs or scat data? Averaged how?
Daily, Weekly, Monthly?

Give some sense in the amount of variability in the averaged route: timing, position, wind speeds (max-min)

Keep it simple enough to program and calculate. Keep the data storage requirements down.

What am I trying to do exactly.. good question. I need to figure this one out.

I think one potential goal is being able to reproduce the world cruising routes using only algorithms and raw data. --YES It would give much more detail, and visual feedback for passage planing, and may in-fact find some routes that were previously overlooked, as well as being able to describe infinite possible routes, not just specific ones. --Yes Also the routes would be specifically tailored to your vessel based on it's characteristics. --Like it

Another is being able to extend the range of grib predictions to complete weather routes for longer passages as it can switch from grib to climatology once the grib data has reached the end. --Excellent It would be nice to use the grib as an initial condition and seed climatology calculations from this state rather than using a general trend, but this goal seems excessively complex. I would love to hear thoughts on this subject.

[Viking Sailor]

Howabout a process something like --

1. Generate route in OpenCPN.

2. Use grib data as input to climatology plug-in to generate extended grib data.

3. Use route and extended grib data as input to weather routing plug-in to generate optimized route.

4. Update OpenCPN with optimized route.

[rgleason]

I think these two ideas might be merging, see 3# above? How can we represent the worst case and the best case, with colored red and blue vector arrows? The the line between which represents a range of where the boat could be is a black line? This would not take much calculation time and it might draw pretty well I think. Maybe we have to simplify it with just drawing the line between and having 1/2 red for slow and the other 1/2 blue for fast.

Maybe we should be calculating variability data somehow from the 30 years base data and store it in the compressed monthly files with the averages? ---I''m still thinking about this, might be too much.

I am starting to realize that the "Wind Atlas" pic in Climatology Post
represents exactly the variables at each spot the rose is located, so we just have to add the boat's sailing characteristics and intention towards a goal for each of the vectors.

That would yield a "sailing variable field" for the next step and it could be represented by a shaded area or a polygon.

How do you then take that "Sailing variable field" and expand it to multiple steps without adding to computing costs exponentially? (-because of the multiple locations the boat could be located).

Is there some simple way of focusing the computing power on the more likely scenarios and throwing out the least likely scenarios?
If thee wind rose has wind barbs which show one or two of these:
> 50% probability in X direction and speed
> 40% probability in X direction and speed
> 30% probability in X direction and speed
> 10% ...
shouldn't the lower ones, 10% and 30% be thrown out?

One thing that is significant, is that boats are going to select routes that tend to be aligned with faster transit and the dominant trade routes, generally. So if the wind tends to blow in a certain direction, they are more likely to be going with it, otherwise they might have decided a different route, beating being a slow passage. How would this affect the weighting of the variability?

Also any calculations that put the boat more than say 25 degrees either side of the rhumbline route when tacking is required, may not be useful, so those calcs can be thrown out. When going downwind, this angle can be tightened up a little.

Additionally, when a boat strays too far from the best calculated average route based upon the scat averages, progress toward destination may suffer later in the course, so you could possibly eliminate calcs that place the boat too far outside the rhumb line route, one side or the other, sort of a bandwidth.

I'm running out of ideas to reduce the calcs for the variability right now. More later.

[Viking Sailor]

There are logically only two valid methods of generating an optimized route. Route optimization can be based upon either historical or current/predicted weather/sea conditions. The problem with historical data is that it only provides a statistical view of what has already occurred and may be nothing like the current conditions. The problem with using predicted conditions is that they will rarely cover the time frame of an ocean passage. Thus, what is need to do proper weather routing is a way to combine current and historical data to provide an extended weather/sea prediction that does cover an entire passage.

Traditionally weather routing uses the most likely historical winds and currents data to generate an optimize route. Without additional input it is statistically unlikely that a more correct route could be generated. And, any route that is based upon less likely weather conditions is unlikely to be useful as there would be no reason to select such a sub-optimal route.

It is possible to start with the current/predicted winds and currents as a basis for deriving a set of extended weather predictions from the historical data. By matching the current weather conditions with historical weather data it is possible to determine how the current weather pattern is most likely to evolve. These extended predictions would then be used as input to generate an optimize route covering the entire passage. This would also allow incorporating new weather/sea data to update the optimized route.

[seandepagnier]

Originally Posted by Viking Sailor

Traditionally weather routing uses the most likely historical winds and currents data to generate an optimize route. Without additional input it is statistically unlikely that a more correct route could be generated. And, any route that is based upon less likely weather conditions is unlikely to be useful as there would be no reason to select such a sub-optimal route.

Well.. yes. This would work well especially for routes in the prevailing winds. For other locations (like the variables) it would not work.

It is possible to start with the current/predicted winds and currents as a basis for deriving a set of extended weather predictions from the historical data. By matching the current weather conditions with historical weather data it is possible to determine how the current weather pattern is most likely to evolve. These extended predictions would then be used as input to generate an optimize route covering the entire passage. This would also allow incorporating new weather/sea data to update the optimized route.

I know it is possible, but I am unsure of the best method for doing this. Could you get more specific. I feel like doing this wrong is worse than not doing it at all.

I feel like I maybe opened a big can of worms with this and they are crawling in all directions.

So back to world cruising routes... some boats sail upwind pretty well, others don't. Some reach 3 times faster than they can run. This could drastically change the world cruising routes generated.

[PjotrC]

Oops, I am falling behind even on reading this interesting thread, to say nothing about coding...

Originally Posted by boat_alexandra

So you mean the route properties already in the main program? There is currently no correct way for a plugin to communicate with the route properties dialog

True, it's a pity, because it makes sharing my tool difficult... Still, I can share some ideas and experience...

I had no option then but to implement the desired functionality in just the way I did. It gave me a nice tool that did serve me well, and will last for years, albeit missing all the novelties. Also, an early tester with racing interest happily reports using its elementary isochrone facility (which I found not very useful for cruising passage planning) ... :-)

I attach an example chart of global currents from Raytech Navigator. It is a freely downloadable demo with almost full functionality, including the monthly global currents, shown for current month. These files are probably proprietary, but a similar set of GRIBs combined with COGOW wind averages would make a nice start for GRIB equivalent of "pilot charts".

As to the method, one should remember that the approach _is_ very different for a 300-mile, 1000-miles or 3000-mile passages... It also may depend on the area and many other factors. For me the main requirement for any automated tool is maximum simplicity and as few clicks as possible

As for optimization (for cruising, not racing) I believe more in hand-optimized routes, so I want rather an evaluation tool than an optimization-tool. The main thing I am missing in automatic optimization is the measure of degree of confidence i.e. what if the actual weather will make me miss that one crucial tack ?

[rgleason]

I've GIT grib_pi.dll and opencpn.exe from origin/master branch
and GIT cloned weather_routing_pi.dll from sean/grib_enhancement.
Then GIT checkout origin/master and cmake and MSCV Rebuild for Release and Debug.
After copying the build/plugin/xxxx_pi/debug DLL to the build directory
tried MSCV Debug and both plugins worked. Did not test extensively.
Then copied the release versions to the this link, so there are new
opencpn.exe (zip)
weather_routing_pi.dll
grib_pi.dll (I don't see Jean's changes here yet)
and the directorys & file under weater_routing

OCPN Download -Weather_Routing Files, etc.

File Thingie now has a login user: 'opencpn' pass: 'betapass'

I have added subdirectorys with WxFax files (sound and wave) and some Grib files for currents (they take a lot of space).

If anyone has some really good wav files for the WxFax plugin, I can send user/pass combo so you can upload them to this link for all to try.