Pilot Grib Files - Similar to Pilot Charts

[rgleason]

I had a question from a friend about planning some long trips with grib files. He has become familiar with OpenCpn and qtVlm. He asks if it is possible to use qtVlm to simulate routes during different times of year?

1. Pilot charts perhaps which OpenCpn has available, but not in a grib format?
2. Use a public archive of Grib files to create "recorded" grib data for different times of the year (typical weather at those times of year).. These would be looking back, not forward, not "predictive" but "recorded". - Pilot Gribs (a full year of the entire world, typical "recorded" weather).

Pilot charts show the average wind speed & direction, whereas these Gribs just show what happened. At some point there could be an averaging of some sort.

Then Maitai had a great suggestion on La Taverne, sending me to a link to a website:
Routing in the past with Old Gribs.

This appears to be a piece of software that appears to compile an entire day from the most accurate hourly gribs for each hour, so that an optimal routing can be done using gribs from dates/times in the past. Despite the clever software, it is going to be a fair amount of work to compile this data.

I am wondering if anyone knows of such a Recorded Routing Grib repository which has this already compiled by year for the world?
OR perhaps there is an easier way to get this complied for last year?

Thanks
Rick

Possible OpenCpn Feature sometime in the future?:
-Display of weather fax (qtVlm does this) and satellite weather in layers with the gribs.
- Join two Saildocs downloads together to reduce the volumn of data... or perhaps to be able to define a polygon with multipple points request to Saildocs. May be more dependent on Saildocs.

[sailorF54]

IMHO, getting the raw data from the CoGow project (end results already available openly here
Climatology of Global Ocean Winds - COGOW) and putting it in grib format would be much simpler and comprehensive

[cagney]

Cogow is the way to go, for now. Why bother squeezing more data or formats from something that haven't been updated for more than 40 years?

Cogow can bee displayed on OpenCPN, in a fashion, already.
Check out this guy Prevailing Ocean Winds » Pitufa.at. Cogow in kml format! Not perfect, but interesting.


When it comes to displaying wx faxes, OpenCPN has done that for years. All wx fax maps in Mercator, including all US faxes, are taken care of by a script and converted to kaps. Check out old thread(s) about this. "Nohal" used to run a script on his server, creating fresh kaps every day for the Atlantic and the Adriatic. I can't find them now...! A sample



The idea of aggregating old gribs seems flawed to me. All the weaknesses and errors in gribs would still be there. To be of any value, updated or modern pilot charts, must continue to be based on real weather observations.

Thomas

[rgleason]

Thanks for the suggestion about Weatherfax overlay in Opencpn. Also COGOW which may be just what is needed.

Thomas, just a reminder, the title is "Pilot Grib" not based on "predicted" but on "actual" historical past - lets say real data instead. That was the whole point...use the gribs that are closest to what actually happened...everything is an approximation anyway, real weather observations are an approximation too.

I agree there are problems with "aggregating" and "averaging" gribs. How do you do it effectively? Perhaps the Cogow is the best thing to try to use for routing. Perhaps compile that information in grib format somehow for the time period of the trip? - I am going to have to read about this more.

I was just trying to get some past grib files (closest to real weather data as possible) that would allow me to run some routes in qtVlm at certain times of year, to answer my friend's question. Rather than complie (and aggregate) the most accurate gribs from last year, perhaps I should look into using Cogow for this.

Thanks both of you for the perspective and suggestions.
The downloads at http://www.pitufa.at/2010/12/prevailing-ocean-winds/ in kml format look good too.

[rgleason]

From JeanLouis on La Tavern. Thanks JeanLouis.

Hi, Rick !
Your 2 links are wrong....
Routing in the past:
http://wiki.virtual-...iens_(archives)

Tool cygwin:
http://www.iges.org/.../downloads.html

[rgleason]

Cagney writes:
Check out this guy Prevailing Ocean Winds » Pitufa.at. Cogow in kml format! Not perfect, but interesting.
------
The author writes:
...have generated kml files for the Atlantic, the Indian, and the Pacific Ocean, one file per month of a year. The files contain prevailing wind arrows at a 2-by-2 degree lon/lat grid. The direction and length of an arrow was obtained by vectorially averaging monthly data over 10 years and over 2-by-2 degree areas (i.e., 32 COGOW data files were averaged for one arrow). The color of an arrow corresponds to the scalar averaged wind speed in knots. In addition, each arrow links to a wind rose image located at COGOW.
----

Is there any way to import this KML into gtVlm which turning it into a longer term grib? Say one for each month?
So we could use it for qtVlm Pilot Grib routing purposes? Testing different routes based on averages?

[Viking Sailor]

At COGOW (Prevailing Ocean Winds » Pitufa.at.) Christian writes:

Download this kmz file (= zipped kml files) to get a world-wide prevailing sea wind atlas which is convenient for off-line usage. You may simply load it into your Google Earth (File > Open…). In your ‘My Places’ panel you will get a list to switch between the months. Displaying them right after loading may take some time. Repeated displaying should work fine.

Might it be possible to use GE2KAP to convert and overlay the GE images as KAP charts in OpenCPN?

[rgleason]

(Prevailing Ocean Winds » Pitufa.at.) At Map » Pitufa.at Select Winds in the upper right then Check North America, and a Month.

This looks very much like it could be used for Pilot - Grib Routing. Is it possible to import as a grib or to convert to grib somehow? The ability to click on a location to see the Wind rose and winds predicted is nice but not essential. This is basically a grib file for the month! Pretty simple.

[rgleason]

These montly files at Map » Pitufa.at , say "prev_winds_north_atlantic_01.kml" are Xml format with the main repetitive part being "Placemarks"

<Placemark>
<name>8.47 kn scalar average</name>
<styleUrl>#tip2</styleUrl>
<LineString>
<coordinates>
8.273842, 1.166456
8.336253, 1.408540
8.110688, 1.300741
8.273842, 1.166456
</coordinates>
</LineString>
<description>
<![CDATA[
<a href="http://cioss.coas.oregonstate.edu/cogow/0101/gif/1.25N_8.25E.gif">View wind rose.</a>
]]>
</description>
</Placemark>


The actual arrows that make up the file are referenced this way
<styleUrl>#tip2</styleUrl>
with each type of arrow being described more completely at the top of the file.

The <description> CDATA link to wind roses would not be needed if it complicated things.

This is a pretty simple structure, with several Lat, Long and references. It may be formatted in accord with OpenKML format but I don't really know.

There must be a pretty direct way to translate to grib format. Should it be Grib1 or Grib2 format?

[rgleason]

Under Pilot Charts this website has some great big PDF Pilot Charts that might be of interest to someone, but not us Sailing Weather - Marine Weather Forecasts for Sailors and Adventurers - PassageWeather . I'd like to get some Pilot Grib files that can be read by qtVlm and OpenCpn.

There is evidently a process to create KML files like above from Grib files. We just need to reverse that.

[rgleason]

Tools and possible Source data and process.
Grib <-- KML or
KML --> Grib or
NOAA Products --> Grib (Ie take Noaa's best grib's from the past, say the 1 hour forecast and string them together for full months. This would be an example and not a Pilot Grib.

1. Take raw data from the CoGow project (end results already available openly here Climatology of Global Ocean Winds - COGOW) and putting it in grib format would be much simpler. -sailorF54
2. Prevailing Ocean Winds » Pitufa.at. Post processed and averaged Cogow in kml format! --Not perfect, but interesting. Cagney.

Possible Tools
PyAOS - Reading Grib on Windows
NOAA's Weather and Climate Toolkit
CygWinWindow

This is getting a little deep I am afraid. It certainly is not importing a KML Route into OpenCPN, but it seems with some database transformation tool we could easily get these into Grib format that could be read by OpenCpn and QtVlm.

[rgleason]

Weather4D - How do these guys do Historical Grib Files (under features), rather than forecast.
KML Output Output (Macro-python)
ECMWF Grib API - this is a little beyond me.

Perhaps if I knew what a Grib format was (it looks binary!!) I could use Notepad++ to work the KML file into the right format?

[rgleason]

Dave about possible use of KML in OpenCpn, (KML Route is done).

I plan to add generic KML overlay capabilities to opencpn. Reading and parsing XML is straightforward with wxWidgets, so I have a starting hook.

Initial plan of attack and proof of concept(simplified): Export a track from opencpn as GPX. Convert to KML. Validate on GE. Write code. Import this track KML to opencpn as overlay. Compare to original track. If all goes well, then...

The next use I have in mind is to generate intermediate KML files from GRIB data as part of the GRIB effort. These intermediate files will then be loaded into opencpn as semi-transparent overlays.

Of course, as a side benefit, these GRIB->KML files could be read in Google Earth, as well. Seems like a useful thing. Maybe I'm not the only one working on this particular path... ...he then mentioned something to the effect that this plan is like jello....

.

[rgleason]

Create a long term 'historic' Actual weather Grib
OBJECT: Collect successive forecasts due immediately so as to constitute a grib file reports, not a long-term forecast, but the weather actually did the sea over a long period.
1. VLM Servers have GFS weather model data stored.
2. ConvGrib Utility - (NOAA) Convert Grib -->Grib2 OR Grib2 -->Grib
3. First Goal Reconstruct a full day with the best actual gribs.
- Select the best gribs and short prediction times.
- Concatenate each of the gribs into one file
4, Procedure is outlined in link very well.
5. A. Grib day based on 8 files. GRIB2 weighs about 3 MB

It would take some time to get a month together and it would be quite big.

It might be simpler and perhaps more useful to take the KML files from
COGOW (Prevailing Ocean Winds » Pitufa.at.) and convert them for each month.
The file would be smaller, and routing could be done with some "Fat" route to represent the degree off variance.

A further development might have an approximation based on some number 1-10 for the wind variability in speed and direction at that time of year at that point.

[rgleason]

Sailover in La Taverne thread below, post 7260
qtVlm - Page 363 - Outils de navigation pour VLM - Navigation tools - La Taverne

Suggests:

A possible way could be the using of the "Wgrib2" (1) utility with the "-import_text data.txt" (2) and the "-grib_out file_name" (3) commands

(1) http://www.cpc.ncep..../wesley/wgrib2/
(2) http://www.cpc.ncep....mport_text.html
(3) http://www.cpc.ncep....2/grib_out.html


I never experienced this way but you should obtain a grib2 file from a KML one throught the following steps:
- Convert KML file into a text file (using KMLCSV utility),
- Convert text file into Grib2 file.
To know what is the requested text file data arrangement, you could convert a grib2 file into a text file using the WGrib2 utility.

Looks like it might be possible to convert KML. Has any one here experience with this? Is it difficult? Thanks.