Modification of Grib File - Irma-Jose

[rgleason]

I have a 14mb grib file of Irma+Jose and would like to reduce its size by the following means:
1. Tighten up the Time period from 8 days to 4 days.
2. Reduce the area about 60%.

Is there an easy way to do that?
Is anyone able to do it for me?

I need it as an example file for weather_routing_pi manual.

Thanks.

PS: I can upload to O Beta File Thingie

[.Paul.]

Rick,

The easiest way to get a smaller grib is to download it from NOMADS:

NOMADS-NOAA Operational Model Archive and Distribution System

I download the "GFS 1.00 Degree" forecasts of wind (10 meters above ground) and pressure (reduced to mean sea level) from this site using the "grib filter". The GFS forecasts initiated over the last 10 days are available at this site.

The oldest GFS run at this NOMADS site was initiated at 2019 09 06 18Z (year month day hour). The runs are initiated every 6 hours. Older forecasts are archived somewhere, but "grib filter" might not be available for those archives.

For the weather_routing_pi you need the wind forecasts (UGRD and VGRD at lev_10_m_above_ground). If you only need the wind from GFS, let me know the left and right longitudes and top and bottom latitudes, the date and time the GFS run was initiated (within the last 10 days) and the last forecast hour needed in the grib, and I'll download it for you. This should be easier than finding a program to reduce size of an existing grib file.

Paul

[rgleason]

.Paul,

I recall trying to do this with RTOFS 3hr, forecast Gulf of Maine, this summer. The Grib Filters were unavailable because they had just changed servers. I see Grib Filters are now available for RTOFS too.
----
The grib I downloaded runs from
Thurs, Sept 7, 2017 18:00 UTC (for 8 days)
GFS, .25 deg, 3hr,
with Wind, Pressure, Wind Gust, Sea temp & Cape
I don't need Wind Gust, Sea temp or CAPE.
---
What I need is:
GFS, .25 res (1 deg would work), 3hr (6hr would work)
Wind + Pressure
Times: Thurs, Sept 7, 2017 18:00 UTC to Tues, Sept 12, 2017 9:00 UTC
Coords: 17.-63 (N17, W63) and 35,-85 (N35, W85)
left long -85 right long -63
top lat 35 bot lat 17
---
One degree GFS Nomads server with Grib Filter
2019 09 06 18Z (year month day hour). The runs are initiated every 6 hours..

Data Transfer: NCEP GFS Forecasts (1.00 degree grid)
the earliest day is Sept 9, 2017 which is too bad, I wish I had an easy tool that would reduce the grib I have and eliminate the unneeded data.

wind (10 meters above ground)
wind forecasts (UGRD and VGRD at lev_10_m_above_ground)

pressure (reduced to mean sea level) (would be nice)
PRES

URL=
http://nomads.ncep.noaa.gov/cgi-bin/...gfs.2017091612

I got gfs.t12z.pgrb2.1p00.anl with 72 bytes.
This is not working too well for me. What is a anl file?

[rgleason]

I might have lucked out Sept 6

Data Transfer: NCEP GFS Forecasts (0.25 degree grid)


URL=
http://nomads.ncep.noaa.gov/cgi-bin/...gfs.2017090618

That is to big, .25 degree,

I guess I need the SUBSET. for time or something.... I just need
"g2subset (grib2 subset) allows you to subset (time, field, level, or region) a GRIB2 file and sends you the result"

[rgleason]

These are big files, and there does not seem to be any
start date ---> end date/time


Needs to be fairly small. 4-5 days.

[rgleason]

I just wanted that area and date/time so the hurricane was running through and weather_routing would react. One would never do this, but its a good way to learn about routing.

[rgleason]

It looks like wgrid might scan using grep searches and outputs a smaller file. Grep is something pretty new. Also wgrid can run scripts.

ftp://ftp.cpc.ncep.noaa.gov/wd51we/wgrib/grib2ieee.txt

ftp://ftp.cpc.ncep.noaa.gov/wd51we/wgrib/tricks.wgrib

I need to figure out what type of file to output to. "binary"? is that what O uses?

Actually I guess wgrid2 is the right one to be using, but it would take some time.
Climate Prediction Center - wgrib2: grib2 utility

[.Paul.]

Rick,

Zipped grib file attached 2017090618.gfs.t18z.pgrb2.1p00.f144.grib2.zip.pdf

2017090618.gfs.t18z.pgrb2.1p00.f144.grib2.zip.pdf

Rename without the ".pdf". Zipped with 7-zip. Should work with built-in Windows zip utility.

This is 1.00 degree grid.

Let me know if I need to change time range, or other parameters.

Paul

[rgleason]

Some Instructions on Grib Filter
Scripting grib_filter on NCEP's nomads server

Scripts from Nov 2014
WX: GFS 0.25 deg GRIB ~ 12km Model Looks Operational - Sailing Anarchy - Sailing Anarchy Forums

Inventory List of Parameters
Inventory of File gfs.t06z.pgrb2.0p25.anl

Parameters I need.

004 10 mb UGRD analysis U-Component of Wind [m/s]
005 10 mb VGRD analysis V-Component of Wind [m/s]

211 surface PRES analysis Pressure [Pa]

[rgleason]

http://nomads.ncep.noaa.gov/cgi-bin/...gfs.2017090618

You must select from both areas or the file will not be limited & filtered properly.
For GRIB-2 data only.
Select the levels desired:
x 2 m above ground
Select the variables desired:
x UGRD
x VGRD

This is the URL that results

URL=
http://nomads.ncep.noaa.gov/cgi-bin/...gfs.2017090618

g2sub 1.1.0.beta-6 and comments: Wesley.Ebisuzaki@noaa.gov, Jun.Wang@noaa.gov

I get this message.

Data file is not present: /common/data/model/com/gfs/prod/gfs.2017090618/gfs.t18z.pgrb2.1p00.anlg2sub 1.1.0.beta-6 and comments: Wesley.Ebisuzaki@noaa.gov, Jun.Wang@noaa.gov

I might have to write Wesley to ask how to establish the data & time period.

[rgleason]

Yet another Try

URL=
http://nomads.ncep.noaa.gov/cgi-bin/...gfs.2017090700

Data Transfer: NCEP GFS Forecasts (1.00 degree grid)

It left a 0 kb gfs.t00z.pgrb2.1p00.anl

[rgleason]

Originally Posted by .Paul.

Rick,

Zipped grib file attached 2017090618.gfs.t18z.pgrb2.1p00.f144.grib2.zip.pdf

Attachment 156188
Rename without the ".pdf". Zipped with 7-zip. Should work with built-in Windows zip utility.
This is 1.00

degree grid.
Let me know if I need to change time range, or other parameters.
Paul

Paul you really are much better than me at this.. Very nice 126kb file showing Irma. Will make learning wxrouting fun. There is even some faster routing by bouncing between Irma & Jose. - No Fear in the polar files.

Then learn how to program wxRoute with prudence and more caution! etc. Very nice file. Thank you.

Do you have a URL link for where you got that? I'd like to learn more how. I think I am almost there.

See below done with your file. Going to be in the WxRoute manual. Now keeping an eye on Jose.

Best
Rick

[seandepagnier]

These storms might make strange sea states which is not really accounted for by weather routing.

I have found that if there are large swells from far away in the same direction as the wind, then it is much easier than normal to beat against the wind for example even in pretty strong winds as it reduces the chop and wind waves, but if they are in a different direction from the wind it can make it much worse.

[.Paul.]

Originally Posted by rgleason

.Paul,

I recall trying to do this with RTOFS 3hr, forecast Gulf of Maine, this summer. The Grib Filters were unavailable because they had just changed servers. I see Grib Filters are now available for RTOFS too.

.
.
.

I got gfs.t12z.pgrb2.1p00.anl with 72 bytes.
This is not working too well for me. What is a anl file?

On the NOMADS web page, the "RTOFS Atlantic" and "RTOFS Atlantic HIRES" have "grib filter" access, but the forecasts are from March 2017. "RTOFS Global" does not have "grib filter". Perhaps someday...

"anl" stands for analysis. I think this file returns initialization data for the forecast run or perhaps an analysis of the initial condition(s).

Paul

[.Paul.]

Originally Posted by rgleason

Paul you really are much better than me at this.. Very nice 126kb file showing Irma. Will make learning wxrouting fun. There is even some faster routing by bouncing between Irma & Jose. - No Fear in the polar files.

Then learn how to program wxRoute with prudence and more caution! etc. Very nice file. Thank you.

Do you have a URL link for where you got that? I'd like to learn more how. I think I am almost there.

See below done with your file. Going to be in the WxRoute manual. Now keeping an eye on Jose.

Best
Rick

I use the "Grib Filter" access to the grib data on the NOMADS page. It lets the user select the date and time the forecast model was initialized, and the hour within the forecast for which output is requested. Then levels (or altitudes) for output are selected, and physical values a selected with check boxes. An option allows a subregion to be selected as a latitude and longitude box. After all of the grib filter values are entered, the grib filter program can output a file of grib data, or a URL may be requested which will obtain the file of grib data if it is submitted within a reasonable amount of time. I request the URL. That URL can be used to obtain a the grib data for a specific hour in the forecast.

I use a Windows batch (CMD) program to replicate the original URL into a text file containing list of URLs each with a different hour in the forecast. Then I use WGET to download and concatenate all of the grib files identified in the list. This produces a single file for the Grib plugin in OpenCPN.

Paul