Shortwave Broadcast of GRIB's ?

[Adelie]

Originally Posted by barnakiel

Barnakiel might have been wrong. He is a sailor, not an IT specialist.

Alas, when he takes a grib file and slightly manipulates some data in it (slightly, say replacing some bits with different values) he finds the manipulated file not to open.

I believe the way a file is sent - if a packet is lost - the file is useless. And to know a packet is lost one has to do the checksums. And to do them one needs a way to send back 'repeat packet' command if a packet is wrong or missing.

???

IT people now - is this picture good enough, or is Barnakiel COMPLETELY wrong?

He IS willing to learn too!

b.

B:
I see where you are coming from now. The problem at this point is that the current Grib readers are using very robust error detection and correction methods that depend on the ability to use backhaul communication to correct errors and they reject the whole file if there are any significant uncorrected errors because for most users requesting a replacement file or appropriate corrections is a given, they have a backhaul connection.

Any broadcast system would have to be more tolerant of uncorrected errors.

To make a Grib broadcast system work, the files would have to be stripped to their minimum size, broadcast would be in packets sent 3 times with checksums for each packet. Broadcast Grib readers would have to be modified to accept and display data despite bad checksums, but noting the results as being questionable in specifiic areas, perhaps by Bold typing or not bold typing the results in those areas or displaying them in a different color line or font. Other error detection and correction methods could also be used.

Since there is no perceived demand nobody is going to the effort of creating the system, (which would be fairly straight forward the way I see it) and broadcasting the info (which would involve money).

A

[barnakiel]

Originally Posted by Adelie

To make a Grib broadcast system work, the files would have to be stripped to their minimum size, broadcast would be in packets sent 3 times with checksums for each packet. Broadcast Grib readers would have to be modified to accept and display data despite bad checksums, but noting the results as being questionable in specifiic areas, perhaps by Bold typing or not bold typing the results in those areas or displaying them in a different color line or font. Other error detection and correction methods could also be used.

OK. If I read it right - there is neither hardware nor software in place to achieve something that can be achieved via other means and methods.

We would need the hardware on the sending end, and proprietary software at both ends. Long way to go. Probably we will never see this happen - there is actually no market enough for such.

BTW It will be interesting to see how well data transfer under the new Inmarsat blue phone is going to work. The phone is cheaper than Iridium, touch would we will get another option to haul some data in while on the passage!

b.

[Adelie]

Originally Posted by barnakiel

OK. If I read it right - there is neither hardware nor software in place to achieve something that can be achieved via other means and methods.

We would need the hardware on the sending end, and proprietary software at both ends. Long way to go. Probably we will never see this happen - there is actually no market enough for such.

BTW It will be interesting to see how well data transfer under the new Inmarsat blue phone is going to work. The phone is cheaper than Iridium, touch would we will get another option to haul some data in while on the passage!

b.

No new hardware needed at the sending end, the existing infrastructure for sending WxFax would work, but airtime is needed, and somebody's time to manage the program same as someone's time is needed to manage the program for WxFax broadcasts.

Software would not be new, it would be modifications of existing software, effort would need to go into the modifications, but not the same effort as writing from scratch. At the receiving end, there are a lot of WxFax viewers that were written by individuals that are now available free or at a nominal price.

It's still not getting done.

[barnakiel]

Originally Posted by Adelie

No new hardware needed at the sending end, the existing infrastructure for sending WxFax would work, but airtime is needed, and somebody's time to manage the program same as someone's time is needed to manage the program for WxFax broadcasts.

Software would not be new, it would be modifications of existing software, effort would need to go into the modifications, but not the same effort as writing from scratch. At the receiving end, there are a lot of WxFax viewers that were written by individuals that are now available free or at a nominal price.

It's still not getting done.

OK, I see.

I believe the Q (power=range) sending hardware is owned by govts and when I look at e.g. US timetable then they are fully booked - not like there are any slots a private party could use (even if any govt could be willing to share the infrastructure with private users). When tropical analysis and other products have been sent they switch on to polar, ice, etc.. PLENTY. And this would be the first limiting factor.

The other thing is if we could use such 'bare' files in e.g. MaxSea or say Deckman? This is the primarily way I use gribs - overlay on the chart against my boats position and course. Then I use the built in tools to advice me on best routing options.

My last grumble will be on timing - now to receive WFax I have to tune in at the right time. Would the procedure be the same for Wgrib? Or could it be organized like the ssb email - I send a request, then receive my files? But then I would need an SSB transceiver, which is expensive compared to a receiver only.

In fact, I would much prefer some sort of software that would allow me to overlay existing WFax transmissions on Max Sea, etc.. - I still do not know how to achieve this.

And with the onset of any-spot, two way sms'ing and cheaper sat data (both Inmarsat and Globalstar new services expected this year) I would think trying to send gribs via an outdated technology might be pretty much art for its own sake. I did the idea, but I think it was due ten years ago. Today there may be better options.

But I do like the idea anyway.

b.

[barnakiel]

Originally Posted by Adelie

No new hardware needed at the sending end, the existing infrastructure for sending WxFax would work, but airtime is needed, and somebody's time to manage the program same as someone's time is needed to manage the program for WxFax broadcasts.

Software would not be new, it would be modifications of existing software, effort would need to go into the modifications, but not the same effort as writing from scratch. At the receiving end, there are a lot of WxFax viewers that were written by individuals that are now available free or at a nominal price.

It's still not getting done.

OK, I see.

I believe the Q (power=range) sending hardware is owned by govts and when I look at e.g. US timetable then they are fully booked - not like there are any slots a private party could use (even if any govt could be willing to share the infrastructure with private users). When tropical analysis and other products have been sent they switch on to polar, ice, etc.. PLENTY. And this would be the first limiting factor.

The other thing is if we could use such 'bare' files in e.g. MaxSea or say Deckman? This is the primarily way I use gribs - overlay on the chart against my boats position and course. Then I use the built in tools to advice me on best routing options.

My last grumble will be on timing - now to receive WFax I have to tune in at the right time. Would the procedure be the same for Wgrib? Or could it be organized like the ssb email - I send a request, then receive my files? But then I would need an SSB transceiver, which is expensive compared to a receiver only.

In fact, I would much prefer some sort of software that would allow me to overlay existing WFax transmissions on Max Sea, etc.. - I still do not know how to achieve this.

And with the onset of any-spot, two way sms'ing and cheaper sat data (both Inmarsat and Globalstar new services expected this year) I would think trying to send gribs via an outdated technology might be pretty much art for its own sake. I did the idea, but I think it was due ten years ago. Today there may be better options.

But I do like the idea anyway.

b.

[jackdale]

Originally Posted by Auspicious

If you can show me a grib model that at least shows fronts I'll be very grateful.

Fronts as drawn on weatherfax by the forecaster. Gribs are the raw data. I compare gribs and weatherfax, and have figured out how to locate a high or low based on isobars, and the fronts by wind shifts and precipitation.

[barnakiel]

Originally Posted by jackdale

(...)

Gribs are the raw data.

(...)

Are you sure this is the case?

I would believe the raw data is the measurements that are fed into mathematical models that will output the gribs.

Gribs are a product, a result, not raw data.

I am also not certain pitching gribs against wefax does any good. These are two different types of information. Probably best to use both unless only one can be gotten due to technical limitations.

b.

[jackdale]

Originally Posted by barnakiel

Are you sure this is the case?

I would believe the raw data is the measurements that are fed into mathematical models that will output the gribs.

Gribs are a product, a result, not raw data.

I am also not certain pitching gribs against wefax does any good. These are two different types of information. Probably best to use both unless only one can be gotten due to technical limitations.

b.

I should have said, "pretty close to raw data", there is no human intervention in the output.

Weatherfax has a human intervention in locating fronts and labelling highs and lows, and the forecaster is human.

I do use both.

[barnakiel]

Originally Posted by jackdale

I should have said, "pretty close to raw data", there is no human intervention in the output.

Weatherfax has a human intervention in locating fronts and labelling highs and lows, and the forecaster is human.

I do use both.

Fine, but the skipper is also human vs. the autopilot being pretty close to raw data.

Who controls your boat in the most demanding conditions? You or the autopilot? And when you use radar, do you say: 'doh, there is nothing, and stay down below, or do you stay in the cockpit and keep watch?.

I would not use 'human' in this aspect. You seem to omit the fact that the mathematical model used to generate the grib is, well, also - human.

b.

[hi3cmz]

I just come across here, was searching for sailboat weather products, for broadcasting over HF, there is a new system called Digital Radio Mundiale, which is new standard for digital broadcasting over HF, and yes, it deliver sound in digital format much like mp3, and there is modes supported for data broadcasting... if one day WxFAX guys decide to broadcast their content digitaly, DRM should be considered. So GRIB over HF is no longer tech problem but financial problem...

[Auspicious]

Originally Posted by estarzinger

See below a current gfs grib off the US east coast with wind, pressure and rain layers.

You will see a high pressure system marked (lower left corner)

And you can clearly see the frontal system in the rain layer diagonally across the picture.

Rather useful IMHO

You mean lower right?

Cool beans. Spelunking in Saildocs is in order. The rain layer is a good indicator. Nice call.

Are you still using Airmail or something else to generate your grib requests? What software version?

[estarzinger]

Originally Posted by Auspicious

Are you still using Airmail or something else to generate your grib requests? What software version?

When ashore I use viewfax beta version or expedition - they do a direct fetch of the grib so you don't have to mess with e-mails. When offshore/iridium I just hand format the gribs, each day's e-mail is only slightly different if at all than the day before's.

For me, the most important thing to understand about the weather forecast, be it from gribs or fax or router is it's probability of occurring (eg how certain it is). If it's quite certain I can do some fancy routing to play it to best effect, but if its quite uncertain then I start looking to minimize downside risk with my routing. Unfortunately none of the met guys or services seem to want to talk about probabilities or level of certainty (except in their hurricane forecasts when they do explicitly show track probabilities) - so I have to make my own judgement - which I do in three ways: (1) Some patterns are typically more predictable (strong discrete systems) than others (weak or complex interacting systems), (2) if the several grib models agree I figure there is more certainty than if the violently disagree, and finally (3) if the forecast system pattern of development is stable over time/day after day I figure its reasonably certain while if yesterday's 2 day out future pattern is quite different than the prior days 3-day out picture I figure its uncertain.

[Auspicious]

I use Airmail 3.4.062 which now integrates Viewfax. Thanks for the pointer to the Get Data function - very convenient when I have a real Internet connection.

Originally Posted by estarzinger

For me, the most important thing to understand about the weather forecast, be it from gribs or fax or router is it's probability of occurring (eg how certain it is).

I agree. I've found using synoptic charts from wefax that I rather quickly develop some opinions about the various forecasters. Some provide more detail in the their forecasts than others. Some quite frankly are often wrong, particularly as they get further out. Just as you compare the various computer models I compare the human forecasters. Just as you compare yesterday's 48-hour grib to today's 24-hour grib I do the same with the synoptic charts, and update my crib sheet on the forecasters.

Originally Posted by estarzinger

Unfortunately none of the met guys or services seem to want to talk about probabilities or level of certainty

Wouldn't that be lovely? I'd really like to see a level of confidence metric.

[barnakiel]

Originally Posted by Auspicious

I agree. I've found using synoptic charts from wefax that I rather quickly develop some opinions about the various forecasters. Some provide more detail in the their forecasts than others. Some quite frankly are often wrong, particularly as they get further out. Just as you compare the various computer models I compare the human forecasters. Just as you compare yesterday's 48-hour grib to today's 24-hour grib I do the same with the synoptic charts, and update my crib sheet on the forecasters.

Hi Guys,

I want to try explain some misconceptions. Not sure I can do it in plain English but will give it a shot. I will briefly touch upon two concepts you discussed:

1) PROBABILITY of the forecast: the figure you will see at some presentations (e.g. rain probability) will most often refer to probability of such an occurrence/development in the database of the relevant model. However, we (final users, not forecasters) hardly ever know how the samples clustered at various analysis levels and how much if at all such clustering has been allowed for in the model. In other words, we never know whether the 50% probability of (say: rain) is measured versus the total rain occurrences, rain occurrences in a year like this (eg. a 'dry' year) or a spring time only, in a dry / any / year. Models vary greatly in how the data is clustered (or not) and hence how the probabilities are to be interpreted. Say next time if you see two consecutive days with 50% chance of rain do not suppose one of the days WILL be rainy. Interestingly, do not expect that a dry day will follow a rainy one, nor that both days will get some rain and some good weather ... Even more interestingly, if you spend any time in a fixed point and use the same forecasting model, you will find that e.g. 20% rain means a sunny clear day. If you go to another location, the 20% may actually mean a grey and drizzly day. So to say, the models are pretty accurate globally, less so locally - and yet - there are general patterns - even in the inaccuracies!

The same applies to all other phenomena that are presented with a probability given.

2) CORRECTNESS of particular human forecasters: the weather analysis will have its fronts / isobars and 24 forecasts of pressure centers movements drawn by a computer, NOT a forecaster. If you see a wefax map drawn manually, it may be so, but expect huge mathematical support practically cancelling any human factor. There will be the human factor in the analysis to some extent, while the forecasts will have less and less to do with what the human thinks will or will not take place. You may notice some forecaster 'styles' by looking at the chartlets - after all, if a forecaster does not want a front at a place indicated by the software, they may (???) have the software omit one, and maybe (???) they can at times add one, where there is nothing indicated by the software. BUT why should they do so?

Thus, your comparing of human forecasters has much to do with your likes and dislikes of a particular forecasting style rather than with any discernible difference in the quality of their output.

Finally, if there is anything I can recommend to anybody willing to better use available weather products then it would be to follow some of the courses as offered online and free of charge at UCAR. If you ever have a chance to visit a NOAA center and get to know the guys behind the names this would be great. I am sure this is possible if you ask some time in advance - NOAA has some walk-thru shows for students and I am sure you will be allowed to join.

Cheers,
b.

[cal40john]

Originally Posted by hi3cmz

I just come across here, was searching for sailboat weather products, for broadcasting over HF, there is a new system called Digital Radio Mundiale, which is new standard for digital broadcasting over HF, and yes, it deliver sound in digital format much like mp3, and there is modes supported for data broadcasting... if one day WxFAX guys decide to broadcast their content digitaly, DRM should be considered. So GRIB over HF is no longer tech problem but financial problem...

My guess is that NOAA won't be interested. They already polled users asking to do away with WEFAX over HF since they thought everyone (important) was using digital communications via sat phone at sea already. They got a solid don't get rid of it, but my feeling is once they're on the track of dumping something it will eventually happen.

John