WeatherFax ...

[HUR450502]

Originally Posted by rgleason

I have checked and fixed all NOAA files including and up to the end of Passageweather.

Can someone else take it from there and check and fix the next segment?

Then we can reissue the XML files.

Hi Rick,

yes, of course..... I will be looking into this over the weekend and send the info to you - either as an attached file, as before or we can exchange e-mail addresses - as you prefer, ok??

Best,

Uwe

[rgleason]

Uwe,
I am going to attach the current XML files for coordinatesets.xml and weatherfaxinternetretrieval.xml

You'll need to remove the doc and expand them.

[rgleason]

weatherfax_pi-1.7.009-ov4.2-win32
Beta release with svg icons.

[stelian]

Originally Posted by rgleason

While checking the many weatherfax urls I had to wait each time for 12 to 15 seconds each time for the opencpn screen to go black (memory overload) and then wait for the regional selections to appear, during which time the user cannot change the selection either. The plugin did not work this way before.
Memory use is better and the plugin does not crash opencpn after many downloads where o's memory use has gone up to 800mb, it just stops downloading and acts weirdly. Then I have to close the plugin and opencpn to clear the memory and get it working normally.

I would really like to have that 12-15 second pause with black screen fixed. On an RPI that is probably 30 seconds of wait time.

Rick,

What are the exact steps you do in order to create the 12-15 seconds pause ?

I tried your latest beta (weatherfax_pi_1.7.009-ov4.2-win32) as well as my own binary compiled from latest git, and I cannot reproduce this. I observe a maximum 2-3 seconds delay, and this is in extreme situations (when clicking on 'everything' in both Servers and Regions tabs). Moreover, this is with a Windows 10 running inside VirtualBox, so I expect native Windows to be even faster.

Stelian.

[rgleason]

What are the exact steps you do in order to create the 12-15 seconds pause ?

I tried your latest beta (weatherfax_pi_1.7.009-ov4.2-win32) as well as my own binary compiled from latest git, and I cannot reproduce this. I observe a maximum 2-3 seconds delay, and this is in extreme situations (when clicking on 'everything' in both Servers and Regions tabs). Moreover, this is with a Windows 10 running inside VirtualBox, so I expect native Windows to be even faster.

Stelian, thanks for responding, this is on my SurfacePro3 with 4gb, it is 6 seconds to bring up the the first screen then 6- to maybe 8 seconds after click on Regions to bring up the weatherfaxes list. It is similar for my Inspiron Laptop XPS15 with 8gb. Both using Win10.

I don't know why. It is even so loggy that if I miss click on a region, I am locked out from picking the right region until the process has completed! Also sometimes the screen goes black.

Since I am using the program to check all the URLs this becomes a pain in the neck, because it really slows down the process.

Can you try just checking the width of the first five entries or something?

[stelian]

Rick,

Does http://popies.net/tmp/weatherfax_pi-...sp20180123.exe make a difference ? (from my tests it is worse than before, but who knows...)

Also, I would really like some more details about this slowness. From what I seen, the GUI is slow when there are many urls in the top list (for example, if you click on "every server" and "every region" buttons).

But if I read you correctly, you experience slowness even if you don't do this, but just click on some random server and region. Right ? How many items do you have in the top list when you get those multi-second delays ?

Can you reproduce this just after starting OpenCPN (without having loaded tons of faxes, because if the faxes themselves take a lot of memory you can experience slowness for everything, not just this particular action) ?

[rgleason]

Also, I would really like some more details about this slowness. From what I seen, the GUI is slow when there are many urls in the top list (for example, if you click on "every server" and "every region" buttons).

I know you could select multiple servers and regions, but it causes huge slow downs black screens and it is totally unproductive.


To answer your question about when I start up fresh, without having downloaded much. Yes, after hitting one "server" it takes about 3 seconds minimum for the regions to appear. Then when I hit a single region it takes about 4 seconds.

Except, when I get into Passage Weather it becomes much slower, on the order of what I was talking about, (5 seconds, and 9 seconds) after downloading and testing urls. Austrailia Dom is similar times.

It may just be my little surfacepro, which I have been using a lot, but I noticed it on my laptop when I updated the xml's the time before too.

I will try your new version next. Thank you for listening.

[rgleason]

Stelian, I think your file is marginally faster. Big Servers like Passage Weather with all the regions, and then hitting regions like Straits of Malacca take still take almost the same amount of time.... but I think it seems a little bit quicker.

---what is that version doing?

I wonder if all the commented out stuff in the XML file from old URL's and things we are working on getting better URLs, strdate, etc, are slowing it down? Maybe I should put all that non-working or working on stuff into a backup file? Maybe that would speed things up?

I'll try it anyway.
Thank you for working on this and thinking about what it might be.

[stelian]

Originally Posted by rgleason

I know you could select multiple servers and regions, but it causes huge slow downs black screens and it is totally unproductive.

Ok, so you're not even using this feature.

Originally Posted by rgleason

To answer your question about when I start up fresh, without having downloaded much. Yes, after hitting one "server" it takes about 3 seconds minimum for the regions to appear. Then when I hit a single region it takes about 4 seconds.

Except, when I get into Passage Weather it becomes much slower, on the order of what I was talking about, (5 seconds, and 9 seconds) after downloading and testing urls. Austrailia Dom is similar times.

We agree that when you're first clicking on the server, the top list is empty ? It will only get populated when you also choose a region. But even with the top list empty it is still slow ?

If this is really the case, the problem cannot be in the auto-resize feature of the columns, because, well, there is no data in the list to resize upon. So this makes me ask: did you experience this slowness forever or it has only appeared at some specific point in time ? (you gave me the impression that it was my auto-size change which caused the trouble, but now I doubt that. I know that there was a version which forced the auto-resize each time you added an item, so the populating times were much much bigger, causing the entire OpenCPN to hang for minutes, but that particular issue was normally solved quickly after having been reported)

Keep in mind that I cannot reproduce this (for me, clicks on server or region are instantaneous), so this is why I ask so much questions.

Originally Posted by rgleason

Stelian, I think your file is marginally faster. Big Servers like Passage Weather with all the regions, and then hitting regions like Straits of Malacca take still take almost the same amount of time.... but I think it seems a little bit quicker.

---what is that version doing?

That version does not do auto-resizing anymore, it manually calculates the width of each column and, once the list is completely populated, it sets the right size.

Originally Posted by rgleason

I wonder if all the commented out stuff in the XML file from old URL's and things we are working on getting better URLs, strdate, etc, are slowing it down? Maybe I should put all that non-working or working on stuff into a backup file? Maybe that would speed things up?

It is certainly worth a try.

Originally Posted by rgleason

It may just be my little surfacepro, which I have been using a lot, but I noticed it on my laptop when I updated the xml's the time before too.

I also measured the time it takes to populate the list and auto-resize the columns, and, surprise, the most of the time is in the loop where the items are inserted, not the resize of the columns (except that - I think - this is not the same issue as yours, because this concerns the top list; for me, when having many many urls ("all" servers, "all" regions), it takes up to 1 second for the list to be populated, but the auto-resize is a few milliseconds.

[rgleason]

Stelian, using your version...

We agree that when you're first clicking on the server, the top list is empty ? It will only get populated when you also choose a region. But even with the top list empty it is still slow ?

Yes, when I first open the top window is clear, with "Scheduled, Server, Region, Contents and Map Area" scrunched together.

When I open O, then open Wxfax, Then Retrieve > Internet (note Ctrl-i does not work) it takes 4-5 seconds to open the window, no matter what server you were using last (Wxfax remembers the last server used.) To get this effect, you must close wxfax and O, and then reopen Retrieve > Internet. Under this type of use, it does not matter what server you picked last, opening Retrieve Internet window takes 4-5 seconds.

If you just change the server and close wxfax, then reopen it is pretty quick to open the empty window at the top with the last used server <1 second, no matter what server.

Ok, lets say I last used UK Met and close wxfax. Then open it without closing O first. It pops up instantly. Then I change to "Passage Weather" it takes 4-5 seconds to have the Regions appear. Then when I select a single "region" and pick "Straits of Malacca" It took about 12 seconds and the screen went black this time with a blue spinner. See attached. -Now that is a PITA, but fortunately that is not always quite so bad. So in summary, I don't know what is causing this but I think it has to do with reading the file or sorting!

[rgleason]

I also measured the time it takes to populate the list and auto-resize the columns, and, surprise, the most of the time is in the loop where the items are inserted, not the resize of the columns (except that - I think - this is not the same issue as yours, because this concerns the top list; for me, when having many many urls ("all" servers, "all" regions), it takes up to 1 second for the list to be populated, but the auto-resize is a few milliseconds.

So you don't think it is the sort. Of course windows is picky, we know this. Let's assume it is not the sort. We are both using similar xml files, why would windows take longer to read a file?

-- I'll make a clean xml file next.

[rgleason]

weatherfaxinternetretieval.xml (uses the same coordinatesets.xml as before)

This file has all blank lines removed and all old unworking servers and regions that were commented out, removed. Dozens of blank lines removed from Passage Weather.

This has improved response a little. From highlighting one server 3-4 seconds before the Regions appear. When a region is highlighted such as Passage Weather "Straits of Malacca" it takes 9 seconds.

I think it is pretty obviously some marginal read file statements that windows is picky about. Maybe it is a wxWidgets read or something that is presenting things that picky Windows does not like digesting.

Here is the file.

[rgleason]

I haven't looked at the code, not sure I'd understand it... but I did do a search on wxWidgets 3.0.3 version which is more recent...

There are routines for xml files

Example: https://forums.wxwidgets.org/viewtopic.php?t=16111

wxXmlNode - Reading from an XML file [SOLVED]

Problem in reading XML file [SOLVED]

Search on wxWidgets Documentation XML+read

I see there is a lot of reading in internetfaxretrievaldialog.cpp but I guess I am not going to be able to do much about it.

[nohal]

Originally Posted by rgleason

I haven't looked at the code, not sure I'd understand it... but I did do a search on wxWidgets 3.0.3 version which is more recent...

There are routines for xml files

Example: https://forums.wxwidgets.org/viewtopic.php?t=16111

wxXmlNode - Reading from an XML file [SOLVED]

Problem in reading XML file [SOLVED]

Search on wxWidgets Documentation XML+read

I see there is a lot of reading in internetfaxretrievaldialog.cpp but I guess I am not going to be able to do much about it.

Rick...
wxXml is not used at all by the weatherfax plugin. What makes you think there is some problem with reading the XML files?

Pavel

[rgleason]

It takes time to read the weatherfaxinternetretrieval.xml file
I took out all the spaces and removed all the OLD commented out URL's that no longer work because of strdate etc. and the read time improved a little.
Currently on SurfPro3 using wxfax internetretrival

This has improved response a little. From highlighting one server 3-4 seconds before the Regions appear. When a region is highlighted such as Passage Weather "Straits of Malacca" it takes 9 seconds.

It takes longer to check all the URLs because of this.