Beta test 4.1.1108 on ARM

[NahanniV]

Vector charts and CM93 charts are working well on my ARM system (OpenGL or NOT).
Raster charts work OK without OpenGL (slowly).
But, there seem to be a few problems with raster charts using OpenGL.

I've been trying to isolate the most easily repeatable ones.

Here is one that I can easily repeat.

Config:
- latest from GIT (224e96d) Nov.23rd.
-no plugins enabled.
- one small US RNC area enabled (14810_1 and 14810_2)
- OpenGL enabled, but not texture compression cache.

To create the problem:
-start with the background map displayed, and zoomed out to a level where the RNC chart outline is displayed.
- zoom in on the RNC chart.

The problem:
- at the point where the chart should be displayed, one CPU usage will go up to near 100%.
- the system will become sluggish.
- memory usage will slowly increase for more than a minute.
- after a minute or more the chart will be displayed, but with square areas missing.

Running in GDB stopped with ^C during the long delay:'
BackTrace:

(opencpn:10558): GLib-GObject-CRITICAL **: g_object_unref: assertion 'G_IS_OBJECT (object)' failed
libGL: built on Nov 23 2015 17:08:31
[New Thread 0xb4a06250 (LWP 10594)]
[New Thread 0xb4206250 (LWP 10595)]
[New Thread 0xb3a06250 (LWP 10596)]
glXGetProcAddress: glGenFramebuffers not found.
glXGetProcAddress: glGetCompressedTexImage not found.
^C
Program received signal SIGINT, Interrupt.
0xb638aa98 in mmap () at ../ports/sysdeps/unix/sysv/linux/arm/mmap.S:42
42    ../ports/sysdeps/unix/sysv/linux/arm/mmap.S: No such file or directory.
(gdb) bt
#0  0xb638aa98 in mmap () at ../ports/sysdeps/unix/sysv/linux/arm/mmap.S:42
#1  0xb66a1e12 in _mali_uku_mem_mmap () from /usr/lib/libEGL.so
#2  0xb66a283c in backend_mmu_get_memory () from /usr/lib/libEGL.so
#3  0xb66a29b8 in _mali_base_arch_mem_get_memory () from /usr/lib/libEGL.so
#4  0xb669f94a in _mali_base_common_mem_alloc () from /usr/lib/libEGL.so
#5  0xb6683a6a in _mali_shared_mem_ref_alloc_mem () from /usr/lib/libEGL.so
#6  0xb6683cf8 in _mali_surface_alloc () from /usr/lib/libEGL.so
#7  0xb6666998 in _gles_texture_miplevel_allocate () from /usr/lib/libEGL.so
#8  0xb6667556 in _gles_tex_image_2d_internal () from /usr/lib/libEGL.so
#9  0xb666769c in _gles_tex_image_2d () from /usr/lib/libEGL.so
#10 0xb6672570 in _gles1_tex_image_2d () from /usr/lib/libEGL.so
#11 0xb6665d64 in glTexImage2D () from /usr/lib/libEGL.so
#12 0x00347d9c in glTexImage2D (target=3553, level=7, internalFormat=6407, 
    width=<optimized out>, height=<optimized out>, border=0, format=6407, 
    type=5121, data=0x3210c68)
    at /home/aruntu/OpenCPN/src/glshim/src/gl/texture.c:166
#13 0x002a1f78 in glTexFactory::PrepareTexture (this=this@entry=0x8e5a90, 
    base_level=0, base_level@entry=3, rect=..., color_scheme=<optimized out>, 
    b_throttle_thread=b_throttle_thread@entry=true)
    at /home/aruntu/OpenCPN/src/glTexCache.cpp:1562
#14 0x00299814 in glChartCanvas::RenderRasterChartRegionGL (
    this=this@entry=0x57d188, chart=chart@entry=0x5bd8e8, vp=..., region=...)
    at /home/aruntu/OpenCPN/src/glChartCanvas.cpp:3103
#15 0x0029a1a8 in glChartCanvas::RenderQuiltViewGL (this=this@entry=0x57d188, 
    vp=..., rect_region=...)
    at /home/aruntu/OpenCPN/src/glChartCanvas.cpp:3198
#16 0x0029b966 in glChartCanvas::RenderCharts (this=this@entry=0x57d188, 
    dc=..., rect_region=...)
    at /home/aruntu/OpenCPN/src/glChartCanvas.cpp:3515
#17 0x0029c42c in glChartCanvas::Render (this=this@entry=0x57d188)
    at /home/aruntu/OpenCPN/src/glChartCanvas.cpp:4305
#18 0x0029d046 in glChartCanvas::OnPaint (this=0x57d188, event=...)
    at /home/aruntu/OpenCPN/src/glChartCanvas.cpp:1510
#19 0xb681c112 in wxAppConsoleBase::CallEventHandler(wxEvtHandler*, wxEventFunctor&, wxEvent&) const () from /usr/local/lib/libwx_baseu-3.0.so.0
#20 0xb692020e in wxEvtHandler::ProcessEventIfMatchesId(wxEventTableEntryBase const&, wxEvtHandler*, wxEvent&) () from /usr/local/lib/libwx_baseu-3.0.so.0
#21 0xb69202b4 in wxEventHashTable::HandleEvent(wxEvent&, wxEvtHandler*) ()
   from /usr/local/lib/libwx_baseu-3.0.so.0
#22 0xb6920532 in wxEvtHandler::TryHereOnly(wxEvent&) ()
   from /usr/local/lib/libwx_baseu-3.0.so.0
#23 0xb692057a in wxEvtHandler::ProcessEventLocally(wxEvent&) ()
   from /usr/local/lib/libwx_baseu-3.0.so.0
#24 0xb69205c0 in wxEvtHandler::ProcessEvent(wxEvent&) ()
   from /usr/local/lib/libwx_baseu-3.0.so.0
#25 0xb69203fc in wxEvtHandler::SafelyProcessEvent(wxEvent&) ()
   from /usr/local/lib/libwx_baseu-3.0.so.0
#26 0xb67c14f2 in wxGLCanvas::OnInternalIdle() ()
   from /usr/local/lib/libwx_gtk2u_gl-3.0.so.0
#27 0xb6bfeb12 in wxWindowBase::SendIdleEvents(wxIdleEvent&) ()
   from /usr/local/lib/libwx_gtk2u_core-3.0.so.0
#28 0xb6bfeb34 in wxWindowBase::SendIdleEvents(wxIdleEvent&) ()
   from /usr/local/lib/libwx_gtk2u_core-3.0.so.0
#29 0xb6bfeb34 in wxWindowBase::SendIdleEvents(wxIdleEvent&) ()
   from /usr/local/lib/libwx_gtk2u_core-3.0.so.0
#30 0xb6b3d662 in wxFrame::SendIdleEvents(wxIdleEvent&) ()
   from /usr/local/lib/libwx_gtk2u_core-3.0.so.0
#31 0xb6b62612 in wxAppBase::ProcessIdle() ()
   from /usr/local/lib/libwx_gtk2u_core-3.0.so.0
#32 0xb6aebd74 in wxApp::DoIdle() ()
   from /usr/local/lib/libwx_gtk2u_core-3.0.so.0
#33 0xb6aebe24 in wxapp_idle_callback ()
   from /usr/local/lib/libwx_gtk2u_core-3.0.so.0
#34 0xb5b792d0 in g_main_context_dispatch ()
   from /lib/arm-linux-gnueabihf/libglib-2.0.so.0
#35 0xb5b79522 in ?? () from /lib/arm-linux-gnueabihf/libglib-2.0.so.0
#36 0xb5b79522 in ?? () from /lib/arm-linux-gnueabihf/libglib-2.0.so.0
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)

Nothing seems odd in log:

23:49:07 EST: 2015-11-23
23:49:07 EST:  ------- Starting OpenCPN -------
23:49:07 EST: Version 4.1.1108 Build 2015-111-08
23:49:09 EST: wxWidgets version: wxWidgets 3.0.2 Linux 32 bit wxGTK
23:49:09 EST: MemoryStatus:  mem_total: 1838 mb,  mem_initial: 13 mb
23:49:09 EST: SData_Locn is /usr/local/share/opencpn/
23:49:09 EST: PrivateDataDir is /home/aruntu/.opencpn
23:49:09 EST: Using existing Config_File: /home/aruntu/.opencpn/opencpn.conf
23:49:09 EST: Setting Viewpoint Lat/Lon 43.2703, -79.4512
23:49:09 EST: Setting Ownship Lat/Lon 33.358, -79.282
23:49:09 EST: Styles loading from /usr/local/share/opencpn/uidata/styles.xml
23:49:09 EST: No styles found at: /home/aruntu/
23:49:09 EST: No styles found at: /home/aruntu/.opencpn/
23:49:09 EST: Detected display size (horizontal): 508 mm
23:49:09 EST: System default Language:  en_US
23:49:09 EST: Opencpn language set to:  en_US
23:49:09 EST: Creating MyFrame...size(1344, 731)  position(0, 35)
23:49:09 EST: Creating glChartCanvas
23:49:10 EST: OpenGL-> Renderer String: Mali-400 MP
23:49:10 EST: OpenGL-> Version reported:  1.4 glshim wrapper
23:49:10 EST: OpenGL-> Texture rectangle format: de1
23:49:10 EST: OpenGL-> glGenerateMipmap unavailable
23:49:10 EST: OpenGL-> Using Vertexbuffer Objects
23:49:10 EST: OpenGL-> Framebuffer Objects unavailable
23:49:10 EST: OpenGL-> Using Depth buffer clipping
23:49:10 EST: OpenGL-> Not Using compression
23:49:10 EST: OpenGL-> Minimum cartographic line width:  1.0
23:49:10 EST: OpenGL-> Minimum symbol line width:  1.0
23:49:11 EST: ChartDB Cache policy:  Application target is 912 MBytes
23:49:11 EST: Loading chart db version: V018
23:49:11 EST: Chartdb: Chart directory list follows
23:49:11 EST:   Chart directory #0: /home/aruntu/Charts/US NY RNCs/BSB_ROOT/14810
23:49:11 EST: GPS Watchdog Timeout is: 6 sec.
23:49:11 EST: Initializing Chart /home/aruntu/Charts/US NY RNCs/BSB_ROOT/14810/14810_1.KAP
23:49:11 EST: OpenCPN Initialized in 4529 ms.
23:49:14 EST: Looking for UserIcons at /home/aruntu/.opencpn/UserIcons
23:49:14 EST: Loading navobjects from navobj.xml
23:49:14 EST: Done loading navobjects
23:49:14 EST: PlugInManager searching for PlugIns in location /usr/local/lib/opencpn
23:49:15 EST:    ***GPS Watchdog timeout at Lat:33.358   Lon: -79.282
23:49:23 EST: Loading World Chart Q=0 in 0 ms.
23:49:23 EST: Background world map loaded from GSHHS datafiles found in: /usr/local/share/opencpn/gshhs/
23:49:23 EST: Loading World Chart Q=4 in 1 ms.
23:49:51 EST: opencpn::MyFrame exiting cleanly.
23:49:51 EST: Chart cache PlugIn purge
23:49:51 EST: Chart cache purge
23:49:51 EST: opencpn::MyApp starting exit.
23:49:51 EST: LOGBOOK:  2015-11-24 04:49:51 UTC OFF: Lat   33.35800 Lon  -79.28200
23:49:51 EST: opencpn::MyApp exiting cleanly...

Any Ideas ?

Thanks,
JM.

[seandepagnier]

A few things (also I will do a bunch of testing on various arm before stable release but am still stuck on other bugs)

If you don't have enough video memory you can get problems. A single chart can use more than 100 megs of video memory just to display without compression. So try to make the split higher if it's an arm board where you set the video memory at boot. I use 256M on my raspberry pi.

Try with compression caching and rebuild the cache. It should work then, if not let me know. I know raster charts worked fine before, at least in single chart mode if the chart was less than 20,000 pixels in dimensions.

Also, I finished neon-optimized intrinsics for mipmap generation which makes this operation about 2x faster, but this is only a 10% or so overall improvement, but with another planned change we can gain another 10%. Then if I implement intrinsics on etcpak that could give a much larger gain probably around 50-60% or so. This is the speed to rebuild the cache so if you don't manually do this, it has to for each tile as it is needed.

Another severe problem which affects all opengles is that it must upload all the mipmap levels. This is particularly bad when zoomed out to view all of a large chart as it must upload all the high detail data that then isn't even used, and also uses all that video memory. We can solve this by using multiple textures, but to do it the best way possible requires a bunch of experiments I haven't had time for yet, so maybe after the stable version. This will significantly improve the performance in some cases though, and also gives the option of getting the chart visible very quickly but a bit blurry maybe, then quickly making it sharper as more data is uploaded.

[NahanniV]

Originally Posted by boat_alexandra

A few things (also I will do a bunch of testing on various arm before stable release but am still stuck on other bugs)

If you don't have enough video memory you can get problems. A single chart can use more than 100 megs of video memory just to display without compression. So try to make the split higher if it's an arm board where you set the video memory at boot. I use 256M on my raspberry pi.

This is on a CubieTruck. I have not been able to figure out how to increase video memory. It may be hardcoded at 80MB?

Originally Posted by boat_alexandra

Try with compression caching and rebuild the cache. It should work then, if not let me know. I know raster charts worked fine before, at least in single chart mode if the chart was less than 20,000 pixels in dimensions.

Compression caching causes the chart to be blurred.
See my other post:
http://www.cruisersforum.com/forums/...ml#post1968366

How can I tell if the chart is more than 20,000 pixels ?

[NahanniV]

Here is an example of a blurred chart when using texture compression with caching.

I also don't understand the logic of the OpenGL config.
Somtimes the Rebuild texture cache button is greyed ?

What is the Software OpenGL option ?

[seandepagnier]

Give me a few days I will deal with this.

Sean

[NahanniV]

Originally Posted by boat_alexandra

Give me a few days I will deal with this.

Sean

Thanks !

JM.

[NahanniV]

Originally Posted by boat_alexandra

...
If you don't have enough video memory you can get problems. A single chart can use more than 100 megs of video memory just to display without compression. So try to make the split higher if it's an arm board where you set the video memory at boot. I use 256M on my raspberry pi.....

It looks like this is done on my system by passing arguments to the kernal at boot time. the arguments related to video are:

It is possible to significantly reduce the amount of reserved memory assigned to various video devices at boot time, resulting in more free system memory, which is especiallly helpful for 512MB devices:
sunxi_ve_mem_reserve=0 -- This eliminates the reserved memory for the video acceleration engine, saving 80MB. You can use this if you don't run accelerated video with programs such as VLC or XBMC or libvdpau-sunxi. (This kernel parameter is ignored in recent 3.4 kernels, if CMA is enabled in kernel configuration. If CONFIG_CMA=y, ve_size is hardcoded to 80MB in sunxi_cedar kernel module.)
cma=96M the amount of memory to reserve for contimuous allocation. Default is 16M so you have to raise it for the Cedar ve_size to fit.
sunxi_g2d_mem_reserve=0 -- This eliminates the reserved memory for the 2D acceleration engine. You can use this if you don't have the G2D accelerated driver enabled in your xorg.conf. Even when G2D is enabled, it may not actually use any of this memory, so this setting may be safe (to be verified).
sunxi_no_mali_mem_reserve -- This eliminates the reserved memory for the Mali400 3D GPU. If you do not have the Mali binary blob driver installed, it is safe to use this and save another 64MB.
sunxi_fb_mem_reserve=16 -- This sets the amount of total reserved memory for the framebuffer to 16MB. The default is 32MB. Because of double buffering Mali may require more than 16MB of framebuffer, so generally only enable this if you don't have Mali installed. 16MB should be sufficient for the largest supported resolution (normally 1920x1080x32bpp).

The only one set in the distribution I am using is:
sunxi_ve_mem_reserve=128

I tried changing that to 256 but it had no effect on these problems.

It is not clear to me which other argument might make a difference ?

Originally Posted by boat_alexandra

...
This is the speed to rebuild the cache so if you don't manually do this, it has to for each tile as it is needed.....

I did not realize that OpenGL with or without cache is the same code (other than saving the cache). To the user this just looks like it is hung up. Shouldn't there be some indication of what's going on, like when ingesting Vector charts ? Maybe include a message like "Use the Rebuild Texture Cache function to avoid this message in the future".

Originally Posted by boat_alexandra

...
Another severe problem which affects all opengles is that it must upload all the mipmap levels. This is particularly bad when zoomed out to view all of a large chart as it must upload all the high detail data that then isn't even used, and also uses all that video memory. We can solve this by using multiple textures, but to do it the best way possible requires a bunch of experiments I haven't had time for yet, so maybe after the stable version. This will significantly improve the performance in some cases though, and also gives the option of getting the chart visible very quickly but a bit blurry maybe, then quickly making it sharper as more data is uploaded.

On the CubieTruck: OpenGL->glGenerateMipmap unavailable

Cheers,
JM

[seandepagnier]

generate mipmap is unavailable yes, but this has nothing to do with what I was talking about

[NahanniV]

Well, at least now you understand how little I understand of what you are talking about

Cheers,
JM

[NahanniV]

Originally Posted by boat_alexandra

Give me a few days I will deal with this.

Sean

Any progress with this ?

It would be good if the blurred texture compression caching on ARM problem could be fixed before the stable release.

Anyone else see this problem?

Cheers,
JM.