MBTiles for OpenCPN

[bcn]

Originally Posted by boat_alexandra

Does this mean we can add the NOAA mbtiles to the chart downloader?

https://github.com/vokkim/noaa-nautical-charts

It will be interesting to compare the mbTile version of the charts against the standard raster/kap charts.
Size and performance wise.

[bdbcat]

nohal...

re:
We will need some more logic besides the current stuff relying on the meta information being present and correct...

I expect all we can do with broken or incomplete tilesets is to bail out. Pretty hard to guess the georef of an arbitrary tile floating in space.

Dave

[bdbcat]

bcn...

I find that the NOAA tilesets are, simply, enormous. 500 MB for south Florida quilted set, for example.

OCPN already does a good job with NOAA standard KAPs, using OpenGL Raster cache. So I'm thinking that the MBTiles use-case for standard NOAA will be limited.

Makes sense, really. MBTiles can give speed advantages because the rasterization from KAP format has already happened at tile creation, at multiple scales. But on O, thanks mostly to Sean, we rasterize once as needed, and then cache the results. Everything gets smaller and probably faster at runtime.


The real MBTiles win comes with custom tilesets for cartography that does not exist in KAP format.

Dave

[sailorF54]

Originally Posted by bdbcat

SailorF54...

re:
Trying to split the same zone in 4 different charts (zoom 14, 15...) with/without quilting creates havoc: not display, boat sent to 0,0 etc...
(The idea being to be able to zoom in /out on the various levels)

Sample tiles showing this problem?

Thanks
Dave

4 charts in same folder : France/North Brittany
This seems to occur when rebuilding chart database / quilting on/off / preserve scale ... on/ off ...
Quitting and restarting O clears the problem...
DL.FREE.FR

[nohal]

Originally Posted by bdbcat

nohal...

re:
We will need some more logic besides the current stuff relying on the meta information being present and correct...

I expect all we can do with broken or incomplete tilesets is to bail out. Pretty hard to guess the georef of an arbitrary tile floating in space.

Dave

Dave...

I think we can do at least some more:

• The tile format can be guessed from the first tile
• The min/max zoom level can be queried from the tiles table
• I would guess the bounds can be calculated from a query to the tiles table and perhaps a dumb assumption of everything being the standard web mercator projection (as I suppose the authors of the tilesets do)

As I now have a few evidently broken publicly available tilesets here, will play with it a bit...

Pavel

[bdbcat]

nohal...

More on:
re:
We will need some more logic besides the current stuff relying on the meta information being present and correct...

I suppose we could scan the entire "tiles" table and deduce the min/max zoom factor. And then take a guess at the image format. Fudge in the chart bounds, etc...

This would allow some display, even if upstream source is questionable.

Dave

[LeaseOnLife]

Feature request:

Could this be expanded to read the chart/tiles cache from SAS.Планета directly, without the user needing to create MBTiles, when the cache is already there, in well structured folders?

Dirk

[rooiedirk]

And another request Is it doable to use other tile formats also?? I am thinking at WMS or WCS.

[seandepagnier]

Originally Posted by bcn

It will be interesting to compare the mbTile version of the charts against the standard raster/kap charts.
Size and performance wise.

I would imagine they would be worse. This is because mbtiles have fixed scales, so it convert the existing kap charts into this format, they would either lose data, or result in much larger files. It would need to go up a few scales to avoid blurry lines.

If NOAA started remaking all their raster charts to the tile scales, the result could be similar in size to the kap charts.

Originally Posted by bdbcat

bcn...

I find that the NOAA tilesets are, simply, enormous. 500 MB for south Florida quilted set, for example.

OCPN already does a good job with NOAA standard KAPs, using OpenGL Raster cache. So I'm thinking that the MBTiles use-case for standard NOAA will be limited.

Makes sense, really. MBTiles can give speed advantages because the rasterization from KAP format has already happened at tile creation, at multiple scales. But on O, thanks mostly to Sean, we rasterize once as needed, and then cache the results. Everything gets smaller and probably faster at runtime.

What about using compressed cache for mbtiles? Decompressing png is slow, and it would save a lot of graphics memory.

The real MBTiles win comes with custom tilesets for cartography that does not exist in KAP format.

Dave

This is a really great start. Initial observations (of the code) make me question.

1) Some mbtiles have sizes other than 256. I think 512 is possible. This should be easy to fix, and in all cases, should fit in an opengl texture.
2) Multitextured blending... While zooming in and out, some tile viewers blend the two tiles from different scales. It looks really nice with raster charts, try the noaa online viewer. I'm not sure how we should handle this exactly since we support zooming to any scale, and when the user "stops" then you don't really want to be looking at a blended view.

[bcn]

For the documentation:
about Zoom levels and the corresponding chart scales
https://wiki.openstreetmap.org/wiki/Zoom_levels
and here
https://wiki.openstreetmap.org/wiki/...tion_and_Scale
(to illustrate Sean's comment how to match mbTile zoom levels and the scales raster charts are designed with)

[BlackSea]

Originally Posted by bdbcat

BlackSea...

Please post your mbtiles files that do not display.

Thanks
Dave

Hi Dave,

here are the samples

XYZ
TMS

Regards
BlackSea

[bdbcat]

Sean...

1. Caching of tiles: Of course could be done. I have not experimented with low-spec hardware yet. Also missing is mipmapping of textures, for sparse tilesets on zoom-out. Could be done just-in-time (another thread?), then multisampled into place. It is just code....

2. I could find no reference to tile sizes other than 256x256. Any reference?

The MBTile spec does not call for a tile size parameter anywhere, AFAICT.

https://github.com/mapbox/mbtiles-spec

3. Multisampling: Nice effect, for sure. We do this on Android at zoom in. Not sure what the level of support for msaa is in general GL implementations. Highly dependent on frame-buffer management in egl backend, etc. Worth a look-see, anyway.

Thanks
Dave

[bdbcat]

Dirk...

re:
Feature request:

Could this be expanded to read the chart/tiles cache from SAS.Планета directly, without the user needing to create MBTiles, when the cache is already there, in well structured folders?

Sounds like a good candidate for a Plugin to me.

Dave

[nohal]

Dave...
Looking into the missing metadata problems now.

• Wrong/missing format info - trivial (we actually do not need m_bPNG at all, load first tile with wxBITMAP_TYPE_ANY and remember what type it was to save some cycles on all the subsequent tiles), patch ready.
• Wrong/missing min/max zoom level - trickier - the tiles table seems to very often contain tiles at lowest zoom levels that are empty etc. Some proof of concept code ready, let's see how well it works.
• Missing bounds - seems quite hard if at all possible to get completely right - low resolution tiles do often, by their theoretical extent, cover a lot of area that is not on the chart they contain. The high resolution tiles almost never cover the whole extent of the tileset. We will have to end up with some compromise.

Pavel

[bdbcat]

Pavel...

Bounds: Error in this parm is not really critical. Not really used for Raster quilting.

In MBTiles, we scan the lowest zoom level tiles to build the valid chart region (for quilting). I think it might be fair to simply toss out tiles that are all black in this scan. We end up with a non-rectangular coverage region, but that is OK.

What will be more problematic is if the minzoom tileset is not contiguous. This will lead to trouble in quilting, I think, as the logic tries to figure out what to do about "holes" in the coverage region.

In the worst case we would need to "logical-OR" all of the tiles in the dB to generate the correct coverage area. And still probably need to detect and deal with holes.


Send up the PRs.

Thanks
Dave