Feature Requests

[bdbcat]

All...


"I don't get this.. Why is a MBTile not suitable for navigation?"


An immediate and obvious problem is depth soundings "unit of measure". Raster charts with soundings in feet can be easily mixed with charts having soundings in Fathoms, all in the same tileset. How is a user to know what is actually being displayed at any particular zoom factor? Tileset metadata will not help us here, unless we define a private "per-zoom-level" metadata structure, and then get the chart creators to support it.


The list goes on..
1. Chart Release date...
2. Chart provenance...
3. etc.



In short, all that finicky stuff in a KAP header is there for a reason: So that the ECS can display the metadata to the navigator, who can then intelligently decide whether this chart is "suitable for navigation".


As nohal says, we stand ready to implement a well thought-out MBTiles metadata scheme, if one appears.


Dave

[Jon Hacking]

But is this reason to change the chart selection paradigm for mbtiles? Why can't we go back to the way KAPs are handled?

[nohal]

Originally Posted by Jon Hacking

But is this reason to change the chart selection paradigm for mbtiles? Why can't we go back to the way KAPs are handled?

Because it would not be possible to implement functionality related to overlays, which is what mbtiles are good for and which is coming in the next phase of the implementation.

[Jon Hacking]

Well, I don't know what's coming. You might stem some of this agony from your users if you enlightened us as to WHY the paradigm has changed.


But to me, there's very little difference between KAPs & mbtiles, so the difference in how they're handled is confusing & not in keeping with the rest of OpenCPN's clean interface.


When I have 4 different mbtiles showing at the bottom of my screen, & none of the "piano keys" have the tiny unreadable X on the left end, which one is actually being displayed? How can I tell? They're not even setup like radio buttons.

[nohal]

What you see depends on the transparency of the mbtiles (You do know they may be and are transparent, right?). And the most detailed, topmost displayed one is on the left, as with any other chart type. Why should they be set up as radio buttons? What would that do?

[bdbcat]

jon...
one critical reason that we changed the chart selection paradigm for mbtiles:


It is easy to create a completely unwieldy huge, but sparse tileset using the available tools. Quilting such a tileset with existing kaps and ENCs is computationally intensive, such that zoom/pan operations can take literally minutes to perform, even on a fast development machine.



Now, I'm sure that YOUR tilesets are thoughtfully constructed.
But we spent a lot of time in Beta testing on this problem. When we saw a problem, we asked the user: "Where did you get this tileset?" Answer, often, was: "I dunno, I guess I downloaded it off the internet..."
And these are the charts that we need to support, somehow, in a way that may be useful, but not misleading.



We support many, many thousands of users worldwide, with experience and knowledge levels spanning the full range. This is a case where we choose to play it safe for the average user experience, and not promise more than the underlying data can deliver.

I think about the Team Vestas "ECS assisted" wreck often....

Dave

[Jon Hacking]

Dave, thanks for this. I still don't understand all of the why, as the reasons given don't seem to justify the confusion, but it seems you folks did discuss it. We were out of internet (north of PNG) during much of the beta program, so couldn't have input then. But my training as embedded dev, MSFT SDET, & Usability Testing was shrieking.

I would have thought that limiting the display to CM93 as the background & ONE other chart type in the possible foreground would have covered the vast majority of cases. This also would have kept computational requirements manageable. Even my i7/GTX860M is having problems with the KAPs & mbtiles I'm throwing at it now.

Pavel, I didn't know that mbtiles could be transparent, but I've noticed that most charts, especially those here in SE Asia, aren't accurate enough to overlay each other well. Well georeferenced satellite imagery is, but I still don't want to overlay it - I want to switch quickly & easily between different sources when one has clouds in the way, or too much reflection on the water. Overlays may work in the developed world, where nautical charts have been corrected over the years, but out here in the boonies, we've seen commercial chart errors of 1nm & errors of 400m are common. So we generally only want 1 chart source at a time.

As I've tried to point out, I'm often loading 5 different mbtiles for a given area: ArcGIS, Bing, GE, Navionics, & CMap, as well as having KAPs & CM93 charts, all for the same area. One problem I have is knowing WHICH mbtile chart is actually displaying, as it's quite possible to have NO little X in the corner of ANY piano key, so potentially ALL mbtiles are displaying. I don't think any of my mbtiles are transparent - they're all made (by me) with Sat2Chart.

But it does seem to me that enabling computationally excessive real-time overlays is not enough reason to change the paradigm. If the user wants overlays, make them offline & display them like any other chart. There are ample tools for that. And most of our boats don't have computational rocket boxes. We select our computers for reliability, not horsepower.

[nohal]

Jon...
While we have trivial solution for your "issue" in form of chart groups, where you may never have the mix of the tilesets you are worried about displayed at the same time (shall you want that), asking the user to "do his overlays offline" by somehow preprocessing the chart material anticipating what he might require in the future, is exactly the thing that can't be done in real world.
And of course displaying all the mbtiles covering the area at the same time is a completely valid use case, maybe not your own, but completely valid, so even if we did what you try to make us do, someone else will appear complaining and will clearly be right with the tilesets he has.

Pavel

[Baksteen]

Originally Posted by bdbcat

An immediate and obvious problem is depth soundings "unit of measure". Raster charts with soundings in feet can be easily mixed with charts having soundings in Fathoms, all in the same tileset. How is a user to know what is actually being displayed at any particular zoom factor? Tileset metadata will not help us here, unless we define a private "per-zoom-level" metadata structure, and then get the chart creators to support it.

I again don't see why a description of the full tileset cannot be useful. Also, it would be just plainly stupid to create a chart with different units at different zoom-levels.

In short, all that finicky stuff in a KAP header is there for a reason: So that the ECS can display the metadata to the navigator, who can then intelligently decide whether this chart is "suitable for navigation".

And indeed, as you say, the KAP header is there for a reason. Currently MBTiles doesn't support all of those entries and therefore would like to use the description entry for it.. For the same reasons as for the KAP.

I also don't see why metadata is ignored (is it?) in the standard? Description is one of the optional entries in the metadata.


Edit:
Someone asked for an example; https://filebin.net/bqdabtdycvb79yto
Very simple one-level example at small resolution, just to clarify some.
For now, I'm putting the description in the name-entry because that is displayed.

Metadata:
  ZOOM_LEVEL=17
  name= Veerbootroute Ameland, Maart 2019
  type=overlay
  description= date: Mar 12 2019 / hDatum: NAP / data: Rijkswaterstaat
  version=1.1
  format=png
  bounds=5.76118964407228873,53.4078281616165071,5.80999511930727408,53.4367457257217353
  minzoom=17
  maxzoom=17

[nohal]

Originally Posted by Baksteen

I again don't see why a description of the full tileset cannot be useful. Also, it would be just plainly stupid to create a chart with different units at different zoom-levels.

NOAA, as an example? Look at any of the composite tilesets.

And indeed, as you say, the KAP header is there for a reason. Currently MBTiles doesn't support all of those entries and therefore would like to use the description entry for it.. For the same reasons as for the KAP.

I also don't see why metadata is ignored (is it?) in the standard? Description is one of the optional entries in the metadata.

Please, read that standard, you can find it at https://github.com/mapbox/mbtiles-sp...er/1.3/spec.md
As you can see, there simply is no metadata for depth unit or datum defined. If you think it is wrong, contribute to that standard.

Edit:
Someone asked for an example; https://filebin.net/bqdabtdycvb79yto
Very simple one-level example at small resolution, just to clarify some.
For now, I'm putting the description in the name-entry because that is displayed.

Metadata:
  ZOOM_LEVEL=17
  name= Veerbootroute Ameland, Maart 2019
  type=overlay
  description= date: Mar 12 2019 / hDatum: NAP / data: Rijkswaterstaat
  version=1.1
  format=png
  bounds=5.76118964407228873,53.4078281616165071,5.80999511930727408,53.4367457257217353
  minzoom=17
  maxzoom=17

This is a perfect example of a bad solution that is not going to work. a) it is incomplete, for example ignores depth units, b) it is not enough, c) it will conflict with what tileset producers not being you think the description field should contain, d) it is error prone while being created, e) it is completely unnecessarily complicating the implementation as everybody will have to parse free text. (metadata table is a table storing key-value pairs, storing key-value pairs in one of the values is simply not a good idea)

[Baksteen]

Originally Posted by nohal

NOAA, as an example? Look at any of the composite tilesets.

Please, read that standard, you can find it at https://github.com/mapbox/mbtiles-sp...er/1.3/spec.md
As you can see, there simply is no metadata for depth unit or datum defined. If you think it is wrong, contribute to that standard.



This is a perfect example of a bad solution that is not going to work. a) it is incomplete, for example ignores depth units, b) it is not enough, c) it will conflict with what tileset producers not being you think the description field should contain, d) it is error prone while being created, e) it is completely unnecessarily complicating the implementation as everybody will have to parse free text. (metadata table is a table storing key-value pairs, storing key-value pairs in one of the values is simply not a good idea)

Ok.... thanks for the kind words.

Of course there's no metadata for depth units or datums. My file was just an example of what I would like to show to the user. It was a workaround only to show the feasibiltiyy of the description entry, and make MBTile actually useful for openCPN. And yes, I've gone through the standard and description is just a perfect valid metadata entry. You say I'm storing key:values in the description, for me it is just a string.
Only thing I was asking if the description entry could be shown in the piano. For some - for me - obvious reasons. But it's OK now, I'll just be using the name-entry.

[nohal]

Originally Posted by Baksteen

Only thing I was asking if the description entry could be shown in the piano. For some - for me - obvious reasons. But it's OK now, I'll just be using the name-entry.

What exactly were you suggesting is the obvious advantage of abusing one field not meant to hold that information over another field not meant to hold that information?
BTW, the last guy with the same idea as yours wanted to do the same with the attribution field, that idea of course was equally bad..

[Baksteen]

Originally Posted by nohal

What exactly were you suggesting is the obvious advantage of abusing one field not meant to hold that information over another field not meant to hold that information?
BTW, the last guy with the same idea as yours wanted to do the same with the attribution field, that idea of course was equally bad..

No, I don't want to abuse anything.
The MBTiles standard is not a nautical standard, but for the sake of giving the user/sailor some information about the image I would like to describe the chart and the information it holds. Seems like 'description' is a proper entry for that... Of course, for an image containing depths buoys etc I want to describe the nature of those, therefore the description is mainly about depth units and datum. While dedicated fields are lacking in the MBTiles format (of course, it is not meant to be dedicated to nautical navigational charts), I need another place for that. The standard supports that ('description'), but it is simply not displayed in openCPN.
I really don't see why there is so much hassle over this. Is there a good explanation why you don't want to show the description?

[nohal]

Originally Posted by Baksteen

No, I don't want to abuse anything.
The MBTiles standard is not a nautical standard, but for the sake of giving the user/sailor some information about the image I would like to describe the chart and the information it holds. Seems like 'description' is a proper entry for that... Of course, for an image containing depths buoys etc I want to describe the nature of those, therefore the description is mainly about depth units and datum. While dedicated fields are lacking in the MBTiles format (of course, it is not meant to be dedicated to nautical navigational charts), I need another place for that. The standard supports that ('description'), but it is simply not displayed in openCPN.
I really don't see why there is so much hassle over this. Is there a good explanation why you don't want to show the description?

If the fields are missing, extend the standard and add those fields, that is what a database table holding key-value pairs is for.

The good explanation is: Because people like you will immediately start to abuse it to store other metadata and this will inevitably lead to another layer of complaints as "Why don't you parse and use the depth units/datum/release/ntm update/... from the description/attribution/name/... field?" This will not happen, not as long as I'm around.

[Baksteen]

Originally Posted by nohal

If the fields are missing, extend the standard and add those fields, that is what a database table holding key-value pairs is for.

The good explanation is: Because people like you will immediately start to abuse it to store other metadata and this will inevitably lead to another layer of complaints as "Why don't you parse and use the depth units/datum/release/ntm update/... from the description/attribution/name/... field?" This will not happen, not as long as I'm around.

It is pretty clear why I wouldn't extend a really general standard with metadata dedicated to navigational charts, I think. Also, why would I add metadata if the currently-implemented standard is and will not be implemented.

But ok, your opinion is clear, although I do not agree with your explanation. Of course, you don't want to parse information from the description. But that's not what I'm proposing.

Then I'll continue to 'abuse' the name entry. Just for the sake of making MBTiles useful in openCPN. For me it's clear now and thanks for your time.