Support GeoServer Vector tile

[sunshine0576]

I want add Geoserver Vector tile layer to QGIS, How should I add a connection?

[sfkeller]

We did not yet test the plugin with a GeoServer connection. Can you provide us with remotely accessible source. A Vector Tile connection requires either a mbtiles file or a TileJSON file typically in the form https://yourdomain.xyz/data/v3.json according to TileJSON 2 spec: is here https://github.com/mapbox/tilejson-spec/tree/master/2.2.0 (which actually still has to be extended in order to reflect the add-ons Mapbox made in order to support Mapbox Vector Tiles, .mvt).

[sunshine0576]

The format of the request for Geoserver.
http://localhost:8080/geoserver/gwc/service/wmts?REQUEST=GetTile&SERVICE=WMTS&VERSION=1.0.0&LAYER=sf:roads&STYLE=&TILEMATRIX=EPSG:900913:{z}&TILEMATRIXSET=EPSG:900913&FORMAT=application/x-protobuf;type=mapbox-vector&TILECOL={x}&TILEROW={y}

[mnboos]

It could work if a TileJSON file is created and the tile url set to the link above. Additionally the required TileJSON values would have to be present (i.e. "vector_layers", "minzoom" and "maxzoom").

[sunshine0576]

Expect to be able to support Geoserver new features.

[mnboos]

@sfkeller Should we implement GeoServer support?

[sfkeller]

Yes, but not "directly" accessing zxy but as data source via the TileJSON metadata file.

[sunshine0576]

@sfkeller Geoserver support TileJSON metadata file.

[sfkeller]

We can't test GeoServer Support without an example of a remote server.

[tomchadwin]

@mtravis Are any of our Geoservers on versions recent enough to support vector tiles?

[mtravis]

I don't believe so. Let me check.

…

On 17 Nov 2017 11:13 am, "Tom Chadwin" ***@***.***> wrote: @mtravis <https://github.com/mtravis> Are any of our Geoservers on versions recent enough to support vector tiles? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#112 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABt0RgMHrjp7a_cTwdNxARi5JbQx9t6-ks5s3WpLgaJpZM4QV6pE> .

[tomchadwin]

@sunshine0576 Do you know which Geoserver version introduced VT support?

[mtravis]

Its seems that the vt extension is only available with 2.11 onwards. https://sourceforge.net/projects/geoserver/files/GeoServer/2.11.0/extensions/ @tomchadwin the shared geoserver is running 2.9

…

On Fri, Nov 17, 2017 at 11:18 AM, Tom Chadwin ***@***.***> wrote: @sunshine0576 <https://github.com/sunshine0576> Do you know which Geoserver version introduced VT support? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#112 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABt0RpGmPGH2Dy3rXSQveY7aN9gfP1-Aks5s3WuWgaJpZM4QV6pE> .

[tomchadwin]

And we can't use yours? @tomchadwin gives @mtravis a winningly persuasive smile

[mtravis]

Shamefully we are even further behind (need to upgrade ubunutu first). I've only just seen the whole thread so see that you need one for testing.

[tomchadwin]

And for fun...

[mtravis]

of course...

[sfkeller]

Dear @flippmoke : Pls. allow me to bump on this again since here at Vector Tiles Reader for QGIS project (to be released end of 2017) we are heavily relying on remote TileJSON and mbtiles metadata, while TileJSON spec. still does not even mention Vector Tiles.

I'd really like to know now which metadata is going to be there - mandatory or optional.

Saying "we" I dare to include OpenMapTiles by @klokan, MapZen by @nvkelso, t-rex by @pka, as well as TileMaker by @systemed and others like GeoServer by @aaime (?) and PostGIS i.e. by @pnorman ...

[pnorman]

fwiw, I don't have any plans on including tilejson data with the projects I'm currently working on.

[sfkeller]

@pnorman Thanks for the info (BTW you probably have been the first realizing this issue in 2014: mapbox/tilejson-spec#14 !)

[systemed]

Yes, would be good to get this finally sorted out - .mvt is in real danger of turning into the new Markdown (everyone uses it, no two clients parse/write it the same way).

[aaime]

Hi, I had no part in adding MVT support for GeoServer, let me add into the loop @DBlasby, @jodygarnett and @tbarsballe (sorry for the long lineup, I'm not sure who's maintaining vector tiles these days)

[jaakla]

It seems that the vector_layers tag in mbtiles/tilejson is mapbox-specific not documented extension. You can consider it proprietary, and hope that their lawyers do not chime in if you use it. Requiring such metadata to be omnipresent by other vendors vector tile implementations sounds like slippery slope to me.

This metadata is very useful indeed for many use cases, without it you may need to scan through all tiles to reverse-engineer data schema. Note: MVT was designed first of all for map rendering, so with using it beyond it brings you to many tech challenges. So any motion to standardize it would be very welcome. But as long as the whole MVT stack is fully under single company control, then there is not much what outsiders can do about it.

p.s. Regarding object IDs in actual data there is some activity in OMT side: openmaptiles/openmaptiles#303

[sfkeller]

It seems that the vector_layers tag in mbtiles/tilejson is mapbox-specific not documented extension. You can consider it proprietary, and hope that their lawyers do not chime in if you use it. Requiring such metadata to be omnipresent by other vendors vector tile implementations sounds like slippery slope to me.

Actually @flippmoke (see above) gave us hope that no lawyers but techies chime in order to update the spec.

[flippmoke]

@jaakla

We are 100% committed to open source and open standards at Mapbox.

https://www.mapbox.com/about/open/

We work hard to keep open standards and open source. This is backed up by our use of Creative Commons license on our specifications. In fact I just added a license file to make it even more clear on the Vector Tile Specification.

Not only do provide these licenses in an open way, we go out of our way to make sure that we spend time to listen to the community as a whole prior to making changes to the specification. While these specs are often important to Mapbox's strategic goals, we also want to spend the time to provide any changes that are needed externally to Mapbox.

As the maintainer at Mapbox of the Vector Tile Spec, I spend quite a bit of time reading and looking at requests for the spec and implementations of the specification. Just like any other body that controls a specification we spend time to ensure that proper time is given for input from others prior to publishing and collect important ideas and comments that determine how the standard should move forward.

But as long as the whole MVT stack is fully under single company control, then there is not much what outsiders can do about it.

This is simply not true -- @sfkeller is a great example of this through this ticket and other efforts. We at Mapbox had not updated some of our standards up to this point and he is calling us out for it. This is greatly appreciated. I have already had several internal meetings about supporting the community needs here and updating items as necessary. The fault here is our own because we did not properly assign a maintainer of some specs as we had some individuals leave Mapbox.

Now that we are attempting to consider how to update these documents we are reviewing community input and issues that have been linked from the specification. We encourage pull requests on all of our standards and we want to be notified when we are not supporting the community properly.

[jaakla]

Thank you for the explanation. By "single company control" I mean different from how e.g. OGC and many other standard setting groups are designed - you have open independent group of people from various backgrounds, who decide about some standard, there is no one big entity who would have final decision for any detail. "Open standard from a company" by definition is different - in addition to the community you also need to consider company interests, and you have every right to keep some things you've invested in or what helps to increase revenues, closed. For example vector data schema or web map style editing tools. It is not special for mapbox, this is how any sensible commercial company does and there is nothing bad about that really. Being too zen with too much openness and forgetting about business realities is dangerous also.

Mapbox has been above average open to share useful things, and to accept community contributions, big kudos to them for this. However, as there has been at least one case where someone has naively tried to hack into tech details which were not explicitly open and they got their fingers burnt. So instead of reverse-engineering I would expect that these de-facto things are specified clearly in the open specification first, to be sure that it is all legal. Also to be sure that it has minimal technical stability. Is see the vector-revision draft is on the way, that's correct?

Going with fast-developing tech like vector tiles were few years back to an organization like OGC can easily kill technical edge, so in short term one strong commercial leader is better. But once they are stable (and vector tiles and mbtiles for example are already 'establishment', have not changed much anymore in last few years), then for long-term safety I hope Mapbox considers handing them over to a neutral standard body, or create new if there is no suitable one.

[gannebamm]

Are there any news how to load by Geoserver served vector tiles by the plugin?

[sfkeller]

No, sorry that I can't report from any activities related to this issue. Perhaps others listening here can correct me.

As this is an underfunded, volunteered project I beg you patience. And to encourage contributors I put a new label "contribute or fund me" on it.

[mnboos]

Apologies for all the delays. We just moved to another town and will soon go to our honeymoon for 3 weeks. I hope I can do something after that.

[GjueAtGit]

I made some attempts to integrate Vector Tiles from GeoServer into QGIS via the Vector Tiles Reader plugin. I was not successful with TMS. However, the tiles can be read via WMTS. A code example and further information (currently only in German) I have described here.

[sfkeller]

Hi/Hallo Gerd.

Thank you very much for your information about your successful approach connecting QGIS MVTs with GeoServer via WMTS (GeoServer 2.14. + VectorTilesPlugin, QGIS 2.18.24 + Vector Tiles Reader QGIS-Plugin 2.0.3).

If you want to help, it would be great when you could deploy a GeoServer on the internet (and send us the URL to the TileJSON file), so that we can test the VT Reader.

It's not yet clear when we have time since, as said above, @mnboos is away for some weeks, and the VT Reader is a hobby project of him and me in our leisure time.

[tomchadwin]

If I can find some time in the next few weeks, I (or we, @mtravis) might be able to get a public-facing Geoserver updated for testing.

[aaime]

If you want to play, this is one you can try out:
https://maps.geo-solutions.it/geoserver/gwc/service/wmts?REQUEST=GetCapabilities

Mind, this server is experimental, works off the development series of GeoServer and can be redeployed at any time. But with some patience, you can use it :)

[GjueAtGit]

@aaime: With this WMTS no format "application/x-protobuf;type=mapbox-vector" is available. So there are no vector tiles served.
You would have to install the Vector Tiles Extension, activate the format "application/x-protobuf;type=mapbox-vector" in "Caching Standards" and then also activate for the corresponding layer under "Tile cache".

[sfkeller]

The challenge here is to get access to a GeoServer instance over internet which has an accomanying TileJSON file (containing metadata, very much like WMTS Capabilities), which is the "access point" for any MVT reader.

@aaime: It would be nice if a GeoServer utility would create such a TileJSON file for a given VT source. This would be a stub which needs to be manually completed (a source like GeoServer/Extension can't guess automatically all mandatory elements of a TileJSON file). The VT server "T-Rex" for example has such a utility.

[aaime]

@sfkeller I know of no plans to generate a tilejson format out of GeoServer, but anyone interested can do a PR, it would be more than welcomed.
Side note, we are currently working with on a OGC pilot on vector tiles, no tilejson files are used, only WMTS capabilities and a variant of that as a WFS3 extension.

[aaime]

@GjueAtGit not sure what you're talking about, here is an excerpt of the capabilities document:

<Layer>
<ows:Title>daraa_vtp</ows:Title>
<ows:Abstract/>
<ows:WGS84BoundingBox>
<ows:LowerCorner>35.8995094299316 32.4131851196289</ows:LowerCorner>
<ows:UpperCorner>36.5781326293945 33.1460647583008</ows:UpperCorner>
</ows:WGS84BoundingBox>
<ows:Identifier>vtp:daraa_vtp</ows:Identifier>
<Style isDefault="true">
<ows:Identifier/>
</Style>
<Format>application/x-protobuf;type=mapbox-vector</Format>
<InfoFormat>text/plain</InfoFormat>
<InfoFormat>application/vnd.ogc.gml</InfoFormat>
<InfoFormat>text/xml</InfoFormat>
<InfoFormat>application/vnd.ogc.gml/3.1.1</InfoFormat>
<InfoFormat>text/xml</InfoFormat>
<InfoFormat>text/html</InfoFormat>
<InfoFormat>application/json</InfoFormat>
<TileMatrixSetLink>
...

[sfkeller]

@aaime

Side note, we are currently working with on a OGC pilot on vector tiles, no tilejson files are used, only WMTS capabilities and a variant of that as a WFS3 extension.

Interesting. I heard about some OGC activities to standardize VT - and I feared they would approach VT standardization the OGC/GeoCapabilities/XML way. I'm much in favour of standards but IMHO using XML is "horseshoe"-ing a bit this webbased use case - which should prefer JSON.

[aaime]

@sfkeller you misunderstand. WFS3 is all OpenAPI/rest/json based and some experiments are being made on serving tiles with an extension of it. This will likely shape a future WMTS 2.0. It's just not going to happen today.

[GjueAtGit]

@aaime: Oh, excuse me. That was a mistake on my part. I searched for mapbox-vector and didn't find the format. I must have typed it wrong.

[brian32768]

Do you still need a GeoServer for testing this? I'm working on a docker-compose project called geoserver-compose to deploy GeoServer+GeoWebCache+PostGIS+pgadmin+nginx. I'm testing it on my home server but if you want to stand up your own complete instance my goal is to make it possible to build your own in 10 minutes. :-) I came here because I added the vector tile plugin to GeoServer today and now I need to test it, so far I test things first in QGIS then write openlayers/react code.
If you still want a server I could stick one up on some public place (probably in AWS)

[mnboos]

Thanks for the offer! A GeoServer for testing would definitly be very nice!

[brian32768]

Worked on my geoserver openlayers test all day and got MVT working. I will put a public geoserver up for and post location.

…

On Fri, Mar 8, 2019, 3:27 AM Martin Boos ***@***.***> wrote: Thanks for the offer! A GeoServer for testing would definitly be very nice! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#112 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AB_oxA_HeeioYzeSDtrdYQtcp_V4WYeuks5vUkkfgaJpZM4QV6pE> .

[mnboos]

Looking forward to it!

[mnboos]

@brian32768 Is this still a possibility?

[kannes]

Hey @brian32768, please share some tidbits at least. I guess it is safe to assume that GeoServer still has no TileJSON thingie nowadays?