getting ArcGIS to recognise the WMTS endpoint

[gtnbssn]

TL;DR: Does anyone have experience and suggestions on making ArcGIS recognize the WMTS endpoint offered by titiler?

We have deployed titiler to provide more usual webservices to our users. An important motivation is that ESRI's software has poor support of cloud optimized geotiffs, and we would like to accomodate their users too.

It all works beautifully in QGIS at the moment, but ArcGIS (pro and online) does not recognize our endpoint as proper WMTS it seems.

We did a bit of url rewriting in order to:

• make urls more readable
• only offer urls that correspond to our COGs, so that our titiler deployment does not get used to tile other COGs out there

So this url: https://libmaps.nus.edu.sg/services/1950/wmts is working very nicely in QGIS, but not in ArcGIS pro. We have made another url to access the tiles directly: https://libmaps.nus.edu.sg/gis/rest/services/Sing_Hist_Maps/1950/MapServer/tile/{z}/{y}/{x} but this means the client does not know the extent of the COG and is potentially requesting many unexisting tiles. (The longer url format mimics the ArcGIS server format we are migrating from, in the hope that some users will be able to keep their setup without intervention.)

Any help appreciated!

[vincentsarago]

Dear @gtnbssn
I would love to help but we don't have access to ESRI product 😓

You are saying it works well with QGIS, and I can also confirm that GDAL reads it 👌

 gdalinfo https://libmaps.nus.edu.sg/services/1950/wmts
Driver: WMTS/OGC Web Map Tile Service
Files: none associated
Size is 114944, 98304
Coordinate System is:
PROJCRS["WGS 84 / Pseudo-Mercator",
    BASEGEOGCRS["WGS 84",
        ENSEMBLE["World Geodetic System 1984 ensemble",
            MEMBER["World Geodetic System 1984 (Transit)"],
            MEMBER["World Geodetic System 1984 (G730)"],
            MEMBER["World Geodetic System 1984 (G873)"],
            MEMBER["World Geodetic System 1984 (G1150)"],
            MEMBER["World Geodetic System 1984 (G1674)"],
            MEMBER["World Geodetic System 1984 (G1762)"],
            MEMBER["World Geodetic System 1984 (G2139)"],
            ELLIPSOID["WGS 84",6378137,298.257223563,
                LENGTHUNIT["metre",1]],
            ENSEMBLEACCURACY[2.0]],
        PRIMEM["Greenwich",0,
            ANGLEUNIT["degree",0.0174532925199433]],
        ID["EPSG",4326]],
    CONVERSION["Popular Visualisation Pseudo-Mercator",
        METHOD["Popular Visualisation Pseudo Mercator",
            ID["EPSG",1024]],
        PARAMETER["Latitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8801]],
        PARAMETER["Longitude of natural origin",0,
            ANGLEUNIT["degree",0.0174532925199433],
            ID["EPSG",8802]],
        PARAMETER["False easting",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8806]],
        PARAMETER["False northing",0,
            LENGTHUNIT["metre",1],
            ID["EPSG",8807]]],
    CS[Cartesian,2],
        AXIS["easting (X)",east,
            ORDER[1],
            LENGTHUNIT["metre",1]],
        AXIS["northing (Y)",north,
            ORDER[2],
            LENGTHUNIT["metre",1]],
    USAGE[
        SCOPE["Web mapping and visualisation."],
        AREA["World between 85.06°S and 85.06°N."],
        BBOX[-85.06,-180,85.06,180]],
    ID["EPSG",3857]]
Data axis to CRS axis mapping: 1,2
Origin = (11550781.529314344748855,151536.408575402601855)
Pixel Size = (0.149291070869485,-0.149291070869485)
Metadata:
  ABSTRACT=Cloud Optimized GeoTIFF
  TITLE=Cloud Optimized GeoTIFF
Image Structure Metadata:
  INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left  (11550781.529,  151536.409) (103d45'44.77"E,  1d21'40.13"N)
Lower Left  (11550781.529,  136860.499) (103d45'44.77"E,  1d13'45.64"N)
Upper Right (11567941.642,  151536.409) (103d54'59.72"E,  1d21'40.13"N)
Lower Right (11567941.642,  136860.499) (103d54'59.72"E,  1d13'45.64"N)
Center      (11559361.586,  144198.454) (103d50'22.24"E,  1d17'42.89"N)
Band 1 Block=128x128 Type=Byte, ColorInterp=Red
  Overviews: 57472x49152, 28736x24576, 14368x12288, 7184x6144, 3592x3072, 1796x1536, 898x768, 449x384, 224x192
  Mask Flags: PER_DATASET ALPHA 
  Overviews of mask band: 57472x49152, 28736x24576, 14368x12288, 7184x6144, 3592x3072, 1796x1536, 898x768, 449x384, 224x192
Band 2 Block=128x128 Type=Byte, ColorInterp=Green
  Overviews: 57472x49152, 28736x24576, 14368x12288, 7184x6144, 3592x3072, 1796x1536, 898x768, 449x384, 224x192
  Mask Flags: PER_DATASET ALPHA 
  Overviews of mask band: 57472x49152, 28736x24576, 14368x12288, 7184x6144, 3592x3072, 1796x1536, 898x768, 449x384, 224x192
Band 3 Block=128x128 Type=Byte, ColorInterp=Blue
  Overviews: 57472x49152, 28736x24576, 14368x12288, 7184x6144, 3592x3072, 1796x1536, 898x768, 449x384, 224x192
  Mask Flags: PER_DATASET ALPHA 
  Overviews of mask band: 57472x49152, 28736x24576, 14368x12288, 7184x6144, 3592x3072, 1796x1536, 898x768, 449x384, 224x192
Band 4 Block=128x128 Type=Byte, ColorInterp=Alpha
  Overviews: 57472x49152, 28736x24576, 14368x12288, 7184x6144, 3592x3072, 1796x1536, 898x768, 449x384, 224x192

it would help if you can share any error message or logs from ESRI

[gtnbssn]

Thanks for the quick answer @vincentsarago !

We took some time to test more on our side, unfortunately ArcGIS pro isn't giving a lot of information about what went wrong... Here is what it looks like:

Since we read somewhere that ArcGIS pro doesn't support webp, we tried to do a url that forces png but no luck:

As you can see, this is not very informative. We found a error log window, but even at the "debug" level it didn't have any relevant information:

Anyway, we are probably going to be more specific in our instructions to our users, to explicitly direct ArcGIS pro users to the TMS service.

Meanwhile if anyone has an idea, I'll be interested!

In particular: does someone have a setup somewhere that works with ESRI's products?

[gtnbssn]

Still exploring this and since it is 2023, I had chatgpt analyse the WMTS spec from OGC available here: https://www.ogc.org/standard/wmts/ and our endpoint's capabilities XML.

One interesting thing that it pointed out is that our endpoint has the same URL for GetCapabilities, GetTile, and ServiceMetadataURL. I asked it to point me to the specific part of the spec detailing how these should be set up but it got lazy and told me to do it myself!

Anyway, could that be the issue here?

This other endpoint has different URLs for each: https://geoinformation.eisenbahn-bundesamt.de/wmts/WMTSCapabilities.xml?

I can see how it would make sense to have different addresses indeed.

Should /cog/tiles/{tileMatrixSetId}/{z}/{x}/{y} be the URL for GetTile?

I have an example integration with openlayers (the only js library that builds up its requests on top of the information it received from the capabilities XML it seems) and it is getting the tiles from there:

https://codepen.io/gbn000/pen/gOEYNKz

I think it is using this URL as it is the ResourceURL for the layer. I did have to specify the layer and matrixSet id in the code.

Not sure what Service MetadataURL is and whether that would be the problem. In the OGC spec, there is an example in annex D, page 98, and it looks pretty much like the capabilities XML.

[danmorley]

Still exploring this and since it is 2023, I had chatgpt analyse the WMTS spec from OGC available here: https://www.ogc.org/standard/wmts/ and our endpoint's capabilities XML.

One interesting thing that it pointed out is that our endpoint has the same URL for GetCapabilities, GetTile, and ServiceMetadataURL. I asked it to point me to the specific part of the spec detailing how these should be set up but it got lazy and told me to do it myself!

Anyway, could that be the issue here?

This other endpoint has different URLs for each: https://geoinformation.eisenbahn-bundesamt.de/wmts/WMTSCapabilities.xml?

I can see how it would make sense to have different addresses indeed.

Should /cog/tiles/{tileMatrixSetId}/{z}/{x}/{y} be the URL for GetTile?

I have an example integration with openlayers (the only js library that builds up its requests on top of the information it received from the capabilities XML it seems) and it is getting the tiles from there:

https://codepen.io/gbn000/pen/gOEYNKz

I think it is using this URL as it is the ResourceURL for the layer. I did have to specify the layer and matrixSet id in the code.

Not sure what Service MetadataURL is and whether that would be the problem. In the OGC spec, there is an example in annex D, page 98, and it looks pretty much like the capabilities XML.

Any luck getting TiTiler working with ArcGIS?

[gtnbssn]

It looks like a fix was made and should be available in the latest versions: #817

I have left my previous job and am not able to test this though!

[Mario-Miguel]

I think the problem is the SupportedCRS property of the WMTSGetCapabilities xml. I tried to substitute the default value of this property (set by default as a url to the xml file where the definition of the CRS system) for the CRS system identifier.

For example:
If WGS1984Quad TileMatrixSet is used I have substituted the default SupportedCRS (http://www.opengis.net/def/crs/EPSG/0/4326) with urn:ogc:def:crs:EPSG::4326.

It is also important to import the WMTS as a server:

Hope this helps.

[vincentsarago]

Ref: https://github.com/developmentseed/titiler/pull/1004/files#r1811175165

We had no idea which one ArcGIS preferred, and the spec is not really clear about this.

I think we could switch to URN instead of URI

cc @AndrewAnnex

[AndrewAnnex]

IIRC most of the examples in the spec did use URNs rather than URIs but in any case tools like GDAL seem to handle both with ease. It's a relatively easy swap to make in the code but I have a few things that will prevent me from contributing for a few days.

On a side note, despite the proprietary nature of ESRI, I have seen that they do respond to bug posts on their forum so it could be worth @Mario-Miguel submitting a linked issue there to see if they are willing to update to be more flexible. Best approach would be to do a fix for titiler (as qgis/gdal seem to work fine with URNs) and post an issue to ESRI to avoid proprietary software from driving open source software development...

[AndrewAnnex]

on a side note, this PR on QGIS qgis/QGIS#59144 aims to fix some of the issues in QGIS that prevent most of the optional parameters from being passed to TiTiler, although I am not sure how well it's changes will work with TiTiler and I don't have the time currently to investigate fully.