Prepare workflow results for map visualisation

Results visualisation in a thematic application based on geobrowser

The geobrowser of the portal uses the metadata of a job processing results to handle data visualisation on the map. It actually exploits the Terradue Cloud Platform API (see Production API here). Any results produced on the platform is analysed to provide with the map visualisation context. The results data are harvested once staged out of the production to find information such as quicklooks, geolocation, time, tags or any useful information. It also groups similar files to present them as one result entry with different format.

In this article, we describe this analysis process and give the main keys to format your workflows result.

Files aggregation

As a first process, the data are listed to find all similar files for regrouping them as a single result entry. The filename without the extension is used for this aggregation.
For instance, if the result folder contains the following files

Flood_Map_20170405_max_extent.shp
Flood_Map_20170405_max_extent.shx
Flood_Map_20170405_max_extent.prj
Flood_Map_20170405_max_extent.dbf
Flood_Map_20170405_max_extent.png
Flood_Map_20170405_max_extent.pngw
Flood_Map_20170405_max_extent.properties
Flood_Map_20170405_freq.tif
Flood_Map_20170405_freq.rgb.tif
Flood_Map_20170405_freq.tif.legend.png
Flood_Map_20170405_freq.png
Flood_Map_20170405_freq.pngw
Flood_Map_20170405_freq.xml
logs.zip

After analysis, the result metadata report will list only 3 items (datasets) but with many different types of data available for download

Flood_Map_20170405_max_extent
  |- Shapefile (zip)
  |- PNG
  |- PNGW
  |- Properties
  |- Legends (PNG)
Flood_Map_20170405_freq
  |- GeoTiff
  |- PNG
  |- PNGW
  |- XML
  |- Legends (PNG)
logs.zip

For the rest of the article, keep in mind that each group is now called a dataset even if the group may contain only 1 file.

Maps Quicklooks

For each dataset in the job results, the harvester searches for a file that it can render as a raster via WMS. Typically a geolocated images such as a GeoTiff or any image geolocated with a world file.
Here is the ranked (first found, first used) list of supported files that can be rendered as a WMS layer and thus visualised on the map

  • GeoTiff: georeferenced tiff raster (extent + projection referential)
  • Shapefiles: set of files of vectorial spatial data
  • Image + World file
  • Image + properties file with the extent defined (see data properties later)

BUT when any of previous files is a raster AND contains RGB bands, it goes in priority for the map quicklook.

If any of the case listed above is found, the system creates a WMS layer in the layer server and link it with the dataset. Therefore, when the geobrowser finds the link, it uses it as map quicklook of the dataset.

More precision about the raster rendering in WMS is given is the following paragraphs

GeoTiff

Basically, the system handles the tifs in 3 ways according to the bands present in the file:

  • RBG(A) : simple case where the the RGB color model with an optional but recommended alpha channel information for transparency is used.
  • Palette band : each pixel value is represented by a color in a palette. Usually 0 value as transparent color if nodata value is not specified.
  • Grayscale band : unique band with raw value. The system try to present the pixel value as a ramped grayscale palette. Usually 0 value as transparent color if nodata value is not specified.

Image + World files

For any image file (preferably PNG with alpha band for transparency), you can geotag it with a world file. It will be a file with the same name as the image suffixed by w (e.g. data.png → data.pngw).

remark : if a geolocated PNG is found along with a greyscale GeoTiff, the system will choose preferably the PNG to be displayed as quicklook on the map.

Data properties

A dataset can also contain a java properties file (key=value), named as the data file it described by replacing the extension or suffixing with .properties, to give additional information about a result file. This additional information will be added into the metadata entry of the output file as metadata information. All keywords / value from the .properties file are added as a table to the summary element (used for metadata display on the geobrowser). Some keywords can set specific metadata such as:

  • date : in the format startDate/endDate with the date in ISO8601 format, it sets StartDate and EndDate values). A single date sets a instant time.
  • title : defines the title visible in the list of result
  • bbox : georeferences the dataset using the format minX,minY,maxX,minY format
  • envelope : georeferences the dataset using the format Env[minX : maxX, minY : maxY]
  • geometry : georeferences the dataset using WKT format
  • image_url (http://<my_image_url> or file://<my_image_path> : add a thumbnail image in the summary table)
  • copyright : add credits in the summary table)

XML metadata

When a dataset contains an XML file, the system tries to identify a known namespace and then exploit the metadata. The list of XML file format supported are:

  • ISO 19115-1:2014 Geographic Information – Metadata
  • Earth Observation Profile metadata model

Legends

In order to attach a legend to describe the dataset, a file suffixed with .legend.png enables a map functionality that display the legend when the data is selected.

Footprint on the map

When displaying the results in the geobrowser and whenever possible, the datasets footprint are displayed on the map according to the harvested information in this ranked order:

  1. geometry property
  2. envelope property
  3. box property
  4. extent derived from the file (Raster, CSV, kml, PNGW)

It is therefore recommended to avoid duplicated georeference (e.g. Geotiff + geometry) as they could result in odd rendering on the map.