Project full title
Data Fusion Portal
Nature of deliverable
Scheduled delivery date`
31st May 2019
The Planmap data access and fusion portal is here introduced. Its architecture, functionalities and components are presented. The planmap map portal codebase (See D7.4) and relevant tools are listed and linked from the present deliverable. Data access is provided via file-based, map and programmatic interfaces. Data citation is going to be available via the ESA Guest Storage facility.
|ESA||European Space Agency|
|ESDC||ESA Science Data Centre|
|GIS||Geographic Information System|
|OGC||Open Geospatial Consortium|
|Portable Document Format|
|WCS||Web Coverage Service|
|Web Feature Service|
|WMS||Web Map Service|
Planmap data comprise raster and vector data (Rothery et al., 2018, D2.1-public; Rothery et al 2019, D2.2-public ; Rossi et al., 2019, D7.4-public). The iterative process of geologic mapping is based on various data, such as images (geomorphology, morphostratigraphy) and the analysis of morphostratigraphy vs. composition (Zambon et al, 2018, 2019, D4.2 (Spectral Index and RGB maps II), D4.2-public). Preliminary products underlying mapping are raster-based, while mapping itself is predominantly vector-based. Final maps are hence constructed on one or more raster layers and one or more vector annotation layers, as well as formatted layout views (map sheets), typically in pdf format.
The Planmap web portal serves the intermediate and mature data sharing needs of the internal project and can be used by the external community, e.g. re-use of data, maps (Figure 1).
Figure 1: Workflow across Planmap Work Packages for data sharing, individual and integrated analysis and sharing at different stages of maturity.
Figure 2: Planmap web app basic components and their relationships.
The backend includes file-base access and OGC-compliant interface access. All products used, or going to be used, for the duration of the project are Open Source-based, and include, but are not limited to:
GeoServer is responsible for providing the geological maps and their underlying basemaps-- raster and vector data -- live to the app. GeoServer provides WMS, WFS services used by the client to display the maps on-demand, publishing multi-resolution and location-based data, styles, and custom projections.
Apache File-server provides the "raw" data packages, composed of pre-processed or mosaicked data from public archives such as NASA PDS or ESA PSA. Hence the users through Apache file-server can browse and download all these data. The interface published was customized to provide a better user experience, allowing the users direct access to the data and metadata describing in fine detail the contents of the packages.
During day-to-day operations and technical/scientific activities of the consortium, data will be stored on each partner's premises as well as, when relevant, on shared network resources (such as cloud, FTP and web mapping data access services). In principle, data will be made publicly available as soon as possible during the project, respecting publication embargoes. Also preliminary mapping products, both in vector and layout map sheet formats are available (see Data and code).
The file-based access web interface is depicted in Figure 3. Map-wide metadata, including author contact details, citation information, map descriptions and companion paper information (when available) are included in plain text and displayed in each subdirectory as needed. The list of available maps is also kept up-to-date on Data and code.
Figure 3: Directory-based access to Planmap raster, vector and document data (see also D2.2 (geomorphological maps) and D7.3 (Data Management Plan Update))
Maps are accessible and also visualised via a dedicated web-mapping interface, whose source code is also available (see D7.4-public) and that will be updated and improved throughout the course of the project. The main interface is occupied by a global view of each planet relevant to Planmap (the Moon, Mercury and Mars; Figure 4).
Additional vector symbologies will be included and updated in future releases.
Extension and homogenisation of vector fields displayed as well as stored on the source vector files will take place and will be reflected in the main web map interface. Major releases of the web map will also result in GitHub data releases and eventual Zenodo/OpenAIRE DOI records for re-use and citation.
Figure 4: Planmap web map interface, exemplified with a map of Mercury. Data visualisation and unit identification, as well as data download, either as a compressed archive or individual files, are provided. Exemplary data access/query notebooks are provided for interactive Jupyter-based analysis. OGC (WMS, WCS, WFS) endpoints information is also provided for use with desktop (QGIS, ArcGis) or Jupyter-based GIS.
Figure 5: Access to an individual map (in this figure a quadrangle map of Mercury, H05, Hokusai as linked from the web map interface.
The backend of the data fusion via Jupyter/Python and OGC (WMS, WCS, WFS) interfaces are described in D7.4-public and references/repositories linked therein. Additional backends and the use of PlanetServer (Figuera et al., 2018) for direct access to relevant data subsets (e.g. for Mars, Moon) are envisaged.
Data access can be obtained with any OGC WMS/WCS/WFS client, e.g. Desktop GIS systems such as Qgis, or Jupyter-based (https://jupyter.org) interactive clients (Figures 6 and 7).
The primary planmap end-point for WMS/WCS/WFS data access is:
Geospatial analysis (vector, raster, data fusion) is primarily envisaged via OGC access to the data and maps, as well as integration with external data sources. The primary client for this is Jupyter/Python. In addition to OGC file access, direct file access via the web is envisaged for relatively lightweight vector files (available as OGC Geopackage individual, self-contained file) using relevant libraries as well as possibly for individual raster files as Cloud-optimised geotiffs (GOC).
Figure 6: Access to Planmap data via WFS via Jupyter/Python and OWSLib (https://github.com/geopython/OWSLib). See D7.4-public and links therein.
Figure 7: Access to Planmap data and Overlay of WMS and WFS. See D7.4-public and links therein.
Data citation is going to be available via the ESA Guest Storage facility from ESA ESDC. Software use is free. In the case of software releases, relevant Zenodo/OpenAIRE entries for each release (e.g. web portal client code, see also D7.4-public) will be provided and linked from relevant web pages.
Marco Figuera, R., et al. (2018) Online characterization of planetary surfaces: planetserver, an open-source analysis and visualization tool, Planet. Space Sci., DOI: 10.1016/j.pss.2017.09.007.
Rothery, D., van der Bogert, C., Luzzi, E., and the Planmap consortium (2019) D2.2, Geomorphological maps, Planmap deliverable, available online at D2.2-public.
Rossi, A. P., Penasa, L., Pozzobon, R., and the Planmap consortium (2019) D7.3-public, Data Management Plan, update 1, Planmap deliverable, available online at D7.3-public.
Rossi, A. P., Brandt, C. H., and the Planmap consortium (2019) D7.4-public, Public data/code dlivery, Planmap deliverable, available online at D7.4-public.
Zambon, F., Carli, C., and the Planmap consortium (2018) D4.1, Spectral Index and RGB maps I, Planmap deliverable, available online at D4.1-public.
Zambon, F., Carli, C., and the Planmap consortium (2019) D4.2, Spectral Index and RGB maps II, Planmap deliverable, available online at D4.2-public.