EU Framework Program for Research and Innovation (SC5-18a-2014 - H2020)
Project Nr: 641538
Coordinating an Observation Network of Networks EnCompassing saTellite and IN‐situ to fill the Gaps in European Observations
Deliverable D5.2
Report on stakeholders and industry challenges
Version 1.0
Due date of deliverable: 31/12/2016 Actual submission date: 02/02/2017
Document control page
Title D5.2 Report on stakeholders and industry challenges
Creator LM_ARMINES
Editors LM_ARMINES
Description
Report describing the stakeholders and industry challenges used during the project. It will also include a report about the ConnectinGEO Energy stakeholder Workshop held on month 12.
Publisher ConnectinGEO Consortium Contributors ConnectinGEO Partners
Type Text
Format MS-Word
Language EN-GB
Creation date 14/09/2016 Version number 1.0
Version date 02/02/2017 Last modified by LM_ARMINES
Rights Copyright © 2017, ConnectinGEO Consortium Dissemination
level CO (confidential, only for members of the consortium) X PU (public)
PP (restricted to other programme participants) RE (restricted to a group specified by the consortium) When restricted, access granted to:
Nature X R (report)
P (prototype) D (demonstrator) O (other)
Review status X Draft Where applicable:
WP leader accepted Accepted by the PTB
PMB quality controlled Accepted by the PTB as public document Coordinator accepted
Action requested X to be revised by all ConnectinGEO partners for approval of the WP leader
for approval of the PMB
for approval of the Project Coordinator for approval of the PTB
Requested deadline
Revision history
Version Date Modified by Comments
0.1 14-09-2016 LM_ARMINES
Created the basic structure of the deliverable, incorporate the content of the WP5 mid-term report section and add elements of the new period.
0.2 16-11-2016 LM_ARMINES Include inputs from EARSC and S&T 0.2 08-12-2017 SJ_52N Include inputs from 52N
0.3 26-01-2017 LM_ARMINES Include inputs from TIWAH 1.0 02-02-2017 LM_ARMINES Include inputs from CNR
Contributors
Acronym Full name
LM_AMRINES Lionel Menard (ARMINES) MML_EARSC Monica Miguel-Lago (EARSC) MV_S&T Marcella Veneziani (S&T) SJ_52N Simon Jirka (52° North) HPP Hans-Peter Plag (TIWAH) MS Mattia Santoro (CNR)
Copyright © 2017, ConnectinGEO Consortium
The ConnectinGEO Consortium grants third parties the right to use and distribute all or parts of this document, provided that the ConnectinGEO project and the document are properly referenced.
THIS DOCUMENT IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS DOCUMENT, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
4 Table of Contents
1 Introduction ... 6
2 Stakeholder and industry challenges ... 7
2.1 Task 5.1: Definition of the focus of the testing ... 7
2.1.1 Introduction ... 7
2.1.2 Analysis of the commonalities in the challenges ... 7
2.2 Task 5.2: The industry energy challenge ... 11
2.2.1 Introduction ... 11
2.2.2 Gaps in in-situ measurements: The platform providing access to in-situ measurements for the energy community... 12
2.2.2.1 Platform’s main features ... 12
2.2.2.2 Additional developments ... 16
2.2.2.3 Tools... 18
2.2.3 Gaps in data model ... 20
2.2.3.1 Metadata ... 20
2.2.3.2 Observation Data ... 22
2.2.3.3 Future Work ... 24
2.2.4 Gaps in collaboration ... 24
2.2.4.1 Energy stakeholder workshop ... 24
2.2.4.2 Targeting the private sector ... 26
2.2.4.3 Scientific dissemination ... 27
2.2.4.4 Link with the GEOSS Common Infrastructure (GCI) ... 28
2.2.4.5 Meta network ... 32
2.2.4.6 Relation with other projects’ tasks ... 33
2.3 Task 5.3: In-situ data compatible to satellite mission data challenge ... 34
2.3.1 In-situ data providers gap analysis ... 34
2.3.2 In-situ data networks harmonization: the NORS project ... 37
2.3.3 Data quality assurance: the QA4ECV project ... 39
2.3.4 Study of the organization and capabilities of major repositories for in-situ data and satellite data 41 2.3.4.1 Ground based Networks for Atmospheric Composition Data ... 41
2.3.4.1.1 NDACC ... 42
2.3.4.1.2 AERONET ... 45
2.3.4.1.3 ICOS ... 46
2.3.4.1.4 WMO – GAW ... 47
2.3.4.2 Oceanic observations ... 48
2.3.4.2.1 GHRSST ... 48
2.3.4.2.2 ARGO ... 48
2.3.4.2.3 GO-SHIP ... 49
2.3.4.3 TERRESTRIAL OBSERVATIONS ... 50
2.3.4.3.1 GCOS Surface Network ... 50
2.3.5 Conclusion ... 52
2.3.6 References ... 53
2.4 Task 5.4: In-situ network integration into the CGI challenge. ... 53
2.4.1 Webservice-Energy Sensor Observation Service ... 53
2.4.1.1 Discoverability ... 54
2.4.1.2 Accessibility ... 55
2.4.1.3 Status ... 55
2.4.2 Ocean Biogeographic Information System ... 56
2.4.2.1 Discoverability ... 56
2.4.2.2 Accessibility ... 57
2.4.2.3 Status ... 57
2.4.3 Global Biodiversity Information Facility ... 57
2.4.3.1 Discoverability ... 57
5
2.4.3.2 Accessibility ... 58
2.4.3.3 Status ... 58
2.4.4 Dynamic Ecological Information Management System ... 58
2.4.4.1 Discoverability ... 58
2.4.4.2 Accessibility ... 58
2.4.4.3 Status ... 58
2.5 Task 5.5: Remote Sensing Private sector challenge ... 59
2.5.1.1 EARSC, European Product Award ... 59
2.5.1.2 EO products taxonomy ... 61
2.6 Task 5.6: Interdisciplinary cooperation on the food-water-energy nexus (FWEN) ... 65
2.6.1 References ... 70
6
1 Introduction
The supporting actions and networking activities in this project must be tested and improved to ensure viability in real world scenarios. Therefore this work package contains several challenges related to industry where it is expected to identify gaps in the procedures and methods, and to demonstrate business opportunities.
The stakeholder and industry challenge WP5 has first conducted a work in order to define, shape and focus about the potential sets of goals that would be shared amongst WP5 subtasks. This has been covered in the task 5.1 and has been completed (D5.1 – “Common criteria in the project stakeholder and industry challenge” report).
The main outcome of the energy industry challenge (task 5.2) is a new platform for accessing, viewing and downloading in-situ measurement for energy practitioners. An additional outcome of this task is the “Energy data model” which has been completed by taking into account existing European projects sharing a similar activity such as NeXOS, FixO3, BRIDGES and ODIP II for a better global harmonization. It is available as “Schematron” file suited for the Energy SensorML profile.
Within the task 5.3 “In-situ data compatible to satellite mission data challenge”
a study has been conducted to assess the availability of in-situ data to complement the Copernicus satellites data acquisition. This workpackage was in close relationship with the task 3.3 (satellite in-situ relation).
During the task 5.4 “In-situ network integration into the GCI challenge” tests were conducted to integrate in-situ data systems into the GEOSS Common Infrastructure. Information about new in-situ data systems that were tested and/or added to the list of brokered systems of the GEO Discovery and Access Broker (DAB) are provided.
During the “Remote sensing private sector challenge” task 5.5 EARSC has organized a competition called "European EO product of the year". The competition has encouraged the use of open data from GEOSS (Global Earth Observation System of Systems) and aimed at rewarding a company which would developed the most innovative product integrating an element of open data ideally discoverable through the GEOSS broker services. In June 2016 the winner has been revealed.
The Task 5.6 “Project outcomes exemplary application challenge”. During the ENEON workshop WS3 the consortium agreed on the “Energy-Food-Water Nexus” as the interdisciplinary challenge to be addressed by ENEON. Tiwah has been included in the participant list for this task and has led the further development of this challenge.
7
2 Stakeholder and industry challenges
2.1 Task 5.1: Definition of the focus of the testing 2.1.1 Introduction
Task 5.1 aimed at better defining and planed the work to be carried out in the other tasks of this work package (2, 3, 4, 5 and 6). Work Package 5 has the objective of supporting the gap analysis by providing use cases that will use or produce GEOSS data.
This is the summary description of the WP 5 industrial challenges:
Task 2 (Challenge 1st) proposes a clear experiment based on Surface Solar Irradiance (SSI) measurements and the associated gaps.
ARMINES is leading this task.
Task 3 (Challenge 2nd) proposes an experiment that combines in-situ and satellite data. The text in the DoA frames the scope to SAR and atmospheric data for CalVal. Sentinel 1 had a role. S&T is leading this task.
Task 4 (Challenge 3rd) proposes to integrate more in-situ networks in the GEOSS DAB with the scope on CZEN and terrestrial ecosystems.
CNR is leading this task.
Task 5 (Challenge 4th) proposes a private sector challenge. It will consist in a competition. EARSC is leading this task.
Task 6 (Challenge 5th) proposes an interdisciplinary cooperation on the food-water-energy nexus (FWEN). Tiwah is leading this task.
2.1.2 Analysis of the commonalities in the challenges
The definition of the five industrial challenges was revisited and made more concrete taking the “Description of the action” material as a starting point. The tasks were defined in a formal way taking into account the following aspects:
Description
Theme (SBA)
EV used
Addressed gap
Types of Outcome
Data (and metadata) produced
Spatial and temporal extent
Open Research Data pilot collaboration
Standards that are going to be used
Mean of Dissemination
Connection to ENEON
Connection with GEO – GEOSS
Stakeholder Profile
8
Though the challenges were relatively heterogeneous, there were some commonalities and gaps that can be detected so we can get the knowledge of what can be expected and the limitations of the approach.
Relevant SBAs:
Many challenges will depend on climate and weather. Two challenges will focus on energy (1st and 5th) and two in land use and agriculture (2nd and 5th). Two of them will consider all the SBAs (3rd and 4th).
EVs used:
The 2nd and 5th will consider the emerging agriculture EVs and also will deal with energy new EVs. 5th will work on water EVs.
Addressed gaps:
Data access and data semantics harmonization are two common gaps addressed by several of the challenges. The first and last will also address lack of collaboration. It is unclear if gaps in Earth observations variables and techniques will be addressed. This should perhaps be the case in the 1st and 5th.
Types of Outcomes:
Foreseen outcomes are heterogeneous: in the 1st outcomes are SOS services and Metadata, in the 2nd harmonization in in-situ – RS, in the 3rd a better DAB, in the 4th the industry as a stakeholder and in the 5th gaps, remedies and costs for combining themes.
Data (and metadata) produced:
Data and metadata will be only produced in the 1st challenge.
Spatial and temporal extent:
Planned extend is Europe on the world.
Open Research Data pilot collaboration:
1st and 4rt will consider participation in Open Research Data pilot.
Standards that are going to be used:
Catalogue CSW, O&M and SOS will be the most used standards.
Mean of Dissemination:
Workshops in ConnectinGEO project framework or organized by others. Other means of dissemination are the DAB, the EARSC email lists, and scientific papers.
Connection to ENEON:
The connection with the ENEON network has not yet been recognized by the responsible of the tasks.
Connection with GEO – GEOSS:
9 Mostly by means of the DAB and the SDG.
Stakeholder Profile:
Stakeholder collectives depend on the challenge and are heterogeneous. This will ensure a good impact in different collectives.
The Figure 1 provides a summary of all tasks compared to this “Description of the actions” as it concentrates the different aspects reported by the challenges in a single table.
10
Theme (SBA) EV used Addressed gap Types of Outcome Data (and metadata) produced Spatial and temporal extent Open Research Data pilot collaboration Standards that are going to be used Mean of Dissemination Connection to ENEON Connection with GEO – GEOSS Stakeholder Profile
2.1 The industry energy challenge Energy Surface Solar Irradiance (SSI) Wind speed. Data models, in in-situ measurements, in collaboration SOS, Metadata SSI data and metadata World, years Yes in principle SOS, O&M, SensorML 19115 Stakeholders workshop AIP8 Scientific papers MACC, CAMS Energy SBA AIP8 SMEs in Renewable Energy Business Big Players in Energy/Electricity Researchers Weather & Climate Networks.
2.2 In-situ data compatible to satellite mission data challenge Climate, Land Crop growth or ice sheets, traces of gases Compatibility of in-situ and RS data access and data standards Gaps in in-situ - RS EO in-situ extension proposal World, long term N/A Workshops Other project connection EO service and data providers Addressed observation networks.
2.3 In-situ network integration into the GCI challenge All All Data access in GEOSS Better DAB N/A N/A N/A Catalogue, SOS, etc Integrated in the GEOSS DAB Workshop GEOSS-DAB Data providers GEOSS Web Portal, GEOSS users GEOSS Client Application developers.
2.4 Remote Sensing Private sector challenge All All Private involvement Research movesto industry Use ofGEOSSDATA core data Catalogue and data access EARSC email lists and communication Companies inthe ENEON GEOSS-DAB EO Service providers including RS operators value adding companies and GI companies.
2.5 Interdisciplinary cooperationonthe food-water-energy nexus Agriculture, Water, Energy New Agric and energy. Emerging water Lack of collaborations Gaps, remedies and costs Europe N/A N/A ENEON workshop Scientific papers Done inside ENEON in a work group The SDGs SDG. Food, energy agriculture networks
Figure 1: Summary of the definition of the five activities During this activity two challenges were also redefined:
Remote Sensing Private sector challenge: In this task EARSC will organized a competition called "European EO product of the year" which encourages the use of open data from GEOSS (Global Earth Observation System of Systems) in the private sector.
11
Project outcomes exemplary application challenge was renamed as
“Interdisciplinary cooperation on the food-water-energy nexus (FWEN)”.
This task will address how we can use ENEON and ConnectinGEO activities to specifically consider indicators related to food, water and energy security and the coordination of the necessary observations of the three domains.
The result of this task is the deliverables 5.1 report ”Common criteria in the project stakeholder and industry challenges” where a detailed description of all tasks regarding the “Description of the actions” framework can be found.
2.2 Task 5.2: The industry energy challenge 2.2.1 Introduction
Task 5.2 aimed at extending an existing GEOSS energy community portal to allow access to data (particularly in-situ) instead of merely metadata and to allow data sharing between companies and organizations. The target community is the renewable energy and more particularly the solar one. The main outcome of the task is a Web platform that has introduced an unprecedented level of collaboration enabling as well gaps detections in European EO networks.
Gaps detection according to the DOW included:
Gaps in in-situ measurements: Extend a catalogue platform with data management functions based on the new metadata profiles and the GEOSS recommendations for interoperability based on 52°North Sensor Web. The platform will allow for (i) visualize sensor locations on a map, (ii) visualize measurements as time series plots and in tabular form, (iii) display sensor metadata at different levels of detail, (iv) upload of data by data owners, and (v) download raw observation data for offline processing.
It will support company to prepare, pre-process and integrate their datasets.
Gaps in data models: Design standardized data and metadata models suitable for in-situ Surface Solar Irradiance (SSI) observations as profiles of OGC Sensor Web Enablement standards. Profiles for XML encodings are defined as Schematron rules. The data and metadata models must take into account spatiotemporal coverage, lineage, data quality, IPR, the existing terminology and units of measure.
Gaps in collaboration: We will present the platform to domain stakeholders at a stakeholder workshop. This demonstration will be exploited as the basis to concretely define the next steps for research in this area and seek funding opportunities to address gaps in data with an added focus on forecasts and projections. The use case will support Tasks 2.2, 3.2, 3.3, 4.4, 6.2 and 7.2. It will help to identify and validate energy related Essential Variables as well as critical observation to latter fill the GEOSS SEE IN Knowledge Base.
12
2.2.2 Gaps in in-situ measurements: The platform providing access to in-situ measurements for the energy community ARMINES has installed and deployed a unique platform providing access to in-situ measurements for the energy community. It is based on the Open Source 52°North SWE (Sensor Web Enablement) solution. It is extending the capabilities of an existing Spatial Data Infrastructure (SDI) that is registered long since into the GCI (GEOSS Common Infrastructure) namely the webservice-energy.org GEOSS Energy Community Portal. Currently, up to 8 data providers have shared their time-series of in-situ measurements on the platform for a total of 17 measuring stations, thus offering around 300 times series ranging from 1 year to up to 11 years of data. This continuously expending list of data providers currently includes:
Universities / Institutions (Uni. Geneva, Uni. Kishinev-Moldova, ENTPE (Ecole Nationale des Travaux Public de l’Etat), Egyptian Meteorological Authority, MINES ParisTech)
World Bank (Energy Sector Management Assistance Program)
IRENA Global Atlas initiative (targeting 39 countries and more than 50 institutes)
Private sector (Solaïs French SME)
2.2.2.1 Platform’s main features
Pyranometric data have been made available for access, display and download to practitioners in electricity generation by PV plants. In addition and following their requests, other variables have been added that are needed for decision-making. They include surface observations of wind speed and direction, air temperature, relative humidity and pressure. An innovation was brought in this pilot compared to similar platforms thanks to the use of OGC standards (SWE, SOS, SensorML) aiming at improving the consistency of all observations and other variables needed in the decision making process and consequently easing this process.
Additional variables were computed using published state-of-the-art models, namely the radiation at the top of atmosphere, the radiation that should be observed under a cloud-free sky, and the solar zenith and azimuth angles, and are associated to the observations. Each raw time-series measurement from data providers is quality checked internally before being made available on the Web platform for the energy community. For each station a metadata record is generated and stored and it is associated with each time-series that is displayed on the platform. The following figures illustrate the main features that are available on the platform: http://insitu.webservice-energy.org
13
Figure 2: Home page of the in-situ platform for station localization
Figure 3: Station's time-series offering
14
Figure 4: Display of time-series as a graph
Figure 5: One click access to metadata
15
Figure 6: Metadata information display for each selected station
Figure 7: Display of tabular values and download option
16
Figure 8: CSV file after download with consistent metadata header information
Not only this new platform provides a user-friendly access to such measurements through the Web client, it also allows direct machine-to- machine access thanks to the respect of recommendations on interoperability as promoted by GEOSS. Access is possible through:
OGC (Open Geospatial Consortium) SOS (Sensor Observation Service) standard operations:
o http://insitu.webservice-energy.org/52n-sos-
webapp/service?request=GetCapabilities&service=SOS
52°North Time-series API (Application Programming Interface):
o http://insitu.webservice-energy.org/timeseries-webapp/api/v1/
2.2.2.2 Additional developments
Based on stakeholders’ feedback at various live presentations of the platform, additional developments have been made enabling a direct access to the
“Show Other Stations” function. This new feature allows partners to share the existence and the detailed description of their own network’s stations without providing access to their in-situ time-series. This useful feature enables various profiles of practitioners to be presented on the same platform and consequently leveraging its awareness and potentially allow further collaborations. The current information associated to these stations includes the following key items (Example of the Garget station in the South-West of France):
Station: CARGET Latitude: 43.552096 Longitude: 1.191413
17
Observations: Solar radiation GTI (2 pyranometers) and ambient temperature (1 sensor)
Time step: 5 minutes Start time: 2014-07-01 End time: Still running
Website: http://www.solais.fr Contact: Solaïs: cvernay@solais.fr
Figure 9: "Show Other Stations" toggle button
18
Figure 10: "Show Other Stations" description display 2.2.2.3 Tools
A set of tools have been developed as well and included in the platform to support the solar energy community regarding access, view and download of in-situ measurements. Among these tools one can find:
Statistical representation of the current time-series
2D view for an easy long time-series visualization
Wind-rose display for wind speed and wind direction time series
19
Figure 11: Statistical tool
Figure 12: 2D view
20
Figure 13: Wind rose display 2.2.3 Gaps in data model
An important objective of this work was to design a standardized set of data and metadata models suitable for in-situ surface solar irradiance (SSI) observations as profiles of OGC Sensor Web Enablement standards. This comprises two aspects: on the one hand the metadata describing the underlying measurement processes as well as the observation data generated by these processes.
2.2.3.1 Metadata
The OGC SWE standard for providing metadata about observation processes is the OGC Sensor Model Language (SensorML)1. This data model offers a broad range of elements and options to describe any type of physical or non- physical observation process (even including models for predicting values).
However, this enormous flexibility to support many different types of observation processes leads to a very high degree of freedom how information is provided in which way. Thus, profiles are necessary to define a subset of SensorML that shall be used for a certain application.
A typical way to define such profiles is the use of the Schematron language2. This language allows to introduce additional restrictions that go beyond the
1 Botts, Mike and Alexandre Robin (2014). OGC Implementation Specification:
Sensor Model Language (SensorML) 2.0.0 (12-000). Wayland, MA, USA, Open Geospatial Consortium Inc.
2 http://www.iso.org/iso/catalogue_detail.htm?csnumber=40833
21
specification of XML schemas. For example, Schematron may introduce certain rules regarding the presence of XML elements (e.g. certain attribute and element values) and further content conditions that need to be fulfilled.
Thus, Schematron is perfectly suited to describe a profile of SensorML: While the SensorML standard defines the syntactic structure, Schematron can ensure that, for example, specific identifiers or characteristics of a sensor are provided. This use of Schematron (as well as the content of the profile) is aligned with further SWE profile developments in other domains (e.g. the currently ongoing definition of marine SWE profiles3 in projects such as NeXOS, BRIDGES, ODIP 2 and FixO3).
One important example of a Schematron-based SensorML profile is the SensorML profile for discovery4. This profile describes for version 1.0 of the SensorML standard which elements of SensorML shall be at least provided to enable the discovery of sensors and sensor systems. For ConnectinGEO this profile was used as a baseline. In addition, it was aligned to conform to basic principles that are also expected to be used for the marine SensorML profile activities.
As the SensorML profile for discovery was only for the SensorML 1.0 standard, a first step within ConnectinGEO was to upgrade this profile to the latest SensorML version (2.0). Based on this version, the additional metadata fields required for the energy domain were identified. In more detail, the following additional fields were added to the SensorML profile so that it was extended to a SensorML Profile for Energy:
Information about the integration period of aggregated measurement data
Link to the Environment Horizon Graph
The following listing illustrates how these additional fields were defined as part of a Schematron document:
<sch:pattern id="EnergyProfileValidation">
<!--
The integration period of the sensor must be provided.
-->
<sch:rule context="//sml:PhysicalSystem">
<sch:assert test="sml:parameters/sml:ParameterList/
sml:parameter[@name =
'integrationPeriod']/swe:Quantity">
Error: The integration period of the sensor is missing.
</sch:assert>
</sch:rule>
3 http://meetingorganizer.copernicus.org/EGU2016/EGU2016-14690.pdf
4 Jirka, Simon and Arne Bröring (2009). OGC Public Engineering Report: OGC OWS-6 SensorML Profile for Discovery 0.3.0 (09-033). Wayland, MA, USA, Open Geospatial Consortium Inc.
22
<!--
A link to the Environment Horizon Graph must be provided as documentation.
-->
<sch:rule context="//sml:PhysicalSystem">
<sch:assert test="sml:documentation/sml:DocumentList/
sml:document[@role =
'environmentHorizonGraph']/
gmd:CI_OnlineResource/gmd:linkage/gmd:URL">
Error: A link to the Environment Horizon Graph is missing.
</sch:assert>
</sch:rule>
</sch:pattern>
As many solar irradiance sensors can belong to larger networks, an additional field was introduced which enables to link from individual sensors to the network they belong to.
2.2.3.2 Observation Data
For exchanging the observation data delivered by in-situ surface solar irradiance sensors, it was possible to rely on the existing OGC Observations and Measurements standard5. This standard offers already all necessary elements for encoding solar irradiance observations. The following listing shows a typical example of an O&M encoded solar irradiance observation:
<?xml version="1.0" encoding="UTF-8"?>
<om:OM_Observation xmlns:om="http://www.opengis.net/om/2.0"
xmlns:gml="http://www.opengis.net/gml/3.2"
xmlns:sos="http://www.opengis.net/sos/2.0"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xsi="http://www.w3.org/2001/XMLSchema- instance"
gml:id="o_4098">
<om:type xlink:href="http://www.opengis.net/def/observationType/
OGC-OM/2.0/OM_SWEArrayObservation"/>
<om:phenomenonTime>
<gml:TimePeriod gml:id="phenomenonTime_4098">
<gml:beginPosition>
2015-09-14T05:25:00.000Z </gml:beginPosition>
<gml:endPosition>2015-09-14T06:10:00.000Z</gml:endPosition>
</gml:TimePeriod>
</om:phenomenonTime>
<om:resultTime>
<gml:TimeInstant gml:id="ti_B6FF992691809597FE805E27D690D880 F41819BB">
<gml:timePosition>
2015-09-14T05:25:00.000Z </gml:timePosition>
</gml:TimeInstant>
5 Cox, Simon (2011). OGC Implementation Specification: Observations and
Measurements (O&M) - XML Implementation 2.0 (10-025r1). Wayland, MA, USA, Open Geospatial Consortium Inc.
23
</om:resultTime>
<om:procedure xlink:href="SIGNES-Pyrano GSM 10.7 horizontal-QC1"/>
<om:observedProperty xlink:href="Global Horizontal Irradiance"/>
<om:featureOfInterest xlink:href="SIGNES" xlink:title="SIGNES"/>
<om:result xmlns:ns="http://www.opengis.net/swe/2.0"
xsi:type="ns:DataArrayPropertyType">
<ns:DataArray>
<ns:elementCount>
<ns:Count>
<ns:value>10</ns:value>
</ns:Count>
</ns:elementCount>
<ns:elementType name="Components">
<ns:DataRecord>
<ns:field name="phenomenonTime">
<ns:Time definition="http://www.opengis.net/def/
property/OGC/0/
PhenomenonTime">
<ns:uom xlink:href="http://www.opengis.net/
def/uom/ISO-8601/0/
Gregorian"/>
</ns:Time>
</ns:field>
<ns:field name="Global_Horizontal_Irradiance">
<ns:Quantity definition="Global Horizontal Irradiance">
<ns:uom code="W_m-2"/>
</ns:Quantity>
</ns:field>
</ns:DataRecord>
</ns:elementType>
<ns:encoding>
<ns:TextEncoding blockSeparator="@@"
decimalSeparator="."
tokenSeparator=","/>
</ns:encoding>
<ns:values>
2015-09-14T05:25:00.000Z,2.0@@
2015-09-14T05:30:00.000Z,3.0@@
2015-09-14T05:35:00.000Z,6.0@@
2015-09-14T05:40:00.000Z,16.0@@
2015-09-14T05:45:00.000Z,25.0@@
2015-09-14T05:50:00.000Z,43.0@@
2015-09-14T05:55:00.000Z,47.0@@
2015-09-14T06:00:00.000Z,73.0@@
2015-09-14T06:05:00.000Z,68.0@@
2015-09-14T06:10:00.000Z,58.0 </ns:values>
</ns:DataArray>
</om:result>
</om:OM_Observation>
It can be seen, how the georeferencing of each observation is ensured through the so called feature of interest which represents the measurement station. In addition, the sensors can be modelled as procedures while the parameter that was observed is provided as an observed property.
Furthermore, the time-value pairs of the result are provided as a data array:
24
the phenomenon time indicates the point in time for which the measured value applies and a Quantity field contains the observed values.
2.2.3.3 Future Work
The developed SensorML energy profile is a valuable step ahead to ensure the consistent provision of metadata for energy related observation data sets and the sensors that have gathered the data. For the future, there are several steps which should be addressed to further improve the defined profiles:
JSON bindings: Currently the usage of O&M and SensorML is focused on XML-based encodings. As a more lightweight alternative, the definition of similar profiles for JSON bindings of the O&M and SensorML standards would be useful.
The profile developments in other domains (especially the marine community) are currently ongoing. Depending on the progress achieved in these activities future updates of the SWE profiles for the energy domain would be advisable.
Provision of more comprehensive tools:
o Metadata editors: In the oceans community, there are several activities to develop SensorML editors for describing the properties of marine sensors (e.g. 52°North smle). It would be useful and increase the acceptance of the developed solutions, if such an editor would also be adjusted to the needs of the energy domain.
o Currently, data viewers such as 52°North Helgoland, offer only quite limited functionality for visualizing sensor metadata. Based on the SensorML profiles in development in different communities, there would be a significant benefit if the corresponding metadata from SensorML files would be seamlessly integrated into Sensor Web viewers.
2.2.4 Gaps in collaboration
The public release of the platform at month 12 (February 2016) has greatly support the start of discussions, feedbacks, questions and new requirements definition among the solar energy community. Since then, numerous promotional and awareness activities have been undertaken towards the largest possible scope of potential users and through the widest possible mean of dissemination activities.
2.2.4.1 Energy stakeholder workshop
The platform has been presented at a solar renewable energy stakeholder workshop through a dedicated session “Europe through the ConnectinGEO project funded under the Horizon 2020 program supports future developments for exchange and dissemination of in situ measurements - SOS, standards and interoperability - Discussion” (http://soda-pro.com/research/training- session-2016). This workshop gathered nineteen engineers from SMEs from Algeria, Belgium, France, Germany and Portugal, as well as twelve academics from the European Union, Egypt, Morocco, Oman and South
25
Korea. During the session a first round of outcomes of the presentation have been gathered. This include discussions that took place at the end of the session were several question were asked:
ConnectinGEO platform
o Usefulness of the existing functionalities?
o Do you need other functionalities?
o Benefit to your daily work?
o Other comments?
ConnectinGEO Quality check, model and export o Relevancy and usefulness in your business?
o Benefit to your daily work?
o Completeness?
ConnectinGEO Contribution of your data to the platform o Data access?
o Data usage policies
o Will you be interested to be a data provider?
o Will you allow to provide access to your data to everyone?
o Could we create an exchange hub possibly with the support of MINES ParisTech?
The following excerpt feedback has been gathered during the stakeholder workshop:
Several participants have already expressed in writing their agreement to provide data to the platform hosted by MINES ParisTech. MINES ParisTech in turn will provide reports on the quality of data. Different licenses of data use and IPR were discussed briefly. The companies AKUO and Martifer expressed interest and asked for more details on the data MINES ParisTech is interested in.
Practitioners see the interest of such a tool. Most of them are ready to use it. It is a valuable tool for them to discover data, to explore these data and eventually take decision about sitting and sizing a sun powered plant.
There was a suggestion made by AKUO backed by Sunpower of using this tool as a means to discover the stations that are measuring the irradiation. A map displaying the locations of the stations would be helpful and if not contained in the MINES ParisTech database, each station will be described not necessarily in details.
Suggestion made by Sunpower. What about providing a tool that estimates the amount of radiation contained in a specific spectral band with the measurements as input to this estimating algorithm?
The general opinion of session’s leaders can be summarized as follow:
The principle of the tool has been well received. There were no questions on the possible benefits but suggestions on improvements that demonstrate indirectly that the tool may be adopted. Several
26
suggestions made may take the tool a bit far from its original objectives.
Following the workshop the same set of questions has been sent by email to enable a more complete test of the platform. The first general trends of email feedback analysis after testing the platform is summarized as follow:
Platform:
o It is a great idea, and can be useful for a variety of actors. The main benefit to my work would be pyranometric data, downloadable in .csv format (as current BSRN data has different formats, sometimes hard to understand).
Quality Check (QC):
o It is a great method to convince institutions with ground stations to share their data, while saving them money and time potentially spent on instruments and algorithm developments.
o QC is crucial as data may be used for comparison. As a future time-series provider, we would use this service to better qualify our measurements.
o It is necessary to have the information of % of good data after QC.
o Vital. Need to have reference to QC methods and be able to flag QC’ed.
Data Contribution:
o I am all for sharing of data. We operate large PV plants but are not owner of the data. So we would need to validate with our customer case by case to provide you with our data.
o Solaïs a French SME PV operator has started to share its measurements coming from their 13 stations in operation.
The full version is available in the mid-term report.
A video providing a summary of this training is also available
Figure 14: Solar Training video (https://youtu.be/p-NCtwkGlZg) 2.2.4.2 Targeting the private sector
The platform has been promoted through the “Avenir Energy PME” program targeting French stakeholders in the field of renewable energies.
27
(http://www.oie.mines-paristech.fr/Recherche/Projets-de-recherche/Projets- de-recherche-acheves/#dissemination).
A one-day workshop Called “Rencontre Avenir Energie PME 2016” has taken place in Grenoble on 28th September. A promotion video of the platform has been realized with the support of the French SMEs Solaïs and THIRDSTEP involved in solar renewable energies exploitation.
Figure 15: Private sector video (https://youtu.be/WpC5C6qnYkg) 2.2.4.3 Scientific dissemination
The platform and been widely presented at scientific events as well as through several publications:
Ménard L, Nüst D, Jirka S, Masó J, Ranchin T, Wald L (2015) Open Surface Solar Irradiance Observations - A Challenge. Geophysical Research Abstracts Vol. 17, EGU2015-6607, 2015:
http://meetingorganizer.copernicus.org/EGU2015/EGU2015-6607.pdf.
European Geosciences Union General Assembly 2015 - EGU 2015, Vienna (Austria).
Ménard, Lionel, Daniel Nüst, Simon Jirka, Joan Masó, Thierry Ranchin, and Lucien Wald. Open Surface Solar Irradiance Observations - A Challenge. European Geosciences Union General Assembly 2015, Vienna, Austria, 12-17 April 2015, EGU2015-6607.
Ménard, L., D. Nüst, K. -M Ngo, P. Blanc, S. Jirka, J. Masó, T.
Ranchin, and L. Wald. 2015. "Interoperable Exchange of Surface Solar
Irradiance Observations: A Challenge".
doi:10.1016/j.egypro.2015.07.867.
Matthes Rieke, Raquel Casas, Oscar Garcia, Simon Jirka, Lionel Menard, Thierry Ranchin, Christoph Stasch and Lucien Wald - Sensor Web Standards for Interoperability between in-situ Earth Observation Networks - Geophysical Research Abstracts Vol. 18, EGU2016-12647, 2016: http://meetingorganizer.copernicus.org/EGU2016/EGU2016- 12647.pdf. EGU General Assembly 2016.
Menard Lionel, Blanc Philippe, Gschwind Benoit, Wald Lucien, A spatial data infrastructure dedicated to the interoperable exchange of meteorological measurements in renewable energies. 16th EMS
28
Annual Meeting, 12-16 September 2016, Trieste, Italy, EMS Annual Meeting Abstracts, 13, EMS2016-369.
GEO XII Ministerial - AIP-8 and Citizen GEO Sessions (Nov. 2015 Mexico).
IEA Task SHC 46: Solar Resource Assessment and Forecasting meeting Sophia Antipolis (France) 6-8 April 2016.
Rencontres Avenir Energie PME 2016 – MINATEC Grenoble (Sep.
2016)
ENEON Plenary Workshop – Building a collaborative ENEON to inform policies and actions to address complex societal challenge - Meta network a contribution to ENEON in solar energy (Oct. 2016 Laxenburg, Austria).
GEO XIII Plenary - In-situ EO networks and its relation to GEOSS (ENEON) and 1st Data Providers Workshop side events sessions.
2.2.4.4 Link with the GEOSS Common Infrastructure (GCI)
At a more technical level, the platform has been registered on the webservice- energy.org SDI catalog (http://geocatalog.webservice-energy.org). A dedicated ISO 19139 metadata record describing the platform and the available interoperable services is available on the catalog (http://geocatalog.webservice-
energy.org/geonetwork/srv/eng/metadata.show?id=3926&currTab=simple).
This enables to promote the GEO recommendations on interoperability.
Indeed the webservice-energy.org catalog is a registered resource of the GCI (GEOSS Common Infrastructure) and it is harvested weekly by the DAB (Discovery and Access Broker). Consequently any resources such as the in- situ platform metadata record that are available on the webservice-energy.org catalog are visible on the GEO Web Portal (GWP) for a larger dissemination and for the good of the solar energy community.
29
Figure 16: ISO 19139 metadata record describing the platform and the available interoperable services (1)
Figure 17: ISO 19139 metadata record describing the platform and the available interoperable services (2)
30
Figure 18: Same ISO 19139 metadata record as available in the DAB
31
Figure 19: Same raw ISO 19139 metadata record as available in the DAB (1)
Figure 20: Same raw ISO 19139 metadata record as available in the DAB (2)
32
Figure 21: Same ISO 19139 metadata record as available in the GWP
Figure 22: Same raw ISO 19139 metadata record as available in the GWP 2.2.4.5 Meta network
As presented at the mid-term review meeting in Brussels on April 8th 2016, new array of potential providers seem to emerge as we move forward presenting the benefit of the platform and its underlying concept of openness and interoperability to the community. This is the case for private companies in the field of PV electricity production. In the European Union hundreds of
33
private ground station are associated with PV plants. Such companies operate PV plants and daily operate in-situ measurement from various types of sensors (Direct, diffuse and global irradiation, wind speed and direction, temperature, humidity…) available on-site for the monitoring and forecast of the production of these plants. They consequently hold very precious time- series that could be further valorized. These in-situ measurements should complement those coming from existing meteorological networks such as WRDC (World radiation Data Center), BSRN (Baseline Surface radiation Network), GAW (Global Atmospheric Watch) for a valued added offer to solar energy stakeholders. The concept we came-up with is called “Meta-Network”
for a combination between well establish Meteorological Network an spread Micro Network. It could be roughly sketch as follow with interactions between such networks, the platform and the relation to gaps assessment and the ENEON effort within the ConnectinGEO project.
To enable such Meta Network further efforts should be carried out that are currently out of the ConnectinGEO objectives.
Figure 23: Meta Network concept diagram
2.2.4.6 Relation with other projects’ tasks
Regarding interactions between this work-package with others tasks of the project the development and the release of the platform as well as all the outreach activities have supported:
• The tasks 2.2 regarding essential variable definition for solar energy.
• The debate of the task 3.2 regarding to shape the ENEON.
• The task 3.3 about Copernicus satellite in-situ relation as the platform is currently used under the MACC/CAMS Copernicus program.
34
• The tasks 4.4 and 6.2.
• The task 7.2 “Exploitation” as illustrated by the numerous references above.
2.3 Task 5.3: In-situ data compatible to satellite mission data challenge
The goal of this study is to assess the compatibility and functional gaps among measured satellite data and in-situ data providers exploitable by the Copernicus program. The study will first analyze the main discrepancies among in-situ measurements on similar quantities performed by different facilities, highlighting the main difficulties for users to exploit those datasets. It will then review relevant projects for harmonizing in-situ repositories, assessing in-situ data quality and making results of the assessment accessible to the scientific community. In the last part of the study, we will review some in-situ data networks which are adopting clear data access policies and harmonization standards and are currently used for Cal/Val in many different space missions.
Conclusions are presented at the end of the document, together with comments on the accessibility and usability of in-situ networks, assessed during our research.
2.3.1 In-situ data providers gap analysis
In-situ is a general definition, which includes all the measurements not taken by a satellite. Ground stations, air-borne and sea-borne sensors, such as ship or buoy based observations, are an example of in-situ data.
In-situ data are indispensable to complement satellite data, as these data provide:
1. Calibration and validation for space-borne information, i.e. as an independent source for detecting trends in instrument performance/degradation.
2. Complement satellite data and fill any kind of gaps that might be present in space sources.
In-situ data are also useful as a primary source of data, because they provide more accurate information on regional scales than satellite data, even if on a less extended spatial scale. Therefore, they are commonly used for local scientific analysis and assimilated into forecasting models to provide high- resolution regional constraints.
The study of the main discrepancies and issues in in-situ data providers has been carried out through the analysis of the intermediate results and conclusions of the HORIZON 2020 GAIA-CLIM project. This section has been written for D3.4 [5] of ConnectinGEO and is also reported here for its relevance to this study.
35
GAIA-CLIM (Gap Analysis for Integrated Atmospheric ECV CLImate Monitoring, http://www.gaia-clim.eu/) established sound methods for the characterization of satellite-based Earth Observation data by surface-based and sub-orbital measurement platforms.
ECV (Essential Climate Variables) are defined in [1] as a list of variables for which data products are required to support climate monitoring, forecasting, research, service provision and policy and whose observation must be technically feasible and cost effective. A non-exhaustive list of ECV for Atmosphere, Land and Ocean is the following:
Atmosphere: Air temperature, wind speed and direction, water vapour, pressure, precipitation, surface radiation budget, Carbondioxide, methane, other long-lived green house gases, ozone and aerosols.
Land: River discharge, water use, groundwater, lakes, snow cover, glaciers and ice caps, ice sheets, permafrost, albedo, land cover (including vegetation type), fraction of absorbed photosynthetically active radiation, leaf area index, above-ground biomass, soil carbon, fire disturbance, soil moisture.
Sea/Ocean: Sea-surface temperature, sea-surface salinity, sea level, sea state, sea ice, surface current, ocean colour, carbon dioxide partial pressure, ocean acidity, phytoplankton, sub-surface temperature, sub- surface salinity, sub-surface current, nutrients, oxygen.
GAIA-CLIM identified, catalogued and studied a selected set of atmospheric ECVs.
In the intermediate results of the project, the assessment of the gaps is presented together with an estimate of their impact and a set of recommendations. In this study, we generalize the outcome of GAIA-CLIM in order to make it applicable to other domains. For more details about the GAIA-CLIM project itself we refer the reader to [2].
Six generic gap types can be identified and applied to any non-satellite data provider. They are:
Coverage: In order to use a network of in-situ repositories as Cal/Val provider to satellite measurements, it is crucial to have a wide distribution of locations as well as consistent long time series. Therefore, the most common gaps falling in this category are:
Geographical/Temporal Coverage: A comprehensive scientific approach assessing the gaps in the current observing capabilities of the system of systems does not exist. Assessments are commonly performed without a scientific basis or using an ad hoc (non-systematic) approach. Often this is done on the basis of the experience gained by the international experts in the frame of research projects.
Knowledge of uncertainties: Limited availability or poor information of uncertainty estimates affects several repositories and propagates to applications when the data are assimilated into models. Progress here is critical to have long time series of consistent data samples, insensitive to the method of measurement and geographically uniform. Moreover, in
