R@Loc 2016
R ESEARCH @L OCATE ’16
Proceedings of the 3
rdannual conference
Melbourne, Australia April 12–14, 2016
Alan Both, Matt Duckham, and Allison Kealy (Eds.)
Editors
Alan Both
School of Science RMIT University Melbourne, VIC 3000 Australia
Matt Duckham
School of Science RMIT University Melbourne, VIC 3000 Australia
Allison Kealy
Department of Infrastructure Engineering The University of Melbourne
Parkville, VIC 3010 Australia
Preface
Locate is the annual conference on spatial information in Australia and New Zealand. The event, running for its third year, is an initiative of the Surveying & Spatial Sciences Institute (SSSI), Spatial Industries Business Association (SIBA), and Geospatial Information & Tech- nology Association (GITA). Locate aims to unite all communities—research, industry, and government—working in the field of spatial information.
Research@Locate is the academic research stream of Locate. Also in its third year, this confer- ence series has been organized independently by the Australasian Spatial Information Educa- tion and Research Association (ASIERA), with the aim of being the premier academic meeting event in the Australasian region.
Research@Locate’16 is organized by the Research@Locate’16 committee, providing carefully selected presentations and peer-reviewed short papers showcasing the latest research advances from researchers at all stages of their career, and from across the breadth of fundamental and applied spatial sciences. Research@Locate makes available its papers in the form of an annual, open-access proceedings.
The presentations this year were held across two days, and organized around four themes (sessions): Build, Prosper, Protect, and Sustain.
Acknowledgements
Research@Locate would not have happened without the support of the institutions behind Locate; SIBA and SSSI. We also wish to thank our colleagues that served on the International Program Committee.
Alan Both, Local Organizing Chair Matt Duckham, Chair
Allison Kealy, Chair April 2016
Organization
Program Chairs
Matt Duckham, RMIT University Allison Kealy, University of Melbourne
Local Organizing Chair
Alan Both, RMIT University
Program Committee
Jagannath Aryal, University of Tasmania Suelynn Choy, RMIT University
Xiaoli Deng, University of Newcastle Amy Griffin, UNSW ADFA
Chris Rizos, UNSW
Pascal Sirguey, University of Otago
Paul C. Sutton, University of South Australia Bert Veenendaal, Curtin University
Stephan Winter, University of Melbourne
Conference Program
Copyright c2016 for the individual papers by the papers’ authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
Day 2 1
Session 1, Build 1
1 Dataset and Feature-Level Provenance Integration for Spatial Datasets Nicholas Car
7 End User Awareness Towards GNSS Positioning Performance and Testing Ridhwanuddin Tengku and Allison Kealy
13 Introducing a Framework for Automatically Differentiating Witness Accounts of Events from Social Media
Marie Truelove, Maria Vasardani, and Stephan Winter
19 Mining the Co-existence of POIs in OpenStreetMap for Faulty Entry Detection Alireza Kashian, Kai-Florian Richter, Abbas Rajabifard, and Yiqun Chen
Session 3, Protect 25
25 Exploring Kimberley Bushfires in Space and Time Ulanbek Turdukulov and Tristan Fazio
30 The Case Study of an Australian Crime Dataset Jessica Liebig and Asha Rao
Session 4, Sustain 36
36 Engaging Communities for Prioritising Natural Resource Management and Biodi- versity Conservation Actions
Robert Milne and Birgita Hansen
Day 3 41
Session 1, Build 41
41 Historic Urban Landscapes and Visualising Ballarat: Citizen Participation for Sus- tainable Urban Planning and Design
Angela Murphy, Peter Dahlhaus, and Helen Thompson
Session 2, Prosper 47
47 The Victorian Digital Cadastre: Challenges and Investigations Hamed Olfat, Davood Shojaei, and Mark Briffa
53 How VGI Intersects with Land Administration
Shima Rahmatizadeh, Mohsen Kalantari, Abbas Rajabifard, Serene Ho, and Ali Daneshpour
Session 3, Protect 59
59 Using GIS Techniques and Quantitative Morphometric Analysis to Evaluate the Groundwater Resources in the Central Flinders Ranges, South Australia
Alaa Ismail and Ian Clark
65 Diurnal and Seasonal Surface Temperature Variations: A Case Study in Baghdad Mustafa Naem, Robert Corner, and Ashraf Dewan
Dataset and Feature-Level Provenance Integration for Spatial Datasets
Nicholas J Car
Geoscience Australia, Symonston, ACT, Australia; Email: [email protected]
SUMMARY
Large, multi-agency projects such as the Foundational Spatial Data Framework are interested in capturing the provenance of their spatial datasets as they are processed and combined to form products. Additionally, work is underway at the CRC for Spatial Information and elsewhere to track the provenance of the production of individual elements (features) within spatial datasets.
How can we reconcile these provenance situations, given the different levels of granularity? Can we relate the provenance from lower-level systems to higher levels? Can we use common tools and methodologies? This paper and talk present provenance modelling work that has taken place at Geoscience Australia and CSIRO to solve these issues. The differing levels of granularity can be related however, for interoperability, a standard must be used and we’ve used PROV.
Keywords: spatial dataset, provenance, multi-granularity, spatial data infrastructure, transparency
INTRODUCTION
Transparency of process and some measure of reproducibility are requirements for information hoping to engender a high degree of trust in its users. A system-independent, international, standard known as PROV [1], now exists to generically represent the provenance of things (i.e. anything that was produced) and can be used to describe the production of national spatial datasets. The use of such standards ensures the interoperability of provenance description across systems and the longevity of the understanding of such descriptions.
A presentation at a previous Locate conference by this author [2] demonstrated the standardised provenance representation of a single map’s production, down to the ‘layers’ level using a formulation of PROV, PROV-O. More recent work by the Cooperative Research Centre for Spatial Information (CRC-SI) has represented individual geoprocessing toolkit actions undertaken to produce elements within spatial datasets using an extension to the PROV-O that they made, called GeoPROV [3].
Additionally, the Foundational Spatial Data Framework (FSDF) project1 intends to use PROV to represent the overall information flow from base data to FSDF data products.
Figure 1. A: The basic PROV-O classes and their relationships. B: A simple implementation of PROV-O describing the clipping of a raster image using ArcGIS2.
1 http://www.anzlic.gov.au/foundation_spatial_data_framework
2 https://esriaustralia.com.au/products-arcgis-software
Clip ArcGIS
Vector mask Raster
Clipped Raster Person X
Entity
Activity
Agent generated/used wasGeneratedBy
wasDerivedFrom wasAttributedTo actedOnBehalfOf wasAssociatedWith
Config
A. B.
used wasGenerateBy actedOnBehalfOf
wasAssociatedWith
These three bodies of work are all use PROV at different granularities and for slightly different purposes, however all three intend to enhance the transparency of the production of spatial products.
In this paper we will demonstrate how standardized provenance information recorded by different processes at different levels of granularity can be conceptually combined. Such combination is necessary in order to provide point-of-truth provenance information for data products.
USING THE PROV DATA MODEL
PROV-O provenance depiction
The PROV Data Model [1] consists of 3 main classes of concepts: Entities (things), Activities (events that act on Entities) and Agents (people or systems that trigger Activities). A diagram of these classes and their basic relationships is given in Figure 1A. An implementation of PROV-O for a simple geoprocessing task exhibiting a granularity similar to the examples in [2] is given in Figure 1B.
PROV-O representations of provenance are graph-based in structure. Graphs3 by their nature, unlike relational databases, contain their schema within the data [4]. This allows for infinitely detailed and infinitely large representations of systems’ provenance with the schema of the graph not limiting extensions of the information stored about items in it, or the links between items. Real limits on the information stored are only imposed by the ability of users to capture provenance information and for storage systems to physically cater for its management.
Additions to provenance graphs can be made by inserting new data into the graph, joining on appropriate prov:Activity4, prov:Entity or prov:Agent nodes. Since PROV-O uses a Resource Description Framework (RDF)5-based graph, each node’s identity is given as a URI6, thus one just needs to discover the URI for a node and graph additions can be made.
Figure 2. A: A high-level dataset provenance graph. B: Two datasets from A with intermediate datasets shown. C: A ‘black box’ Activity consuming 3 datasets and producing 1, D: The same datasets as C with the ‘black box’ broken down into two parts and an intermediate dataset shown.
PROV-O used at different levels of granularity
Detail insertion
If a system records the provenance of a dataset at a high level – perhaps just recording which datasets are a target dataset’s ancestors (see Figure 2A) – and this information is stored, additions to that can
3 https://en.wikipedia.org/wiki/Graph_(abstract_data_type)
4 PROV-O objects are denoted prov:{CLASS_NAME}, e.g. a PROV Agent is denoted prov:Agent
5 https://en.wikipedia.org/wiki/Resource_Description_Framework
6 https://en.wikipedia.org/wiki/Uniform_Resource_Identifier
Ancestor Dataset 3 Ancestor Dataset 4
Ancestor Dataset 2 Ancestor Dataset 1
Target Dataset
A. all links are prov:wasDerivedFrom
Dataset C Ancestor
Dataset 1
Dataset B
Dataset D
Dataset A Target Dataset
Target Dataset Activity
Ancestor Dataset B Ancestor Dataset C Ancestor Dataset A
B. all links are prov:wasDerivedFrom
Intermediate Dataset Sub
Activity Ancestor
Dataset B
Ancestor Dataset C Ancestor
Dataset A
Sub Activity
Target Dataset
C. D.
used
wasGeneratedBy
wasGeneratedBy wasGeneratedBy
used
used
be made later that fill in intermediate steps (see Figure 2B). Additionally, if a process records high- level provenance noting an activity that has taken place and that consumes (prov:used) and produces (prov:generated) datasets (see Figure 2C) which is then stored, that too can be added to later by recording activities at a finer granularity and any intermediate datasets (these don’t necessarily have to be persisted: their existence may only be represented) (Figure 2D).
As well as increasing the granularity of provenance graphs by filling in details, detailed provenance graphs can have their granularity decreased by querying. The SPARQL query protocol7 is for RDF- based graph databases what SQL is for relational databases. It is able to skip over nodes in provenance graphs by using path-based, transitive queries. This skipping of intermediate nodes allows one to, for example, discover the ultimate ancestor of a dataset, despite there being any number of intermediate ancestors. For the scenario shown in Figure 2B, a path-based SPARQL query can tell the user that
“Ancestor Dataset 1” is the ancestor of “Target Dataset”.
Dataset Subsetting
Representing dataset subsetting is important for linking provenance at different granularities as subsetting can be the tie-in points for systems’ reporting provenance at different scales.
There are a range of options regarding the recording of provenance for datasets that are subsets of other datasets. The PROV data model doesn’t directly prescribe how one should represent subsetting of datasets or how a part of a dataset is related to the larger whole: such instructions require far more detail than the generic PROV data model can deliver. One method of representing detailed dataset subsetting is shown in Figure 3A. As per that diagram, a dataset subset is created via a prov:Activity subsetting procedure with instructions as to how the sub-setting was undertaken recorded in a prov:Plan class object which is a specialised prov:Entity used to denote methodology. The prov:Plan object could hold computer code, detailed manual methodology or other instructions.
Another method for representing subsetting is shown in Figure 3B. In this formulation, instructions for performing the subsetting are not given with additional input data but are described by typing the subsetting prov:Activity. An example could be a prov:Activity of a hypothetical class such as
“TemporalExtentSubsetting” where the instances of such always subset the Large Dataset with some selection of a temporal extent. Sufficient metadata for the types subsetting activity, such as actual temporal extents, would need to be provided elsewhere (i.e. not in the provenance graph) in order to remove ambiguity from the action. One location for such metadata could be a register of typed activities maintained for use by a certain set of workflows. Figure 3C presents a combined formulation in which the typed prov:Activity demands that certain inputs to the subsetting action, in addition to the dataset from which a subset was taken, be represented in the provenance graph.
Figure 3. PROV-O Representations of subsetting actions. A: Using a prov:Plan object to hold subsetting instructions. B: By classifying the subsetting prov:Activity instance. C: Formulation combining A & B where required inputs are specified by the typed subsetting prov:Activity.
7 https://en.wikipedia.org/wiki/SPARQL
Dataset Subset Parent
Dataset Subsetting Instruction
Activity prov:Entity
prov:Plan
A
Class Key
Parent Dataset
Typed Subsetting
Activity Instance
Dataset Subset
B.
Classed Subsetting
Activity Instance Parent
Dataset Subsetting Instruction Additional Required input
Dataset Subset
C.
Dataset Merging & Splitting
Dataset merging and splitting can be modelled like dataset subsetting with either prov:Plan objects or typed prov:Activities, or a combination of the two, providing the instructions the action. It follows that the representations of dataset merging & splitting are akin to that of dataset subsetting shown in Figure 3 but with multiple input (merging) or multiple output (splitting) datasets.
REPRESENTING FEATURE AND DATASET PROVENANCE
Limited sets of typed actions for features
Where the provenance of features manipulated via a limited set of actions is to be represented, the representation shown in Figure 3A or B may be used and then aggregated to dataset-level provenance.
Figure 4 shows a representation of a hypothetical set of feature manipulation actions using the formulation given in Figure 3B: “selected”, “not-selected”, “merged”, “split” and the generic “alter”
typed prov:Activities are shown. These actions may have been carried out against features in one or more datasets and the results stored in a resultant dataset. They may be the result of specialized spatial tools, such as ArcGIS, certain actions of which are modelled using PROV-O in [3].
For a scenario in which features from one dataset (perhaps classes of vectors in a cadastral dataset) may be manipulated to form features in another dataset, such actions and their associated features may be represented as in Figure 4. Figure 4A shows feature-level manipulation and parts B, C & D dataset- level integration of feature-level provenance.
Figure 4. A: Feature manipulation actions as per Figure 3B. B: Aggregation of features manipulated into datasets with feature/action mappings preserved as prov:Activity inputs and outputs, C:
Aggregation of features manipulated into datasets with feature/feature mappings preserved as a prov:Activity, prov:Plan input and, D: Aggregation of features manipulated into datasets with feature/feature mappings preserved by annotating output features with links to actions performed and
features within the input dataset.
Identifier handling
The three feature-level provenance integration strategies presented in Figure 4B, C & D all rely on feature identification in order to link input and output features to their manipulation actions and each other. All three strategies are therefore dependent on either a mechanism for minting IDs for features that, although they are part of a dataset, are referenceable from outside that dataset or a feature register
Output feature Input
Feature Selected
Input Feature
Not‐
selected
Output feature Input
Feature X
Merged Input
Feature Y
Output Feature A
Output Feature B Split
Input Feature
A
Output feature Input
Feature Alter
Feature Manipulation Input
Dataset
Output Dataset
Feature‐action mapping
Action‐feature mapping
B.
Feature Manipulation Input
Dataset Output
Dataset Feature‐action‐
feature mapping
C.
Feature Manipulation Input
Dataset
Annotated Output Dataset
D.
that records feature identity independently from any particular dataset. The first case is implementable by URI patterns in accordance with Linked Data8 principles where the feature-level URIs are mapped to a higher level dataset-level URI via a relative, logical path. The second case requires a master feature register that can mint identifiers for features which can be referred to by any dataset containing them. Such a register may provide access to authoritative copies of their data, but this is not necessary.
In addition to the requirements listed above, the part B scenario also relies on the identification of, and storage of, the instance of each typed prov:Activity in order to preserve feature-level provenance since the feature linking is not directly coupled – it is in two parts: input feature(s) action then action output feature(s). The part C scenario conceptualizes the input and output feature mapping as a prov:Plan object for such a mapping if it contains feature-to-action-to-feature mappings that act as the entire instructions for the “Feature Manipulation” prov:Activity.
The part D scenario annotates each feature in the output dataset with the identity of its relevant manipulation actions instance as well as the input features manipulated. Such a formulation is also dependent on the identification and storage of the instance of each typed prov:Activity, as per part B, but it also has a shortcoming not present in parts B & C: actions that result in no output feature, such as feature non-selection, will not be identifiable in the annotated output dataset.
FSDF DATASET PRODUCTION CASE STUDY
Detail insertion, dataset subsetting, aggregating and splitting actions, as described two sections above, can easily be used in specific spatial data scenarios. Feature-level action recording and feature/action mapping as outlined in the section above can be applied to spatial datasets if the feature manipulation systems are able to record it and if the dependencies, also outlined above, are met.
Figure 5 shows the processing of two hypothetical FSDF source datasets (A & B) into an FSDF product. Part A shows simple dataset-level provenance, part B shows dataset-level provenance but with more details PROV-O formulation, as per Figure 3A. 5C implements many of the techniques described above, specifically:
The whole of 5C shows detail insertion (Figure 2D);
The path from Source Dataset A to Intermediate X shows detail addition (3A) and either 4B or 4C formulation, depending on whether feature-action + action-feature mapping (4B) or feature- action-feature mapping (4C) is used;
The Intermediate X to Intermediate Y path shows typed prov:Activity formulation (3B) and could use annotated output dataset (4D) mapping;
Intermediate Y plus Source Dataset B fusing to form the FSDF product could be a 3C-type exercise where the types prov:Activity, “Merging” specifies two input datasets and an feature mapping prov:Plan which preserves feature origin knowledge. This formulation is also a feature- action-feature mapping (4C).
Figure 5. A hypothetical FSDF product generation scenario modelled with different amounts of detail and at different levels of granularity.
8 http://www.w3.org/TR/ld-bp/
Source Dataset A
Source Dataset B
FSDF Product
A.
Production
Source Dataset A
Source Dataset B
FSDF Product
B.
Merging Feature
Manipulation (Selection)
Feature Manipulation
Type M Selection
Criteria
Ancestor/Descen dent mapping
Inter‐
mediate X
Inter‐
mediate Y Source
Dataset A Source Dataset B
FSDF Product wasDerivedFrom
Plan
C.
PROVENANCE DATA MANAGEMENT
It’s also not possible to write in generalities about provenance data collection or generation – in-depth knowledge of specific systems is required in order to make sensible descriptions – and collecting provenance data in standardised formats is far harder than managing and storing it [5, see Discussion].
Once collected however, there are a range of generic tools available to manage and manipulate it. The PROMS family of tools and their associated methodology [6]9 allow any number of systems to report PROV-O-based provenance information and have it stored in a graph database. The system will automatically join provenance graphs where the same node URIs are used, thus detail insertion, as per Figure 2, can easily be achieved. Similarly, the joining of small provenance graphs into larger super- graphs can be achieved which allows independent systems to assemble continuous graphs across their individual processes, as long as they can share dataset or feature identifiers in order to report against them. Most RDF-based graph database allow querying via SPARQL thus the abstraction of detailed graphs into simpler ones can take place when detail insertion or multi-process reporting has taken place. Installations of PROMS Server make the SPARQL endpoint of its underlying RDF graph database available for such use thus allowing fine to coarse granularity translation out of the box.
CONCLUSIONS
We have presented a range of PROV-O-based modelling formulations (ontology design patterns) to help provenance data managers meld provenance information at varying levels of granularity. We focused on dataset and feature level provenance, as these are the two obvious granularities for spatial data products, but the principles could apply to information at other granularities. We have presented alternative methods for the integration of provenance information of different granularities and pointed out some of the logical and system dependencies that certain patterns require. We have given a very brief FSDF case study implementing many of the techniques and also finally described several aspects of provenance data management referencing a particular tool.
ACKNOWLEDGEMENTS
This paper is published with the permission of the CEO, Geoscience Australia.
REFERENCES
[1] Moreau, L. & Missier, P. (eds.) PROV-DM: The PROV Data Model. W3C Recommendation 30 April 2013 W3C (2013). Online at http://www.w3.org/TR/prov-dm/. Accessed 2015-12-08.
[2] Car, N.J. Map data lineage: provenance concepts, tools and future shared infrastructure.
Locate2015 Conference presentation (2015).
[3] Sadiq, M.A., West, G., Arnold, L., McMeekin, D.A. and Moncrieff, S. Spatial data supply chain provenance modelling for next generation spatial infrastructures using semantic web technologies.
MODSIM2015, Gold Coast, Australia, 29th Nov – 4th Dec, 2015. (2015) Online at http://mssanz.org.au/modsim2015/.
[4] Robinson, I., Webber, J. & Eifrem, E. (2013) Graph Databases. O’Reilly Media. ISBN 978-1- 4493-5626-2. Online at http://graphdatabases.com. Accessed 2015-12-11.
[5] C. Wise, N. J. Car, R. Fraser and G. Squire. Standard Provenance Reporting and Scientific Software Management in Virtual Laboratories. MODSIM2015, Gold Coast, Australia, 29th Nov – 4th Dec, 2015. (2015) Online at http://mssanz.org.au/modsim2015/.
[6] Nicholas J Car, Matt Stenson, Mick Hartcher, Simon Cox, Peter Fitch, and David Lemon. A provenance management methodology and example architecture for science projects containing heterogeneous automated and manual processes. In HIC 2014 – 11th International Conference on Hydroinformatics, page 8, New York, USA, 2014. International Water Association. URL http://academicworks.cuny.edu/cc_conf_hic/57/. Accessed 2015-12-11.
9 See http://promsns.org for up-to-date information on the PROMS family of provenance tools
End User Awareness Towards GNSS Positioning Performance and Testing
Ridhwanuddin Tengku and Assoc. Prof. Allison Kealy
Department of Infrastructure Engineering, University of Melbourne, VIC, Australia;
Emails: [email protected] (R.T.); [email protected] (A.K.)
SUMMARY
The accessibility of positioning information derived from Global Navigation Satellite System (GNSS) is arguably one of the main drivers that has transformed the way spatial data is currently generated and consumed. As the form factor and power consumption of GNSS enabled devices starts to scale down, the technology will further become pervasive and ubiquitous. This is applicable to both safety and convenience related applications. Depending on what receiver is used, the correctness and reliability of the positioning information may greatly vary and end users may not necessarily be aware of the capabilities and limitations of the receiver being used. In this respect, the blind reliance of end users toward this technology has raised concerns within the GNSS community. As a part of a broader study on end user needs for GNSS testing, a survey was conducted to investigate the level of awareness amongst end users towards GNSS performance. This paper elaborates on the results of the survey.
The findings of this study, together with this needs analysis are used as a basis for principles and recommendations of end user GNSS testing standards and certification guidelines in the context of Australia.
Keywords: GNSS Questionnaire, GNSS Testing, GNSS Receiver Performance
INTRODUCTION
Global Navigation Satellite System (GNSS) derived coordinates are used in most spatial related applications. From data collection and analysis to real time position tracking, users rely on correctness and reliability of locational data to be presented within the desired tolerances. As locational data become increasingly ubiquitous, it is often difficult to diagnose the weakness of an operational system.
However, users can make a conscious decision in choosing and maintaining a suitable GNSS receiver which complies with the required positioning performance. Positioning performance is measured according to a system’s accuracy, integrity, continuity, availability, interoperability and timeliness [1,2].
The term positioning performance is perceived differently according to application, and research is being conducted to examine the needs and principles for end user GNSS testing. The scope of end users in this paper is taken from a broad perspective, and is not limited to handheld device users, but also GNSS derived data and non-manufacturer aligned GNSS data correctional service providers.
This paper presents the outcomes from an online survey that was conducted as part of this investigation. The focus of analysis is centred on the level of awareness of end users towards the technology and in-depth discussion on the overall results will be presented in a separate publication.
This paper will elaborate on the need to educate and inform end users on standardised receiver testing guidelines and certification.
GNSS USER QUESTIONNAIRE
An online questionnaire, focused on GNSS user testing requirements, sampled 70 individuals representing different GNSS dependent sectors: construction and surveying (CS) with 25 respondents;
land management (LM) with 14 respondents; aviation (AV) with 2 respondents, maritime (MR) with 4 respondents; agriculture (AG) with 2 respondents; road and rail transportation (RRT) with 3 respondents; location based services (LBS) with 9 respondents; and academic research (AR) with 11
respondents. The plain language statement of the questionnaire can be accessed at [3]. The volunteer respondents were either approached individually or received an email via sector specific mailing lists and forums. Australian participants totalled 81% of the respondents, with overseas respondents from Canada, India, Pakistan, Sweden, Azerbaijan, South Korea, United Kingdom, Japan and New Zealand comprising the remainder. Filter questions were used to identify the perspective and nature of the operations conducted by the respondents. Qualitative data fields were analysed to gauge opinion.
Questions and Results
Participants were presented with direct or classification type questions. The questions chosen for the paper focused on the aspects of GNSS confidence, awareness of receiver weakness, and the need for device testing. Each question is presented below, with a corresponding explanation of the given answers.
How confident are you with the positioning information given by a GNSS receiver?
Figure 1. Response of confidence in GNSS receivers.
GNSS receivers have become commonplace for many applications and this question is posed to investigate the level of trust users have towards the coordinates being presented. Figure 1 demonstrates that users are confident with positioning information of a GNSS receiver, with 82%
indicating confidence: very confident (13%) and confident (69%). Out of these 82% responses, 21 out of 25 are from construction and surveying, 11 of out of 14 are from land management, 7 out of 9 are from location based services, 7 out of 11 are from academic research, and all respondents from agriculture, road and rail transport, maritime and aviation sectors. This reflects the maturity of GNSS technology and its wide use in many day to-day-operations.
Classify the impact of the following GNSS receiver weaknesses on your operations:
Occasionally erroneous coordinates or unrealistic coordinate quality indicators.
For the subsequent questions, participants were asked to identify the impact of the GNSS receiver weakness on their operations. A scale of 1 to 5 was presented, with 1 indicating low impact and 5 indicating high impact. Participants were also allowed to choose ‘Not Applicable’ as a response.
Figure 2. Response of impact on unrealistic quality indicators
Response Number of
Response
Percentage (%)
Participant Breakdown (Count)
No Response 4 6 CS (1), LM (1), LBS (1), AR (1).
No Confidence 0 0 -
Low Confidence 1 1 CS(1)
Neutral 8 11 CS (2), LM (2), LBS (1), AR (3).
Confident 48 69 CS (18), LM (9), LBS (7), AR (7),
AG (2), MR (3), AV (1), RRT (1).
Very Confident 9 13 CS (3), LM (2), MR (1), AV (1),
RRT (2).
Response Number of Response
Percentage (%)
Participant Breakdown (Count)
Not Applicable 5 7 CS (1), LM (2), AR (2).
(Low) 1 15 21 CS (6), LM (3), LBS (1), AR (1),
MR (2), RRT (2).
2 11 16 CS (5), LM (3), LBS (2), AR (1).
3 16 23 CS (5), LM (2), LBS (4), AR (3),
AG (1), RRT (1).
4 14 20 CS (4), LM (2), LBS (1), AR (4),
MR (1), AV (2).
(High) 5 9 13 CS (4), LM (2), LBS (1), AG (1),
MR (1).
Coordinate quality indicators of the receiver are vital in providing the usability of the coordinates for any location. The results in Figure 2 indicate varied opinion on the impact of those coordinate quality indicators. A total of 33% of responses provided high impact on quality indicators, 37% provided low impact and 23% of respondents provided a neutral response. A possible explanation for 60% of respondents providing a neutral or below response is that users may be unable to quantify the impact.
This 60% constitutes 16 out of 25 from construction and surveying, 8 out of 14 from land management, 7 out of 9 from location based services, 5 out of 11 from academic research, 1 of out 2 from agriculture, 2 out of 4 from maritime, and 3 out of 3 from rail and road transport. These results also indicate that GNSS receivers are mostly reliable for its intended purposes and unrealistic quality indicators are uncommon, even in sectors with trained users such as construction and surveying.
Classify the impact of the following GNSS receiver weaknesses on your operations:
Unpredictable receiver behaviour under non-optimal signal environments
Figure 3. Response of impact on receiver unpredictability.
This question is closely related to the previous as receivers have a tendency to behave erratically under difficult GNSS environments. Shown in Figure 3, 65% of the responses were 3 and above, with the remaining 34% being 2 and below.
The response indicates that users are aware of the direct relationship between a GNSS receiver’s operational environment and its impact on receiver weaknesses. With 38% of respondents indicating high impact, standards which address unpredictable receiver behaviour under non-optimal signal environments should be adopted. This 38% includes 9 out of 25 from construction and surveying, 9 out of 14 from land management, 2 out of 9 from location based services, 6 out of 11 from academic research, 1 of out 2 from agriculture, 2 out of 4 from maritime, 1 out of 2 from aviation and 1 out of 3 from rail and road transport. In this respect, the transportation and land management sectors appear to be more aware and affected by such environmental limitations.
Classify the impact of the following GNSS receiver weaknesses on your operations:
Older hardware models do not perform as well as newer models
Figure 4. Response of impact on older over new hardware on performance.
Response Number of Responses
Percentage (%)
Participant Breakdown (Count)
Not Applicable 5 7 LM (1), LBS (2), AR (1), AG (1).
(Low) 1 11 16 CS (4), LM (2), AR (2), MR (1),
AV (1), RRT (1).
2 8 11 CS (4), LM (2), LBS (1), RRT (1).
3 19 27 CS (8), LM (4), LBS (4), AR (2),
MR (1).
4 24 34 CS (9), LM (3), LBS (2), AR (6),
AG (1), MR (1), AV (1), RRT (1).
(High) 5 3 4 LM (2), MR (1).
Response Number of Responses
Percentage (%)
Participant Breakdown (Count)
Not Applicable 14 20 CS (3), LM (4), LBS (4), AR (2),
AG (1).
(Low) 1 18 26 CS (8), LM (2), LBS (1), AR (2),
AG (1), MR (1), AV (1), RRT (2).
2 12 17 CS (4), LM (4), LBS (1), AR (2),
MR (1).
3 13 19 CS (4), LM (2), LBS (2), AR (3),
MR (2).
4 11 16 CS (5), LM (2), LBS (1), AR (2),
AV (1).
(High) 5 2 3 CS (1), RRT (1).
As shown in Figure 4, users’ perception of the physical receiver indicates that the age of the hardware is not indicative of its performance. With half the respondents acknowledging that older models work as well as newer models, testing for this variable may be unnecessary for the user. It is important to note that 20% of respondents selected ‘Not Applicable’, implying that a significant portion of users have not used older hardware, given GNSS’ relative newness. Indirectly, the results may signify difficulty for users to compare hardware performance and positioning performance. This is also reflected in the wide distribution of responses throughout different sectors, particularly from the construction and surveying, and land management sectors which constitutes the majority of participants.
Apart from testing coordinates against a known position, are there any other GNSS related tests routinely conducted?
Figure 5. Response of respondents conducting routine GNSS related tests.
Having considered the limitations of the receivers, users were then asked to identify whether any GNSS related tests were conducted to validate positioning performance. The results in Figure 5 show that 69% of users conduct no further testing, which validates the users’ perception of confidence in positioning performance. Qualitative investigation users who conduct further testing are mostly focused on data quality control and radio frequency interference detection. However, this is limited to more specialised users.
Would independent tests directly benefit the organisation?
Figure 6. Response of respondents benefiting from independent tests.
Whilst the previous response indicates that further testing is limited within the user base, the results from this final question shown in Figure 6, signify that users perceive the usefulness for further receiver testing. With 57% of respondents indicating that further testing would benefit their organisation and 69% having responded previously that no further testing was being conducted, the results indicate that guidelines for testing may not be sufficient. This is evident within the construction and surveying and land management sectors where data traceability is of importance.
The 43% of respondents who answered ‘No’ were asked to provide a reason why they saw no benefit. The qualitative analysis revealed that most specified that manufacturer tests are sufficient and coupled with the cheap price of the receiver, the cost of testing was unjustified. Users believe there are more critical aspects to the system that need to be examined. This is particularly true within the location based services sector where GNSS positioning is perceived as a secondary function.
DISCUSSION AND LIMITATION OF STUDY
Currently, users’ confidence in GNSS receivers is derived from the manufacturer’s testing. Whilst users indicate that manufacturer tests are mostly sufficient, there is evidence that users recognise the benefits of independent testing and certification for more specialised applications; particularly those
Response Number of Responses
Percentage (%)
Participant Breakdown (Count)
Yes 22 31 CS (9), LM (4), LBS (1), AR (2),
AG (1), MR (2), AV (1), RRT (2).
No 48 69 CS (16), LM (10), LBS (8), AR (9),
AG (1), MR (2), AV (1), RRT (1).
Response Number of Responses
Percentage (%)
Participant Breakdown (Count)
Yes 40 57 CS (17), LM (8), LBS (3), AR (6),
MR (1), AV (2), RRT (3).
No 30 43 CS (8), LM (6), LBS (6), AR (5),
AG (2), MR (3).
involving navigation and safety-of-life purposes. In addition, typical operational end user environments could vastly differ from manufacturer testing environments, thus justifying independent end user testing. In this respect, the aviation [4] and maritime [5] industry requires all GNSS receivers used for commercial-based navigation to be independently certified. The research aims to introduce such guidelines and standards beyond this scope.
Despite being aware that performance is limited by environmental factors, respondents find it difficult to gauge the performance of one receiver model to another. Users would be aided by standardised testing procedures and principles underlying an independent test bed, allowing individual receivers to be reliably validated and certified. This issue will be addressed in future papers.
One of the main challenges of the survey was to populate a large sample of respondents. This is due to the relative high level of knowledge required to understand the principles of this technology and it appears that many end users may be intimidated by the field specific questions. The researchers also acknowledge that it was difficult to approach participants within all transportation and agriculture sectors. Due to the small number of responses and similarity of its applications, the analysis from these sectors were treated as one generalised sector.
Although a small overall population sample of responses were gathered, the significance of the results are justified from the quality of responses given and the moderate level of expertise of chosen participants. The filter questions aided the process to exclude erroneous and irrelevant entries. From this filtering process, participants who use GNSS for more generic and mobile phone-based applications were categorised as location based services users.
Despite having this questionnaire widely distributed, the small number of respondents gathered also validated that the level of GNSS ubiquity has made many end users disconnected from the expectations of a GNSS receiver. Consequently, specialised end users such as system developers, service providers and system integrators bear the onus to ensure the receivers are operational to their minimum standards.
Only a small portion of the original questionnaire results were included in this paper so as to highlight the level of awareness of end users towards performance and testing. Responses from GNSS equipment providers and manufacturers were also excluded due to the end user focus of this paper. A separate publication will be written to explain the detailed results along with the qualitative study conducted.
CONCLUSION
Overall results of the questionnaire is discussed, and it is shown that most GNSS users are confident with its positioning performance. However, there is a wide distribution of user knowledge and expectation on positioning performance under difficult operational environments. From the results, it appears that users are not overly concerned about comparing relative performance between receivers.
The outcome of this investigation is used to validate the criteria needed to establish an independent end user test and certification, particularly in the context of Australia. The varying levels of responses suggest that a hierarchy of tests need to be defined, depending on the level of complexity required.
With such guidelines on standards being established, the test bed envisages users to reliably test their GNSS receivers, and in turn provide more confidence and traceability on the receivers being used.
ACKNOWLEDGMENTS
This research is funded by the Cooperative Research Centre for Spatial Information and its industry partners; ThinkSpatial, Geoscience Australia, and DELWP Victoria. The first author thanks the supervisors; Allison Kealy, Simon Fuller, Mark Moreland and Phil Collier in providing insight and guidance with the research. A special mention is given to Victoria Petrevski for the many hours spent to proof read the questionnaire documents and papers. Finally, to all survey participants: the outcomes of the study would not have been possible without your contribution.
REFERENCES
[1] AUSROADS Evaluation of the Potential Safety Benefits of Collision Avoidance Technologies Through Vehicle to Vehicle Dedicated Short Range Communications in Australia, Austroads Research Report, 2011.
[2] ACIL ALLEN CONSULTING The Value of Augmented GNSS in Australia: An Overview of The Economic and Social Benefits of the Use of Augmented GNSS Services in Australia, Prepared for the Department of Industry, Innovation, Climate Change, Science, Research and Tertiary Education, 2013, 12.
[3]TENGKU, R. Initiating the Development of a Test Facility for Global Navigation Satellite Systems (GNSS) Positioning Validation and Certification: Plain Language Statement, 2014. Available at https://www.dropbox.com/s/408bnw740yo08uc/PLS%20Online%20Survey.pdf?dl=0
[4] ICAO AERONAUTICAL COMMUNICATIONS, Annex 10 to the Convention on International Civil Aviation, International Standards and Recommended Practices, Radio Navigation Aids, International Civil Aviation Authority (ICAO), 2008
[5] IEC, IEC 61108:2003 Maritime Navigation And Radiocommunication Equipment And Systems - Global Navigation Satellite Systems (GNSS) Geneva, Switzerland: International Electrical Commisson (IEC), 2003.
Introducing a Framework for Automatically Differentiating Witness Accounts of Events from Social Media
Marie Truelove, Maria Vasardani, and Stephan Winter
Department of Infrastructure Engineering, The University of Melbourne, Australia;
Emails: [email protected] (M.T.); [email protected] (M.V.); [email protected] (S.W.)
SUMMARY
Identifying Witnesses of events from social media is an opportunity to crowdsource real-time information to enhance numerous applications including emergency response in a crisis, filtering sources for journalism, and enhancing marketing services. Using a sporting event broadcast live to a proportionally much larger audience, this research demonstrates a significant increase in the number of Witnesses identified posting from the event venue, in comparison to the number identified from geotags alone. This is achieved by considering the text and image content of micro-blogs as additional evidence. This paper also reports progress towards the automatic categorisation of the additional text and image evidence, and modelling and testing this evidence for corroboration or conflict, using Dempster-Shafter Theory of Evidence.
Keywords: Crowdsourcing, Social Media, Witness Accounts, Supervised Machine Learning, Dempster-Shafer Theory of Evidence
INTRODUCTION
Crowdsourcing information about events from social networks such as Twitter is recognised as an opportunity to harvest detailed real-time information, for example enhancing situational awareness for emergency response and management [18] and creating news summaries of large sporting spectacles [19]. However, these opportunities come with many problems to solve, including detecting the fraction of relevant micro-blogs, and assessing the credibility and location of the micro-bloggers who posted them. This research makes unique contributions by proposing a framework towards distinguishing those micro-blogs which are Witness Accounts (WA) of events. WA are defined as those micro-blogs which contain an observation of the event or its effects [17], for example a statementI see the bushfire smoke!
or an image conveying the same information. The micro-blogger who posted the WA is considered a potential Witness to the event, and it can be inferred they are on-the-ground (OTG) [15], that is they in close proximity to the event [17]. Impact Accounts (IA) are defined for those micro-blogs which do no contain an observation of the event, but from which it can also be inferred that the micro-blogger who posted it is OTG. IA statements may be as explicit asI’m being evacuated from my home due to the bushfire. Formally modelling the witnessing fundamentals of observation and spatial relationship separately enables a generic model for a range of event types including unpredicted natural disasters to scheduled events broadcast live from dedicated venues, such as the case study presented in this paper.
All micro-bloggers who post observations of the event whether viewed direct from the grandstands or via television are by definition Witnesses. The research in this paper questions whether it is possible to differentiate those Witnesses which are physically at the event from those watching a broadcast. Such differentiation is supported by micro-blogs with geotags, but typically they are present in only a fraction of micro-blogs, for example 1% [1]. This research demonstrates that including the text content and linked images as evidence, the sample of micro-blogs posted from the event location can be increased significantly from those identified by geotags alone. Additionally, this research questions whether text content and linked images can be automatically categorised, and used to test whether they corroborate
the inference they were posted from the event.
In order to automatically differentiate those micro-blogs which are WA or IA, and test the Witness categorisation of the micro-bloggers who posted them, a framework is proposed with the following parts:
1. Machine learning approaches to categorise micro-blogs with text and linked images that are likely WA or IA;
2. Combine the evidence extracted for each individual micro-blog to determine those which can be ranked as containing corroborating or conflicting evidence;
3. For each micro-blogger found to have posted micro-blogs containing evidence, combine these to rank their likely status as a Witness OTG; and
4. For likely Witnesses, seek further evidence, for example from micro-blogging history posted during the event.
This paper presents progress to date on parts 1) and 2). To demonstrate part 1) supervised machine learning approaches are used to categorise the text and image content. A model of the micro-blog text, linked images and geotags using Dempster-Shafer Theory of Evidence [3] is developed to demonstrate part 2). The results indicate a significant improvement on the recognition rate of micro-blogs posted from an event from geotags alone. And where multiple evidence is present for an individual micro-blog their combination does produce intuitive results, including identifying conflict due to GPS error. Enhancements and alternative approaches to those presented in this paper, as with parts 3) and 4) of the framework is the subject of future work.
BACKGROUND
Communication technologies have been described as space-adjusting techniques [14], as they enable events to be witnessed by proportionally much larger audiences than the capacity of the venues in which they are held. In these scenarios, unlike previous case studies such as those in [16], it is not possible to infer a Witness is OTG for the dominating category of observations, that is of the play on the field [19].
It has also been determined that the live broadcast delay of approximately 12 seconds cannot be detected in micro-blogs, ruling this feature out as a method to distinguish those witnessing via a broadcast [19] . In addition to sport, differentiating Witnesses of crisis events has gained much interest from researchers.
A journalistic approach describes extracting observation features from text to identify Witnesses [2], whereas spatial presence in the city of the event is the criterion in other work [10].
Supervised Machine Learning for Categorisation
Natural language processing (NLP) using bag-of-words approaches from unigram, bigram and parts-of- speech (POS) models, can be utilised as baseline text categorisation features [10] [18]. These research report success, comparable in many scenarios to more sophisticated features [10] [18]. A visual bag-of- words approach to categorise images linked to micro-blogs has also been tested [9]. The disadvantage of bag-of-words approaches is that although the methodology can be applied generically, the resulting model is not generic, for example, a model developed from training data for a football game cannot be used for a bushfire. Approaches which extract semantic meaning, for example locative expressions from text [7] would enable a generic model, but their success to-date is limited in domains such as social media [7]. Detecting micro-blogs posted from OTG is also recognised as a unbalanced class problem [15]. Approaches taken to mitigate class imbalance typically involve balancing the data via sampling [10] [5] [18], or algorithmically introducing a miss-classification cost to the under-represented class [15].
Dempster-Shafer Theory of Evidence
Dempster-Shafters Theory of Evidence is one method that has found application in classifier fusion, and managing uncertainty and incomplete reasoning [3]. The theory models the power set for the frame of discernment of the hypothesis [3]. A mass function is assigned for each subset in the power set from which the belief interval can be derived [3]. The mass function can be assigned from various classifier results, including the overall accuracy, class statistics or individual instances [12]. The mass functions for independent evidence can then be combined [3]. Dempster’s Rule of Combination has been shown to produce uninituitve results in scenarios with conflict [13], resulting in many enhancements being proposed including PCR6 [13] based on proportional conflict resolution.
1 METHODOLOGY
Data Collection and Training Set Creation
The case study event is an Australian Football League (AFL) match played at the Melbourne Cricket Ground (MCG) on the annual ANZAC Day public holiday. In 2015, this match attracted a near capacity crowd of 88,3981and television ratings of 1.298 million2. The corpus was collected using the AFL’s promoted hashtag #afldonspies, utilising the Twitter Data Analystics software packages [6].
Pre-processing samples the micro-blogs to those which can be identified as individual and original, that is not a retweet or posted by a non-individual such as the media [17]. To collect a sample of linked images, all micro-blogs in the corpus with a URL to Twitter or Instagram were inspected as these are more likely to contain WA [16]. To create the training set, two expert annotators coded the tweet text and linked images with one of three categories, examples for which are presented in Table 1. The three categories are:
1. No Evidence (NE) when no evidence of being posted from OTG or another place could be detected.
2. When evidence is detected, it is categorised as either evidence posted from OTG (E-OTG);
3. Or counter-evidence indicating that it is not posted from OTG (E-NOTG).
Table 1. Example text and image content for each category. (Source: twitter.com, access date:
25-26/04/15.)
No Evidence (NE) Evidence OTG (E-OTG) Evidence not OTG (E-NOTG) Fletcher goes bang with a 60
metre monster! #AFLDon- sPies
Not the best seats in the house but just glad to be here at @MCG #AFLDonsPies
In front of TV with chips for next 3 hours! #AFLDonsPies
Supervised Machine Learning for Categorisation of Text and Images
Pre-processing of the text included word tokenisation and parts-of-speech tagging using Ark NLP [11].
WEKA’s [4] default pre-processing filters were used to experiment with unigram and bigram models.
WEKA default feature selection filters are utilised to reduce the number of redundant dimensions, and experiment with a range of classifiers indicated by previous research including Naive Bayes, Random Forest and Support Vector Machines (SVM). All experiments were completed with 10-fold cross valida- tion. As expected, class imbalance was an issue in particular for the text corpus. Sampling to micro-blogs posted by micro-bloggers with at least one piece of evidence detected in the the training set was used to mitigate the imbalance. The classifier selected was that which maximises precision of E-OTG and E-NOTG classes, at the expense of recall if necessary, to minimise conflict due to miss-classification in the Dempster-Shafer modeling.
Categorisation of Geotags
Geotag evidence was cateogorised as E-OTG or E-NOTG based on whether it was contained within or in the immediate vicinity of the MCG, the place of the event. It was necessary to create a decision boundary for this categorisation, which was informed primarily by the boundaries of places bordering the MCG, for example train lines, roads and other venues.
1twitter.com/MCG/status/591859347891748865
2http://footyindustry.com/files/afl/media/tvratings/2015/2015AFLRatings.png
Dempster-Shafer Modelling of Evidence Extracted from Micro-blogs
The frame of discernment is modelled as{E-OTG, E-NOTG}with power set (null, E-OTG, E-NOTG, {E-OTG, E-NOTG}). The categorisation of NE is not modelled in the frame of discernment. For example, if a micro-blog has a geotag categorised as E-OTG, and text and image categorised as NE, the text and image do not corroborate or produce conflict with the E-OTG categorisation provided by the geotag. For demonstraton mass functions are set manually, with derivation from classifier results left to future work.
The mass functions assigned to geotags represent greater certainty than that assigned for images, which are greater than that assigned for text. The combination rule PCR6 implemented in Matlab [8] is then used to compute the combinations for analysis, and again, decision algorithm testing left for future work.
2 RESULTS AND DISCUSSION
The corpus contained 3260 micro-blogs, 265 with linked images and 133 with geotags. Table 2 presents the categorisation results, both training and predicted by classifiers. The annotator agreement for the text and image content was high with Cohen’s Kappa of 0.895 and 0.929 respectively. Combining the three content sources, or evidence, from the training data indicates the number of micro-blogs categorised as E-OTG and E-NOTG can be increased significantly from those with geotags alone. The increase for E-OTG is from 21 to 176 micro-blogs, and the increase for E-NOTG is from 112 to 241 micro-blogs.
This corresponds to an additional 125 potential Witnesses OTG from 16. 54 tweets had more than one piece of evidence which could be checked for conflict. Conflict did exist for a fraction of tweets, found to be due to GPS error. The geotag indicated the micro-blog was posted from a nearby venue, when the image and text indicated it was posted from the MCG.
The combined results correctly predicted are fewer than the training data, but still a significant increase from those with geotags alone. The number of micro-blogs categorised as E-OTG increased from 21 to 125, and the number of micro-blogs categorised as E-NOTG increased from 112 to 182. This corresponded to an additional 77 potential Witnesses posting from OTG and an additional 50 potential Witnesses via the broadcast. From the predicted results of the classifiers, 26 micro-blogs had more than one piece of evidence, with five in conflict. In addition to GPS error, these conflicts are now also attributed to miss-classification.
Table 2. Number of micro-blogs categorised for each content individually and in combination. The number of miss-classified micro-blogs in the class are presented in parenthesis.
Content E-OTG E-NOTG NE
Training Predicted Training Predicted Training Predicted
Geotag 21 - 112 - - -
Image 95 95 (11) 26 17 (10) 146 173 (27)
Text 99 34 (10) 129 66 (6) 3032 1088 (143)
Combination 176 125 (15) 241 182 (17) 2328 876 (132)
From the classifier experimentation it was found the WEKA default SVM, feature selection filter, and a unigram model maximised precision of the E-OTG and E-NOTG classes for text content. Using the methodology described by [9] the SVM classifier was additionally selected for the image content.
The precision and recall results for each class are presented in Table 3. These results indicate the image categorisation failed for the E-NOTG class, which is attributed to the insufficient number of samples in the training data. For future experiments this category could be removed, or the sample increased from other events, both options are to be tested in future work. In comparison, the E-OTG category proved acceptable for both precision and recall. The better precision for text E-NOTG compared to E-OTG could in part be explained by the topics contained in these micro-blogs were dominated by explicit statements critiquing the television coverage or the medium via which the broadcast was being viewed, enabling a more representative unigram model. In comparison, there was not a dominate topic for E-OTG.
More robust feature development based on previously identified witnessing characteristics [16] is being developed in future work. Additionally, the results indicate improvements could be made if the class imbalance were further addressed.
Table 3. Class precision and recall results for adopted classifier.
Corpus Statistic E-OTG E-NOTG NE Text Precision 0.706 0.909 0.869
Recall 0.242 0.465 0.984
Image Precision 0.844 0.412 0.844
Recall 0.896 0.280 0.896
Table 4 presents example mass functions manually assigned for text and image evidence and combina- tion mass functions corresponding to predicted combination results. Manual analysis concludes that these results appear intuitive, for example, when there are multiple evidence present supporting a categorisation, the increased values indicate corroboration. Additionally, when conflict exists the values reflect this, suggesting the conflict redistribution of the PCR6 algorithm is appropriate for the modelled scenario.
Hybrid methods for deriving the mass functions which can model the uncertainty of the evidence in addition to the automatic classification results are in progress.
Table 4. Example mass functions assigned or PCR6 combination results for evidence combinations detected in micro-blogs. (- indicates no data.)
Evidence Categorisat. Mass Function Comment
Text Image Geotag E-OTG E-NOTG E-OTG,E-NOTG
E-OTG - - 0.70 0.15 0.15 Assigned Mass Funct.
E-OTG - E-OTG 0.95 0.04 0.01
E-OTG E-OTG E-OTG 0.97 0.02 0.01
E-OTG E-OTG E-NOTG 0.55 0.43 0.02 GPS error example
E-OTG - E-NOTG 0.35 0.64 0.01 Miss-classified Text
- E-NOTG - 0.10 0.80 0.10 Assigned Mass Funct.
E-NOTG E-NOTG - 0.07 0.91 0.02
3 CONCLUSION
This paper presented progress on a framework to automatically extract WA and IA of events from social media. Baseline supervised machine learning techniques to categorise text and images were demonstrated, enabling micro-blogs posted from OTG or via the broadcast to be identified in signficantly greater numbers than with geotags alone. Additionally, a method based on Dempster-Shafer Theory of Evidence was demonstrated to combine the extracted evidence to test corroboration or conflict in the categorisation of the micro-blogs. Many areas for enhancements are identified, including machine learning approaches that further mitigate class imbalance and enable generic model development. In addition to seeking these enhancements, future work will include modeling the combination of evidence from multiple micro-blogs to identify the status of potential Witnesses.
REFERENCES
[1] CHENG, Z., CAVERLEE, J.,ANDLEE, K. You are where you tweet: A content-based approach to geo-locating Twitter users. InProceedings of the 19th ACM International Conference on Information and Knowledge Management(2010), ACM, pp. 759–768.
[2] DIAKOPOULOS, N., DECHOUDHURY, M.,ANDNAAMAN, M. Finding and assessing social media information sources in the context of journalism. InProceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems(2012), pp. 2451–2460.
[3] GARGIULO, F., MAZZARIELLO, C., ANDSANSONE, C. Multiple Classifier Systems: Theory, Application and Tools. Spinger-Verlag, 2013, ch. 10, pp. 335–378.
[4] HALL, M., FRANK, E., HOLMES, G., PFAHRINGER, B., REUTEMANN, P.,ANDWITTEN, I. H.
The WEKA data mining software: An update.SIGKDD Explorations 11, 1 (2009), 10–18.
[5] KUMAR, S., HU, X.,ANDLIU, H. A behaviour analytics approach to identifying tweets from crisis regions. InProceedings of the 25th ACM Conference on Hypertext and Social Media(2014), pp. 255–260.
[6] KUMAR, S., MORSTATTER, F.,ANDLIU, H. Twitter Data Analytics. Springer, 2014.
[7] LIU, F., VASARDANI, M.,ANDBALDWIN, T. Automatic identification of locative expressions from social media text: A comparative analysis. InProceedings of the 4th International Workshop on Location and the Web (LocWeb)(2014), pp. 9–16.
[8] MARTIN, A. Implementing general belief function framework with a practical codification for low complexity. InAdvances and Application os DSmT for Information Fusion, F. Smarandache and J. Dezert, Eds., vol. 3. American Press Rehoboth, 2009, pp. 217–273.
[9] MCLEAN, S. Identifying witness account in social media using imagery. Master’s thesis, 2015.
The University of Melbourne.
[10] MORSTATTER, F., LUBOLD, N., PON-BARRY, H., PFEFFER, J.,ANDLIU, H. Finding eyewitness tweets during crises. InProceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science(2014).
[11] OWOPUTI, O., OCONNOR, B., DYER, C., GIMPEL, K., SCHNEIDER, N.,ANDSMITH, N. A.
Improved part-of-speech tagging for online conversational text with word clusters. InProceedings of NAACL 2013(2013).
[12] PARIKH, C. R., PONT, M. J., AND JONES, N. B. Application of Dempster-Shafer theory in condition monitoring applications: a case study.Pattern Recognition Letters 22(2001), 777–785.
[13] SMARANDACHE, F.,ANDDEZERT, J. On the consistency of PCR6 with the averaging rule and its application to probability estimation. InProceedings of the 16th International Conference on Information Fusion(2013), pp. 1119–1126.
[14] SPENCER, J. E.,ANDTHOMAS, W. L. J. Cultural Geography. John Wiley & Sons, Inc., 1969.
[15] STARBIRD, K., GRACE, M.,ANDLEYSIA, P. Learning from the crowd: Collaborative filtering techniques for identifying on-the-ground Twitterers during mass disruptions. InProceedings of the 9th International ISCRAM Conference(2012), pp. 1–10.
[16] TRUELOVE, M., VASARDANI, M.,ANDWINTER, S. Testing a model of witness accounts in social media. InProceedings of the 8th Workshop on Geographic Information Retrieval(2014), no. 10.
[17] TRUELOVE, M., VASARDANI, M.,ANDWINTER, S. Towards credibility of micro-blogs: charac- terising witness accounts.GeoJournal 80, 3 (2015), 339–359.
[18] VERMA, S., VIEWEG, S., CORVEY, W. J., PALEN, L., MARTIN, J. H., PALMER, M., SCHRAM, A.,ANDANDERSON, K. M. Natural language processing to the rescue? Extracting ”situational awareness” tweets during mass emergency. InProceedings of the 5th International AAAI Conference on Weblogs and Social Media(2011), pp. 385–392.
[19] ZHAO, S., ZHONG, L., WICKRAMASURIYA, J., AND VASUDEVAN, V. Human as real-time sensors of social and physical events: A case study of Twitter and sports games. Tech. rep., Rice University and Motorola Labs, 2011.