Share - Publish - Store - Preserve. Methodologies, Tools and Challenges for 3D Use in Social Sciences and Humanities

(1)

HAL Id: hal-02155055

https://hal.archives-ouvertes.fr/hal-02155055

Submitted on 13 Jun 2019

HAL is a multi-disciplinary open access

archive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Distributed under a Creative Commons Attribution| 4.0 International License

Humanities

Anas Alaoui M’Darhri, Vincent Baillet, Bastien Bourineau, Alessio

Calantropio, Gabriella Carpentiero, Medhi Chayani, Livio de Luca, Iwona

Dudek, Bruno Dutailly, Hélène Gautier, et al.

To cite this version:

Anas Alaoui M’Darhri, Vincent Baillet, Bastien Bourineau, Alessio Calantropio, Gabriella Carpen-tiero, et al.. Share - Publish - Store - Preserve. Methodologies, Tools and Challenges for 3D Use in Social Sciences and Humanities. PARTHENOS. Share - Publish - Store - Preserve. Methodologies, Tools and Challenges for 3D Use in Social Sciences and Humanities, Feb 2019, Marseille, France. 2019. �hal-02155055�

(2)

(3)

(4)

3 WHITE PAPER

Share — Publish — Store — Preserve. Methodologies, Tools and Challenges for 3D Use in Social Sciences and Humanities

This White Paper is the result of the PARTHENOS Workshop held in Marseilles at MAP Lab. (France), February 25th-February 27th, 2019.

During the organization of this workshop, we were all strongly affected by the premature death of Matteo Dellepiane. Matteo had participated in the first PARTHENOS workshop on 3D. He was an outstanding researcher whose work on geometric reconstruction and visual appearance was recognized by the entire community working on 3D and cultural heritage internationally. As a member of the Visual Computing Lab of CNR-ISTI, he contributed to the development of several tools now widely used by our community for the processing, visualization and publication of 3D models for the purpose of documentation and dissemination of Cultural Heritage. Even though Matteo was not with us in Marseilles during this workshop, the fruit of his work has served us well and will support our scientific community facing the long-term challenges in this field.

(5)

(6)

5

C

OORDINATORS

Hélène GAUTIER, CNRS, TGIR Huma-Num (PARTHENOS project), France. Xavier GRANIER, IOGS, Huma-Num’s 3D Consortium for Humanities, France. Adeline JOFFRES, CNRS, TGIR Huma-Num (PARTHENOS project), France. Livio DE LUCA, CNRS-MAP & LIA MAP-ISTI, France.

Stéphane POUYLLAU, CNRS, TGIR Huma-Num (PARTHENOS project), France. Laurent ROMARY, Inria (PARTHENOS project), France.

Roberto SCOPIGNO, CNR, ISTI Lab (PARTHENOS project), Italy. Dorian SEILLIER, Inria (PARTHENOS project), France.

(7)

6

A

UTHORS

Anas ALAOUI M’DARHRI

CNRS-MAP, France.

Vincent BAILLET

Université de Bordeaux Montaigne, ArchXant, France.

Bastien BOURINEAU

I-Maginer, France.

Alessio CALANTROPIO

Politecnico di Torino, Italy.

Gabriella CARPENTIERO

Soprintendenza Archeologia Belle Arti e Paesaggio del Molise, Italy.

Mehdi CHAYANI

CNRS, Archeovision Lab, France.

Livio DE LUCA

CNRS-MAP & LIA MAP-ISTI, France.

Iwona DUDEK

UMR 3495 CNRS/MC MAP, France.

Bruno DUTAILLY

Hélène GAUTIER

CNRS-TGIR Huma-Num, France.

Eleonora GRILLI

FBK-3DOM, Italy.

Valentin GRIMAUD

Université de Nantes-LARA, France.

Christoph HOFFMANN

ACDH-ÖAW, Austria.

Adeline JOFFRES

CNRS-TGIR Huma-Num, France.

Nenad JONČIĆ

University of Primorska, Slovenia.

Michel JORDAN

ETIS, UMR8051, Université Paris-Seine, Université de Cergy-Pontoise, ENSEA, CNRS, France.

Justin KIMBALL

Kulturhistorisk Museum, Norway.

Adeline MANUEL

CNRS-MAP, France.

Patrick MCINERNEY

Waterford Institute of Technology, Ireland.

Imanol MUÑOZ PANDIELLA

Universitat de Girona, Spain.

Ariane NEROULIDIS

(8)

7

Erica NOCERINO

LiS/ETH Zürich, France.

Anthony PAMART

CNRS-MAP, France.

Costas PAPADOPOULOS

Maastricht University, The Netherlands.

Marco POTENZIANI CNR-ISTI, Italy. Emilie SAUBESTRE CNRS-CFEETK, Egypt. Roberto SCOPIGNO CNR-ISTI, Italy. Dorian SEILLIER Inria, France Sarah TOURNON-VALIENTE

Martina TROGNITZ

ACDH-ÖAW, Austria.

Jean-Marc VALLET

CICRP-MAP, France.

Chiara ZUANNI

ZIM-ACDH, Karl Franzens Universität Graz, Austria.

(9)

8

Executive Summary

Through this White Paper, which gathers contributions from experts of 3D data as well as professionals concerned with the interoperability and sustainability of 3D research data, the PARTHENOS project aims at highlighting some of the current issues they have to face, with possible specific points according to the discipline, and potential practices and methodologies to deal with these issues.

During the workshop, several tools to deal with these issues have been introduced and confronted with the participants experiences, this White Paper now intends to go further by also integrating participants feedbacks and suggestions of potential improvements.

Therefore, even if the focus is put on specific tools, the main goal is to contribute to the development of standardized good practices related to the sharing, publication, storage and long-term preservation of 3D data.

(12)

11

1. Introduction

1.1. Rationale

Today, 3D data have become an inherent part of research material in Humanities and Social Sciences. The 3D technologies are progressing rapidly but are not easy to manage and a balance has to be found between stable and cutting-edge tools. Furthermore, there are still issues particularly with interoperability, and hence sharing, publishing and long-term preservation.

This White Paper is the result of a second workshop organized by CNR (Italy), CNRS (France), Inria (France), Huma-Num's Consortium 3D for Humanities (France) and the CNRS-MAP/CNR-ISTI International Joint Lab within in the scope of PARTHENOS Work Package 4 on Standardization, with support from the technical partners and on behalf of the PARTHENOS Research Infrastructure. This was held in Marseilles (France), from February 25 to February 27, 2019, and entitled "Share — Publish — Store — Preserve.

Methodologies, Tools and Challenges for 3D Use in Social Sciences and Humanities". After a first workshop in 2016 dedicated to the challenges of creation, interoperability and preservation1, this one mainly focused on interactive 3D data management tools, most of them developed by the organizers, highlighting their strengths and weaknesses.

More precisely, the objectives were to:

● Share general knowledge on 3D issues in Humanities and Social Sciences; ● Deepen technical knowledge about specific 3D tools;

● Contribute to a better understanding of technologies potential and users’ needs; ● Confront practices and experiences and raise for discussion;

● Promote and contribute to best practices and standards to ensure 3D data interoperability and sustainability.

1

Pierre Alliez, Laurent Bergerot, Jean-François Bernard, Clotilde Boust, George Bruseker, et al.. Digital 3D Objects in Art and Humanities: challenges of creation, interoperability and preservation. White paper: A result of the PARTHENOS Workshop held in Bordeaux at Maison des Sciences de l’Homme d’Aquitaine and at Archeovision Lab. (France), November 30th - December 2nd, 2016. PARTHENOS. Digital 3D Objects in Art and Humanities: challenges of creation, interoperability and preservation, Nov 2016, Bordeaux, France. pp.71, 2017. 〈hal-01526713v2〉

(13)

12

We selected four main topics to focus on during the workshop, corresponding to the life cycle and the various uses of 3D data in Humanities and Social Sciences: sharing, publishing, storing and preserving.

For each one, the related theme, the issues it deals with and specific tools addressing these issues, has been introduced by domain specialists in brief presentations followed by training sessions where participants used the tools presented with their own data and discussed the limitations and potential improvements of the tools. Those themes and discussions are the basis of the core chapters of this white paper.

A specific aspect that deserves to be highlighted compared to the first edition is the way we proceeded to get a wider impact, beyond PARTHENOS. Indeed, we decided to open the discussion through launching an open call2 for participation in order to gather interdisciplinary profiles of people who could contribute various viewpoints that were presented during a break out session3. Thus the workshop was attended by 16 participants representative of different domains, with complementary backgrounds. Indeed, we aimed at including not only archaeology - the reference domain- but also Humanities and Cultural Heritage at large with all related domains such as digital humanities, architecture, geomatics or computer graphics. The goal was to ensure coverage of a wide range of use cases from different research communities and to ensure that the background of the participants would not bias the discussion.

As a consequence, this white paper is based on contributions from all the participants, reporting the main conclusions of the discussion. They also contribute to feed one of the main outputs of the PARTHENOS project, the Standardization Survival Kit (or SSK : http://ssk.huma-num.fr/#/) produced in the WP4 framework, through the creation of research scenarios which are fully accessible online.

Such a framework may be further enriched by other experts in the field who will take advantage of the video recording of the master classes. Indeed, you can access the videos that have been recorded and edited through the PARTHENOS collection of

2

This call received nearly 60 applications from 11 European countries among which we selected 17 participants according to criteria of knowledge of 3D technologies and their uses, their projects carried out or in progress, their disciplinary background, and their motivation to participate according to our own expectations around the PARTHENOS project. One participant finally has to cancel his participation and we ended up with 16 participants.

3

The videos of this session will be available online at the address: https://www.canal-u.tv/producteurs/humanum/parthenos.

(14)

13 Num’s Canal-u’s channel (the French digital video library for higher education and research) following this link: https://www.canal-u.tv/producteurs/humanum/parthenos. All the presentations preceding the training sessions will also be available through the PARTHENOS Internet platform4.

We report here the results of the discussion at the workshop, further improved and extended by subsequent work done after the workshop by the speakers, the organizers, and the participants involved. Our aim with this White Paper is to briefly review the status of key 3D tools used by different communities willing to share, publish, store and preserve data they produce and to promote further discussion and action regarding their use and the data produced.

4

(15)

14

1.2. Publishing

and

Accessing

Knowledge

through

the

PARTHENOS SSK, Another Way of Sharing

In the context of research, standards usually take the form of documents informing about practices, protocols, artefact characteristics or data formats that can be used as reference for two parties working in the same field of activity in order to produce comparable (and interoperable) results.

The experience gained within the various communities and infrastructures represented in PARTHENOS5 that have been in need of adopting existing standards, has led to the identification of three use cases where it would be possible to reconcile standards and researchers in the Humanities:

● In the context of a research project, there is always an initial phase during which scholars should be made aware of some core standards that are systematically related to the definition of interoperable digital objects.

● On the other hand, when researchers, individually or within teams, have built a research workflow based on the use of the best practices and relevant community standards, we lack a place to publish and share such information.

● Finally, some of these workflows, or protocols, can at some point become themselves standards, and therefore a "pre-standardization" framework that stabilizes the knowledge is essential.

The Standardization survival kit6, or SSK, addresses these issues by supporting and providing expertise to researchers in their use of standards. It encourages the publication and sharing of the best practices in a given research context, and it acts as a preservation system for the submitted workflows as they can become standards for the communities.

5

http://parthenos-project.eu/

6

(16)

15

1.2.1. General Presentation and Features

To document the close link to be made between the use of a given standard, or a subset of it, and a corresponding research activity, the natural choice is to use the Taxonomy of Digital Research Activities in the Humanities (TaDirah7) (Schöch 2015, Borek et al. 2016), exactly designed to cover the core tasks of Digital Humanities Research, even though, when it comes to Heritage Science, other vocabularies have to be used.

To link those activities between one another, there emerged the concept of research scenarios which are the descriptions of research processes built on the use of standards. These scenarios aim at providing contextual information and relevant examples on how standards can be applied in a given research project. They have been created and added by domain experts, from real life researcher-oriented use cases. Each scenario within the SSK works like a high-level research guide for scholars. They are made up of successive steps involving specific tasks, and can be followed as a complete process to solve a given problem with the most standardized means. Scenarios can be seen as a living memory of what should be the best research practices in a given community. For that reason, the SSK can be considered as a framework showing concrete use of standards, rather than simply a catalogue of resources.

For each step, relevant resources to perform the given task are proposed, divided into two categories:

- the “general resources” that include the primary documentation and tools

- the “project-specific resources” that point to concrete use cases in which a similar task was accomplished.

The material contained in these sections is of various kinds, among which the most important is a state-of-the-art bibliography, which includes all the documentation needed to carry out a given task. The bibliographical references are up-to-date and gathered within a Zotero library, which was specially created for this project. The SSK also offers more technical resources, such as stylesheets, code samples, software or services, and training materials like tutorials.

7

(17)

16

1.2.2. Navigating the SSK

As a tool, the SSK is dedicated to the researchers in the wide scope of Social Sciences, Humanities and Heritage disciplines, regardless of their skills or practices with digital tools. The exact disciplines addressed in the scenarios are taken from the AURéHAL8 ontology, in the « Humanities and Social Sciences » section. Besides the disciplines, the researchers can find the relevant material by selecting:

 the research objects (images, artifacts, manuscripts, etc.)  the research techniques (mapping, encoding, scanning, etc.)  the research activities (discovering, archiving, publishing, etc.)  the standards (ISO, EAD, TEI, etc.).

Users can browse by using those facets, and switch easily between scenarios, steps and resources:

Figure 1: Navigation page in the SSK

8

(18)

17

1.2.3. Actual Limitations and Suggested Improvements

Among the concerns and flaws of the tool listed by the participants:

- The validity and reliability of the scenarios, since they come mostly from individuals (or labs), without clear peer or institutional review. There could be also a problem of consistency between scenarios, depending on the level of expertise and/or the amount of time allocated by the author(s). That would be problematic as scenarios should be seen as best references in their domain.

- Lacks of terms & vocabularies to describe specific fields or objects. We should add more exhaustive ontologies or allow users to propose new ones.

- Need of more connection between scenarios in order to build more global workflows.

- Risk of redundancy: if scenarios are too similar, it will be difficult to discriminate between them and to find the best one.

- Including the communities by allowing comments, remarks, suggestions on the scenarios, to allow potential enrichment.

- Inserting images within steps would be good.

Reinforcing the links between the SSK and similar digital initiatives, like Open Methods, or Dariah Teach.

1.3. Impact on PARTHENOS: the New Scenarios Envisioned

Resulting from the work of all the participants to the workshop, split into small groups, five scenarios have been proposed to be included in the SSK. They are still in progress but should be added to the platform in April 2019.

1.3.1. RTI Archiving

(19)

18

Reflectance Transformation Imaging, a computational photography method based on the principle of raking light has been extensively used in fields such as archaeology, museum studies, and art history to capture and enhance subtle surface details. Low-angled and multidirectional light creates texture details and enhances planar irregularities. There are two methods to capture RTI images: the Dome Method and the Highlight RTI Method. In both methods, objects are photographed with a single camera in a fixed position. The only element that changes in the process is the position of light. In the Dome method, the light position is known since lights are fixed. In the highlight method, the position of the light is captured by using reflective spheres that are then used by specialised software to create the RTI image. The images stack is then computed into a polynomial texture map. In a dedicated viewer, the resulting file can be viewed under individual light positions and different rendering algorithms.

This scenario focuses on the preparation of captured RTI data for archiving. The main component of an RTI dataset is photographs. Additionally, some software-dependent formats will have to be considered for archiving. Resulting formats for RTI images were developed in 2001 by the Hewlett Packard Laboratories and further extended by Cultural Heritage Imaging (CHI).

1.3.2. Export of ortho-photos

Authors: Christoph Hoffmann, Nenad Jončić, Michel Jordan, Justin J.L. Kimball

In this scenario, we demonstrate a methodology to translate a 3D model into a 2D orthophoto using Agisoft’s Metashape/Photoscan. The dataset that we use here is based upon a studio-created photogrammetric 3D model, however, it can be utilised broadly within any photogrammetric 3D model. This scenario assumes that the reader has a photogrammetry of a real-world object which they can use along with this methodology.

1.3.3. Doing 3D Reconstruction in Archaeological Context

(20)

19 3D reconstructions have a long history in archaeology and cultural heritage, although there are varying practices, data capture and processing methods, tools, and visualisations. There are not clear ways of determining the accuracy and precision of the reconstructions. Furthermore, scientific and engagement purposes inform different dissemination practices, and therefore pose different challenges to the preservation of these outputs.

1.3.4. Classification of 3D Models: Working from 2D to 3D

Authors: Eleonora Grilli, Imanol Muñoz-Pandiella, Erica Nocerico

The use of 3D models is, nowadays, widely diffused for the documentation and dissemination of cultural and archaeological heritage. Nevertheless, to provide useful 3D data, it is necessary to associate some semantic information that can help operators to understand the heritage. This scenario aims at pointing out a pipeline to manage various types of 3D survey data and to associate them with heterogeneous information and attributes to characterize and describe the surveyed object.

1.3.5. UAV Data Acquisition and Management for Multitemporal Surveys of Archaeological Sites

Authors: Alessio Calantropio, Gabriella Carpentiero, Lorenzo Teppati Losè

Aerial photogrammetry is a powerful technique that can be used for recording and analyzing excavation phases of archaeological sites. Based on the authors experience, there are some strategies that can speed up the data acquisition, processing and analysis steps. This works aims to underline best practices and lesson learned in the scenario of UAV data acquisition and management for multitemporal surveys of archaeological sites with a photogrammetric approach.

(21)

20

2. Collecting and Sharing Data Linked with 3D

2.1. Mission Statement

For the last few years, the cross-disciplinary community working on digital technologies applied to cultural heritage has been involved in national and international projects for building next generation tools and information systems for analysing the state of conservation of heritage artefacts, studying their temporal transformations, extracting their morphological features, and correlating heterogeneous data coming from several disciplines. Even if the impressive and fast evolution of digital technology has produced more and more proficient hardware and software for automated data acquisition and processing, at the same time, the main actors of everyday production of knowledge on cultural heritage need today a new generation of tools for embedding their knowledge into digital representations. Moving from a data-driven to a semantic-driven digital documentation approach is an essential challenge, especially in Cultural Heritage, where knowledge is always the result of a combination of complementary skills and disciplinary profiles. The semantic enrichment of digital resources is generally achieved through an annotation process that links a document with additional information that can take several forms (keywords, semantic classes, process description...). Semantic annotation is the process that creates semantic labels to support advanced research. The annotations, thus defined, make it possible to create a complex information structure by the use of different data models (tags, attributes, relationships, ontologies...). The annotation of 3D models raises problems related to their manipulation, especially for the definition of complex annotations. On the other hand, images have proved to be an effective and easy-to-use media for defining annotations and for recording other information from scientific imagery but do not provide accurate geometric information. The integration of these different media into a common annotation methodology would offer a promising approach. For an overview of the different systems and approaches for annotating visual models, see this Overview Box9.

9

A. Manuel, R. Scopigno, “Overview Box 3 - Tools supporting annotation on 3D models” in Pierre Alliez, Laurent Bergerot, Jean-François Bernard, Clotilde Boust, George Bruseker, et al.. Digital 3D Objects in Art and Humanities: challenges of creation, interoperability and preservation. White paper: A result of the PARTHENOS Workshop held in Bordeaux at Maison des Sciences de l’Homme d’Aquitaine and at Archeovision Lab. (France), November 30th -

(22)

21

2.2. Collect and Share - Aïoli, a Reality-Based 3D Annotation

Platform

Archaeologists, architects, engineers, materials specialists, teachers, curators and restorers of cultural property, contribute to the daily knowledge and conservation of heritage artefacts. The management of multi-dimensional and multi-format data introduces new challenges, in particular the development of relevant analysis and interpretation methods, the sharing and correlation of heterogeneous data among several actors and contexts, and the centralised archiving of documentation results. Despite their different approaches and tools for observation, description and analysis, the actors of cultural heritage documentation all have a common interest and central focus: the heritage object, the physical one, whether it is a site, a building, a sculpture, a painting, a work of art, or an archaeological fragment. This is the starting point of Aïoli, a reality-based 3D annotation platform, which allows a multidisciplinary community to build semantically-enriched 3D descriptions of heritage artefacts from simple images and spatialized annotations coupled with additional resources. Developed by the CNRS-MAP Lab, this platform introduces an innovative framework for the comprehensive, large-scale collaborative documentation of cultural heritage by integrating state-of-the-art technological components (fully automatic image-based 3D reconstruction, 2D-3D spreading and correlation of semantic annotations, multi-layered analysis of qualitative and quantitative attributes...) within a cloud infrastructure accessible via Internet interfaces from PCs, tablets and smartphones online and onsite.

Our approach puts the heritage object (the physical one) at the heart of the documentation process by considering reality-based 3D and semantic description in a strongly integrated way. By correlating simple images (probably the stablest support for registering field observations since the invention of the photography), our platform generates a dynamic 3D morphologic scaffolding for structuring data and annotations. At the same time, this

December 2nd, 2016. PARTHENOS. Digital 3D Objects in Art and Humanities: challenges of creation, interoperability and preservation, Nov 2016, Bordeaux, France. pp.71, 2017. 〈hal-01526713v2〉

(23)

22

data structure becomes a way of enabling the communication between several actors involved in the documentation and the study of the same object. This is made by interlinking all phases that characterise the overall documentation process within an ‘informative continuum’ by combining three essential features:

● A continuous 3D mapping and annotation process

Our approach creates a sort of bridge between the real object and its digital representation by introducing a solution for memorising spatialized annotations made by different actors.

● A morphology-based documentation framework

Based on a spatial overlapping factor, our multi-layers description model allows the simultaneous structuring of data and geometry as well as the continuous correlation of semantic annotations (with relative attributes).

● A flexible & scalable technology

Our platform is built on a cloud computing service allowing the gathering, processing and sharing of semantically-enriched 3D data within online and onsite documentation scenarios.

Our system introduces an original informative linkage between the physical object space and its digital representation by facing to two interconnected technical issues: the onsite retrieval of structured information according to the physical object's annotation; the onsite collection and processing of new data to be spatially referenced and semantically correlated with previous data. This is done by integrating the following features:

● An incremental image-based 3D spatialisation process to manage the geometric merging of several images coming from different actors at different temporal states; ● A 2D/3D annotation framework enabling users to draw, visualise and register

relevant surface regions by handling simple 2D images spatially oriented around a dynamic 3D representation;

(24)

23 ● A multi-layered morphology-based data structuring model to accurately describe real objects in all their geometric complexity and according to multidisciplinary observations.

Our platform is today running on a server appliance composed of 90 cores (developing 180 high frequency threads equivalent to 250 gigaflops) of computing power, completed by 256GB of RAM and supported by 8TB of storage. Given its technical implementation, the platform can be easily installed in larger server appliances.

Further information about the aïoli project (including demonstration videos, case studies, beta testing programme) are available on the platform website: www.aioli.cloud.

2.2.2. Use Cases

In order to try out the experimental features of aïoli, a beta-testing programme started on October 2017. This programme is currently ongoing and is involving selected actors of cultural heritage scientific and professional community. During this programme, a wide range of heritage artefacts, belonging to different scales and photographed by different sensors, have been used for experimenting the implemented features. This beta-testing programme was also an opportunity to explore the potential uses of the platform within the CH community. In the case of curation and restoration of cultural property, building specialists could analyse, for instance, the type of stone, alteration morphology, lost material and biological colonisation, in order to help preventive conservation (Fig. 2 cf usage 1). In the frame of study and restoration of paintings, experts could use the annotation tool to map the materials such as pigments identified thanks to techniques such as Scanning Electron Microscopy (SEM) or Laser-Induced Breakdown Spectroscopy (LIBS) and also examine the underdrawings using technical imaging (Fig. 2 cf usage 2). The 2D/3D annotation framework can be very useful for isolating and analysing the morphological features of archaeological fragments (Fig. 2 cf usage 3). Finally, aïoli may be used to help stratigraphic studies by spatializing and retrieving 3D information (dimensional layouts, orientation attributes …) such as lapidary signs (Fig. 2 cf usage 4).

(25)

24

Figure 2 : Screen captures of the case studies carried out within the beta testing programme: usages. Usage 1 left) – Mapping of biological colonization on the East wall of the Caromb’s church, France; Usage 2 (top-right) – Spatialization of a pictorial sample from the painting Triptyque de Venasque, Musée du Petit Palais, Avignon, France; Usage 3 (bottom-left) – Morphological analysis of a theatrical mask of the archaeological Museum of Lipari, Italy; Usage 4 (bottom-right) — Annotation of lapidary signs on the meridional wall of the

castle of Les Baux-de-Provence, France

The next step in this beta-testing programme is to move towards a progressive integration of collaborative experimentations, to put forward new scenarios of comparative and co-operative analysis of cultural heritage.

The technical implementation of the platform presents some limitations today: 1) it currently works only online and needs at least a 4G connection;

2) it currently works in arbitrary georeferencing, then the computed descriptors are only locally expressed, with an important limitation in terms of correlation potential. A new version of the platform should be available soon including some new features.

(26)

25 Given the emphasis put on the multi-users aspects, the interface for aïoli must ease the collaborative work on shared projects. This will be enabled by adopting a social network-like approach for the user-end uses in future releases.

A guide to the photographs acquisition for the successful reconstruction of the point cloud by our platform is being written to help users to understand the constraints related to the specificities of this acquisition type.

Since many photogrammetric reconstruction tools exist and many projects have already been processed and reconstructed using these tools, we plan to allow the import of data from other photogrammetric processing chains such as Agisoft Metashape.

The multiplicity of actors involved in the description of the same object requires that the meaning of the terms used for each of the specialities can be established in a reliable way. Controlled vocabularies are used to determine terminology that is intended to promote consistency of terms used within the same domain. Many vocabularies exist and are structured according to RDF data models to allow easy publication within the Semantic Web. Thus, in the description sheets, the integration of a new type of field linked to a controlled vocabulary is currently being developed.

We also plan the development of a off-line/online synchronisation process in order to extend the use to some particular work conditions, as well as a complete approach for scaling/georeferencing issues.

(27)

26

3. Visualize and Publish

3.1. Mission Statement: From Local Visualization to Web

Publishing of 3D Data

3.1.1. Introduction

Visualization and analysis of 3D data are moving from desktop to Internet. Several approaches have been proposed over the last years for effectively driving the publication of 3D material on the Internet.

Luckily, the first solutions - closed, plugin-based, non-standard, and poorly performing - are just a distant memory. The introduction of WebGL (the standard API for Computer Graphics Internet programming) brought a complete revolution, igniting the development of Web3D solutions efficient, standardized, plugin-free, and able to transform 3D content in a standard component of any webpage.

Nowadays, the resulting panorama is composed of plenty of libraries, tools, frameworks, and platforms, enabling the immediate and interactive rendering of 3D data directly inside all common Internet browsers. This ensemble is composed of a very heterogeneous set of online 3D viewers: from low to high-level software, from proprietary to open-source systems, from commercial to academic platforms.10

Unfortunately, at a closer inspection the Web3D ecosystem reveals several empty spots (unsolved issues, uncovered users, and neglected fields), mostly due to the fact that this large variety of solutions is often just focused towards mainstream publishing actions, and thus unable to cope with the needs of more specialized and technical 3D users.

Cultural Heritage professionals constitute a representative sample of users belonging to this community.

10

Potenziani M., Callieri M., Dellepiane M., Scopigno R., 2018. Publishing and Consuming 3D Content on the Web: A Survey, in Foundations and Trends® in Computer Graphics and Vision, Volume 10 (4), December 2018, Pages 244-333. https://doi.org/10.1561/0600000083

(28)

27

3.1.2. General Issues about 3D Internet Publishing

Thanks to the democratization of the technologies aimed at 3D acquisition (like photogrammetry or laser scanning), nowadays Cultural Heritage research teams produce a huge amount of digital 3D models.

Sharing all these data over the Internet is more and more felt as a primary need, and not just for following dissemination purposes (e.g. to cope with museums aims, or to enable exploiting digital reconstructions in teaching activities), but mostly because of more technical and professional uses (e.g. to support the remote and collaborative study of artworks, or to enable the creation of a comprehensive digital documentation of a restoration project).

To efficiently stream and render CH 3D datasets over the Internet is not trivial (because they are reality-based 3D data, and so high-resolution and intrinsically complex). Even now, there are few solutions specifically tailored to specific fields (cf. the tools presented in the next sections) that allow users to visualize and manipulate these 3D models online with excellent results.

Thus, the needs of the CH community are now going beyond the plain Internet visualization functionalities, starting to focus on those second-order features (analytical tools, data enrichment, copyright protection, etc.) that mark the border between “visualization” and “publishing”.

Enabling these more sophisticated visualization modes prompts basic questions and open issues, like for instance:

● Data enrichment is fundamental for transforming 3D models from mere data to a fully qualified content. It allows linking other content to 3D, enabling transmedia interlinking, multimedia annotations, documentation, etc. Currently very few platforms can manage data enrichment in a complete, structured, and standardized way. How an effective enrichment strategy could be implemented? What are the real needs of the community?

● 3D data have to be stored on a Internet server. The hosting service often is demanded by external institutions, either commercial or public. The related service can be provided by a tier under specific conditions of use, or also under specific costs. The conditions of use may not guarantee the protection of data in terms of

(29)

28

property and copy. Is this really perceived as an issue from the CH community? May it have any impact on the publication of their 3D datasets?

● Managing reality-based 3D models over the Internet is not simple. Fixed resolution datasets must be completely downloaded to be available in the viewer. This may strongly impact on the performances of the Web3D applications (bandwidth requirements and latency times). Efficient multi-resolution data management based on compressed and progressive encoding and streaming, does exist and can mitigate these issues. On the other hand, multi-resolution formats need data pre-processing, and make difficult implementing sophisticated visualization strategies (i.e. multi-texturing, collaborative editing, etc.). Are these issues critical for CH professionals? What is it possible to do for overcoming them?

The work of this session is organized consequently, with the aim to present the results of the discussion and the insights gathered on these topics.

3.2. OpenSpace3D

OpenSpace3D11 is an open platform with a large range of uses, from a simple 3D model visualization to an advanced Augmented Reality or Virtual Reality application.

Developed and published by the I-Maginer / Arinfo company, OpenSpace3D is accessible to people without skills in computer development, to create applications for computers and mobile. OpenSpace3D is aimed at helping to study, evaluate, share and present to the public 3D data, work and concepts.

It is possible to create industrial grade applications or prototypes quickly through the visual programming method.

OpenSpace3D has a multitude of pre-programmed functions to create applications for: ● Augmented Reality;

● Virtual Reality; ● virtual tours;

11

(30)

29 ● serious games;

● video games;

● connected objects or robots with Raspberry pi or Arduino.

OpenSpace3D is developed from the Scol language, modular and scalable and distributed under LGPL licence.

This general framework was used to configure a generic interface for a viewer dedicated to examine 3D models in the field of Cultural Heritage, even though it can fit other purposes. The way to produce this kind of application was thought in order to make it easier and can be synthesized in three steps:

1. Drag and drop your 3D model inside the OpenSpace3D’s window; 2. Import the template of the 3D viewer;

3. Export your application into Windows, Mac, Android. The main features of the 3D viewer template are:

● About object properties:

○ Switch between a set of materials in order to propose different kinds of visualization or layers of information;

○ Get some characteristics of the mesh or the point cloud selected (size in meters, number of triangle or dots);

○ Define the position, orientation and scale of the object, by entering the coordinates or by moving with manipulators.

● About light settings:

○ Activate and define the light of the sun; the 3D scene can be geolocalized; ○ Activate and explore a surface with a light moving alongside a dome; ○ Activate or deactivate shadows and / or ambient occlusion.

● About camera settings:

○ Switch between a fly or a first person point of view to explore the scene; ○ Switch between a navigation designed for objects (free orientation) and

architectural exploration (where the horizon stay horizontal); ○ Choose the focal length or an orthographic view.

(31)

30

○ Create and orientate a plane section; ○ Define the size of dots in point clouds; ○ Annotate the model;

○ Measure some distance in the model.

OpenSpace3D is used in many professional sectors – Industrial, Marketing / Events, Education, Research, Architecture and Games. Concerning Cultural Heritage, it allows visualization of 3D and point cloud models, and by adding a dedicated template, it adds tools to review and study these models that can also be enabled in Virtual Reality. OpenSpace3D can be used for museums and teaching to make playful applications to share with the general public.

3.2.2. Use Cases: Rock Art Visualization

Context:

Rock art documentation in Brittany (France) is the object of a research programme led by University of Nantes. The highest density of engravings is around the Carnac area and can be superposed with the territory concerned with a UNESCO project.

Figure 3: Screenshot of OpenSpace3D for the visualisation of neolithic architecture with Rock Art (cross-section of the Mané er Groez dolmen, Carnac)

(32)

31

Main issues OpenSpace3D allows to resolve:

Engravings can be found in a variety of situations, from isolated stela to complex architectures spreading over several kilometers in the landscape. Neolithic funerary architecture has a particularity strong relationship with the landscape, so we have to take this scale of representation into account. The difficulty of representation for this area can be solved thanks to a 3D representation of this territory in such a way that it is possible to go through a wide range of scales: from the infra-millimetric detail of the engravings to several hundred square kilometers of the geographical area.

It should be possible to show different layers of visualisation or information on a surface. For instance, for an engraved slab of stone, it could be interesting to switch between a coloured and relief-based representation, and to show different steps of rock art analysis coming from the ICEO protocol (Images Compilées sous Eclairages Obliques), such as outline synthesis, material removal synthesis and the interpretation of signs and symbols engraved. This is even more relevant since the engravings could be eroded and really difficult or impossible to detect with the naked eye.

It is important to share the data with other specialists (archaeologists, curators, etc.) who don’t necessarily have much experience with 3D tools. Interacting with 3D models could then be difficult. The application created with OpenSpace3D gathers a wide bunch of tools in order to extract the information they are looking for. It is important too to transmit this knowledge to visitors, and especially when a UNESCO project is going on. With the same scene as basis, it is possible to create a new application designed for a museum, with all possible ways to interact with the 3D content, from augmented or virtual reality to connected objects.

In order to provide an answer to these issues, 3D models should be prepared in a specific way (Grimaud and Cassen 201912). Then 3D data will be compatible with real-time visualization.

12

Grimaud V., Cassen S., 2019. Implementing a protocol for employing three-dimensional representations in archaeology (PETRA) for the documentation of neolithic funeral architecture in Western France, in Digital Applications in Archaeology and Cultural Heritage, vol. 13, june 2019.

(33)

32

Offering support for 3D models directly compatible from 3DHop could simplify the process between the online visualisation tool and end user applications.

Also adding more dedicated documentation and examples that specifically aim at 3D models presentation, museums, architecture and teaching to help users to start with Openspace3D.

Some developments about the conservation of some information such as translation and rotation, and annotation should be perfected in order to make a standard useable with other tools.

3.3. 3DHOP

3DHOP - 3D Heritage Online Presenter13 (http://vcg.isti.cnr.it/3dhop) is a software solution aimed at building advanced presentations of high-resolution 3D datasets on the Web (Figure 4).

Developed by CNR as a publishing framework, it has been expressly designed for the Cultural Heritage (CH) domain. In this specific application field, digital 3D models are nowadays widespread, both for “technical” use (documentation, restoration support, study, and measurement), as well as for dissemination, teaching, and presentation to the public. Enabling these uses require reality-based digitized geometries, essential to convey to the public the correct and qualitative information that lower-resolution hand-modeled 3D objects may not be able to provide. For this reason, the capability to efficiently stream and render high-resolution 3D models makes 3DHOP particularly suitable to the online presentation of CH artifacts.

In fact, the ability to work with gigantic 3D structures is one of the key features of 3DHOP. For doing this, the framework implements a cluster based view-dependent multiresolution

13

Potenziani M., Callieri M., Dellepiane M., Corsini M., Ponchio F., Scopigno R., 2015. 3DHOP: 3D Heritage Online Presenter, in Computers & Graphics, Volume 52, November 2015, Pages 129-141.

(34)

33 approach, Nexus 14 (http://vcg.isti.cnr.it/nexus), on top of the SpiderGL 15 library (http://vcg.isti.cnr.it/spidergl), obtaining very good performances, both in rendering and data transfer.

Thanks to Nexus, 3DHOP enables the interactive visualization of huge 3D meshes and point clouds (tens of million triangles/points), eventually equipped with per-vertex colors, normal vectors, and multiple textures, exploiting a compact representation in a compressed format.

However, efficient data handling is not the only characteristic that makes 3DHOP a valuable tool for the specific niche of CH users.

Indeed, exploiting intuitive interaction/manipulation paradigms and sophisticate linking schemes, 3DHOP simplifies the creation of interactive transmedia webpages in which 3D data are strongly connected with the other multimedia information typically composing a webpage.

14

Ponchio F., Dellepiane M., 2015. Fast decompression for web-based view-dependent 3D rendering, in

Web3D 2015, Proceedings of the 20th International Conference on 3D Web Technology, June 2015, Pages 199-207. https://doi.org/10.1145/2775292.2775308

15

Di Benedetto M., Ponchio F., Ganovelli F., Scopigno R., 2010. SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW, in Web3D 2010, Proceedings of the 15th Conference on 3D Web Technology, July 2010, Pages 165-174. https://doi.org/10.1145/1836049.1836075

(35)

34

Figure 4: A 3DHOP application developed by CNR-ISTI. In this example, a high-resolution 3D digitization (10 million triangles) of the Capsella Samagher, an antique reliquary, is presented with additional information (images and text). These information are linked to the 3D geometry through

geometric hotspots (light-blue regions).

Moreover, thanks to its modular nature and a declarative-like setup, it is easy to learn and may be configured and customized at several levels, making it accessible for people with different programming skills, without requiring solid knowledge in Computer Graphics (CG) programming.

Finally, 3DHOP also provide a full set of interactive tools, most of them expressly focused on CH applications. A representative selection of these tools includes:

● light tool: enables the control of the lighting on the virtual scene. In addition to the global lighting control, through this tool the user can also interactively change the position of a point light source, moving and placing it everywhere around the 3D scene;

● measurement tools: two different tools enable the interactive retrieval of metric information. Specifically, the point picker tool gives access to the 3D coordinates of

(36)

35 a specific point selected in the virtual scene, while the distance tool enables the calculation of the linear distance between two 3D points ();

● sectioning tool: enables the visualization of planar sections of the model on the scene. Three orthogonal sectioning planes are provided, the proper use of them not only give accurate references points (for measurement or other needs, for instance), but also allows to visualize the internal parts of the 3D objects composing the virtual scene;

● hotspot tool: enable the interaction with geometric points of interest placed on the 3D scene. This tool allows the user to setup geometric hotspots on the virtual scene. These clickable objects can be used to link additional information to a 3D model or specific parts of it.

3DHOP is based on the WebGL component of HTML5, i.e. the JavaScript standard API oriented to Computer Graphics programming on the Web. Thanks to this, 3DHOP works without the need of plugins on the most common platforms (Windows, MacOS, Linux), on the most modern browsers (Chrome, Firefox, Edge, Safari), and on all the available devices (personal computers, tablets, mobile phones).

3.3.2. Use Cases

3DHOP has been released as open source (GPL licence) package. The downloadable archive, with a series of tutorials, how-tos, examples, and other resources, are freely available on its website.

Since the release date (April 2014), the framework has been welcomed by the user community, as the numbers concerning the accesses to the official website and the the software downloads, can easily witness. To date, the 3DHOP website counts around 48,000 accesses (an average of 800 accesses per month), while in the same period the 3DHOP package has been downloaded more than 4,400 times (an average of 74 downloads per month).

From these numbers, it can be seen that 3DHOP is currently used in many real-world projects, able to cover a heterogeneous set of publication domains, needs, and purposes. A couple of these has been selected as representative use cases to be shown here.

(37)

36

The first case study presented16 is a publishing action commissioned by the Peggy Guggenheim Collection museum and focused towards dissemination purposes.

In the specific case, this contribution retraces the steps of a real campaign that, starting from a very peculiar artifact (the modern-art painting Alchemy, by Jackson Pollock), 3D digitized for conservative and diagnostic purposes, led to the design of a Web3D application (see Figure 5) used in a virtual exhibition, as well as museum kiosk in a real exhibition. This interactive application aimed at exploiting the high-resolution 3D model for explaining to the museum public the gestures, actions, and techniques characterizing Jackson Pollock's art.

In pursuing this purpose, the 3DHOP code has been extensively customized for obtaining a 3D viewer tailored to the most interesting aspects of the analysed artwork. By exploiting the modularity of the framework, this project showed how to use very few interaction tools to set up a minimal interface, suitable for not overloading the museum visitors with too many technicalities.

16

Callieri M., Pingi P., Potenziani M., Dellepiane M., Pavoni G., Lureau A., Scopigno R., 2015. Alchemy in 3D – A Digitization for a Journey Through Matter, in Proceedings of 2015 Digital Heritage International Congress, Volume 1, October 2015, Pages 223-231.

(38)

37 Figure 5: The Alchemy interactive 3D viewer developed using 3DHOP. Three different 2D interface elements are visible, superimposed to the 3D model: the main menu pane, the 2D mini-map, and the

informative component.

The resulting Alchemy interactive 3D viewer uses the 3DHOP hotspots to create interactive elements, while custom components were developed to ease the navigation (the mini-map, the modified trackball), and enhance the visualization (custom shaders). This use case also highlights another interesting 3DHOP feature. Indeed, exploiting the fact that 3DHOP is a client-side system, this project shows that is possible to develop a viewer both accessible on-site as a (museum) kiosk running on a local Internet server, and on the Internet as a classic Web3D application running on a remote Internet server, with almost no changes to the code.

Both Internet and kiosk applications have been exploited in the Guggenheim temporary exhibition “ALCHEMY BY JACKSON POLLOCK. Discovering the Artist at Work”, running in Venice (Italy) from February to September 2015, totaling more than 180,000 visitors. Later on, the kiosk was used in a similar initiative at the Guggenheim Museum in New York (USA), while the Web3D app is still accessible today (http://vcg.isti.cnr.it/alchemy).

(39)

38

Apparently, similar to this case study, the second contribution presented17 is actually very different in the publishing purpose. Indeed, even though both the proposed use cases explore the possibilities of a specialized Web3D content creation, the first one proposes an example of publishing aimed at supporting dissemination to the public, while the second one has been developed for supporting scientific analysis.

More specifically, this contribution introduces a publishing action aimed at a technical data presentation. In this case, the main goal was to build a tool supporting experts in their work of study and documentation of an archaeological find (the “Annona” Roman marble sarcophagi, II-IV century AD).

Figure 6: The Annona Sarcophagus Web3D platform developed using 3DHOP. Detail of the specialized panel enabling the visualization of the selected areas where the VIL specialistic analysis

were performed.

The project exploits all the data produced during the campaign of study on this artefact, integrating them with a high-resolution 3D digitization of the same, so as to support enhanced analysis, comparisons, and even a digital 3D polychrome reconstruction by virtual painting.

17

Siotto E., Palma G., Potenziani M., Scopigno R., 2015. Digital Study and Web-based Documentation of the Colour and Gilding on Ancient Marble Artworks, in Proceedings of 2015 Digital Heritage International Congress, Volume 1, October 2015, Pages 239-246.

(40)

39 Finally, all those data are made accessible on the Internet, using the 3DHOP framework to design and develop a visual media platform (Figure 6) aimed to provide domain's practitioners with a new interactive 3D assessment tool.

Through a substantial customization of the 3DHOP original user interface, the resulting Internet viewer is able to exploit the 3D models as main channel to manage and display all the achieved scientific analysis and historical information about the artefacts, integrated into a single reference system.

3DHOP can be considered a fully qualified solution. Since the beginning it has been well accepted by the community, and nowadays it is largely used by many universities, important institutions, and freelancer Web3D content creators.18

Despite being an established tool, 3DHOP is still an ongoing project: it has significantly grown since the first release and is still growing.

Since it is a modular framework, many features and components were added over time, and many others will be added in the future for supporting the creation of more flexible and effective interactive visualizations. The enhancements and extensions on the to-do list include:

● Novel navigation and visualization components (new trackball types, new scene manipulation functions, configurable shaders to be selectively attached to each instance);

● Dynamic definition of the 3D scenes (novel methods for dynamically adding/removing entities, or for modifying their transformations and properties); ● Authoring services (visual editor or wizard tools to make easier the setup of a

visualization scheme).

18

Potenziani M., Callieri M., Scopigno R., 2018. Developing and Maintaining a Web 3D Viewer for the CH Community: An Evaluation of the 3DHOP Framework, in Proceedings of 16th Eurographics Workshop on Graphics and Cultural Heritage, Volume 1, November 2018, Pages 169-178.

(41)

40

The PARTHENOS workshop on the uses of 3D data in Humanities and Social Sciences has been a good opportunity to submit the 3DHOP to a selected and qualified audience, gathering impressions and suggestions about the framework and its future development. Analysing the participants feedback, there particularly emerges a general request for focusing the effort on the 3D data enrichment through an improved annotations management (“...it’s important to put the same level the 3D object with the archaeological

context...”, “...a more intuitive annotation system would definitely be of benefit...”, “...in the

future, I think that the annotations and the sharing of metadata could be developed further...”), somehow confirming the need/willingness to converge towards authoring helpers, possibly implemented as a service.

Again on annotations, remarks concern also their standardization (“...to develop an

annotation standard for 3D models just as TEI was developed for annotating text documents in XML format...”), and a goal to possibly join forces with other platforms presented during the workshop (“...I would love to see is that Aioli and 3DHOP join forces

in how annotations are stored, in order to get to a more standardised way of annotating 3D models...”).

Finally, minor indications involve the 3D data pre-processing (“...what I found difficult is,

sometimes, to create and compress texturized models with NEXUS...”), the opportunity to rely on coding structures that are more Internet-based (“...maybe for the future it would be

great to avoid the coding for the user and to use some tag representation like XML...”), and the possibility to manage different textures on the same geometry.

However, generally speaking, once again the 3DHOP framework has been very well received, with much positive feedback witnessing the participants global satisfaction:

● “...great tool with very wide possibilities that can be used each in a customized way [...] very smart...”

● “...this tool is a nice solution to share our 3D acquisition...”

● “...it is a very powerful tool which allows users to adapt it for very different and specific applications...”

● “the tool is very well designed and I have already used it under several occasions [...] the documentation is very clear and even a user with a very basic knowledge of HTML can set up and run a model in minutes...”

(42)

41 ● “...I've been using 3DHop since several years and for different dissemination projects [...] in my opinion, one of its greatest strength is its versatility and customization...”

● “...3DHOP has proven its potential and is widely used in heritage for both student, as well as professional projects [...] it is great that its developers are committed to its sustainability and user-friendliness...”

● “...great tool, easy to use, with very wide possibilities that 3DHOP can be used like solution for publishing 3D models of archaeological objects...”

● “...I really liked 3DHOP, I think that it has a huge potential and, the most important, not only on a CH context...”

● “...3DHOP is fantastic tool, and in my mind is the go-to tool for sustainable 3D visualisation...”.

3.4. Visual Media Service

The Visual Media Service19 (https://visual.ariadne-infrastructure.eu) offers a very easy solution for publishing online different visual media and for visualizing them under a common framework (Figure 7).

Developed by CNR in the framework of the EC projects ARIADNE and PARTHENOS, it is implemented on top of the 3DHOP (http://vcg.isti.cnr.it/3dhop) and Relight (http://vcg.isti.cnr.it/relight) technologies. It was designed as a service-oriented platform for assisting the deployment of Internet presentations pivoted on three complex visual media assets: high-resolution 2D images, RTI, and 3D models.

These types of media are still not very common on the Internet, because their visualization may require the download of the entire file (long transmission times discourages the users), and possibly having to select/install dedicated software (viewers).

19

Ponchio F., Potenziani M., Dellepiane M., Callieri M., Scopigno R., 2016. ARIADNE Visual Media Service: Easy Web Publishing of Advanced Visual Media, in Proceedings of the 43rd Annual International Conference on Computer Applications and Quantitative Methods in Archaeology, Volume 1, March 2016, Pages 433-442.

(43)

42

The Visual Media Service approach is to build an automatic service able to transform these media into progressive formats that will allow easy and efficient access and visualization on the Internet.

Figure 7: The Visual Media Service Internet site landing page. Accessing the service, the user finds a simple Internet form that allows to upload its own data (3D models, hi-res 2D images, or RTIs) and to

provide some basic information about the published media.

The efficiency of the final presentations is gained by endorsing these efficient data representation schemes, but also by the design of proper GUI (viewers). With this setup, even unskilled users can easily create an efficient webpage to display complex 2D or 3D content.

The service is based on a simple Internet interface that allows uploading the visual media file of interest and (after some processing) to receive the URL where the specific data has been stored and could be accessed for the visualization.

Alternatively, for more experienced users, these basic webpages may be the starting point for the development of more complex visualization, or for the integration of this