Publisher’s version / Version de l'éditeur: GIM International, 2009-12-31
READ THESE TERMS AND CONDITIONS CAREFULLY BEFORE USING THIS WEBSITE.
https://nrc-publications.canada.ca/eng/copyright
Vous avez des questions? Nous pouvons vous aider. Pour communiquer directement avec un auteur, consultez la
première page de la revue dans laquelle son article a été publié afin de trouver ses coordonnées. Si vous n’arrivez pas à les repérer, communiquez avec nous à PublicationsArchive-ArchivesPublications@nrc-cnrc.gc.ca.
Questions? Contact the NRC Publications Archive team at
PublicationsArchive-ArchivesPublications@nrc-cnrc.gc.ca. If you wish to email the authors directly, please see the first page of the publication for their contact information.
NRC Publications Archive
Archives des publications du CNRC
This publication could be one of several versions: author’s original, accepted manuscript or the publisher’s version. / La version de cette publication peut être l’une des suivantes : la version prépublication de l’auteur, la version acceptée du manuscrit ou la version de l’éditeur.
Access and use of this website and the material on it are subject to the Terms and Conditions set forth at
A framework for the processing, visualization and analysis of large
2D/3D datasets: modeling the Erechtheion
Beraldin, J. Angelo
https://publications-cnrc.canada.ca/fra/droits
L’accès à ce site Web et l’utilisation de son contenu sont assujettis aux conditions présentées dans le site LISEZ CES CONDITIONS ATTENTIVEMENT AVANT D’UTILISER CE SITE WEB.
NRC Publications Record / Notice d'Archives des publications de CNRC: https://nrc-publications.canada.ca/eng/view/object/?id=042dd8f3-4e96-4636-8b24-71c7ba1c8038 https://publications-cnrc.canada.ca/fra/voir/objet/?id=042dd8f3-4e96-4636-8b24-71c7ba1c8038
Author:
J-Angelo Beraldin
A framework for the processing, visualization and analysis of large 2D/3D datasets: modeling the Erechtheion.
NRC R&D team members in alphabetical order:
Blais, F.; Borgeat, L.; Cournoyer, L.; El-Hakim, S., Godin, G.; Massicotte, P.; Picard, M.; Poirier, G. Photography: Dan Gamache
[Introduction]
Modern terrestrial laser scanners (TLS) generate between 10 and 50 million 3D coordinates per minute. This corresponds roughly to about 500 Mbytes of data in the same lap of time. When combined to the acquisition of high resolution texture images, three-dimensional modeling projects lasting several hours generate enough information to choke high-end PCs running commercial software packages. We describe a framework for the processing, visualization and analysis of such very large 2D/3D datasets. The recently developed framework is applied to the highly detailed and complex monument located on the Acropolis in Athens (next to the Parthenon) and is known as the Erechtheion. The fully processed model contains 348 million polygons.
Figure 1. Several views of the 3D model shown with the processed laser intensity mapped onto the model.
[3D imaging and modeling: a key enabling technology]
In recent years, sensors and algorithms for three-dimensional (3D) imaging and modeling of real objects, structures and sites have received significant attention, not only in a number of research fields, but also increasingly as an important tool for a variety of commercial applications. The ability to efficiently create an accurate digital representation of the shape (e.g. envelop) and appearance (e.g. color texture) of an existing object is a key enabling technology in many fields, e.g. automotive industry, space exploration, medical, manufacturing, forensics, and construction to name a few. In this article, we describe the results of a research and development (R&D) project where a famous heritage site located in Greece, the Erechtheion, is modeled in 3D to a spatial resolution of 2 mm. The fully coloured model contains 348 million polygons. We can interactively visualize and analyze this 3D model in real-time.
[The monument]
The restoration of the Erechtheion (aka Erechtheum) was completed in 1987. The monument is made mostly of marble but artificial and natural stones were also used. It remains impressive in
spite of the fact that it is only a remnant of what it used to be when it was completed in 406 B.C.E. The Erechtheion is a large complex structure (about 20 m × 10 m × 5 m) with many challenges to 3D data acquisition, appearance processing, and interactive visualization. On the north side, there is a large porch with columns. On the south side, there is the famous "Porch of the Maidens" composed of six draped female figures (aka Caryatids). The Caryatids are cast copies made from the six original ones (five are displayed at the Acropolis Museum and one at the British Museum). The capitals and entablature are composed of small decorations. The surface is marked by a number of cuts, gunshot damage and impact from cannon balls.
A) B) C)
Figure 2. Surface markings visible on the 3D model: A) Graffiti of crosses, B) Cannon ball impact, C) Damage due to gunfire. Renderings were created from the coloured 3D model; source of information Lesk, 2004.
[NRC and cultural heritage]
The cultural heritage sectorhas been of one of the National Research Council of Canada’s (NRC) key area for its R&D program in 3D technologies. The application of NRC’s 3D technologies to the cultural heritage sector creates stringent requirements on the underlying technologies in terms of accuracy, performance and usability. Through a number of demonstration projects, each of which posing specific new challenges, technologies were developed, tested and refined, and served to test and expand innovative solutions. These projects cover a wide range of application categories and contexts. They include archaeological and historical sites such as a decorated Neolithic cave “grotta dei Cervi,” a Byzantine crypt, 3D modelling of castles, and paintings such as the Mona Lisa. Many of the technologies developed were adapted specifically for commercial applications such as NASA’s Space Shuttle inspection laser scanner and metrology software used in major corporations. These are only a few of the successful accomplishments.
[2D/3D data acquisition of the Erechtheion]
This cultural heritage project required the creation of a richly detailed accurate 3D model of the Erechtheion. The resolution and visual quality of the rendered model should ideally match what is perceptible by the human eye on a physical visit, preferably when at close proximity. For interactive visualization, the 3D model must be viewable on a standard workstation.
The size, setting and the monument surface created several problems. The height of the Erechtheion made coverage from ground level difficult on top parts. Some problems were due to obstructions and the terrain, which caused delays and missed areas. Some parts with shape
complexity caused self-occlusions, and impediments from plants/trees created holes in the coverage. Deciding on the next best view was a time consuming step that could only be tackled by proper planning before the actual work. A preliminary photographic campaign helped the team plan the locations for 3D scanning and to calculate the time necessary for the on-site work.
A phase-based time-of-flight (TOF) mid-range terrestrial laser scanner (Surphaser 25HSX) for most parts (1-5 mm lateral data spacing at a range of 5 m) and a long-range pulse-based TOF laser scanner (HDS3000) for sections (mainly the top) unreachable by the first scanner were used. Once the spatial resolution (lateral and depth) was selected from the range for best laser beam foot print and the knowledge of the dimensions of the small bas-relief sculptures, a study of material properties effects on 3D image data quality was conducted. It was found that time-of-flight based laser scanners produce an apparent laser penetration of about 6 mm on marble. The mechanism behind this phenomenon is different from what we found 10 years ago with triangulation–based systems (see Digital Michelangelo project at: http://www.nrc-cnrc.gc.ca/eng/ibp/iit/about/digital-michaelangelo.html#9). Work is underway to further study and correct this apparent penetration.
Figure 3. Views of the Erechtheion with team members and 3D terrestrial laser scanner.
Photo-realism, defined as having no difference between the view rendered from the model and a photograph taken from the same viewpoint, goes much further than simply draping static imagery over geometry. For instance, photographic images need to be processed for varying illumination before being mapped onto the 3D model. During the five campaigns in Athens, we not only acquired 3D images but also high-resolution images for texturing purposes.
[Modeling with large 2D/3D datasets]
Within the Erechtheion project, NRC developed efficient and robust algorithms for 3D image processing, management and real-time visualization of multi Giga-triangle meshed models of large and complex sites. These algorithms are part of Atelier3D.ca, a general framework developed for the acquisition, modeling, visualization, and analysis of very large, multi-scale 3D datasets built from data gathered with 3D and color sensors. This suite of tools is compatible with Innovmetric ModelerTM. The use of a commercial package allowed speeding up the development of the tools
and helped us streamline operations that didn’t need reimplementation. Atelier3D.ca emphasizes high automation, interactivity across the entire processing pipeline, maximum scalability, and measurement accuracy. It facilitates or fully automates many of the tedious operations necessary
in the typical modeling pipeline (e.g. registration of the multiple scans, processing the photographs before texture draping, accurate registration of texture images with the geometric model, interactive visualization of the full resolution 3D model).
Processing and modeling of large 2D/3D datasets:
This part of the framework aims at providing a set of tools to model large datasets acquired using 2D and 3D sensors. The main challenge we tackled is the enormous size of the datasets that current sensor can produce: billions of 3D samples and hundred of gigabytes of digital photographs for single objects or scenes. To achieve this, we had to rethink existing algorithms so that they work at this new scale. One key aspect was to automate the different steps of the process as much as possible, since any form of human interaction becomes extremely costly and time-consuming for such large datasets.
Specifically, we improved on 2D and 3D alignment, data integration, and correction of errors and deformation due to intrinsic scanner characteristics. One key recent advance is our work on automatic high precision alignment of 2D and 3D data, where the pose, camera parameters, and residual deformation linking a 2D photograph and a 3D model are computed and optimized automatically whenever possible, and a very efficient user interface allows an expert operator to complete the work in minimal time. This is an important development as this is one of the most difficult and time consuming activities for a human operator to perform when processing sensor data for texture mapping purposes.
Interactive Visualization and display of large 2D/3D datasets:
For visualization, we have developed in the recent past and are currently improving algorithms capable of displaying those datasets interactively at full resolution and quality on inexpensive laptops or desktop computers. We are now improving the processing steps of those technologies to make them entirely automated for use by non-experts. We also have worked to overcome the resolution limitations of current projectors by implementing advanced stereoscopic multi-projector display technologies capable of showing the full resolution of our 3D models.
The ability to interact with 3D models is a continuing problem due to the fact that the demand for detailed models is growing at a faster rate than computer hardware advances. The rendering algorithms have to be capable of delivering images at real-time frame rates of at least 20 frames-per-second even at the full resolution of both geometry and texture. The NRC-developed Atelier 3D.ca system, a view-dependent real-time system for multi-resolution models was used for the Erechtheion project. For instance, when the 3D model is viewed at close range, the full resolution is shown. As the model is moved away from the observer, the resolution is decreased smoothly so no artefacts are visible. This part of the software is based on the Geomorphing of Levels of Detail (GoLD) system.
Analysis of large 2D/3D datasets:
We have built on our work on processing and visualization for large datasets by adding an analysis layer over the visualization one. Indeed, the complexity of large meshes built from measurements does not only result from their typically huge size. High resolution and accurate 3D scanning provides information not only about overall shape and color, but also of fine surface details and
variations that need to be accessed within a larger context and can be hidden within global surface features. We therefore seek to do more than only displaying it in a photorealistic manner, but also transforming it in order to enhance its understanding by the viewer and therefore maximize the value of the dataset. In this project, we implemented numerous real-time data transformation techniques on the GPU that enabled real-time extraction of all the information available in the models. The following figures show different such representations computed in real-time for a 348 millions polygons representation of the Monument.
[Results]
A) B)
Figure 4. 3D image sampling and filtering. A) Lateral resolution at 2 mm versus a 20 mm sampling. B) Shape preserving filtering performed in real-time on GPU using the 2 mm resolution model.
A)
B)
C)
Figure 5. A section of the Erechtheion showing a view of the porch of the maidens. A) The 3D model is displayed using a rendering based on synthetic shading. B) Same section shown with the corrected laser intensity. C) Colour texturing. These renderings were created from the 348-million polygon model of the monument using Atelier3D.ca.
[Acknowledgement]
This work is part of the project “Development of Geographic Information Systems at the Acropolis
of Athens,“ financed by the European Union and the Government of Greece, and supervised by the
Acropolis Restoration Service, Hellenic Ministry of Culture. The partners in this project are Elliniki Photogrammetriki Ltd (Elpho), Athens; Geotech O.E., Athens; ETH (Swiss Federal Institute of Technology), Zurich; National Research Council of Canada, Canada; Institute for Mediterranean Studies, Foundation for Research & Technology (FORTH), Rethymno, Crete; with external co-operators, Leica Geosystems, Switzerland and Basis Software Inc., USA. Special thanks to Innovmetric Software Inc.
[References]
Beraldin, J.-A., Blais, F., and Lohr, U. 2010, Laser Scanning Technology. Chapter 1 of Airborne and Terrestrial Laser Scanning (Vosselman and Maas, eds.), Whittles Publishing, UK.
Borgeat, L.; Godin, G.; Massicotte, P.; Poirier, G.; Blais, F.; Beraldin, J.-A., Visualizing and Analyzing the Mona Lisa, IEEE Computer Graphics and Applications, vol. 27, no. 6, 2007. pp. 60-68, NRC 49834 Borgeat, L., Godin, G., Blais, F., Massicotte, P., and Lahanier, C. GoLD: Interactive Display of Huge Colored and Textured Models. ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH 2005). 31 July - 4 August, 2005. Los Angeles, USA.
Borgeat, L., Poirier, G., Beraldin, J.-A., Godin, G., Massicotte, P., Picard, M., 2009, A framework for the registration of color images with 3D models, ICIP 2009, Cairo, Egypt, 7th – 11th Nov
2009.
Godin, G., Massicotte, P., Borgeat, L. 2006, High-Resolution Insets in Projector-Based Display: Principle and Techniques, Stereoscopic Display and Applications XVII Conf. Part of: IS&T/SPIE 18th Annual Symp. On Electronic Imaging Science and Technology. San Jose, California, USA. January 15-19.
Godin, G., Rioux, M., Beraldin, J.-A., Levoy, M., Cournoyer, L., Blais, F., 2001. An assessment of laser range measurement on marble surfaces, In 5th Conf. Optical 3D Measurement Techn., 1-4 Oct, Vienna, pp. 49-56.
El-Hakim, S., and Beraldin, J.-A., 2007, Sensor Integration and Visualization. Chapter 10 of Applications of 3D Measurement from Images (Fryer, Mitchell & Chandler, eds.), Whittles Publishing, UK, pp.259-298.
El-Hakim, S., Beraldin, J.-A., Picard, M., Cournoyer, L. 2008. Surface reconstruction of large complex structures from mixed range data – The Erechtheion experience. ISPRS XXI Congress, 3-11 July, Beijing, China, pp. 1077-1082.
Moullou, D., et. al, 2008, Recording, Modeling, Visualisation and GIS Applications Development for the Acropolis of Athens. ISPRS XXI Congress, 3-11 July, Beijing, China, pp.1073-1076.
Accessed 02-2010, http://www.isprs.org/proceedings/XXXVII/congress/5_pdf/184.pdf
Fani Mallouchou-Tufano, 1987, The restoration of the Erechtheion: 20 years later, Newslatter 7, Ed.Prof. Emeritus Ch. Bouras, YSMA-The Acropolis Restoration Service, Athens, pp.35-39. Accessed 02-2010, http://www.ysma.gr/En/pdf/Newsletter_7_eng.pdf
Alexandra L. Lesk, 2004, A diachronic examination of the Erechtheion and its reception, PhD, McMicken College of Arts and Sciences and the Department of Classics of the University of Cincinnati, Accessed 02-2010,http://erechtheion.org/index_003.htm
