Report
Reference
Semantic Enrichment of Scene Graphs for Task Based Adaptation in 3D User Interfaces: A Proposal
FALQUET, Gilles, MOCCOZET, Laurent, BAZARGAN, Kaveh & CUI - Centre Universitaire d'Informatique, Université de Genève
Abstract
There are currently no well-established techniques or tools to build adaptive 3D user interfaces. The aim of this research is to overcome identified shortcomings by providing a model and framework for the semantic enrichment of scene graphs for task based adaptation in 3D user interfaces. The semantic enrichment of scenes can play an extremely important role in enabling the viewers to understand and interact with the usually complex and incomprehensible visualized information, in simple, intuitive and user-friendly ways. The goal of our research is to develop a general model and framework supporting the adaptations of 3D user interfaces. The general model and framework relies on the formalization of the vertical correspondence between two layers: the spatial layer (the graphics) and the semantic layer (the concepts).
FALQUET, Gilles, MOCCOZET, Laurent, BAZARGAN, Kaveh & CUI - Centre Universitaire d'Informatique, Université de Genève. Semantic Enrichment of Scene Graphs for Task Based Adaptation in 3D User Interfaces: A Proposal . Geneva : CUI - Centre Universitaire d'Informatique, Université de Genève, 2008, 7 p.
Available at:
http://archive-ouverte.unige.ch/unige:46617
Disclaimer: layout of this document may differ from the published version.
1 / 1
Semantic Enrichment of Scene Graphs for Task Based Adaptation in 3D User Inter- faces: A Proposal
Gilles Falquet, Laurent Moccozet, Kaveh Bazargan
Centre Universitaire dInformatique (CUI)
University of Geneva Battelle Bat. A 7, route de Drize CH-1227 Carouge Switzerland
SMV technical report series, N
o202, February 2008
Centre Universitaire d’Informatique Battelle Bat. A
Route de Drize 7 1227 Carouge
Switzerland
Semantic Enrichment of Scene Graphs for Task Based Adaptation in 3D User Interfaces:
A Proposal
Contents
Software Modeling and Verification Department of Computer Science University of Geneva
Battelle Bat. A Route de Drize 7 CH-1227 Carouge Switzerland
tel: +41 22 37 90 161 fax: +41 22 37 90 250 http:/smv.unige.ch/
Corresponding author:
G. Falquet
1
Semantic Enrichment of Scene Graphs for Task Based Adaptation in 3D User Interfaces: A Proposal
Gilles Falquet, Laurent Moccozet, Kaveh Bazargan Centre Universitaire d’Informatique (CUI), University of Geneva Battelle Bat. A, 7, route de Drize, CH-1227 Carouge, Switzerland {gilles.falquet, laurent.moccozet, kaveh.bazargan}@cui.unige.ch
ABSTRACT
There are currently no well-established techniques or tools to build adaptive 3D user interfaces. The aim of this research is to overcome identified shortcomings by providing a model and framework for the semantic enrichment of scene graphs for task based adaptation in 3D user interfaces. The semantic enrichment of scenes can play an extremely important role in enabling the viewers to understand and interact with the usually complex and incomprehensible visualized information, in simple, intuitive and user-friendly ways. The goal of our research is to develop a general model and framework supporting the adaptations of 3D user interfaces. The general model and framework relies on the formalization of the vertical correspondence between two layers: the spatial layer (the graphics) and the semantic layer (the concepts).
Author Keywords
3D scene, semantic model, adaptive user interface, adaptation model
ACM Classification Keywords
H.5.1 [Information Interfaces and presentation] Multimedia Information Systems: Artificial, augmented, and virtual realities; H.5.2. [User Interfaces]: User-centered design INTRODUCTION AND MOTIVATION
Scene graphs have become an established tool for developing interactive 3D applications. They offer an object-oriented and structured approach to describing the application’s graphical needs and interactions with the 3D presentation. According to Reitmayr and Schmalstieg [1], the original idea of the scene graph is to allow for rapid and
flexible implementation of 3D user interfaces.
Leisser et al. affirm that in a visual application, two layers can be identified as outline structure: a semantic layer and a visualization layer [2]. The semantic layer describes what is to be visualized, namely, the relation between concepts within a knowledge base, for example, as a domain ontology or a concept map. The visualization layer describes what is visible on the screen.
In [3], a generalized model and methodology to incorporate domain knowledge into 3D scenes in web based environments is presented. The related methodology raises the description of graphics and virtual reality content to the ontologies layer of the semantic web offering new interesting techniques to manipulate and query the scenes.
Informally, the task is a collection of activities in a domain, with a common goal. Task-aware interfaces represent, in machine-readable form, the knowledge about: what the task is, how it is represented in the interface, how it is decomposed into actions, and how these actions correspond to the applications functionality.
RELATED WORK
According to Chittaro and Ranon, augmenting virtual environments with adaptive capabilities could greatly increase their usability and effectiveness [4,5,6]. Li and Hsu propose an intelligent 3D UI adapting to user control behaviors is presented [7]. They propose two mechanisms to make the intelligent 3D user interface with virtual forces adaptive to users’ control behaviours in order to further improve the navigation performance. Santos and Osorio propose an intelligent, adaptable, three-dimensional and virtual environment that explores the resources of Virtual Reality, seeking to increase the interactivity degree between the users and the environment [8].
Bilasco et al. present an open framework supporting rule- based adaptations of 3D scenes is used [9]. The main role of the adaptation framework is to arrange the adaptation process following the adaptation strategy materialized by adaptation rules that come with the scene request. The adaptation framework relies on the 3DAF (3D Annotation Framework) that handles the identification of objects matching the rule criterion, and external engines that either adapt individual objects or regions [10].
Di Giacomo et al. have contributed to the assisted extraction of semantic and knowledge information from 3D spatial information [11, 12]. Garcia-Rojas et al. have contributed to the development of a global semantic representation connecting 3D spatial information with domain knowledge [13, 14, 15]. Moccozet et al. propose scenarios that demonstrate the benefit that can be expected from this type of representations for storing, retrieving and querying 3D models [16].
Falquet and Métral have presented a global semantic model to integrate non-geometric urban knowledge and data into 3D city models [17]. In addition to this semantic integration, the authors have defined a declarative specification language that enables the designer, or the expert user, to specify 3D representations for non-geometric knowledge elements and to include them into the 3D scenes that represent 3D city models. This knowledge integration can take the form of semantic navigation links between 3D elements, thus accelerating the access operations within 3D scene. Métral et al. we have achieved the semantic integration of urban knowledge for urban planning communication [18]. The authors have defined an ontology- based model whose main characteristic is to the semantic integration in a knowledge base of the urban knowledge coming from various sources such as geographical information systems databases, master plans, local plans and any other document.
PROPOSAL FOR A GENERAL MODEL AND FRAMEWORK
The goal of our research is to develop a general model and framework supporting the adaptations of 3D UI. The general model and framework relies on the formalization of the vertical correspondence between two layers: the spatial layer (the graphics) and the semantic layer (the concepts).
The general model and framework relies also on the formalization of the horizontal correspondence between three main components: the static component, the adaptation component and the dynamic component (Figure 1).
Figure 1. Overview of the general model and framework
The main role of the general model and framework is to arrange an automatic adaptation process in the 3D UI following the adaptation model materialized by adaptation rules. The research steps are essentially composed of defining and building the correspondence models between and within each layer. The two final steps consist in first, the implementation of the proposed general model and framework, and then, experiments with the model in form of case studies and usability tests with the model.
EXPECTED CONTRIBUTIONS
One of the originality of this research, compared to the state of the art, is to focus on the dynamic aspect of the 3D scene. If the 3D scene is intended to convey knowledge about processes, procedures, actions, tasks, etc. of a particular domain (e.g. how to remove a part of an engine, how does a mechanical adding machine works, ...) then the scene must be coupled with animation and interaction features. This dynamic aspect of a scene must be represented both at the scene and at the conceptual levels.
At the scene level, interaction and animation are often represented through the notions of sensor (to detect user actions), events and event propagation routes, and actions that transform the attributes of some scene components (position, orientation, size, colour, texture, etc.). At the conceptual level the objective is to model the domain processes through their decomposition into subprocesses, the synchronization of these sub-processes, and their actions on the domain objects. Formalisms such as Petri Nets, state charts, etc. are adequate to represent domain processes.
The tasks we intend to model belong to two categories:
- domain tasks or processes (e.g. show how an engine functions),
- visualization tasks: these are tasks that have no counterpart in the domain but that help the user visualize and understand the 3D scene, and hence the domain of interest. Typical visualization tasks are: making an object transparent or moving objects around to enhance the visibility of some part of the scene, move the user's point of view (the camera) to an appropriate location, etc.
CONCLUSION
In this paper we have presented an overview our proposed general model and framework relies on the formalization of the vertical correspondence between two layers: the spatial layer (the graphics) and the semantic layer (the concepts).
Our future work consists in: (a) connecting a conceptual model to a scene graph model (static), (b) connecting a conceptual task model and a scene graph animation model, (c) defining and building the adaptation model, (d) developing a design and implementation framework and finally (e) testing the usability of the model.
3 REFERENCES
1. Reitmayr, G. and Schmalstieg, D. Flexible Parametrization of Scene Graphs. In Proceedings of the 2005 IEEE Conference 2005 on Virtual Reality. VR.
IEEE Computer Society, Washington, DC, 51-58.
2. Leissler, M., Müller, A., Hemmje, M. and Neuhold, E.
A VR Component Architecture for Visual Multi-User Applications. In: ISCA 2nd International Conference on Information Reuse and Integration (IRI-2000) Honolulu, Hawaii, USA, November 2000.
3. Kalogerakis, E., Christodoulakis, S., and Moumoutzis, N. Coupling Ontologies with Graphics Content for Knowledge Driven Visualization. In Proceedings of the IEEE Virtual Reality Conference (VR 2006) – Volume 00 VR. IEEE Computer Society, Washington, DC, 6.
4. Chittaro and L., Ranon, R. Adaptive 3D Web Sites, In Brusilovsky, P., Kobsa, A., Nejdl, W. (eds.), The Adaptive Web – Methods and Strategies of Web Personalization, Lecture Notes in Computer Science Vol. 4321, Springer-Verlag, Berlin, June 2007, pp.
433-464.
5. Chittaro, L. and Ranon, R. Adaptive Hypermedia Techniques for 3D Educational Virtual Environments, IEEE Intelligent Systems, July/August 2007, Vol. 22, No. 4, pp. 31-37.
6. Chittaro, L., Ranon R., Web3D Technologies in Learning, Education and Training: Motivations, Issues, Opportunities, Computers & Education Journal, August 2007, Vol. 49, No. 1, pp. 3-18.
7. Li, T. and Hsu, S. 2004. An intelligent 3D user interface adapting to user control behaviors. In Proceedings of the 9th international Conference on intelligent User interfaces. IUI ’04. ACM Press, New York, NY, pp. 184-190.
8. Trojahn dos Santos, C. and Osorio, F. S. An Intelligent and Adaptive Virtual Environment and its Application in Distance Learning. In Proceedings of the 6th International Conference on Advanced Visual Interfaces (AVI 2004), Gallipoly, Italy. ACM Press (2004) 362-365.
9. Bilasco, I. M., Villanova-Oliver, M., Gensel, J., and Martin, H. Semantic-based rules for 3D scene adaptation. In Proceedings of the Twelfth international Conference on 3D Web Technology. Web3D '07. ACM Press, New York, NY, 97-100.
10. Bilasco, I. M., Gensel, J., Villanova-Oliver, M., and Martin, H. An MPEG-7 framework enhancing the reuse of 3D models. In Proceedings of the Eleventh international Conference on 3D Web Technology.
Web3D '06. ACM Press, New York, NY, 65-74.
11. Di Giacomo, T., Kim, H., Moccozet, L. and Magnenat- Thalmann, N. Control Structure and Multi-Resolution Techniques for Virtual Human Representation. In book Shape Analysis and Structuring, edited by L. De Floriani & M. Spagunolo, published by Springer Verlag, 2007.
12. Di Giacomo, T., Moccozet, L., Magnenat-Thalmann, N., Boulic, R. and Thalmann, D. Towards Automatic Character Skeletonization and Interactive Skin Deformation. Eurographics, EG'07 State-of-the-Art Reports. October 2007.
13. Gutiérrez, M., García-Rojas, A., Thalmann, D., Moccozet, L., Magnenat-Thalmann, N., Mortara, M.
and Spagnuolo, M. An ontology of virtual humans:
incorporating semantics into human shapes. The Visual Computer, vol.23 (3) pp.207-218, 2007.
14. Garcia-Rojas, A., Vexo, F., Thalmann, D., Raouzaiou, A., Karpouzis, K., Kollias, S., Moccozet, L. and Magnenat-Thalmann, N. Emotional face expression profiles supported by virtual human ontology.
Computer Animation and Virtual Worlds, vol.17 (3-4) pp.259-269, 2006.
15. Garcıa-Rojas, A., Thalmann, D., Vexo, F., Moccozet, L., Magnenat-Thalmann, N., Mortara, M. and Spagnuolo, M. An Ontology of Virtual Humans:
incorporating semantics into human shapes. In Proceedings of the IEEE Workshop on the Integration of Knowledge, Semantic and Digital Media Technologies, pp. 7-14, Nov. 2005.
16. Moccozet, L., Garcia-Rojas, A., Vexo, F., Thalmann, D. and Magnenat-Thalmann, N. 2006. In Search for Your Own Virtual Individual. In: Semantic Multimedia, Springer Verlag pp.26-40, 2006.
17. Falquet, G. and Métral, C. 2005. Integrating Urban Knowledge into 3D City Models. In Proc. First International Workshop on Next Generation 3D City Models, Bonn, Germany, June 2005.
18. Métral, C., Falquet, G. and Vonlanthen, M. 2007. An Ontology-based Model for Urban Planning Communication. In Ontologies for Urban Development, Studies in Computational Intelligence, Volume 61, Springer, Berlin, 2007, pp 61-72.
