E xperiments and R esults
6.3 Limitations and Future research
The use-case scenarios presented here were conducted in localized areas and tested with up to 30 end-clients. Albeit the presented architecture took into consideration the ability to scale to a multi-fold of active end-clients it was never applied. Mainly due to the issues already present at the small scale deployment deemed to be more challenging than anticipated. It is therefore for future research on how to extend this to a greater area, while retaining all the same capabilities. In some sense the issue can be solved by increasing the amount of server hardware to accommodate the increase of end-clients. This however is a poor solution as it only solves the rendering and simulation demands, but the handling of user events and interaction within the VE will increase significantly.
Improvements can be achieved in having a more dedicated streaming processing software. As currently direct frames are grabbed from a buffer on the graphics card, depending on the ren-dering engine used this can be done in several ways. The fastestimage grabmethodology was to render to a texture buffer and have this then copied to a local buffer. Possibly faster methods can include dedicated hardware capable of capturing and direct compressing into a specific format.
Albeit this was not the focus of the framework, utilizing the jpeg and VP8 codecs mainly in software mode did limit the frame-rate. Later updates to the VP8 codec offered some hardware acceleration, but there are faster compression codecs available (mostly commercial).
The adaptive user interface integration proofed to be difficult and is currently the biggest limita-tion. Using several methods throughout the application scenarios, the embedded interface into the 3D scene shows the easiest solution. However this requires that the interface is displayed within the rendering context (OpenGL for example). This is no problem when using Qt and building an application from ground up. But the dependency on using a library that is capable of rendering the interface in the 3D scene can be interpreted as too strict. Other solutions might be found in the use of HTML 5 and the use of JavaScript and web-sockets for custom implemen-tations and functionality. Albeit these can be very power consuming, where a native interface performance-wise is insignificant.
In continuation of using nodes and the possibility of running them in their own respective pro-cess came from the idea of upgrading services remotely and without interruption. By having a node saving its state to disk and spawn the new version of the same node which then takes the old state. Converting it to its own state and takeover from the old node which then can be shut down. The initial implementation shows that this is quite possible but it becomes rather quickly very complex especially with platform independence in mind and having active networking connections. However we feel this is a viable direction.
A cronyms
CAAR Context Aware Adaptive 3D Rendering CAVE Cave Automatic Virtual Environment
DVMRP Distance Vector Multicast Routing Proto-col
ITK Insight Segmentation and Registration Toolkit JNI Java Native Interface
MBone Multicast Backbone MHz megahertz
MiB mebi bytes (1024 kibi bytes) MIP Mobile IP
MMOG Massive Multi-player Online Game MMO Massive Multi-player Online
MMVE Massively multi-user virtual environment MOSPF Multicast Extensions to OSPF (Open
Short-est Path First)
PIM-DM Protocol Independent Multicast - Dense Mode
PIM-SM Protocol Independent Multicast - Sparse Mode SAIL Sensor Abstraction and Integration Layer SDK software development kit
SDL Simple DirectMedia Library
SERVIVE Service oriented intelligent value adding network for clothing-SMEs embarking in Mass-Customisation
F rameworks
0 2 4 6 8 10 12
1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
Figure B.1:Chronological overview of publications on CVEs (Appendix B)
Table B.1:CVE Architectures
Name Year Description References
A3 2008 Interest Management Algorithm for Distributed Simulations of MMOGs [45]
Advanced Distri-bution Simulation
1995 ADS
ALSP 1997 "Aggregate Level Simulation Protocol describes message filtering Two classes of data: - Attributes of persistent objects Entities in VE - Interactions Non persistent events
Class filter, filter on class (regardless of value) Attribute value filter (e.g. range checks)
Filtering at sender and receiver Uses broadcast"
[117]
ANTS (MOVE) 2002 ANTS/MOVE Multiuser Oriented Virtual Environments [118]
ARIVU 2010 Power-aware middleware for multiplayer mobile games [119]
ATLAS 2002 A Scalable Network Framework for Distributed Virtual Environments [34]
Aviary 1994 distributed obiect system. [120]
Avocado 1999 A Distributed Virtual Reality Framework [121], [122]
Bamboo 1998 Virtual Environment Toolkit, a portable system for dynamically extensible, real-time, networked, virtual environments
[123], [124]
Beach 2004 application model and software framework for synchronous collaboration in ubiq-uitous computing environments.
[125]
Continued on next page
Table B.1 –Continued from previous page
Name Year Description References
Bricknet 1994 Based on an object sharing strategy allowing users to set-up their own private work-spaces, populated by shared and private objects. Virtual worlds are not restricted to sharing an identical set of objects; a virtual world manages its own set of objects, some or all of which may be shared with the other virtual worlds on the network.
[126]
Calat 1996 Intelligent tutoring system. server side and a multimedia viewer on the client side. [127]
Calvin 1996 Prototype of Cavern. multiple users to synchronously and asynchronously exper-iment with architectural room layout designs in the Cave Automatic Virtual Envi-ronment (CAVE).
[128]
Cavern 1997 A distributed architecture for supporting scalable persistence and interoperability in collaborative virtual environments
[129]
CavernSoft-G2 2000 (CavernSoft G1 is Cavern)Toolkit for High Performance Tele-Immersive Collabora-tion.
[130]
CCTT 1995 "Close combat tactical trainer DIS compliant Interest Management – pre-defined grid and packet type based.
Uses multicast per grid-cell"
[131]
Colyseus 2006 a distributed architecture for online multiplayer games [132]
Community Place 1997 Formerly known as Cyber Passage. Community Place is a shared multi-user VRML system de- signed to work in the Internet. It consists of a VRML2.0 browser, a multi-user server architecture and an application support environment.
[133]
Continuum 2007 An architecture for user evolvable collaborative virtual environments [134]
CoUniverse 2009 Framework for Building Self-Organizing Collaborative Environments Using Extreme-Bandwidth Media Applications
[135], [136]
Cove 2006 Collaborative object-oriented visualization environment [137]
Coven 2001 Based on DIVE; prototype large-scale applications of CVE. [138]
CSPray 1997 Provides different levels of information sharing, incremental updates to reduce network traffic, an intuitive floor control strategy for coordinating access to shared resources, a built in session manager to handle participants who either join late or leave early and a host of collaborative 3D visualization aids.
[139]
CVD 1999 Together with CAVE5D and tightly integrated with Cavern. Cave5D/Virtual Direc-tor, integrates the capabilities of both existing VR applications, Cave5D and Virtual Director, in order to enable the user to view and interact with the data from within the data set, visualize the data in real time and easily record the experience.
[140]
CyberWalk 2003 a web-based distributed virtual walkthrough environment. [141]
DACIA 2000 A mobile component framework for adaptive groupware application. [142]
DEVA3 2000 Architecture for a Large-Scale Distributed Virtual Reality System. [31]
DEVRL 1996 Distributed extensible virtual reality laboratory [143]
DIS 1993 Distributed Interactive Simulation, the follow up of SIMNET and focuses on mili-tary simulations.
DISCIPLE 1999 a framework for multimodal collaboration in heterogeneous environments. [144]
DIVE 1993 "The Distributed Interactive Virtual Environment (DIVE) is an internet-based multi-user VR system where participants navigate in 3D space and see, meet and interact with other users and applications. The DIVE software is a research prototype cov-ered by licenses. Binaries for non-commercial use, however, are freely available for a number of platforms. The first DIVE version appeared in 1991.
DIVE supports the development of virtual environments, user interfaces and ap-plications based on shared 3D synthetic environments. DIVE is especially tuned to multi-user applications, where several networked participants interact over a network."
[145]
Dive 3 1999 Latest (and last) version DIVE 3.3x (eXperimental) [24]
DOOVIE 1997 an architecture for networked virtual environment systems [146]
Drone 2009 A Flexible Framework for Distributed Rendering and Display [147]
DSG 2010 "Distributed Scene Graph to Enable Thousands of Interacting Users in a Virtual Environment"
[148]
DVS 1991
EQUIP 2002 a software platform for distributed interactive systems [149]
ERCIS 1998 Distributed interactive simulation for group-distance exercises on the web. [150]
EventHeap 2000
FlowVR 2004 A framework for distributed virtual reality applications [151], [152]
FUSE 2002 ubiquitous collaboration within diverse mobile environments [153]
GreenSpace 1995 a distributed virtual environment for global applications. [154]
HLA 1995 "Higher Level Architecture; general purpose architecture for distributed computer simulation systems. Two types of filtering Class based and Value based"
[155]
HyClass 1997 Interactive Cooperative Learning System Based on Virtual Shared Space [156]
Hydra 2007 A Massively-Multiplayer Peer-to-Peer Architecture for the Game Developer [157]
ICOME 1999
iRos 2002 A dedicated meeting space with large displays, wireless or multimodal devices, and seamless mobile appliance integration.
[158]
Continued on next page
JPSD 1995 "Joint Precision Strike Demo Prototype simulator Uses DIS protocol A node sim-ulates entities Per node Interest management Entity broadcasts(publish) their in-terest manager to other nodes Static: priori simulation Dynamic: upon need Uses point-to-point unicast"
LEARS 2011 lockless, relaxed atomicity state parallel game server [159]
MACVE 2007 A Mobile Agent Based Framework for Large-Scale Collaborative Virtual Environ-ments
[160]
MaDViWorld 2002 A software framework for massively distributed virtual worlds [161]
MASSIVE 1 1994 Initially aimed at teleconferencing, supports multiple users, applications and worlds connected by "portals"
[22]
MASSIVE 2 1996 * [162]
MASSIVE 3 2000 flexible support for data consistency and world structuring [162]
Matrix 2005 Adaptive Middleware for Distributed Multiplayer Games [163]
MAVERIK 1999 A micro-kernel for large-scale virtual environments [164]
MERCURY 2002 a scalable publish-subscribe system for internet games [165]
Mobihoc 2007 a middleware adopt- ing VFC to support multiplayer distributed games in ad-hoc networks.
[166]
ModSAF 1995 "Modular Semi-Automated Forces Designed for US Army Initial: LISP on BBN ButterFly Progress: SIMNET SAF -> OdinSAF Final: ModSAF
Used for STOW-ED-1 and NPSNET
Filter by Static Spatial based, cells And filter packet on Type (class) Uses broadcast Interest management done by aggregate level (not individual)
“destination based filtering”"
MOPAR 2005 a mobile peer-to-peer overlay architecture for interest management of massively multiplayer online games.
[167]
MPACC (gaia) 2001 Model-Presentation- Adapter-Controller-Coordinator, an application model that ex-tends the MVC pattern to Ubiquitous Computing scenarios. This new model takes into account issues such as the non-existence of a single interaction device, con-textual properties associated to the user and the space where the application runs, automatic model-view data type adaptation, mobility of the view, model and con-troller, and applications running on behalf of a user or a space, instead of in the context of a particular device.
[168]
MPCW-REALTIME
1992
MUVEES 2003 a Multi-User Virtual Environment for Learning. [169]
NATIVE 1999
NetEffect 1997 a network architecture for large-scale multi-user virtual worlds [170]
NOMAD 2000
NPSNET I 1991 "Navel Postgraduate School NET DIS compliant Uses Hexagonal grid, per cell mul-ticast address
AOI, radius of grid cells. Oldest active entity in grid cell -> group leader for adding, deleting members to grid cell"
[21]
NPSNET II 1993
NPSNET III 1993
NPSNET IV 1993
NPSNET V 2000 "a platform for research on infrastruc- ture technology for networked virtual envi-ronments."
[171]
Octopus 2001 A cross-platform, object oriented API designed to abstract the complex details in-volved with CVEs including soc ket management, shared data handling, shared object control and representations of other users (avatars).
[172]
OdinSAF 1995
OpenPing 2004 a reflective middleware for the construction of adaptive networked game applica-tions
[173]
P2LIFE 2010 An Infrastructure for Networked Virtual Marketplace Environments [35]
PaRADE 1997
PAULINGSWORLD 1998 An Immersive Environment for Collaborative Exploration of Molecular Structures and Interactions
QUICK 2000 Framework for optimizing fidelity on a per-task basis. Integrates ratings of repre-sentational Quality, scene node Importance, and ma- chine resource Cost.
[175]
RAVE 2008 Resource-aware visualization environment [176]
Reality Built for Two
1990 Development platform for designing and implementing real-time virtual realities . [177]
Rendezvous 1990 an architecture for synchronous multi-user applications [17]
Repo-3D 1998 A distributed 3D graphics library [178]
Continued on next page
Table B.1 –Continued from previous page
Name Year Description References
RING 1995 A client-server system for multi-user virtual environments. [179]
Rubber Rocks 1992 Interactive simulation in a multi-person virtual world [180]
SCAPE 2003 Stereoscopic Collaboration in Augmented and Projective Environments [181]
SCORE 2004 a scalable communication protocol for large-scale virtual environments. [50]
SEMMO 2008 A Scalable Engine for Massively Multiplayer Online Games [182]
SIMNET 1987 Miltary real-time distributed combat simulations. Mainly based on broadcast net-working and filtering. Introduces the concept of "dead-reckoning"
[15]
SmallTool 1998 a toolkit for realizing shared virtual environments on the Internet. SmallView was realized with this toolkit.
[183]
SmartCU3D 2001 a Collaborative Virtual Environment System with Behavior Based Interaction Man-agement
[184]
SPLINE 1996 Used for social virtual environment "Diamon Park". Support audio and video in-teraction.
[23]
Steve Benford Spa-tial model of inter-action
1993 COMIC Project; A Spatial Model of Interaction in Large Virtual Environments [185]
STOW 1994 Synthetic Theater of War [16]
STOW ACTD 1997 "Synthetic Theater of War Advanced Concept Technology Demonstration Uses experimental HLA, and grid based, variable pre-defined grid cell sizes. Each cell with multicast"
STOW ED-1 1995 "Synthetiic Theater of War Demo 1 Uses DIS PDU (but not fully DIS compliant) Uses multiple grid layers (fidelity) with multicast Similar to ModSAF
Complete data when subscribe to all grid-layers. Agent at LAN-WAN, as long entity has IE in group, agent wil stay in multicast group."
STOW-E 1994 "Synthetic Theater of War - Europe One of the STOW programs DIS 2.03 with custom PDU Uses grid-based and PDU type filtering AOI, grid cell set Uses broad-casting for lan-wan-lan data (node-node)"
StudierStube 1998 An environment for collaboration in augmented reality [186]
Three-Tiered IM 1999 Based on Bamboo, providing an extensible interest management for scalable per-sistent distributed virtual environments
[187]
TRADE 1997 A transatlantic research and development environment. [188]
UCAVE 2012 Ubiquitous collaborative activity virtual environments [189]
Urbi et Orbi 2000 An asynchronous architecture to manage communication, display, and user inter-action in distributed virtual environments.
[190]
VAST 2007 Voronoi-based Adaptive Scalable Transfer; a fully-distri- buted peer-to-peer proto-col, designed to handle event mes- sages in MMOGs
[191]
VELVET 2003 An Adaptive Hybrid Architecture for Very Large Virtual Environments [192]
VEOS 1994 Virtual Environment Operating Shell. emphasizes rapid prototyping, heteroge-neous distributed computing, and portability.
[193]
VERN 1991 Virtual Environment Realtime Network, object oriented testbed for interconnection of envirnments over a network of graphical workstations.
[194]
Vesuvius 2007 Interactive Atmospheric Visualization in a Virtual Environment [195]
Virtual Society 1995 Based on Community Place, [196], [197]
VISINET 1997 collaborative 3D visualization and VR over ATM networks [198]
VISTEL 1993 In VISTEL, the 3D models of the participants at different sites are combined into an artificially created 3D image of a virtual space, and by displaying the 3D image of the virtual space on the 3D screen at each site, the participants will get the feeling of meeting each other in a common space.
[199]
VLNET 1995 a shared virtual life network with virtual humans that provides a natural interface for collaborative working.
[200]
Von 2006 A Scalable Peer-to-Peer Network for Virtual Environments [33]
VPARK 1999 a Networked Virtual Environment (NVE) System, called W-VLNET (Windows Vir-tual Life Network) and an Attraction Building System, able to create and modify attractions used in the NVE System.
[201]
WAVES 1993 Design of architectures for low-end distributed virtual environments [202]
Whiskey 2007 Synchronous collaborative systems for distributed virtual environments in Java [203]
Wolverine 2005 A distributed Scene-Graph Library [204]
H ardware
This section lists an overview of all the hardware mentioned in more detail.
(i) Workstation, Intel Pentium 4 CPU at 3.40GHz, 2GiB RAM, NVidia GeForce 7800GT
(ii) Workstation, 2.4GHz Intel Core2 CPU, 2GiB RAM and an NVIDIA GeForce 7800 GT GHz, Win-dows XP.
(iii) Workstation, 2.4GHz Intel Core2 CPU, 2GiB RAM and an NVIDIA GeForce 8800GTX GHz, Win-dows XP.
(iv) Workstation, Intel Core i7 950 CPU. NVIDIA GeForce GTX 480, Windows 7.
(v) Workstation, Intel Core i7 X 980 CPU. NVIDIA GeForce GTX 480, Windows 7.
(vi) Workstation, Intel Core i7 3770 CPU. NVIDIA GeForce GTX 660, Windows 8.
(vii) Ultra Mobile Personal Computers are based on the Microsoft Origami specification released in 2006 and have been developed jointly by Microsoft, Intel, and Samsung, among others. UMPCs are basically small form factor mobile PCs running Microsoft Windows XP. The UMPC used is a Asus R2H. Intel Celeron M Processor ULV : GHz 900MHz;Intel 910GML Express Chipset;DDR 533 MHz SDRAM, 256MB on board;Embedded Intel GM965.
(viii) Tablet, Asus Slate EP121, Intel Dual-Core i5 470um, DDR3, 4GB SO-DIMM, 64GB SSD, Intel HD Graphics, Windows 8.
(ix) Motion tracker, Type Mtx.a small and accurate 3DOF Orientation Tracker. It provides drift-free 3D orientation as well as kinematic data: 3D acceleration, 3D rate of turn and 3D earth-magnetic field.
(x) Mobile phone HTC Hero, 288 MB RAM, 512 MB ROM, Qualcomm MSM7200A, 528 MHz, Adreno 130 GPU, Android OS, v1.5 (Cupcake).
(xi) Mobile phone HTC HD2, 448 MB RAM, 512 MB ROM, Qualcomm QSD8250 Snapdragon, 1 GHz (Scorpion) , Adreno 200 GPU, Microsoft Windows Mobile 6.5 Professional.
(xii) Mobile phone Samsung Omnia 7, 512 MB RAM, 8GB Flash, Qualcomm QSD8250 Snapdragon, 1 GHz (Scorpion), Adreno 200 GPU, Microsoft Windows Phone 7.5.
(xiii) Mobile phone Nokia Lumia 820, 1 GB RAM, 8GB Flash, Qualcomm Snapdragon S4 MSM8960, 1.5 GHz dual-core Qualcomm Krait, Adreno 225 GPU, Microsoft Windows Phone 8.
(xiv) Head-mounted display
(xv) Surface pro 2 tablet, Microsoft Windows 8.1, 4 GB Ram, Intel Core i5-4200u CPU 1.6Ghz, Intel Graphics.
P rojects
The work presented in this thesis was supported by several projects. Each of these projects brought their own domain, limitations and requirements to the research topic. It broadened the topic, made it more varied and diverse as a whole level of complexity in overall design of the framework was added, which made it challenging and intriguing. Each section presents a description of the project and includes a section with the contributions correlated with the work presented in this thesis. Starting with the main project Intermedia bringing the main focus on user centric media, followed by the second most important contributing project 3DAH in terms of telemedicine, and last the two supportive projects Servive and Leapfrog for e-commerce and real-world applications. The descriptions given are the official descriptions from the project websites.
D.1 InterMedia
“
There have been considerable efforts to have Audio Video systems and applications converge, in particular in home environments with homes as spaces of convergence, and for nomadic users with advanced mobile devices as points of convergence. These trends are important but also have limitations that we seek to address and overcome: home-centric systems fail to account for increased mobility and the desire to provide continuous service across spatial boundaries outside the home; device-centric convergence, e.g. in 3G phones, supports nomadic use but provides a very limited user experience as no single device and interface will fit many different applications well.In this Network of Excellence 12 we seek to progress beyond home and device-centric convergence toward truly user-centric convergence of multimedia. Our vision is The User as Multimedia Central:
the user as the point at which services (multimedia applications) and the means for interacting with them (devices and interfaces) converge. Key to our vision is that users are provided with a personalized interface and with personalized content independently of the particular set of physical devices they have available for interaction (on the body, or in their environment), and independently of the physical space in which they are situated. Our approach to this vision is to investigate a flexible wearable platform that supports dynamic composition of wearable devices, an ad-hoc connection to devices in the environment, a continuous access to multimedia networks, as well as adaptation of content to devices and user context.
The concept of user-centric convergence liberates a nomadic user from carrying a range of mobile de-vices by providing personalized access to multimedia regardless of dede-vices and networks. It is
accom-1Network of Excellence web-link:http://cordis.europa.eu/fp6/instr_noe.htm
2Project Intermedia web-link:http://cordis.europa.eu/projects/rcn/79765_en.html
plished not only by removing spatial constraints but also by making multimedia contents adapted to diverse devices and networks in our daily activities. An overview of the three key challenging issues
plished not only by removing spatial constraints but also by making multimedia contents adapted to diverse devices and networks in our daily activities. An overview of the three key challenging issues