STATE-OF-THE-ART 121

Figure 5.8: Modeling of the control board.

using the property window (see figure 5.8). The relationships between CIOs is inferred automatically from direct manipulation (e.g. when the designer drops an object on a table, this is interpreted as an object being attached on the top of the table. The designer is able to introduce a new compound AIO based on the specified CIOs on the current design world. The relationships between its components are analysed and some of their properties become frozen to preserve the structure of the composition (this is called ”locking” a compound AIO). Of course, the designer is also able to ”unlock” some values while instantiating the compound AIO on the working surface area.

The XIML

The user interface scene is stored in XIML format. It is a descriptive language of the user interface without worry about how user interfaces will be effectively implemented. It’s purpose is to describe and manage the complex interactions among users, applications, devices and user interfaces.

XIML defines five basic interface components, namely, the task, domain, user, dialogue and presen-tation. The first three ones are contextual and abstract, while the last ones are concrete and implemen-tational. Envir3D focus especially on these last two components. Based on this XIML, the 3D scene can be automatically generated in VRML97.

User Interface Evaluation

The XIML model of the virtual world can then be used to model checking, consistency checking, au-tomatic verification of certain properties and model quality. The authors are particularly interested in verifying if the designed virtual world is compliant with the recommended domain-based usability guides. For instance the NUREG-0700 [27], is an official US standard on the quality of interfaces in control rooms. The guidelines expressed in this standard typically govern the location, size, the format of objects and if they are accessible enough. In Envir3D, it is done some static analysis, over values of

122 CHAPTER 5. RAPID PROTOTYPING OF USER INTERFACES...

the attributes of the CIOs and their relationships, to detect variations between their values and those recommended by the standard.

Some practical applications

The main application of Envir3D is modeling virtual user interfaces for control rooms (like control rooms of nuclear power plants), to help engineers during the design and evaluation of the user interfaces from the point of view of ergonomics.

Figure 5.9: Visualization of the control room of a Japanese nuclear plant.

Figure 5.10: The control board of a Volkswagen Touran car rendered in VRML.

As shown in figure 5.9, they have modeled a control room of a Japanese PWR Mitsubishi nuclear plant, which was composed basically into five identical workstations sharing same presentation and behaviour (see figure 5.8). They have also modeled a control board of a Volkswagen Touran car (see figure 5.10) - they have not modeled all the control environment, like the rearview mirror, wing mirrors and glasses, which are also part of the control environment.

5.5. COMPARISON 123

5.5 Comparison

As we have just presented, there have been several attempts in the past to tackle the problem of au-tomatically generating user interfaces for control systems. Several aspects have been approached and interesting techniques have been tried out in different projects. We will now review these projects first summarising the aspects concerning the specification of models, then in those concerning automation, simulation and user profiling.

5.5.1 Models

Domain User task Presentation Dialogue 3D Geometric User Design

Project Model Model Model Model Model Model Model

TRIDENT √ √ √ √

Table 5.1: Comparison of the projects on the GUI Modeling.

On table 5.1 we present a comparison over the researched projects on user interface modeling field.

As shown in the table, MOBI-D seems to be the most general-purpose of all the studied model-based systems, as it is the one which as all the components of theinterface model. Both TRIDENT (business oriented) and ERGOCONCEPTOR (process control oriented) imposes a specific design order which also guides the designer through all the design process. MOBI-D also drives the design process from abstract models to concrete ones, but it uses a specific model (design model) to glue link all models and an assistant (TIMM) to help the designer pruning the space of choices. This way of organising the mappings can be interesting if we are to support iterative design due to the fact that all modeling components are intrinsically independent from each other. In DWARF, the specification of both multi-modal tasks (task model) and interface objects (presentation model) is done using Petri Nets. Finally in Envir3D, they have limited their work to presentation and dialog models, but they left doors open to the other interface model components by adopting the XIML format on their model-based engine.

Automation

Table 5.2: Comparison of the projects on the Automation, Simulation and Verification Features.

Automation is a way to speed up design process by automatically generating a user interface based on a formal specification of some kind. As shown in table 5.2, DWARF presents some automation by generating the GUI based on a specification of the user interface logic through a Petri net-based formalism. TRIDENT presents almost full automation in several aspects, since it uses configurable production rules to guide the designer, and automatically take some options on the interface design.

The positioning of the widgets on the dialogues is automatically done given the desired layout strategy, effectively reducing the design time of a user interface. Clearly, there must be a balance in a RUIP system between flexibility and the design time wasted in specifying all the details on the user interface design. MOBI-D uses a tool (TIMM) to assist the designers in their design choices, not limiting however the artistic options that the designer may take. ERGOCONCEPTOR, balances the capability of rapidly

124 CHAPTER 5. RAPID PROTOTYPING OF USER INTERFACES...

generate a first version of the GUI formed by abstract user controls (and with several design options left open) to the flexibility of instantiating those abstract user controls in an external design editor - this way joining the best of two worlds. Finally, Envir3D can also generate a user interface (VRML code) based on the XIML specification. Their AIO library can effectively speed up the design process in a considerable way.

Verification and Simulation

ERGOCONCEPTOR uses production rules to post validate the generated interface, in terms of usability and human factors, proposing changes to the next design iteration step. The user interface verification is done by static evaluation of the interfaces using an ergonomic knowledge base built by the referred production rules, and dynamic validation (usability tests) through simulations on the control room site.

Envir3D can also use the XIML specification to statically analyse CIO’s attributes and relationships to detect variations of their values and the recommended control system’s interfaces ergonomic guidelines.

Simulation is a requirement in MOI, which uses a valuable anticipation system (on-line simulations). In MOI the anticipation and recovery system is used to test all the user inputs predicting the impact of the operator’s commands on the system. It is built using a reference model of a safe successful state evolution of the system to distinguish between safe actions from unsafe ones. Enabling an anticipation and recovery system is another way to ensure semantic correctness of the generated interfaces.

Project User Profiling Multi-Modality Navigation

Table 5.3: Comparison of the most common operational features on the studied projects.

2D/3D user interfaces

Although it is very rare to find RUIP systems that deal with 3D interfaces, there is some interesting work done in the area of RUIP for the purpose of Knowledge Bases [4]. Most of the studied projects (see table 5.3) focus only 2D interfaces since their concerns are placed on a relatively general scope. MOI mix 3D interfaces with 2D interfaces, but it presents no particular formal methods to model them. DWARF presents a cutting edge 3D interface in the field of ubiquitous and tangible interfaces, with a flexible design architecture of its internal components (some of them configurable with Petri-Nets in XML).

Envir3D is the only one which fully supports the design of both 2D and 3D interfaces in a model-based fashion. In fact, 2D interfaces can easily be transformed into 3D interfaces, since they abstract the final layout of the objects, leaving the fact of being a 2D object or a 3D object a mere implementation detail.

5.5.2 User profiling

Different user profiles will need different kinds of user interfaces according to their skill in a determinate area of the control system. From the studied projects, MOBI-D presents a user model to explicitly specify different users, which in turn can be an elegant and formal way to specify user profiles and most of the adaptability concerns. MOI is referred to be user- task- adaptable by means of which the operator can choose the interface with the most suitable automation level to his skill or profile. This is done in MOI and DWARF using multi-modal interfaces. Also, their need for increasing control may change dynamically according to context operation. Multi-modal interfaces are such interfaces. From the researched projects, DWARF and MOI are the only ones that focus in multi-modal interfaces. They have identified most of the relevant modes of interaction, and their projects support them, however they do not seem to present a strong formal framework to support them. Envir3D can also support user profilling with XIML, but to date they do not fully support it.