TRAMA: A Traceability Analysis Method for (Interactive) Applications
A dissertation presented by Giovanni Randazzo
Supervised by Prof. Paolo Paolini
Submitted to the
Faculty of Communication Sciences University of Lugano
For the degree of
Ph.D. in Communication Sciences
November 2005
Board
Thesis supervisor: Prof. Paolo Paolini PhD Coordinator: Prof. Marco Colombetti First reviewer (external): Prof. Giuseppe Visaggio Second reviewer (internal): Prof. Carlo Ghezzi
Final version – March 2006
Abstract: if you have no time, read just this
This dissertation presents TRAMA, a TRaceability Analysis Method for (interactive) Applications. The method was originally conceived for being used on interactive application projects, but some experiences showed that its tools and its concepts could be applied in wider domains, including information systems, knowledge management systems, educational applications, etc. This is the reason why the word interactive has been put between brackets in the title.
TRAMA is a method, i.e. it is a systematic procedure to perform analysis for documenting tracing, and not a methodology, i.e. a body of rules and postulates which analyse the principles of inquiry in the traceability field. TRAMA is a Requirements Traceability method, i.e. a method to explicitly trace and document relationships between requirements and the different phases of a project’s life-cycle, helping ascertain how and why system development products satisfy stakeholder requirements. In particular, TRAMA focus on Design Tracing, i.e. on analysing and documenting the impact of requirements on design elements and the reasons for design choices.
The first attempts in this research went in the direction of finding a way to record the process that brings from requirements to design. In fact, a common opinion in the Requirements Traceability (RT) field is that solution design, i.e. the design of the application solutions, may be derived directly from a requirements refinement activity; some works [Pohl et al., 1994; Egyed et al., 2000] consider design as the result of a refinement process and try to document this process establishing a requirements-to-design traceability.
Despite this trend, experiences and case studies1 conducted for TRAMA highlight a different situation: the design process does not seem to be a fully rational and explicit sequence of actions; according to Arciszewsky [Arciszewsky et al., 1995], “design is an intuition and induction process more than a derivation one”. In fact, at least in the TRAMA case studies, designers keep requirements in mind as a background knowledge, and they build up the application architecture almost from scratch, as a result of an inductive and a partly intuitive activity. Some requirements remain implicit at the beginning of the project but they are considered in design; often, at the end of the project, designers do not remember the actual reason for that choices. Since requirements are understood as “base information” about how the application should be and why, designers are able to draw a design that satisfies those requirements at least partially, thanks to their skills and to their professional experience. In common industrial cases these relationships are anyway still not explicitly specified; this problem makes it very hard to verify, to evaluate, to revise, and to reuse efficiently design solutions in relation with high-level requirements.
Taking into account these elements, TRAMA has intentionally not been developed to explicitly record the mental processes that brings us from general requirements to concrete design solutions; on the contrary, this method’s tracing activity tries to move intuition and induction to a more rational cause-and-effect motivation. This kind of analysis must not repress or stiffen the design process, but it helps us to better understand the reasons for design choices and it forces us to make explicit requirements that are both implicit or unexpressed. In particular, TRAMA is intended to allow a trace documenting activity from five different points of view:
1 Se also section 3.
• client validation: the method helps in gathering a structured argumentation to show to the client that all the needs have been taken into consideration;
• design versioning: the method allows analysts to highlight different design areas, identifying the application elements which satisfy the goals of a specific stakeholder;
• non-traceable design: the method provides conceptual tools to document the motivations of design elements that do not derive from requirements;
• “negative” design: the method allows to keep track of old design choices that have been eliminated or modified during the project;
• reverse requirements specification: the method provides tools to check the consistency between design and requirements, to “tune” requirements specification according to the real stakeholders’
goals and to extract consistent requirements specification from design;
• evaluating usability based on design documents: the method helps in selecting the elements in the design involved for a specific task, in evaluating the quality of the product with respect to the high- level goals and in identifying test procedures which should be rerun to validate an implemented design change.
TRAMA is therefore an effective method to discover, elicit, analyse and document “ex-post” traces, i.e. the method does not record the design process but it helps designers in understanding both the impact of requirements in their projects and the motivations or “sources” of specific design decisions after the design has been drawn. The method is based on traceability matrices which cross requirements with design in a forward direction and design with its sources (requirements, visions, constraints, etc.) in a backward direction.
Requirements-to-Design matrix called RIM (Requirements Impact Model/Matrix) and illustrated in Table 1, can be filled and read both horizontally, highlighting how single requirements are taken into account into the design, and vertically, showing how a single design element satisfies the project requirements.
DESIGN ELEMENTS
Content 1 Content 2 Access path 1 … VISIONS
Vision 1 Vision 2
… GOALS Goal 1 Goal 2
…
REQUIREMENTS Requirement 1
REUQIREMENTS-RELATEDINFORMATION
Requirement 2
Table 1. A template for the RIM matrix
Design-to-Sources matrix called DMM (Design Motivations Model/Matrix) and illustrated in Table 2, traces back single design elements to the motivation why a certain decision is relevant for the project. These motivations can be:
• the designer expertise, i.e. particular “good design” principles that are part of the designer’s skills and that she/he applies in any case;
• a specific understanding of the domain, i.e. recurring good solutions in a domain that the designer applies because she/he learnt it in other cases in the same domain;
• a particular constraint, e.g. budget limitations, time, technology limitations, etc.;
• a law obligation, e.g. copyright issues, personal data treatment, etc.
• requirements-related information, i.e. a vision, a goal, a requirements, etc.
• an arbitrary choice, i.e. a choice without particular reasons, usually a single detail that could be set any of a number of way, e.g. the structure of a game was in three steps (instead of four or two).
DESIGN MOTIVATIONS Visions Goals /
Requirements
Designer expertise
Understanding of the domain
Constraints Law obligations
Arbitrary choices Content 1
Content 2 Access path 1
… Negative design element 1 DESING ELEMENTS Negative
design element 2
Table 2. A template for the DMM matrix
As a kind of self-standing process, the TRAMA activity workflow is structured as follows:
• Preliminary plan: understanding the stakeholders of the traceability analysis, the traceability goals, the constraints (time and budget, related to ROI) and the expected results.
• Information re-organisation: understanding requirements and design from documents or from interviews with designers and organising it in terms of structured specifications.
• Information “normalisation”: structuring requirements and design information in “normal” terms, bases on a strong methodology (e.g. AWARE for requirements and IDM for design).
• Elicitation: surfacing relationships between requirements and design in terms of the impact of requirements on the design (“How these requirements have been considered in the design?”) and of motivations for design choices (“Why this solution has been adopted?”).
• Analysis: tracing relationship and developing the Requirements Impact and the Design Motivations Matrices (RIM and DMM).
• Specification: documenting stakeholders, goals and analysis results.
• Validation: checking the results with requirements analysts, designers, project managers, and clients.
Benefits in the use of TRAMA are mainly the following:
• TRAMA is a powerful communication means to show to the clients that all their requirements have been considered and how, and that there are no unmotivated elements in the design;
• TRAMA is a structured practice for checking requirements and design consistency for revision, for surfacing missing design elements and missing requirements; the method supports reverse requirements engineering;
• TRAMA is an advanced tool to tune up and re-align a design in the maintenance phase and to assign priorities to design elements;
• TRAMA specifications provide a complete project knowledge summary of requirements-related information, of design elements and of relationships between them, as vital information allowing effective system reengineering, workflow organisation, and more focused verification procedures to be performed.
Acknowledgements
The author is grateful to all the people who contributed directly or indirectly to this dissertation, and in particular:
• the TEC-Lab team, Luca Triacca, Marco Speroni, Nicola Piccinotti, Daniele Gobbetti and Davide Bolchini for sharing contents, ideas, and visions;
• the L@E team for their proactive support collaboration to the traceability analysis process; in particular, I thank Nicoletta Di Blas, Paolo Paolini, Camilla Fontana and Caterina Poggi.
• my family, Rosario, Rita, Gloria, Giusi, Mauro and Carlo for their invaluable support and love
• all the other people in Milano, Como, Lecce and Lugano whom I did not mention but who gave me precious help in this long and winding road.
Lugano, 24 October 2005
Contents
ACKNOWLEDGEMENTS 9
0. INTRODUCTION: JUST A BEGINNING 13
1. TRAMA RESEARCH FOUNDATIONS 21
2. REVIEW OF RELATED WORKS 49
3. CASE STUDIES 67
4. THE TRAMA METHOD 93
5. TEACHING TRAMA 125
6. CONCLUSIONS & FUTURE WORKS 143
REFERENCES & BIBLIOGRAPHY 149
0.Introduction: just a beginning
<<There’s no point in being exact about something if you don’t even know what you’re talking about.>>
John von Neumann
0.1. The traceability problem
In the software requirements engineering community, traceability has been for a long time studied as a crucial quality factor for software development projects. In particular, a big push in this field was given by the works of Gotel & Finkelstein [Gotel & Finkelstein, 1994] and Pohl [Pohl et al., 1994] in the early 90’s.
In these last 15 years, traceability for interactive applications has been studied as a part of the requirements analysis process (see also [Jarke, 1998]) and perceived as the activity to trace relationships from and to the requirements specification. A number of models and methodologies have been developed in order to manage and record these information during a project’s life-cycle [Potts & Bruns, 1988;
McLean et al., 1989; Lee, 1991; Pohl et al., 1994; Gotel & Finkelstein, 1995, Egyed et al., 2000;
Grünbacher et al., 2001; Arkley et al., 2002; Dick, 2002].
Thanks to thus research, Requirements Traceability (RT) is viewed as a measure of system quality and it is mandated by many standards governing the development of systems (e.g., IEEE Std 830-1993 and IEEE/EIA 12207). The importance of RT is highlighted by the fact that, for instance, the US Department of Defence has spent about 4% of its IT costs on traceability [Ramesh & Jarke, 2001] – often without getting an adequate value for this money, as traceability in many organisations is haphazard, the standards provide little guidance, and the models and mechanisms vary to a large degree and are often poorly understood. In fact, unfortunately, the penetration degree of these approaches in industrial cases and in companies’ workflows is very low. In current industrial practices, RT is still perceived as extra-work, without clear advantages in terms of its return on investment (ROI).
“It is a very valuable but seldom used technique in today’s development processes. Traceability analysis is rarer still in the Internet development industry, where it is even more essential” [Leon, 2000].
“It is rare to find a software project team that can honestly claim full requirements traceability throughout a project, especially if the team uses object-orientated technology” [Ambler, 1999].
The reasons for this situation can be identify in the fact that current traceability methods requires a large amount of time to be spent in order to keep track of requirements along a full project’s lifecycle. The effort to maintain traceability is not perceived by developers and managers to be cost-effective. It is considered by management to be extra, optional work, for which insufficient resources are allocated. This position agrees with the opinion of engineers who believe that maintaining traceability information is costing them too much work. Capturing information on the design history of a project may take over 50%
of an engineer’s time [Wieringa, 1995]. Despite all this work, the benefits of maintaining traceability are not clear: in projects where tools and techniques to maintain traceability are used, problems with traceability are still reported. According to Pinheiro [2002], the less intrusive the tracing activity is the more efficient and accurate the tracing process will be. In large part this occurs because any intrusive process will be rejected or neglected by developers trying to deal with tracing.
Another problem related to the introduction of RT methods in industrial practices could be linked to the fact that some of these methods have a tool-based approach. I refer here to a kind of “myth” that has
never been wrote down but that circulates in several forms in the RT community. According to the Merriam-Webster dictionary, a myth is “a usually traditional story of ostensibly historical events that serves to unfold part of the world view of a people or explain a practice, belief, or natural phenomenon”.
This myth is a story about a world were the problem of checking the quality of a software application has been solved by the mean of a tracing practice; a world where software developers write down in detail every step of their work, the reasons of every choice, their assumptions, their goals and their beliefs related to the piece application they are working on; a world were these people can spend half of the project time in documenting and recording all these information using complex tools or formal languages to link it each other in a (more or less) meaningful way; a world where, at the end of the day, someone could draw useful conclusions for the quality of the application from this huge network of relationships.
In pragmatic terms, most of the currently available RT approaches give to the traceability problem these answer: “while it is difficult maintaining the huge mass of dependences among the many objects produced by a large software system development effort, some current approaches require the use of a software tool to become usable and manageable; so, bring all your documents, specifications and artefacts produced during the project, record it into a support tool and trace all the relationships that you consider meaningful; other relationships will be automatically created by the tool itself” [typical traceability solution]. Unfortunately, this tool-based solutions do not consider that in the actual practice, some specifications are not taken, some documents are not written or are written after the application is implemented and that some “knowledge” (about reasons, beliefs, etc.) is never recorded or explicitly considered. Furthermore, current tools have problems in maintaining relationships concerning artefacts expressed in natural language, often ambiguous, or artefact created independently by non-interoperable tools and that evolves autonomously. Besides, some tool-based practices have access problems for the user (communication problems) and methodologies are often not clear, not complete or too formal for their adopters.
0.2. Requirements to Design
The research presented in this dissertation proposes a method to keep trace of “requirements-to-design”
traces. With “design” I mean here the conceptual, high-level description of the functionalities of an application and of the system’s solutions to strategic needs. In the last years, some RT models proposed software-based approaches directed towards the automation of requirements to design tracing processes (see also [Dick, 2002]). These models are based on the assumption that a system design activity moves in a explicit and structured way, considering one by one possible solutions to requirements and refining these requirements in a continuum to design elements. On the contrary, the case studies conducted for this research2 and “real world” experiences performed with professional designers in the last 3 years, highlight that requirements do not fade naturally into system solutions; requirements and design stand into separate and different conceptual area instead: the firsts are in the space of problems, the latter in the space of solutions. During a design process there are not explicit relationships between these two
“spaces”; in fact, designers take in account requirements as background information but then they design the system following their own skills and competences. In other words, in order to consider stakeholders
2 See also section 3.
needs, requirements are understood and “absorbed” by designers as a whole. Therefore, the design process is not a fully rational and explicit sequence of actions; designers build up the application architecture almost from scratch, as a result of a an inductive3 and in part intuitive practice. In fact, in such a decision making activity not only the general requirements knowledge described before is involved, but also a wider knowledge about the specific application domain, some project constraints, “good design”
and usability principles, designer skills, etc. – in most of the cases in a implicit or almost unconscious way. On the other hand, design structure and requirements structure are inhomogeneous, while a requirement does not impact on a single design element but on a number of design elements and on their interactions. Since requirements are understood as base information about how the application should be and why, skilled and experienced designers are able to draw a design that satisfy in a certain measure those requirements. In common industrial cases these relationships are anyway still not explicitly specified; this problem make very hard to verify, to evaluate, to revision and to reuse efficiently design solutions in relation with high-level requirements. This fact includes that in some cases it is not clear why a certain piece of the system has been developed, and how the product answer to needs stressed by strategic requirements; in other words it is not clear how to evaluate, to validate or to motivate the quality4 of the final product. Another problem raised out from this situation is that revision activities are very hard to bring on, while it is not clear the impact of requested changes and their effect on the application requirements compliance.
A proposal to find a reasonable and usable solution to these problems is presented in this dissertation as TRAMA, a TRaceability Analysis Methodology for (interactive) Applications. The method is a first attempt to reduce the complexity of current methodologies considering requirements-to-design relationships between objects of adequate granularity; TRAMA can be applied even in case of lack of documentation: it is also useful to write an ex-post specification of the work done; TRAMA can be used without any specific software tool: objects are related each other using simple matrices; TRAMA analysis discover or highlight the main reasons for conceptual design choices and which is the impact of a goal or of a requirement on the application. This traceability approach does not focus on modelling the process of evolving requirements into design, but it pretend to provide to designer an effective tool conceived to discuss and analyse the design choices after they have been taken, in order to refine it according to the main requirements and in order to eliminate unmotivated elements. TRAMA consists in a structured analysis process, in a general conceptual model of entities and relationships to trace and in a set of conceptual tools supporting traces inquiry, analysis and documentation. The case studies where TRAMA has been applied have shown that the methodology is easy to use and to learn, and that the tracing activity is reduced to an average of the 5% of the time spent for the entire project.
3 See also [Arciszewsky et al. 1995].
4 For an explanation of the concept of quality, see also Cantoni et al. [2003].
0.3. Dissertation’s structure
The TRAMA research presentation is organised in 6 main sections:
1 TRAMA research foundations
This section illustrates the main concepts related to the traceability field, with a particular focus on the Requirements Traceability (RT) domain; the section describes the scope, the focus, the hypothesis and the goals of the research treated in this dissertation as well.
2 Review of related works
This section proposes a review of the state-of-the-art in the RT field, with a particular emphasis on those works that influenced in some way the TRAMA research. A short review of the main traceability conceptual tools and software tools is also provided. Finally, some open problems in current traceability practices are highlighted.
3 Case Studies
In this section all the case studies on which the different versions of TRAMA was applied are described.
Since this is an empirical research, each experimentation bring key elements to improve the method, to modify it, to refine it, to test it and to provide at the end a general approach. The sequence of case studies traces therefore an history of how TRAMA has been developed, as well as examples of use of the method.
4 The TRAMA method
In this section the current version of the TRAMA method is described, both as a process, i.e. as a sequence of actions divided in phases, and as a tracing approach, i.e. as a model including conceptual structures, tools, purposes, etc. In the section is presented first a tracing activity workflow allowing TRAMA to be properly applied; then, a TRAMA approach is described in terms of purposes, processes, conceptual trace model and tools. Finally, the benefits of the method and some of its limits are discussed.
5 Teaching TRAMA
This section presents the modules, the activities and the courses conceived to teach the TRAMA method to different targets and in different situations. In particular, three modules and four courses are provided.
6 Conclusions & Future works
This section wraps up the proposal in its key elements, highlighting the problems and the solutions described in the dissertation, its sources, its hypothesis, its main characteristics and benefits and limits of the approach. Finally, an input for future research is provided.
ANNEXES
Annex 1 - Educational Material
In this annex, all the slides packs and the slides for the courses related to the TRAMA educational plan are collected.
Annex 2 - TRAMA in a nutshell
In this annex, a short document describing TRAMA and 10 slides summarising the approach are provided.
Annex 3 - Case studies reports
In this annex, all the traceability reports related to the case studies presented in section 3 are collected.
1.TRAMA research foundations
"Walking on water and developing software from a specification are easy if both are frozen."
Edward V. Berard
Abstract
This section illustrates the main concept related to the traceability field, with a particular focus on the Requirements Traceability (RT) domain; the section describes also the scope, the focus, the hypothesis and the goals of the research described in this dissertation.
Traceability is the ability to explicitly trace and document relationship between the requirements phase and other phases of a project life-cycle. In the literature, a major distinction is highlighted between pre- and post-Requirements Specification traceability and between backward and forward traceability. A number of purposes for a tracing activity may be highlighted, according to the point of view of different actors involved in a project life-cycle: the client, the project manager, the designers, etc. A general meta- model describing a tracing approach can also be described, highlighting its purposes, the processes supported, the tools used and the conceptual trace model at the heart of the approach.
The TRAMA approach proposed in this dissertation is a method for tracing designs of interactive applications, supporting post-Requirements Specification Traceability in both a forward and backward direction. The research method bases on a empirical approach: an iterative sequence of experimentations and case studies will modify, improve, refine and test the method to provide at the end a general model.
1.1. Contextualisation
1.1.1. Traceab… what?
Traceability is the degree to which a relationship can be established between two or more products of the development process [IEEE, 1990]
Traceability can be simply defined as the ability to explicitly trace and document relationships between the different phases of a project’s life-cycle. A specification can be considered as “traceable” if the origin of each of the artefacts or objects described in such a specification is clear and if it facilitates the referencing of each object in future development or enhancement documentation [Gotel & Finkelstein 1994]. A great contribution in research for traceability comes from the Requirement Engineering field (RE); in this context the definition of Requirements Traceability (RT) can be adopted. According to Palmer [1997], RT helps ascertain how and why system development products satisfy stakeholder requirements. In short, RT is the ability to determine which documentation entities of a software system are related to which other documentation entities according to specific relationships; from this point of view traceability can be seen as the mean whereby an analyst is able to discover from one hand the impact of such entities on the application and from the other hand the reasons or the “sources” of specific entities [Spence & Probasco, 1998]. Formally speaking, a traceability system can be defined as a semantic network in which nodes represent objects (also stakeholders and sources), among which traceability is established through links of different types and strengths [Ramesh et al., 2001]. From a more “dynamic” point of view, RT is defined as the ability to follow a specific item at input of a phase of the software lifecycle to a specific item at the output of that phase; RT enables each requirement to be traced to its origin in other documents and to the software components satisfying the requirements. Traceability gives essential assistance in understanding the relationships that exist within and across various artefacts produced during the acquisition process. These relationships help establish traces of the process through which critical acquisition decisions are made and help ascertain how and why outputs of an acquisition process satisfy stakeholder requirements. According to Hamilton & Beeby [1991], traceability can be viewed as the ability to discover the history of every feature of the outputs of an acquisition activity so that the impacts of changes in acquisition requirements can be identified.
In literature a major distinction is highlighted between pre- and post-Requirements Specification traceability and between backward and forward traceability [Gotel & Finkelstein, 1994].
Figure 1. A simplified picture of traceability types
Pre-Requirements Specification traceability
According to Gotel & Finkelstein [1994], pre-Requirements Specification traceability (pre-RST) is concerned with those aspects of a requirement’s life prior to its inclusion in the requirements specification (i.e. requirements production and refinement). It is a technique that attempts to document the rationale and socio-political context from which requirements emerge, thus linking the business world with that of information technology [Jarke, 1998]. Pre-RST also serves to answer questions that arise during the project’s life-cycle, including: “Who is responsible for including this requirement?”, “To whom should I refer to for more information?”, “Who was responsible for copying this information into this document?”, and “Was this requirement a result of a meeting of stakeholders or just one individual?” [Gotel &
Finkelstein, 1995]. Pre-RST facilitates the reopening of previously closed specifications, tracing back to the sources of requirements, and then the (possible) reworking of a specification in the forward direction;
sources of requirements may be the following:
• Stakeholder Visions: stakeholders are those who have a direct interest in the success of the website (e.g. clients, sponsors, representatives, opinion makers, etc.); stakeholder visions are the assumptions of a stakeholder which dictate his/her “weltanshaung” on the project [Bolchini et al., 2005b].
• User Motivations: they shape the emotional, psychological, social or individual elements which can trigger a person (a final user) to use an interactive application [Bolchini et al., 2005b].
• Goals: they are defined as high-level targets of achievement for a user or a stakeholder; goals may represent a wished state of affairs (for main stakeholders) or a wished experience (for users) and may arise from visions or motivations.
• Constraints: they are defined as those elements that implicate a restriction on the degree of freedom the requirement analyst have in providing a solution; constraints can be economic, political,
technical, or environmental and pertain to project resources, schedule, target environment, or to the system itself.
In literature, major contribution to pre-RST comes from Contribution Structures [Gotel & Finkelstein, 1995] and PRO-ART [Pohl et al., 1994] methodologies; a deeper explanation of these two works will be carried on in section 2.
Post-Requirements Specification traceability
Post-Requirements Specification traceability (post-RST) is concerned with those aspects of a requirement’s life which result from its inclusion in the RS (i.e. requirement deployment and use). This kind of traceability provides a way to elicit and discover the impact of requirements and how requirements have been taken into account on the following project elements:
• conceptual design: high-level definition of the information structure, of the features and of the services/capabilities that the application will own;
• technical design: in-detail definition of the software (and/or hardware) components the application will be made of;
• experience design: definition of all the elements contributing in building the user experience, including organisational concerns, technical set-up and use scenarios;
• implementation: it’s the “tangible” part of the application, i.e. classes, routines, lines of code, interfaces, etc.
• tests: including technical test verifying if the application works properly, usability tests and accessibility test.
In literature, major contribution to post-RST comes from CBPS methodology [Egyed et al., 2000] and from the idea of rich traceability applied in the DOORS tool [Dick, 2002]; a deeper explanation of these two works will be carried on in section 2. The TRAMA method is an example of post-RST.
Backward traceability
Backward traceability records information and data on the past history of the product, providing knowledge about the sources of a specific element (e.g. a requirement, a design element or a piece of code) and about the reasons of a specific decision in the previous project items (e.g. in goals, requirements, etc.). In other words, backward traceability to previous development stages depends upon each requirement explicitly referencing its source in previous documents.
• Backward from requirements - This trace type lets the analyst verify that the system meets the user community’s needs, an important consideration attempting to justify the budget; therefore it is important to understand the source of requirements (e.g.: a requirement from a key customer likely has a different priority than one from a junior programmer).
• Backward to requirements - This trace type verifies compliance of design, software or tests built to requirements; this approach do not take into account software features that cannot be traced back to requirements because their source is another element such as a constraint or an organisational aspect.
Forward traceability
Forward traceability explains what will happen to a certain product and all the processes and output that the product in question went into. Forward traceability to all documents spawned from the software requirement specification depends upon each requirement in the software requirement specification having a unique name or reference number.
• Forward to requirements - This trace type maps stakeholder needs, visions and goals to the requirements, so that the analyst can determine the impact to requirements as needs change;
changing needs, either from a change in strategy, an increased understanding of the problem domain, or an environmental change, is a reality of the software industry and an important issue that must be managed effectively.
• Forward from requirements - With this trace type, the analyst assign responsibility for fulfilling a requirement to the design or to the various system components that will implement it, letting the responsible ensure that each requirement is fulfilled.
Nowadays, it is widely agreed that tracing requirements is essential in developing large systems and RT is intended to become an important feature of software systems. Many standards governing the development of such systems (for example, IEEE Std 830-1993 and IEEE/EIA 12207) require the development of RT documents; the US Department of Defence has produced standards with the same goal (e.g., MIL-STD-2167-A and MIL-STD-498) and spends about 4% of its Information Technology (from now on: IT) costs on traceability [Ramesh, 2001] – often without getting an adequate value for this money, as traceability in many organisations is haphazard, the standards provide little guidance, and the models and mechanisms vary to a large degree and are often poorly understood.
1.1.2. Quality of Service
The concept of RT is deeply related to the Quality of Service (QoS) and to the Software Quality (SQ) concerns [Kenny, 1996]. Quality per se can be seen from two points of view which are strongly intertwined and which affect each other:
• ad intra: quality is considered by mean of the intrinsic characteristics of the application (e.g.
performance, accuracy, up-to-date);
• ad extra: quality is the correspondence between services offered and stakeholders'5 goals; it can see as the combination of the quality of the user experience, the user satisfaction and the main stakeholder's6 satisfaction
5 The word stakeholder is here used as the sum of main stakeholder and of final users (see next footnote)
6 A main stakeholder is anyone who has an interest on the success of the application, e.g. clients, sponsors, decision makers, etc. [Bolchini, 2003]
EXAMPLE 1.1. Let’s try to consider the quality of a chair7. I will take for instance my office chair: it is a standard, plain office chair, with wheels and a back. Is it comfortable? Yes, it is: it let me work without pains for an entire day. Is it stable and solid? Yes, it is: it can support more than 100 Kg.
Therefore, it can be said that the intrinsic quality of my chair is good. But what if I should use the same chair to relax at evening watching TV? Maybe I would not feel it very comfortable. And what if it should be used in a western movie in a scene where the good crashes a chair on the head of the bad? Please, don’t do it. The extrinsic quality of this artefact is therefore deeply related to the use of the object and to the goals and motivations the users of that object have.
EXAMPLE 1.2. There is another case where the comfort ability of a chair doesn’t make its quality. In a US company, the time spent for meetings was too much; in fact, people usually drank coffee and chatted for part of the time, sitting around the meeting table. One day, the CEO of that company decided to change all the chairs in the meeting rooms with very uncomfortable ones. The result was that meetings time reduced of 50%. In this case, the quality of the chairs was good in relation to the goals of the main stakeholder.
In the EX1.1 it can be introduced the concept of usability, i.e. “the effectiveness, efficiency and satisfaction with which specified users can achieve specified goals in particular environments” (ISO 9241- 11)8. My chair is usable if I use it as office chair, but it is extremely unusable in the western movie. In the EX1.2 the new chairs have a very low level of usability in a strict sense, but their quality can be considered anyway good: they match with the needs of the main stakeholder. Therefore it can be introduced a more extensive definition of quality; according to Kenny [1996], Software Quality is:
• the totality of features and characteristics of a software product that bears on its ability to satisfy given needs, for example to conform to specifications;
• the degree to which software possesses a desired combination of attributes;
• the degree to which a customer or user perceives that software meets his or her composite expectations;
• the composite characteristic of software that determine the degree to which the software in use will meet the expectations of the customer.
In other words, it can be said that quality is a multifaceted characteristic of an application; the quality degree of a project may depend on services and features provided, user satisfaction and context of use, customer and main stakeholders satisfaction, compliance with strategic goals and impact on the organisation. Therefore, it becomes crucial to keep in a global picture the relationships between these elements: traceability can improve the quality of the systems development process, providing a global picture under control.
7 Adapted from Cantoni et al. [2003]
8 For a further explanation of the concept of usability and of how it is concerned with quality, I refer to Cantoni et al.
[2003].
1.1.3. Traceability purposes
Traceability can be seen as a powerful communication mean, whereby software producers can prove to their client that the requirements have been understood, that the product will fully comply with the requirements and that the product does not exhibit any unnecessary feature or functionality. Traceability can also facilitate communication among various stakeholders involved: project manager and project planner, customer, requirement analyst, designer, verifier and maintainer. Each one of these actors has his/her own view on traceability and can use traceability information for different purposes, as described in the following lines [Dömges, 1998; Gotel, 1994] and in Figure 2.
Figure 2. Traceability as communication mean and its use among the different actors involved in a project life-cycle
(i) Client/Customer/Stakeholder
Clients have in most of the cases a certain number of problems in evaluating the quality and the effectiveness of a software application a priori, i.e. before its effects have been produced. Usually there is a knowledge and understanding gap between stakeholders and the development team; clients can hardly see how and where the applications provided may fit to their needs and goals. Traceability analysts can guide these people in evaluating such applications; traceability is a communication “bridge” between a client (usually with marketing or economics background) and a software house, a web agency or anyway the internal development team (with engineering or informatics background) that allows to check the following elements:
• Requirements compliance – Traceability can shows the relationships between strategic goals, requirements and solutions in the application, allowing clients evaluating the compliance degree of the product with their needs. Therefore, the overall quality of the application can be understood without any need to consider single technical or software details.
• Requirements covering – Relationships between requirements and elements or pieces of the application may highlight the progress state of the project. Clients can understand which percentage
of the stated requirements are met and which part of the job is completed. A thorough traceability analysis may also provide stakeholders that all the strategic goals have been satisfied and how the application will address to their needs.
• Goldplating check – According to Dömges [1998], goldplating is adding superfluous features that aren’t motivated by actual requirements; since requirements are not the only source for design choices9, this definition can be limited excluding those elements that becomes from constraints, stakeholder visions, etc. Goldplating is therefore defined as the presence of features that are not motivated by any explicit reason. Traceability analysis highlights goldplating by linking all the application features with their motivations; if no explicit reasons are specified, two options can be considered: there is a reason but it has not been make explicit yet or it is a case of actual unmotivated feature. This kind of analysis lets clients ascertained that (costly) goldplating have been avoided because all components of the implementation can be traced to at least one reason, but it avoid also the risk of eliminate useful but only implicitly motivated features.
• Changes impact – It is not unusual to observe that after the end of a project, clients may ask to developers further changes to the applications. Reasons can be identify in lack of proper needs analysis or in lack of proper communication to the client. Anyway, this is something that happens in most of the cases for interactive application, and in particular for web applications. In this context, traceability analysts can help clients in evaluating the consequences of their requests, i.e. the impact of a requested change on the entire application and on the way the system meets stakeholders goals.
• Tests effectiveness - If the tracking information system records which requirements are satisfied by which parts of the implementation, and which tests must be performed to ascertain the “presence”
of a requirement, then clients can better understand the value, the results and the implications of technical tests and usability evaluations. In addition, acceptance testing can refer directly to the user requirements being tested for, making it relevant from a stakeholder point of view.
(ii) Project manager and project planner
Project managers can use the traceability information to control project progress. In a project life-cycle, project managers are supported by a correct traceability approach in accomplishing their different tasks:
• Project definition - An early traceability analysis during the work definition allows project managers to control that the work team and the client have the same perception of the project; this includes the delivered and the not delivered artefacts, how much does it costs, who will perform the work, how the work will be done and which benefits will be achieved.
• Workplan definition, development and managing - Matching goals with design elements is crucial to organize efficiently the time plan, giving priorities to the development of the core elements of the application and avoiding useless or superfluous features. Project managers can prevent conflicts and check the progresses of the different tasks related each other, with test procedures and with the main strategic goals. Conflicts between requirements can be discovered earlier and unexpected product delays avoided.
9 This concept will be extensively treated in section 4.
• Communication and quality management – Traceability is a powerful communication mean with clients, providing to project managers arguments and evidences of the project quality in terms of satisfaction of goals, needs and expectations [Palmer, 1997].
• Documentation management – Traceability analysis allows complete and refined documentation and specifications; the traceability chain10 provides a preferential way to order, link and organize each document or deliverable.
• Metrics management – All the relationships traced between parts of the application, features and services on one hand, and test procedures on the other hand, becomes crucial to give to project managers quantitative data to identify trends, support decisions and as indication of the good health of the project.
• Impact analysis and reuse – Project planners use a tracing approach to perform impact analysis;
requirements can be tracked to determine the impact of a required change on the entire project, on the workplan, on other feature of the application, on goals, etc. Requirements not yet satisfied by the implementation can be collected, and the work to be done to satisfy these remaining requirements can be estimated. Future systems will have reduced development time and effort because past implementation decisions can be reused.
(iii) Requirements engineer
Requirements engineers keep and elicit visions, strategic goals, constraints, user profiles, etc. from stakeholders and motivations, user goals, etc. from users11. A pre-requirements specification traceability analysis is needed to keep these relationships between stakeholders and goals, between users and goals, between goals and sub-goals in the refinement process and between sub-goals and requirements12. These people use the traceability information for the following purposes:
• Consistency check – Traceability analysis is used by requirements engineers to keep the consistency between the different information they consider, and in particular between requirements as indications for the design on one hand and goals and constraints as source and motivations for requirements on the other hand.
• Conflicts management – Conflicts between goals are usual, in particular between stakeholders goals and user goals. Traceability helps the analyst in finding a good compromise between conflicting goals, considering the relevance of stakeholders that own such goals and evaluating the impact that changes may have on other goals, sub-goals or requirements.
• Refinement management – During goals refinement activities it is crucial to keep all the relationships between high-level goals and derived or refined sub-goals. Traceability may also help in keeping an history of all the refinement changes performed in different moments of the project life-cycle and for different reasons (technology changes or constraints, budget constraints, timing, etc.).
• Prioritization – The traceability chain links as in a flow, stakeholders with goals and requirements; if all the relations are kept and updated, the requirements analyst can give a relative priority to each
10 According to Triacca [2001] a traceability chain shows the relationships traced between the components of an application framework, i.e. goals, requirements, tasks and design, and their influence on subsequent components.
11 For the definitions of vision, motivation, etc. see also Bolchini et al. [2005b].
12 Some structured methods (such as i*, KAOS or AWARE) provides conceptual tools to document the relationships between a stakeholder and the goals it express and between a requirement and the goal(s) it fulfil.
requirement or to groups of requirements that meet the needs of a certain stakeholder;
requirements related to more relevant stakeholder should be considered with higher priority respect to others.
(iv) Designer
Designers of software products are responsible to shape the information architecture of the application, considering the content structure, transitions between pieces of contents, interactive features, access to contents and features and navigation architecture. To keep the consistency of the entire project, designers take in consideration goals and requirements highlighted during the requirements analysis;
nevertheless, a major part of the final design has other motivations than requirements: for instance, some elements could have pure technical reasons or being just based on “good design” principles.
Usually, part of these reasons are not recorded and part are not explicitly perceived or understood. A traceability analysis allows eliciting hidden or unconscious knowledge and helps designers to show that the elements indicated in the conceptual design are not unusual, unnecessary or unmotivated. In particular, designers use the traceability information for accomplish the following tasks:
• Consistency check – A tracking information system should record the results of design, the justification of the results, the alternatives considered, and the assumptions made in a decision;
therefore a traceability analysis prevents from consistency problems between different parts of the project and may help in solve inconsistencies with technical implementation or with strategic goals.
• Requirements and goals compliance – Designers use traceability to understand dependencies between the requirements and to check whether all requirements are considered by the design;
therefore, they can more easily verify that a design satisfies the requirements or not. If a design element is not directly liked to a specific requirement, they can find arguments in traceability documents to justify their decisions in a more general relation with strategic goals or with non- functional requirements. A traceability approach force designer to ask themselves the “why”
question (before the client do it…).
• “Negative” design management – With “negative” design [Randazzo, 2004] I refer to the design elements that for any reason have been rejected or eliminated from the application. In most of the cases, the knowledge of which are these elements and why they have been deleted is crucial to measure their impact on the project. Traceability analysis support designers in keeping these kind of
“design history”, avoiding time-consuming features that for the same reasons would be rejected and considering alternate solutions for other similar cases.
• Impact analysis – Traces between the different elements of a project allow designers to evaluate possible consequences for changing a design feature in terms of compliance with requirements and goals or in terms of needed changes in implemented prototypes and applications. From another point of view, designers can understand the impact on the design of a change in requirements and take consequent decisions. Designers can use traceability information also to estimate the impact of a change in available implementation technology on the design assumptions and hence on the design alternatives.
• Solutions acceptance analysis – Starting from traceability documents, designers can understand the reasons why a certain design was accepted and another rejected, even when the design was
produced long time ago by a no more present designer. These reasons may relate design decisions to non-functional requirements, to unexpressed constraints or to more general stakeholders’ visions.
• Patterns reuse management - A traceability chain relates a specific need with a certain design solution; if the design is accepted, such a solution can be considered as a good one at least from a stakeholder point of view. Therefore, designers may reuse design components for similar needs in other projects because the assumptions under which the component will work are recorded in the traceability report. Besides, the tracking information system may become a kind of “corporate memory”, i.e. a library of solutions patterns and a way to refers to specific solutions in a fast and direct way; this “corporate memory” can be used in the work team to speed up decision-making in future development projects.
• Design revision – Traceability documents keep the knowledge about the relationships between requirements and design in a structured way; if there is a need to tune up or to revise a former project, designer can understand and/or remember previous decisions taken and properly “adjust”
the application.
(v) Verifier / validator
Verifiers in large projects provides a further consistency check of the final application; they base their job on traceability information to verify that all the strategic goals have been properly satisfied, that all the requirements have been taken into account, that design doesn’t have goldplating, that software meets with design specifications and that the application has been properly tested.
Validators use traceability relationships between requirements and test plans to prove that the system
"completely" meets the needs of the customer. In addition, test procedures can be identified that should be rerun to validate an implemented change. This saves test resources and allows the schedule to be streamlined.
(vi) Tester / usability inspector
Testers perform a detail evaluation of the system’s technical performances: the application should not
“crack” or generate errors in any condition of use. Usability inspectors are concerned with the application
“easy of use”: they check that the declared goals can be reached by users by the mean of the application in a efficient end effective way.
Testers can use traceability information from two points of view. First, they can perform their tests in a more systematic way; e.g. they can test features in relevance order or organize tests grouping features by stakeholder or by goal they meet. Second, in case of problems surfaced during the tests, they can indicate which exactly are the pieces of software or the design elements to review; they can also suggest a priority order for these problems based on the impact they have on the satisfaction of strategic goals.
Usability inspectors have to taken into account high-level goals of the product, evaluating it according to its real scope. Keeping traces between these two activities can help usability inspectors performing a more effective and efficient evaluation and showing that the main goals have been consistently tested.
Usability experts can also use entire parts of the traceability analysis to plan and prepare their evaluation:
in fact, inspectors need to know dependencies between user profiles, goals and features in the application to properly test the usability of that solutions. As for the testers case, to usability problems can be assign a priority and the inspectors can indicate on which element of the project they have an impact.
(vii) Maintainer
Maintainers “keep alive” the application; this is particularly true for interactive and web-based applications, where key success factors are to be up-to-date and always to adapt the communication and the business channels to new user or stakeholders’ needs.
Maintainers use the traceability information to decide how a required and accepted change will affect a system, i.e., which modules are directly affected and which other modules will experience residual effects.
Documenting an engineer’s design rationale helps the maintainer to understand the system. If a required change is implemented, understanding the existing solution structure helps to prevent the system from degrading. A maintainer can this way estimate the impact of a change in requirements on other requirements, discover conflicts dependencies, estimate the impact of a change in requirements on the implementation and estimate the permissibility of a change in implementation with respect to (unchanged) requirements.
1.2. Tracing approach
According to the Merriam-Webster dictionary, a method is “a way, technique, or process of or for doing something” and “a systematic procedure, technique, or mode of inquiry employed by or proper to a particular discipline or art”. In order to conceive, to apply or to understand a traceability method, the only knowledge about the meaning of the word “traceability” itself and about how traceability can be applied by different actors is not enough. A wider understanding about the elements composing a general traceability method and about processes and tools characterizing it is needed. Such a knowledge is indicated in literature with the name of tracing approach.
The concept of tracing approach (TA) refers to a generic term for methods, techniques and models enabling tracing activities [von Knethen & Paech, 2002]. A general TA, e.g. a traceability method, is characterised by (a) the purposes the activity may have, (b) the processes involved in the tracing activity, (c) the conceptual trace model on which the activity is based and (d) one or more tools enabling, facilitating or documenting such an activity [von Knethen & Paech, 2002]; Figure 3 summarises these four aspects shaping a kind of meta-TA.
(a) Purposes
As it has been shown in the previous paragraph, different stakeholder may have different goals and benefits in adopting a tracing technique. Therefore, different methods may differ because of stakeholder and goals supported. In other words, a tracing approach can be characterise by:
• Who wants to trace, who is the user of a traceability chain - Many different stakeholders (project sponsors, project managers, analysts, designers, maintainers, end users, etc.) are involved in the system development life cycle. The traceability needs of these stakeholders differ due to differences in their goals and priorities, and many problems of traceability stem from these differences in interest and understanding [Ramesh et al., 1993].
• Why and when traceability is provided - Traceability activity can be performed in different moments of a project’s lifecycle: during the system evolution (to refine the application design) or after the system evolution, i.e. at the beginning of a new project for a design tuning activity or for a reengineering process.
Figure 3. Core concepts of a tracing approach, adapted from [von Knethen & Paech, 2002]
(b) Process
Different tracing techniques may be characterised by the kind of activity or task supported; the process can involve one or a combination of the following:
• Define “entities”, i.e. elicit and define with stakeholders the objects to keep related each other, e.g.
requirements, design elements, test procedures, etc.
• Capture traces, i.e. trace the relationships between the different elements of the trace model.
• Analyse traces, i.e. interpret the relationships and highlight problems or weaknesses raised out from traceability, e.g. poor requirements covering, useless or unjustified design elements, etc.
• Represent traces, e.g. provide tools, procedures, checklists, etc. helping stakeholders and analysts in document, illustrate and display the traceability knowledge; summarise the results in a traceability report.
• Maintain traces, e.g. keep tracing information up-to-date as far as new decisions are taken or any change is made to the system status.
(c) Conceptual trace model
A conceptual trace model, also called reference model [Ramesh & Jarke 2001], defines what “trace entities” and “traces” are and which traces should be captured. Therefore, a conceptual trace model determines what is relevant and it identifies and formalises which aspects of the system are to be recorded and worked with. Such a model should provide also guidelines to identify a common way of dealing with the traces. Two sub-concepts, "entity" and "relationship", refine the concept.
(c.i) Entity
An entity is an object or an item of the traceability activity that represents the input or output of the system development process. According to Spence & Probasco [1998], a traceability item (i.e. an entity) is defined as “any textual, or model item, which needs to be explicitly traced from another textual, or model item, in order to keep track of the dependencies between them”. Examples of various types of entity include goals, requirements, assumptions, designs, system components, decisions, rationale, alternatives, critical success factors, etc. A trace may also capture the human co- operation in the design process, that is, how stakeholders contribute to the development. The entities that should be traced are determined by the purposes supported by the TA; according to von Knethen
& Paech [2002] the concept considers three aspects of an entity:
• the kind of the entity: it describes which software documents (e.g., requirements, test cases, design elements, etc.) should be involved in the conceptual trace model; examples of kinds of entity are: temporary work products and permanent work products [Lindvall, 1994];
requirements, specifications, and implementation [Ramesh & Edwards 1992];
• the granularity of the entity: also called "different levels of traceability" [Lindvall 1994], it determines the detail level of the entities involved, e.g. classes or attributes/methods of an object-oriented analysis, paragraphs or sentences of a textual requirement document;
• the attributes of the entity that should be added: they are traceability information because they allow, for example, tracing a requirement back to its source; examples of attributes are: effort [Carlsharmre & Regnell, 2000], priority (determined by the customer) [Tvete & Sundnes, 1999], source [Kirkman, 1998], status proposed/approved/designed/incorporated/ validated [Tvete &
Sundnes, 1999], status captured/specified/planned/realised [Carlsharmre & Regnell, 2000, status new/assigned/classified/selected/applied/rejected [Carlsharmre & Regnell, 2000], status optional/mandatory/deleted/desirable [Kirkman, 1998].
(c.ii) Relationship
The concept of "relationship" investigates different tracing approaches concerning the relationships that are suggested to be captured/maintained. The concept considers four aspects:
• the kinds of relationships described: from the literature, three general kinds of relationships can be identified: relationships between documentation entities on the same abstraction, relationships between documentation entities at different abstractions and relationships between documentation entities of different versions of a software product [Wieringa, 1997];
• the relationship direction: backward and forward, in pre- and post-RST, as shown in the first paragraph of this section;
• the relationship attributes: e.g., status, completion date, authorisation, responsible, priority, etc.
• the setting of relationships: this concept distinguishes between two kinds of approaches for setting relationships:
o implicit relationships - links that do not require manual setting, e.g. name tracing, where if names and abbreviations are used in the same way and are meant to denote the same things in two documents, then a degree of traceability between them may be established;
o explicit relationships - they are manually implemented references between documentation entities and came from external considerations supplied by the developers; so, for example, the linkage, or relationship, between a textual requirement and a use case that describes the requirement is determined solely by the decision of the developers that such a relationship has meaning.
Figure 4. Some kinds of link that may be represented by a traceability approach, adapted from [Wieringa, 1997]
(d) Tools
Traceability tools answer to the following problem: in what way providing access to and presenting the traced information? These tools may be conceptual tools, software tools or a combination of the two.
Conceptual tools are general techniques to represents entities and relationships of a conceptual trace model; according to von Knethen & Paech [2002] and to Wieringa [1995] the techniques used to keep trace of requirements include the following:
• Traceability matrices - A simple way to represent links between items is a matrix in which the horizontal and vertical dimensions list the items that can be linked, and the entries in the matrix represent links between these items; the items in both dimensions may or may not be the same.
