Poster: Behavioral Model-Based Testing of Variability Intensive Systems

RESEARCH OUTPUTS / RÉSULTATS DE RECHERCHE

Author(s) - Auteur(s) :

Publication date - Date de publication :

Permanent link - Permalien :

Rights / License - Licence de droit d’auteur :

Bibliothèque Universitaire Moretus Plantin

Institutional Repository - Research Portal

Dépôt Institutionnel - Portail de la Recherche

researchportal.unamur.be

University of Namur

Poster: Behavioral Model-Based Testing of Variability Intensive Systems

Devroey, Xavier

Publication date:

2015

Document Version

Peer reviewed version

Link to publication

Citation for pulished version (HARVARD):

Devroey, X 2015, 'Poster: Behavioral Model-Based Testing of Variability Intensive Systems', Journée des

Doctorants en Informatique 2015, Namur, Belgium, 26/07/14 - 26/07/14.

General rights

Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain

• You may freely distribute the URL identifying the publication in the public portal ?

Take down policy

If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.

Behavioral Model-Based Testing of

Variability Intensive Systems

Xavier Devroey, Gilles Perrouin, Pierre-Yves Schobbens, Patrick Heymans

Contact:

[email protected]

https://projects.info.unamur.be/

vibes/

▪ Focus on Variability-Intensive Behavioural Models (software product lines, plugin-based systems, etc.)

▪ Formal Foundations (Featured Transition Systems) suitable for Analysis and Checking

▪ Combining Model-checking and Test-case Generation techniques in VIBeS Framework

▪ Cover various Testing Activities (test-case seleciton, prioritization, mutation assessment, etc.)

Bibliography

Future Work

[1] Devroey, X., et al. (2014). A Variability Perspective of Mutation Analysis. In Vision &

Challenges @ FSE'14. Hong Kong: ACM.

[2] Devroey, X., et al. (2014). Abstract Test Case Generation for Behavioural Testing of

Software Product Lines. In SPLat @ SPLC'15. Florence, Italy: ACM.

[3] Devroey, X., et al. (2014). Coverage Criteria for Behavioural Testing of Software Product

Lines. In ISoLA'14. Corfu, Greece: Springer.

[4] Devroey, X., et al. (2014). Towards Statistical Prioritization for Software Product Lines

Testing. In VaMoS'14. Nice, France: ACM.

[5] Devroey, X., et al. (2012). A Vision for Behavioural Model-Driven Validation of Software

Product Lines. In ISoLA'12. Crete, Greece: Springer.

▪ Higher order and equivalent mutants detect./gen.

▪ Test cases generation using counter examples in

ProVeLines FTS model checker

▪ Abstract test cases concretization

▪ Empirical assessment on an industrial case study

Test Model

CPTerminal t

Identification id

DirectDebit

dd

CreditCard

cd

Online

on

Offline

off

PIN

pin

Signature

sig

req.

[1..*]

PaymentSchema ps

Connectivity c

opt.

Init

Card_in

Aborted

insert_card

abort

remove

_card

App_init

abort

CH_verified

check_PIN

_online

/

on

∧

pin

check_PIN

_offline

/

off

∧

pin

check

_signature

/ ¬

dd

∧

sig

Go_online

Go_offline

NO_GO

no_go

go_offline /

off

go_online /

on

abort

Issuer_responded

ask_issuer

issuer_rejects

Completed

issuer_accepts

accepts

remove_card

init_schema /

dd

∨

cr

Featured Transition System Java DSL

public

class

CPTerminalFts

extends

FeaturedTransitionSystemDefinition {

@Override

protected

void

define() {

initial("i");

from("i").action("insert_card")

.to("CardIn");

from("cardIn").action("init_schema")

.fexpr("dd || cr").to("App_init");

...

}}

Feature Model

Boolean Conjunctive Normal Form (CNF) in DIMACS

format used as input to SAT and BDD solvers

Models Def.

Model-Based Testing Process

Test

Engineer

Requirements

Test

Model

Test Case

Generator

1.

models

2. selection

criteria

Abstract

Tests

Test Script

Generator

Test Scripts

3. concrete mapping

System

Under Test

Test

Results

4. analyses

1. Models definition (Java DSL)

2. Selection criteria definition

■ Maximizing

- Structural coverage (state, transition, paths, ...) [2,3]

- Usage coverage (based on a usage model) [4]

- Dissimilarity measure (based on distance def.)

- Number of mutants killed by the tests [1]

■ Minimizing

- Number of test cases in the test set [3]

- Number of valid configurations needed by the test set [3]

3. Concrete mapping definition (TBD)

4. Tests results analysis (TBD)

Structural coverage

FeaturedTransitionSystem

fts

=

new

CPTerminalFts().getTransitionSystem();

SolverFacade

solver

=

new

Sat4JSolverFacade(dimacsFile);

TestSet

set

= allStatesSelection(

fts

,

solver

);

Dissimilarity measure maximization

...

set

= dissimilarLocalMaximumDistance(

fts

,

solver

,

nbrTestCases

,

time

);

...

set

= dissimilarGlobalMaximumDistance(

fts

,

solver

,

nbrTestCases

,

time

);

Selection Criteria Def.

■ Goals: Quality Assessment (QA)

- Tester: QA of a set of abstract test cases

- Researcher: QA of a test case selection method

■ Mutant = Test Model with injected fault(s)

■ Result of the application of a mutation operator

- Act. Exchange, State Missing, Trans. Dest. Exchange, etc.

■ Execution of the test cases on the mutants

Number of mutants detected

■ Mutation score =

Total number of mutants

Model Based Mutation Testing

Mutant Execution using

Featured Mutants Model

Variability Intensive system

Behavioural teSting Framework

VIBeS

toolbox

DSL

selection

transformation

execution

mutation

core

fexpression

<<uses>>

●

●

Alive

Killed FMM

5e+01

5e+02

5e+03

5e+04

Claroline

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Alive

Killed FMM

1e+02

1e+04

1e+06

AGE−RR

●

●

●

●

●

●

●

●

●

●

●

●

●

Alive

Killed FMM

1e+03

1e+04

1e+05

1e+06

Random

T

ime

i

n

mi

cro

se

c.