Assessing the quality of company’s maintenance data reporting: a case study

(1)

4th International Conference on Data Management Technologies and Applications

Assessing the quality of company’s maintenance data reporting: a case study

Manik M ÂDHIKERMI ¹ , Sylvain K ÛBLER ^1,2 , Andrea B ûda ¹ , Kary F ^r¨ âmling ¹ &

Aalto University, School of Science and Technology, Finland

University of Luxembourg, Interdisciplinary Centre for Security, Reliability & Trust, Luxembourg Corresponding Author: manik.madhikermi@aalto.fi

Problem in context

B ^u siness are increasingly using their enterprise data for strategic decision-making activities. In fact, information, derived from data, has become one of the most important tools for businesses to gain competitive edge. Data quality assessment is now a hot topic in numerous sectors and considerable research has been carried out in this respect.

Nonetheless, existing frameworks often need to be adapted with respect to the use case needs and features. Given this, the present work develops a methodology for assessing the quality of enterprises’ daily maintenance reporting, relying both on an existing data quality framework and on a Multi-Criteria Decision Making (MCDM) technique. This work is applied in cooperation with a Finnish multinational company in order to evaluate and rank different company sites/office branches (carrying out maintenance activities) according to the quality of their data reporting. Based on this evaluation, the industrial partner wants to establish new action plans for enhanced reporting practices.

Objectives

T ^he paper’s contribution is to develop a methodology combining an existing data quality framework (Krogstie’s framework [1]) and an efficient MCDM technique (Analytic Hierarchy Process – AHP [2]) in order to assess the quality of enterprises’

daily maintenance reporting. The overall methodology is depicted in Figure 1.

S

ITE

1 S

ITE

2 . . . ^S

^ITE

ⁿ

x x x

x y ^x ^x ^x y x ^y ^x ^x y ^x

OEM Database

Maintenance Operator (Site 1) Report ID : 1D

Asset Location Description Actual End-Date Target Start Date

. . .

Scheduled Start Date Scheduled End Date Maintenance Operator (Site 1)

ID : 1389706

. . .

Scheduled Start Date Scheduled End Date

Maintenance Operator (Site 1) ID : 1389706

. . .

Maintenance Operator (Site 1) Report ID : 1A

. . .

Power Controller 4v Done

28/08/2014 23/08/2014

24/08/2014 27/08/2014

Front axle 34.8YH Done

02/05/2014

07/06/2014

Maintenance Operator (Site 1) Report ID : 1A Asset Location Description Actual End-Date Target Start Date

. . .

Report ID : 1A Asset Location Description Actual End-Date Target Start Date

. . .

Report ID : 1A

. . .

Report ID : 1A

. . .

Maintenance Operator (Site2) Report ID :2A Asset Location Description Actual End-Date Target Start Date

. . .

Scheduled Start Date Scheduled End Date Maintenance Operator (Site2)

Report ID :2A

. . .

Report ID :2A Asset Location Description Actual End-Date Target Start Date

. . .

Report ID :2A

. . .

Report ID :2A

. . .

Maintenance Operator (Siten) Report ID :nA Asset Location Description Actual End-Date Target Start Date

. . .

Scheduled Start Date Scheduled End Date Maintenance Operator (Siten)

Report ID :nA

. . .

Report ID :nA Asset Location Description Actual End-Date Target Start Date

. . .

Maintenance Operator (Siten) Report ID :nA Asset Location Description Actual End-Date Target Start Date

. . .

Report ID :nA

. . .

Report ID :nA

. . .

Report ID :nA

. . .

Report ID :nA

. . .

Report ID :nA

. . .

Report ID :nA

. . .

Maintenance Operator (Siten) Report ID :nD

. . .

ID : 1389706

. . .

Maintenance Operator (Site 1) ID : 1389706

. . .

Maintenance Operator (Siten) Report ID :nA

. . .

Fuel System 01X.2 System changed by...

28/08/2014 23/08/2014

24/08/2014 27/08/2014

Chassis has been re...

28/08/2014 23/08/2014

24/08/2014 27/08/2014

Maintenance Operators per OEM’s Site

Example when comparing operator reports between Site 1 and Site n

Data Quality Assessment of OME’s Maintenance Reporting

C_B₁ : Length of Work Description One world (”Done”) is too short to properly

describe the maintenance opration

The description seems to be long enough in reportsnA & nD

C_B₂ : Work Log Variation No variation between the operator reports,

i.e. between report 1A & 1D in that example

The content of the work description reported by the operator often vary C_C₁ : Asset Location Reported

Field “Asset Location” filled out in report 1A as well as in report 1D

Field “Asset Location” filled out in reportnA but not in report nD. . . . . .

C_T : Average Delay of Reporting Reports 1A was made 1h after the task, while

report 1D was made with a delay of 3 weeks Both ReportsnA and nD have been

made with a delay inferior to 2h

MCDM technique Site ranking considering all reports, from all operators, from all sites : {Site 1, Site 2, Site 3. . . Siten}

2 3

S^ITE 1 S^ITE 2

1

S^ITE n

Figure 1: Stages composing the maintenance reporting quality assessment framework

Figure 2 shows how our MCDM ranking problem has been broken down into a hierarchical structure consisting of four levels :

➫ Level 1 : overall goal of the study is to rank the different OEM company sites in terms of maintenance reporting quality ;

➫ Levels 2 and 3 : set of data quality criteria/sub-criteria, used to assess the mainte- nance reporting quality (derived from Krogstie’s framework and listed in Table 1) ;

➫ Level 4 : alternatives that are the OEM company sites ;

Site 1 Site 2 Site 3 Site 4

. . . . . . . . . . . . . . .

_{Site 54}

CB1 CB2 CB3 CC1 CC2 CC3 CC4 CC5 CC6 CC7 CC8 CT

Believability Completeness Timeliness

Reporting Quality Assessment and Ranking of OEM Sites

Level 1

Level 2

Level 3

Level 4

Figure 2: AHP structure of the maintenance reporting quality assessment problem

Table 1: Criteria and its sub-criteria description related to the data quality dimensions

.2 Criteria Sub-Criteria Description

Believability (C^B)

Length of Work Description (C^B1) Length of the work description related to a work order

Work Log Variation (C^B2) Work Description variation among the different operator reports Technician Log Variation (C^B³) Technical log variation among the different operator reports

Completeness (C^C)

Asset Location reported (C^C1) Location of asset within product where maintenance has been done Description reported (C^C2) Description of work to be done in particular maintenance work Actual Finish Date reported (C^C3) Actual Finish date and time of work completed

Target Start Date reported (C^C4) Targeted start date of the maintenance work Target Finish Date reported (C^C5) Targeted finish date of the maintenance work DLC Code reported (C^C6) Actual location of the defect within product Schedule Start Date reported (C^C7) Scheduled start date of the maintenance work Schedule Finish Date reported (C^C8) Scheduled Finish date of the maintenance work Timeliness (C^T) This is average delay of reporting on individual site

Case study & Results

The analysis on the maintenance reporting data has been carried out considering 54 maintenance sites of the Finnish company, and datasets collected during two years.

The different sites have been assessed with respect to each criteria, and the resulting scores have been aggregated in order to get an overall score of the maintenance re- porting quality. Figure 3 gives insight into the comparison of sites 11 , 32 , 37 and 47 when aggregating the scores at level 2 of the AHP structure (i.e., with respect to the three criteria : Believability, Completeness and Timeliness ; cf. Figure 2).

Timeliness

1st

12th

23th

34th

1st 12th

23th 34th

45th 54th

45th 34th 23th 12th 1st

Site 32

Site 11 Site 47

Site 37

Completeness Believability

Figure 3: Comparison of sites 11, 32, 37 and 47

The overall ranking of the maintenance sites (i.e., when aggregating the scores at level 1 of the AHP structure) is presented in Figure 4. The histogram shows that some quality scores dropped below “ 0 ” ; the reason being that a penalty score has been introduced when a report field was left empty.

Site 11Site 1Site 18Site 34Site 25Site 33Site 10Site 43Site 42Site 49Site 32Site 4Site 35Site 9Site 12Site 5Site 22Site 52Site 39Site 21Site 2Site 46Site 36Site 38Site 51Site 40Site 16Site 50Site 27Site 13Site 41Site 23Site 3Site 28Site 47Site 20Site 26Site 44Site 6Site 7Site 29Site 24Site 17Site 19Site 14Site 8Site 30Site 31Site 45Site 53Site 54Site 37Site 48Site 15 -1

-0.8 -0.6 -0.4 -0.2 0 0.2

O ve ra ll D a ta Q u a li ty S co re

Figure 4: Site ranking based on the maintenance reporting quality assessment

References

[1] Krogstie, J., Lindland, O. I., and Sindre, G.(1995) Defining quality aspects for conceptual models, Proceedings of the IFIP8.1 Working Conference on Information Systems Concepts : Towards a Consolidation of Views, 1995 :216-231.

[2 Saaty, T. L. (1980) The analytic hierarchy process : planning, priority setting, resources allocation, New York : McGraw.

The 4th International Conference on Data Management Technologies and Applications, July 2015, Colmar,Alsace, France

Assessing the quality of company’s maintenance data reporting: a case study

4th International Conference on Data Management Technologies and Applications