T E C H N I C A L A R T I C L E — P E E R - R E V I E W E D
Steel Strip Surface Defect Identification using Multiresolution Binarized Image Features
Zoheir Mentouri.Abdelkrim Moussaoui.Djalil Boudjehem. Hakim Doghmane
Submitted: 2 March 2020 / in revised form: 22 July 2020 / Accepted: 3 September 2020 ASM International 2020
Abstract The shaped steel strip, in the hot rolling pro- cess, may exhibit some surface flaws. Their origin could be the internal discontinuities in the input product or the thermomechanical transformation of the material, during the shaping process. Such defects are of a random occur- rence and may lead to costly rework operations or to a downgrading of the final product. So, they should be detected and identified as soon as possible, to allow a timely decision-making. For such a quality monitoring, the used vision systems are mainly based on an image description and a reliable classification. In this paper, we explore pre-defined image filters and work on a procedure to extract a discriminant image feature, while realizing the best trade-off between the improved recognition rate of the surface defects and the computing time. The proposed method is a multiresolution approach, based on the Binarized Statistical Image Features method, employed to date in biometrics. The filters, pre-learnt from natural images, are applied to steel defect images as a new surface structure indicator. They provide a quite discriminating
image description. A relevant data reduction is used toge- ther with a classifier to allow an efficient recognition rate of the defective hot rolled products.
Keywords Computer visionStatistical features ClassificationStrip surface defectsHot rolling process
Introduction
Computer vision systems are widely used in steel surface inspection. The general concept retained is that the defect images are processed in a way that allows the extraction of the most discriminant features, to ease the defect catego- rization. The published works in this field, mention numerous approaches. For instance, in an earlier study, Ref. [1] used a cost matrix method to optimize features selection and to facilitate the choice of a classification method of steel flat products, in cold rolling process.
Depending on the type of defects and the specificity of applications, different feature extractors have been applied, later. As reviewed by [2], morphological operations, joint spatial/frequency domain and spatial domain filtering have been found interesting for all types of steel surfaces.
Indeed, the true position of micro-defects, under complex conditions, has been detected on plates, at a rate of 93%, in [3]. In this study, a segmentation method based on math- ematics morphology filtering and Fisher discriminant approaches has been applied. Whereas to recognize scales and water marks, at a rate of 90.23%, on the same type of product, an undecimated wavelet transform and mathe- matical morphology operator have been applied in [4].
Moreover, in [5], where different wavelet feature sets were evaluated in detecting 24 types of defect classes, of hot rolled steel surfaces, it has been shown that a three-level Z. Mentouri (&)
Research Centre in Industrial Technologies CRTI, P.O. Box 64, 16014 Che´raga, Algiers, Algeria
e-mail: zoheirmentouri@yahoo.com Z. MentouriD. Boudjehem
Laboratory of Advanced Control-LABCAV, Universite´ 8 Mai 1945 Guelma, BP 401, 24000 Guelma, Algeria
A. Moussaoui
Laboratory of Electrical Engineering-LGEG, Universite´ 8 Mai 1945 Guelma, BP 401, 24000 Guelma, Algeria
H. Doghmane
Laboratory of Inverse Problems-PI:MIS, Universite´ 8 Mai 1945 Guelma, BP 401, 24000 Guelma, Algeria
https://doi.org/10.1007/s11668-020-01012-7
Haar feature set, outperformed the other tested wavelet sets as Daubechies, bi-orthogonal and even a texture-based segmentation and thresholding techniques. As for the Gabor filters, they have been the means widely used. Aided either by a specific lighting system or some transformation of the filtered images, they have allowed an accurate detection of some defects of a lumpy surface and other scratches in the hot rolling process [6,7].
In many other studies, the spatial domain filtering has been applied to allow a subsequent statistical analysis. The developed binarization method, in [8], has been interested in the target defect shape and image histogram, to construct the feature vector, detect and classify, by SVM, plate oil- marks and pseudo-defects. Whereas different statistical approaches, based on a background difference, region growing, clustering algorithm or pixel distribution, have given a good rating in the detection of periodical surface defects of steel strips [9–11].
Typically, a free-defect steel surface is rather regular, and any irregularity is, most likely, related to certain sur- face abnormalities that represent a defective region. In other words, the defect detection is often considered, as a texture analysis issue [12], and several studies have dealt with from this point of view. Thus, for such analysis, some statistical techniques as local binary pattern (LBP) and its extensions as the completed local binary pattern (CLBP) or the smoothed LBP have been used to resolve some prob- lems of detection and noise sensitivity enhancement of the features [13,14]. Such techniques have been considered in [15] as powerful means of texture measurement.
Therefore, there are various approaches for steel strip flaw detection and classification. However, despite their proved relative effectiveness, none of them can be con- sidered a unique or a standard method, which could be relevant for all surface defect types, whatever are their origin, size, position and orientation. It is why, the field still captures researcher attention to find, for a given case, the most appropriate technique or method combination. For instance, in [16], a uniform LBP has, recently, been used as a feature space element, together with a gray-level co-oc- currence matrix, Gabor filter and other feature extractors, to design an evolutionary classifier of steel surface defects, with a small sample set.
In the paper, we are presenting, a multiresolution approach, based on the recent image encoding method, namely the Binarized Statistical Image features [17–19], is used to describe surface defect images of hot rolled strips.
It combines BSIF filters in a single bank, which is applied to images to extract the textural information at different scales and resolutions. The proposed approach is assumed presenting some advantages, summarized as follows:
• The use of BSIF filters, in a multiresolution approach for the description of steel defect images, is a novelty in the industrial product surface inspection.
• The proposed approach achieves a better overall performance in terms of classification accuracy than the widely used methods.
• More than one defect type are considered in this multi- class defect categorization, unlike some other applica- tions, where, a good performance is reached, but concerns just a binary classification (defective region/
defect free), or finds out one or two specific types of steel flaws.
• The proposed approach has proved to be suitable for real-time application.
In the remainder of this paper, the sections ‘‘BSIF Pre- learnt Filters’’ and ‘‘One-Scale BSIF Application’’ pre- sents, respectively, an overview on the pre-learnt filters and the basic BSIF operation. In the section ‘‘BSIF Multires- olution Approach’’, the multiresolution BSIF approach is introduced as well as the general procedure of our appli- cation. The experimental study is presented in the section
‘‘Experimental Study’’. It includes the description of the used database and the defect image representation. As for the results, they are summarized and discussed in the section ‘‘Results and Discussion’’. The paper ends with a conclusion.
BSIF Pre-learnt Filters
In the BSIF method, the applied filters are pre-leant instead of manually designed, as in other techniques. The deter- mination procedure of these filters [17], which uses natural images provided by [20], is summarized as follows: A filter response is given by:
h¼Fx ðEq 1Þ
wherexis the vector of an image patch of sizem9mand Fan9m2matrix ofnstacked filters. The main property of the responseh is the maximum statistical independence of its individual responses hi. Thus, a filter matrix is assumed giving the response defined by:
h¼MPx¼My ðEq 2Þ
whereP is a canonical preprocessing matrix and M is an estimated orthogonal matrix. The first matrix (P) is com- puted with the use of principal component analysis (PCA), whereas the independent component analysis (ICA) algo- rithm is used to estimate the orthogonal matrix M. Then, the filter matrix is obtained by the simple product of the two matricesMandP. Figure1shows the representation of a filter sample response.
One-Scale BSIF Application
Based on the above pre-learnt filters, the BSIF application consists firstly in convolving a filter ofm9mof size with the image to obtain thehiresponse. The pixels new values are then binarized via a threshold at zero, as follows:
B¼1 iffTx[0 B¼0 Otherwise
ðEq 3Þ Moreover, with the use of, rather, a bank of kernels of a same sizem9m, a weight is affected to each individual binarized responses Bi. Then, all the binarized and weighted responses are summed to get a new code or the new pixel value mp of each image pixel, as shown in the formula4.
vp¼Xr
l¼1
½Code(ConvðFl;XÞÞ 2ðrlÞ ¼Xr
l¼1
ðBl2ðrlÞÞ ðEq 4Þ where in the used filter,ris the number of kernels defining the length of the bit string, and l its rank. A sample of filtered images is presented in Fig.2.
BSIF Multiresolution Approach
Principle
Usually, applying a filter to an image, for some objectives, may result in other unintended effects. As using an increased size filter, to remove more noise and gather information from a large image region, results in more blurred edges. Whereas a small spatial support area as 393 of size, for instance, enhances edges, but allows capturing the information of only small-scale structures.
This issue has been addressed in many studies, and robust multiscale approaches have been proposed, as with the local binary pattern. The extended form, noted LBP (P, R) has allowed the capture of more information and low frequency contents than the basic form [21].
The same concern may be with the individual use of a single BSIF filter. With the provided set [17], each three- dimensional ‘‘m9m9r’’ filter is defined by a window of size m9m and r kernels. The ‘‘r’’ also represents the number of bits of the computed pixel code. A large choice is provided, with the different window sizes (from 393 to 17917) and different bit string numbers (rfrom 5 to 12).
Then, a whole filter bank, for instance, ‘‘59595, 7 97 912,…, 1791798’’ is supposed to represent a maximum of spatial image frequencies, thanks to the multiple window sizes; and on the other hand, to capture more image details since the r bit string expresses the resolution of the pixel new codeword. Hence, such a combination of filters is assumed to make the feature vectors more representative of the texture variation.
Defect Image Representation
There are various local image descriptors as LBP, LPQ or BSIF. It is worthy of notice that with some descriptors as LBP, only the image itself constitutes the support of information. Whereas in the BSIF descriptor, the different filters stand for a new texture indicator. This provides a more extensive description of the pixel, based not only on its neighborhood, but also on the external information, brought by the natural images, used in the filters learning step.
Figure3 presents the followed procedure for the mul- tiresolution approach MR-BSIF. It consists in successive convolutions of each image, with a bank of filters of Fig. 1 Sample of pre-learnt filter of 797912 of size Fig. 2 Image samples—1st and 3rd columns: original defect images;
2nd and 4th columns: Filtered images
multiple scales and bit strings, followed by a zero-thresh- olding operation of the responses and the histogram calculation.
Note that the invariant properties of the histogram to image transformation, as rotation and translation, make of this tool the suitable means to describe such images, where the occurring defects are of a changing size, location and orientation.
Therefore, for a given filter from the selected bank ofn filters, the corresponding histogram of the processed image m, as defined in Eq. 5, is extracted:
Hf1ðs;rÞ ¼hh0s;r;h1s;r;. . .;hP1s;r iT
ðEq 5Þ where s, r and P represent, respectively, the filter scale (window size), the bit string number (pixel codeword res- olution) and the pixel intensity.
The stacked normalized elements of H(s, r) are com- puted by:
hps;r¼1 P
XP
j¼1
QpðjÞ ðEq 6Þ
QpðjÞ ¼ 1 ifVp¼p 0 Otherwise (
ðEq 7Þ
The final image representation is then built by the con- catenation of the histograms obtained by the application of all the filters of the multiresolution bank, as follows.
Hm¼ Hf1;Hf2;. . .;Hfn
T
ðEq 8Þ where Hfi correspond to the computed and concatenated histograms of the responses, obtained with the applied filter bank.
All the considered defect images, of all classes, are processed in the same way; and the large constructed his- tograms are grouped columnwise in a single representative matrix, as shown in Eq.9, to serve as a base for the sub- sequent steps of data reduction and classification.
H¼
Hf11 . . . Hf1M
... .. .
... Hfn1 . . . HfnM
2 64
3
75 ðEq 9Þ
where the length of columns is 2r= 2(r1?r2?_?ri)
,riis the bit string of each applied filter andMis the number of the processed defect images.
Image Feature Space
In order to optimize the data representation and speed the classification task, all encoded-image vectors are projected into a new reduced subspace which is created by the use of
Fig. 3 General scheme of a multiresolution BSIF procedure
the two most popular techniques [22,23], and carried out in three steps. The principal component analysis (PCA), known for its powerful data description, is firstly applied to a training set HT[2r, N], where the first dimension is the length of the randomly selected histograms from the matrix H, andNthe overall number of the training defect samples.
Then, the covariance matrix ofHTis computed with Eq. 10 and eigen-decomposed.
C¼uuT ðEq 10Þ
whereuis the centered data matrix ofHT.
Keeping the Z (\N) most significant eigen-vectors (reduction level around 83%), a PCA subspace computa- tion is executed as well as a whitening operation.
The linear discriminant analysis (LDA) is applied in a second step. Based on the class distinction principle, it makes use of the between-class scatter matrix of Eq.11.
Sb¼Xc
i¼1
ðuciuÞðuciuÞT ðEq 11Þ
and the within-class scatter matrix of Eq.12 Sw¼Xc
i¼1
Xqi
k2ci
ðckuciÞðckuciÞT ðEq 12Þ
withck: thekth sample in classci,uci: the mean vector in classci,u: the overall mean of the data-classes, c: the number of classes andqi: the vector number in classci.
Instead of employing the two actual scatter matrices above, the LDA method uses their respective projections into the obtained PCA subspace to compute the final and optimal projection matrixWfld. Thus, a ratio between the projections of Sb and Sw is computed according to the Fisher criterion formulated by Eq.13.
Wfld¼WTWpcaT SbWpcaW
WTWpcaT SwWpcaW ðEq 13Þ where WpcaT SbWpca and WpcaT SwWpca are, respectively, the projection ofSbandSw matrices into the PCA subspace.
Finally, all the images of the training set are projected into the new PCA-based LDA subspace (LDAPCA), to be ready for the comparison with the input images of the test set. These are projected in the same way into the created subspace before the computation of the matching distance by the classifier.
Experimental Study
Steel Surface Defect Dataset
In steel production, the surface defects always depend on the process parameters and the input product
characteristics. These defects are countless, diverse and of a random occurrence, what may explain the fact that in studies, it has been dealt, each time, with only some of them [2]. In this application, the used data includes six defect classes of hot rolled steel surfaces, from the Northeastern University (NEU) surface defect database.
Figure4 presents some variants of these defects.
The dataset, below, counts 1800 images with 300 grayscale images of 2009200 of size. It presents, within each class, enough variabilities in terms of defect orienta- tion, size and grayscale level.
These defects consist in Crazing: a type of fine cracks network; Inclusions: non-metallic particles that show through at the surface of the steel; Patches: a surface with the oxide not completely removed by a faulty pickling process; Pitted surface: sharp depressions in the surface, related to chemical attacks; Rolled-in scale: a scale par- tially rolled into the surface of the steel sheet and Scratches: sharp indentation in the surface, caused by a machine parts. Furthermore, the surface defects may be, as shown, in the above samples, localized, compact and with well-defined edges such as scratches, or sparse and affect the whole surface such as a pitted surface or crazing.
Filter Evaluation
A point that should be emphasized is that a highly dis- criminant descriptor would significantly support the testing step in reaching a good identification rate. To that end, all filters of m9m9rof size are, before all, individually, used in computing the resulting classification errors, pre- sented in Fig.5.
As shown in this figure, more than 5% of the 900 defects are misclassified when filters of low bit string are used (r= 5, 6 or 7), and this applies for all filter window sizes.
On the other hand, a meaningful decrease in the classifi- cation errors is shown when the r number is higher (r= 8–
12). This means that the higher is the bit string the slighter are the detected pixel variations, and the more the infor- mation is captured. Note that these individual scores are obtained with the KNN3 (three neighbors) classifiers.
Filter Bank Application
In the proposed multiresolution approach, which consists in applying rather a bank of filters, one may think gathering a large amount of features by a systematic application of the maximum available filters (i.e., filters of all scales and of a same bit string, or filters of a same scale with all bit strings). However, this may lead to more time-consuming and even a dimensionality curse [24] because of the pos- sible feature correlation.
The solution lies in finding the filter combination that brings the best trade-off regarding these considerations.
Then, driven by the filter individual results obtained in the section above, many filter combinations are considered using the algorithm of Fig.6. The procedure consists in:
• Randomly choose a group of filters among those that have allowed individual error rates less than 5%.
• All the possible filter combinations are considered to use a bank that may include 3–8 filters.
• The encoding operation and classification with each selected bank are executed to obtain the mean rate of many trials as well as the standard deviation.
Fig. 4 Example of NEU database defect images. One defect type per row: crazing, inclusion, patches, pitted surface, rolled-in-scale and scratches
Fig. 5 Filter individual scores in NEU defect recognition
Fig. 6 Searching procedure of the optimal set of filters
Results and Discussion
The evaluation procedure is carried out by the filter bank application, the LDAPCA data reduction and the classifi- cation by two classifiers: The non-parametric K-nearest neighbor classifier (with K= 3), based on the Euclidean matching distance and the supervised learning machine algorithm: multi-class SVM, with a radial basis kernel function (Rbf).
Table 1 summarizes the results corresponding to the main variants of the tested filter banks. In the column
‘Filter bank composition’, the ‘S’ letter refers to the filter window size and the ‘B’ to the bit string or codeword resolution; i.e., the S7B12 label refers to a single filter of 797912 of dimension, whereas S5-7-9_-17B12 refers to a bank including the filters as follows: 595912, 797912, 999912, 17917912.
As specified in the second column, a filter bank may include filters of all available scales from the filter dataset, or filters of all available resolutions with the same filter scale or may be mixed to be a multiresolution and multi- scale bank. For a reminder, in a pair of filters as 797 98 and 797 912, for instance, the twelve kernels of the second filter do not include the eight kernels of the first one. They are completely different since all filters have been previously learnt from a different set of natural image patches.
Each recognition rate, in Table 1, as well as the fol- lowing ones, represents the average of the one hundred scores, obtained by as much execution of the algorithm.
Further, three criteria are considered in assessing the
results: The recognition rate, the standard deviation, and the total processing time of an image defect.
As shown, the lines 1 and 3 confirm that increasing only the filter window size does not, necessarily, improve the results of KNN classification. Whereas an increased codeword resolution, in the lines 4–6, allows capturing slighter details. However, this needs more computing time (line 6). As for mixed banks, with filters of multiple win- dow sizes and bit strings, they seem to be more interesting, with a recognition rate exceeding 99.50%; and among the different tested filter combinations, a bank of a minimum of filters provides the highest score (line 8) with the KNN classifier.
To better appreciate its performance, our proposed defect labeling application is compared with other pattern recognition methods. The results are presented in Table2.
As reported in other studies, the Gabor transform has revealed interesting results, mainly in detecting defects on thick plates and slabs, [6,7,25]. Thus, in the first line of Table 2, the whole complex response of this Gabor trans- form is used to extract image features. The method parameters: the scale, orientation and downscaling factors, are tuned to allow capturing the maximum of the image abrupt changes. This image description allows moderate and quite similar recognition rates, with the two used classifiers. However, even with a downscaling operation, the description vectors remain important and result in memory and computing time constraints, particularly when an important number of Gabor filters is used.
This first comparison is followed by the assessment of the local phase quantization (LPQ) technique, which uses Table 1 Filter selections for NEU database defect recognition
Method
number Filter-bank type Filter bank composition
Number of
filter Classifier Results (%)
Time/Image (ms)
1 One scale of all resolutions S5B5-6-7-8-9-10-11-12 08 KNN 98.68±0.42 50.80
SVM 98.32±0.47 117.23
2 S7B5-6-7-8-9-10-11-12 07 KNN 99.09±0.33 58.34
SVM 98.84±0.42 122.39
3 S17B5-6-7-8-9-10-11-12 08 KNN 98.54±0.35 135.47
SVM 98.15±0.46 204.65
4 All scales of one resolution S5-7-9-11-13-15-17B5 07 KNN 98.10±0.42 42.12
SVM 97.42±0.50 116.84
5 S5-7-9-11-13-15-17B8 07 KNN 99.26±0.29 62.19
SVM 99.10±0.34 114.30
6 S5-7-9-11-13-15-17B12 07 KNN 99.43±0.27 105.70
SVM 99.25±0.31 162.57 7 Multiscale, multiresolution S3-11B8?S13B11?S5-7-9-15-17B12 08 KNN 99.55±0.24 112.13 SVM 99.33±0.26 165.63
8 S3,5B8?S13B11?S9,15B12 05 KNN 99.60±0.20 58.97
SVM 99.04±0.41 116.15
the local phase information and helps in capturing the local structure while being invariant to the image blur con- straints. The method shows a fast feature extraction with a relatively improved result. Better yet, the defect recogni- tion rate, exceeds 90%, and the computing time is improved, when the real part of the Gabor wavelets and the LPQ methods are combined.
As for the proposed approach (MRBSIF_LDAP-
CA_KNN), it allows the highest recognition rate, with the lowest standard deviation and an interesting overall com- puting time.
It is worth noting that the choice of rather a PCA-based LDA method, for data reduction, contributes to achieve the best score. As shown in the table above, the use of the basic PCA data reduction technique leads a relative lower defect identification level.
Then, the obtained results reflect the reliability and the robustness of the proposed approach. It performs consis- tently better than the other method combinations, as shown in Fig.7.
In this illustration, the proposed approach is highly rated, even with a training partition of a minimum size.
Table3is the generated confusion matrix. It presents the result that corresponds to the last identification rate of 99.64%, among the one hundred trials, of which the aver- age is 99.60% mentioned above. It shows that few defect classes are erroneously predicted, despite the high method efficiency. The shape, size and orientation characteristics of surface flaws are so random that some may present more similarities with the defects of the other classes than with
Fig. 7 Identifications rates of NEU database defects achieved by different methods
Table 3 Confusion matrix of NEU Database defects classification
Actual class
Predicted class
Cr In PS Pa RS Sc
Cr 150 0 0 0 0 0
In 0 150 0 0 0 0
PS 0 1 149 0 0 0
Pa 0 0 0 150 0 0
RS 0 0 0 0 150 0
SC 0 2 0 0 0 148
Table 2 Recognition rates of strip surface defects of the NEU database
Method num F. descriptor Parameters Classifier Labeled samples (/900) Results (%)
01 Gabor_LDAPCA O= 4,S= 2 KNN 667 74.16±1.26
SVM 674 74.90±1.28
O= 8,S= 5 KNN 800 88.86±0.95
SVM 791 87.89±0.99
02 LPQ_LDAPCA W.S = 999 KNN 866 96.18±0.57
SVM 861 95.71±053
03 Gabor_LPQ_LDAPCA D= 4,S= 2, W.S = 999 KNN 892 99.12±0.29
SVM 888 98.67±0.32
D= 8,S= 5, W.S = 999 KNN 891 98.98±0.25
SVM 886 98.44±0.86
04 Gabor_LPQ_PCA D= 4,S= 2, W.S = 999 KNN 829 92.16±0.84
D= 8,S= 5, W.S = 999 KNN 829 92.15±0.94
05 BSIF_LDAPCA S7B12 KNN 893 99.18±0.30
SVM 761 84.50±1.07
06 MRBSIF_LDAPCA S3,5B8?S13B11?S9,15B12 KNN 896 99.60±0.20
SVM 893 99.03±0.41
07 MRBSIF_PCA KNN 855 95.62±0.61
those of the same class. As in the table below, some scratch defects may be predicted as thin and elongated diverse inclusions.
Comparison with Previous Works
The comparison with some recent works that have dealt with the same database and applied different image descriptors and classifiers shows that the results obtained by previous methods are lower than the one obtained by our proposed approach. For more relevance of this com- parison, the training set is chosen of equal size as the one of the compared applications. Table 4 summarizes some reported results.
Real Time Considerations
Vision systems for online inspection usually require special image processing equipment, with parallel processing capabilities and optimized software; to allow meeting the application speed requirements. That is, in defect recog- nition application, the different tasks have to efficiently classify the defect while being completed in a stipulated time, with regard to the object maximum speed. Some reported works have mentioned an image processing time varying between 7 and 178 ms for steel process speeds of 2–100 m/s and have been qualified as suitable for real-time operations [2].
To get an idea about the execution speed, the time per defect sample of the different processing tasks has been evaluated on a machine endowed with an Intel i5-4590S CPU, 3.00 GHz, and 8 MB of dual channel memory. A particular care has been paid to the code organization as using functions rather than scripts, preferring local vari- ables instead of the global ones, variables pre-allocation, loops optimization and so on.
To approximate as much as possible this time, the averages of processing times are computed for different image defects, selected from different positions in the processed dataset, as shown in Fig.8.
The computing time is calculated as follows:
tj¼max (mean½ 100ðtijÞÞi¼1:3; ðEq 14Þ
T ¼mean6ðtjÞj¼1:6; ðEq 15Þ
where tij is the processing time of an image defect of classj,tjis the maximum processing time among those of three images taken at different positions from thejth class, andTis the processing time overall average.
Table 5 shows the computed times, broadly taken by feature extraction (filtering and projection operations) and classification. Each indication, in this table, represents the average time in milliseconds of the maxima obtained with different defect variants. As shown, the filtering operation lasts more time compared to the other tasks. Clearly, this time can be easily, reduced with the use of dedicated industrial equipment. Nevertheless, as it is, this time remains interesting about the computing time information reported in [2].
Conclusion
In steel strip surface inspection, the effectiveness of an automatic defect identification procedure highly relies on the feature extraction step. This paper deals with the BSIF
Table 4 Comparison of identification rates (%) of NEU surface defect database
Work Refs. Features descriptor Classifier Results (%)
Kenchen [14] AECLBP SVM 98.93
Kenchen [14] CLBP SVM 98.28
Mang [16] ULBP, GLCM, HOG, Gabor filter SVM, Bayes Kernel 96.30
Kenchen [26] SCN SVM 98.60
Li [27] CNN CNN 99.05
Achour [28] DST-GLCM_PCA SVM 94.11
Fei [29] Bilinear model CNN 99.44
Suggested MRBSIF_LDA KNN 99.60±0.20
Fig. 8 Test defect selection from the beginning, middle and the end positions within a class
method, used to date in biometrics. With this pixel encoder, the different kernels, of the applied filter set, stand as an external tool for image structure revealing. They allow a description based on the processed image itself and on the external information brought by the natural images used in the filter learning step. These encoders have been applied in a multiresolution approach which has clearly outper- formed the compared methods and some others of previous works. In fact, the study has shown a good overall per- formance. It consists in a higher average rate in the recognition of six defect types in the steel hot rolling process, a lower standard deviation revealing its robustness and a reasonable lasting time of the sample processing. The obtained results have demonstrated the suitability of the proposed approach for an industrial application.
References
1. F. Dupont, C. Odet, M. Carton, Optimization of the Recognition of Defects in Flat Steel Products with the Cost Matrices Theory.
NDT&E Int.30(1), 3–10 (1997)
2. N. Neogi, D.K. Mohanta, P.K. Dutta, Review of Vision-Based Steel Surface Inspection Systems. EURASIP J. Image Video Process.1, 50 (2014)
3. J.H. Guo, X.D. Meng, M.D. Xiong, Study on Defection Seg- mentation for Steel Surface Image Based on Image Edge Detection and Fisher Discriminant. J. Phys. Conf. Ser.48, 364–
368 (2006)
4. W. Xiu-yong, X. Ke, X. Jin-wu, Application of Undecimated Wavelet Transform to Surface Defect Detection of Hot Rolled Steel Plates, inIEEE Congress on Image and, Signal Processing (2008), pp. 528–532
5. S. Ghorai, A. Mukherjee, M. Gangadaran, P.K. Dutta, Automatic Defect Detection on Hot-Rolled Flat Steel Products. IEEE Trans.
Instrum. Meas.62(3), 612–621 (2013)
6. Y.J. Jeon, D.C. Choi, J.P. Yun, C. Park, H. Bae, S.W. Kim, Automated Inspection Algorithm for Thick Plate Using Dual Light Switching Lighting Method. Int. J. Electr. Comput. Eng.
6(12), 1475–1478 (2012)
7. C. Tikhe, J.S. Chitode, Metal Surface Inspection for Defect Detection and Classification using Gabor Filter. Int. J. Innov. Res.
Sci. Eng. Technol.3(6), 13702 (2014)
8. D.C. Choi, Y.J. Jeon, S.J. Lee, J.P. Yun, S.W. Kim, Oilmarks Detection Algorithm in Steel Plates, Latest Trends on Systems, in Proceedings of the 18th International Conference on Systems (part of CSCC ’14), vol 1 (2014), pp. 239–242
9. G. Wu, H. Kwak, S. Jang, K. Xu, J. Xu, Design of Online Surface Inspection System of Hot Rolled strips, in Proceedings of the IEEE International Conference on Automation and Logistics (Qingdao, China, 2008), pp. 2291–2295
10. F.G. Bulnes, R. Usamentiaga, DF Garc’ia, J Molleda, Vision- Based Sensor for Early Detection of Periodical Defects in Web Materials. Sensors 12, 10788–10809 (2012). https://doi.org/10.
3390/s120810788
11. D. Djukic, S. Spuzic, Statistical Discriminator of Surface Defects on Hot Rolled Steel, inProceedings of Image and Vision Com- puting(University of Waikato, Hamilton, New Zealand, 2007), pp. 158–163
12. X. Xie, A Review of Recent Advances in Surface Defect Detection Using Texture Analysis Techniques. Electron. Lett.
Comput. Vis. Image Anal.7(3), 1–22 (2008)
13. M. Chu, R. Gong, Invariant Feature Extraction Method Based on Smoothed Local Binary Pattern for Strip Steel Surface Defect.
ISIJ Int.55(9), 1956–1962 (2015)
14. K. Song, Y. Yan, A Noise Robust Method Based on Completed Local Binary Patterns for Hot-Rolled Steel Strip Surface Defects.
Appl. Surf. Sci.285, 858–864 (2013)
15. T. Maenpaa,Surface Quality Assessment with Advanced Texture Analysis Techniques (Proceedings of International Surface Inspection Summit, Luxembourg, 2006)
16. M. Xiao, M. Jiang, G. Li, Li Xie, Li Yi, An Evolutionary Clas- sifier for Steel Surface Defects with Small Sample Set. EURASIP J. Image Video Processing (2017). https://doi.org/10.1186/
s13640-017-0197-y
17. J. Kannala, E. Rahtu, BSIF: Binarized Statistical Image Features, in Proceedings of 21st International Conference on Pattern Recognition (ICPR 2012) (Tsukuba, Japan, 2012), pp. 1363–
1366.
18. J. Ylioinas, J. Kannala, A. Hadid, M. Pietika¨inen, Face Recog- nition Using Smoothed High-Dimensional Representation, in Image Analysis,SCIA. Lecture Notes in Computer Science, vol 9127, ed. by R. Paulsen, K. Pedersen (Springer, Cham, 2015) 19. R.A. Shervin,Multiscale Binarised Statistical Image Features for
Symmetric Face Matching Using Multiple Descriptor Fusion Based on Class-Specific LDA(Springer, London, 2015) 20. A. Hyvarinen, J. Hurri, P.O. Hoyer,Natural Image Statistics. A
Probabilistic Approach to Early Computational Vision(Springer, Berlin, 2009)
21. T. Ojala, M. Pietika¨inen, T. Maenpaa, Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns. IEEE Trans. Pattern Anal. Mach. Intell.24(7), 971–987 (2002)
22. P.N. Belhumeur, J.P. Hespanha, D.J. Kriegman, Eigenfaces vs.
Fisherfaces: Recognition Using Class Specific Linear Projection.
IEEE Trans. Pattern Anal. Mach. Intell.19(7), 711–720 (1997) 23. A.M. Martinez, A.C. Kak, PCA Versus LDA. IEEE Trans. Pat-
tern Anal. Mach. Intell.23(2), 228–233 (2001)
24. P.H. Bugatti, A.J.M. Traina, J.C. Felipe, C. Traina, A New Method to Efficiently Reduce Histogram Dimensionality, in Medical Imaging 2008: Computer-Aided Diagnosis, vol. 6915, ed. by M.L. Giger, N. Karssemeijer (2008), pp. 1605–7422.
https://doi.org/https://doi.org/10.1117/12.770512
25. Y. Jeon, D.-C. Choi, J.P. Yun, C. Park, S.W. Kim. Detection of Scratch Defects on Slab Surface, in1th International Conference on Control, Automation and Systems, 2011, pp. 1274–1278 26. K. Song, S. Hu, Y. Yan, Automatic Recognition of Surface
Defects on Hot-rolled Steel Strip using Scattering Convolution Network. J. Comput. Inf. Syst.10(7), 3049–3055 (2014).https://
doi.org/10.12733/jcis10026
27. L. Yi, G. Li, M. Jiang, An End-to-End Steel Strip Surface Defects Recognition System Based on Convolutional Neural Networks.
Int Steel Res (2016).https://doi.org/10.1002/srin.201600068 Table 5 Average time per processed NEU defect image
Processing task
Time per surface defect sample (ms)
Average time (T)
Cr In PS Pa RS SC
Filtering operation
59.13 58.47 58.80 58.63 59.03 58.33 58.73 Projection 0.08 0.06 0.06 0.06 0.06 0.07 0.06 Classification 0.18 0.18 0.18 0.18 0.18 0.17 0.18 Total 59.39 58.71 59.04 58.87 59.27 58.57 58.97
28. M.W. Ashour, F. Khalid, A.A. Halin, L.N. Abdullah, S.H. Dar- wish, Surface Defects Classification of Hot-Rolled Steel Strips Using Multi-directional Shearlet Features. J. Sci. Eng. Arab.
(2018).https://doi.org/10.1007/s13369-018-3329-5
29. F. Zhou, G. Liu, F. Xu, H. Deng, A Generic Automated Surface Defect Detection Based on a Bilinear Model. Appl. Sci.9, 3159 (2019).https://doi.org/10.3390/app9153159
Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
