HAL Id: hal-03170968
https://hal.archives-ouvertes.fr/hal-03170968
Submitted on 16 Mar 2021
HAL is a multi-disciplinary open access
archive for the deposit and dissemination of
sci-entific research documents, whether they are
pub-lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d’enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.
A Fast Segmentation Method for Fire Forest Images
Based on Multiscale Transform and PCA
Lotfi Tlig, Moez Bouchouicha, Mohamed Tlig, Mounir Sayadi, Eric Moreau
To cite this version:
Lotfi Tlig, Moez Bouchouicha, Mohamed Tlig, Mounir Sayadi, Eric Moreau. A Fast Segmentation
Method for Fire Forest Images Based on Multiscale Transform and PCA. Sensors, MDPI, 2020, 20
(22), pp.6429. �10.3390/s20226429�. �hal-03170968�
sensors
Article
A Fast Segmentation Method for Fire Forest
Images Based on Multiscale Transform and PCA
Lotfi Tlig1,* , Moez Bouchouicha2 , Mohamed Tlig1,2, Mounir Sayadi1 and Eric Moreau2
1 Member of SIME Laboratory, ENSIT University of Tunis, Tunis 1008, Tunisia; mohamed.tlig@ensit.u-tunis.tn (M.T.); mounirsayadi@yahoo.fr (M.S.) 2 Aix Marseille Univ, Université de Toulon, CNRS, LIS, 83041 Toulon, France;
moez.bouchouicha@univ-tln.fr (M.B.); eric.moreau@univ-tln.fr (E.M.) * Correspondence: lotfi.tlig@isimg.tn
Received: 15 October 2020; Accepted: 5 November 2020; Published: 10 November 2020 Abstract:Forests provide various important things to human life. Fire is one of the main disasters in the world. Nowadays, the forest fire incidences endanger the ecosystem and destroy the native flora and fauna. This affects individual life, community and wildlife. Thus, it is essential to monitor and protect the forests and their assets. Nowadays, image processing outputs a lot of required information and measures for the implementation of advanced forest fire-fighting strategies. This work addresses a new color image segmentation method based on principal component analysis (PCA) and Gabor filter responses. Our method introduces a new superpixels extraction strategy that takes full account of two objectives: regional consistency and robustness to added noises. The novel approach is tested on various color images. Extensive experiments show that our method obviously outperforms existing segmentation variants on real and synthetic images of fire forest scenes, and also achieves outstanding performance on other popular benchmarked images (e.g., BSDS, MRSC). The merits of our proposed approach are that it is not sensitive to added noises and that the segmentation performance is higher with images of nonhomogeneous regions.
Keywords: Gabor filtering; PCA morphological transformations; fuzzy clustering; color image segmentation; fire forest
1. Introduction
The conventional detection systems of smoke and fire use sensors [1]. One of the major drawbacks, is that the systems do not issue the alarm unless the particles reach the sensors [2]. Recently, as an appropriate alternative to conventional techniques, vision-based fire and smoke detection methods have been adopted. Here, smoke and fire are regarded as a specific kind of texture. It is difficult to accurately detect the appearance of mentioned regions from images due to large variations of color intensities and texture. Although, many research works confirmed that texture features play a very important role in smoke and fire detection [3,4]. A wide recent work demonstrated that the multi-scale based techniques play an important role in smoke and texture classification [5,6]. Developed methods cover both areas; images and videos processing [4,7,8]. In this work, we aim to segment images into significant regions. This will be used to generate useful information for our project. In this paper, we propose a new segmentation approach based on Gabor filtering and Principal Component Analysis (PCA). The proposed method is based on the modification of the superpixels extraction methodology to increase the robustness to added noises and to improve the segmentation accuracy of the fire forests color images. For the extracted features clustering, we used the new version of the fuzzy classifier recently proposed in [9]. Our choice is done regarding the
Sensors 2020, 20, 6429 2 of 27
higher performance of the mentioned fuzzy method compared to a large variety of clustering methods; FCM, FGFCM, HMRF-FCM, FLICM, NWFCM, KWFLICM, NDFCM, FR- FCM, and Liu’s algorithm. In [9], Lei et al. introduced a fast fuzzy clustering algorithm to address the problem of computational segmentation complexity of color image segmentation with higher resolution. This was conducted by the use of adaptive local spatial information provided by a pre-segmentation task. Despite its higher accuracy compared to a large number of used algorithms, this method remains limited due to many drawbacks. This is experimentally noted with blurred images and images with nonhomogeneous regions. In this work, our research is focused on the part where we have cited a lower efficiency. It is the task of superpixels extraction. As mentioned above, we have introduced the multiscale image transformation based on the Gabor filtering.
Many applications approved that Gabor feature sets are high-dimensional (typically defined by the number of orientations and frequencies). Concatenating all the feature images tends to exacerbate the dimensionality problems and computing complexity [4,6]. To overcome this issue, we introduce a dimensionality reducer before the superpixels extraction stage. In literature, there are many proposed dimensionality reduction methods; Independent Component Analysis (ICA), Principal Component Analysis (PCA), and Canonical Correlation Analysis (CCA) [10,11]. In this work, we find that PCA is sufficient.
Note that our intention was not to develop an end to end new segmentation approach of color image. Rather, we propose improving this task in general, by integrating several methodologies. As a first goal, these methodologies (multi-resolution filtering, superpixels computing and fuzzy clustering) work together to provide reliable segmentation results, characterized by a higher segmentation accuracy and robustness. The second is to reduce the computing complexity and speedup the segmentation process. By this work, we present two contributions as given below:
(1) A multiresolution image transformation based on 2-D Gabor filtering combined with a morphological gradient construction to generate a superpixel image with accurate boundaries. This proposition integrates a multiscale neighboring system to solve the problems of rotation, illumination, scale, and translation variance. This is very useful specially with images of high resolution.
(2) We introduce a Principal Component Analysis (PCA) to reduce the number of extracted Gabor features. From obtained regions we compute a simple color histogram to reduce the number of different intensities (pixels) and achieve a fast clustering for color image segmentation.
In summary, image segmentation methods can be roughly classified into two categories: supervised and unsupervised. In this paper, we mainly discuss a fuzzy unsupervised framework. No features learning is involved. This task remains one of the most challenging research topics because there is no unified approach to achieve fast, robust, and accurate segmentation.
In this work, a detailed study of existing color image segmentation approaches was carried out to investigate the most common stages in segmentation’s techniques. In Section2, we discuss the motivation for using the different implemented techniques. Furthermore, we thoroughly described each phase and introduced ideas for improvements. Next, we describe the development of the proposed method. Section4presents an evaluation study of the proposed improvement using a set of synthetic and real color images from the well-known dataset (BSD 500 and MSRC). As a validation stage, the developed method is applied on fire forest images and compared to the standard and recent methods.
2. Motivation
2.1. Motivation for Using Superpixels with Gabor Filtering
In color image segmentation, non-texture areas are relatively uniform, and it is easy to obtain the accurate boundaries. Color and spatial information are sufficient for the clustering task. In texture areas,
Sensors 2020, 20, 6429 3 of 27
the boundaries are the combination of micro and macro adjacent regions. Here, texture edges cannot be incorporated only by characteristics of single pixels: intensities and spatial coordinates. Hence, to obtain these boundaries requires a combination of multi scale characteristics. Many researchers have verified that multi-resolution features are able to get the main outline for various texture regions of the image [12,13]. In the last decade, the Gabor filters, firstly proposed by Dennis Gabor in 1946 in 1-D and extended, in 1985, to 2-D by Daugman, have received much attention. Their wide usage in multiple fields can be taken as proof of their success: image analysis, compression and restoration, object tracking and movement estimation, face recognition, smoke detection, texture retrieval, contour extraction, or image segmentation [14–17].
2.2. Motivation for Using Color Images Histograms
For C-means oriented algorithms, the clustering task has to compute the distance between each pixel and centers of different clusters. This task leads to a high computational complexity especially with images of higher resolution. Moreover, it is difficult to extend this idea of FCM for color image segmentation. This is due to the number of different colors which is usually close to the number of pixels in a color image. Compared to a grayscale image, the c-means clustering algorithms require a longer execution time to segment its corresponding color image. Because the histogram level is far less than the whole image pixels, the use of histogram-based features reduces the computational complexity of the clustering procedure. In [9], an enhanced FCM method for grayscale images was proposed. It is called the Spatial Fast Fuzzy C-means clustering algorithm (SFFCM). Authors demonstrate that it is faster to implement FCM on histogram-gray-levels than on pixel’s intensities. This novel extension of fuzzy clustering algorithm is used in our segmentation pipeline (see Figure1).
Sensors 2020, 20, x FOR PEER REVIEW 3 of 28
researchers have verified that multi-resolution features are able to get the main outline for various texture regions of the image [12,13]. In the last decade, the Gabor filters, firstly proposed by Dennis Gabor in 1946 in 1-D and extended, in 1985, to 2-D by Daugman, have received much attention. Their wide usage in multiple fields can be taken as proof of their success: image analysis, compression and restoration, object tracking and movement estimation, face recognition, smoke detection, texture retrieval, contour extraction, or image segmentation [14–17].
2.2. Motivation for Using Color Images Histograms
For C-means oriented algorithms, the clustering task has to compute the distance between each pixel and centers of different clusters. This task leads to a high computational complexity especially with images of higher resolution. Moreover, it is difficult to extend this idea of FCM for color image segmentation. This is due to the number of different colors which is usually close to the number of pixels in a color image. Compared to a grayscale image, the c-means clustering algorithms require a longer execution time to segment its corresponding color image. Because the histogram level is far less than the whole image pixels, the use of histogram-based features reduces the computational complexity of the clustering procedure. In [9], an enhanced FCM method for grayscale images was proposed. It is called the Spatial Fast Fuzzy C-means clustering algorithm (SFFCM). Authors demonstrate that it is faster to implement FCM on histogram-gray-levels than on pixel’s intensities. This novel extension of fuzzy clustering algorithm is used in our segmentation pipeline (see Figure 1).
Gabor filtering PCA Original Image If1 If2 Ifn Dimensionality Reduction Multisclae Transformation Gabor Features Segmented Image Fuzzy clustering Superpixels Extraction Presegmentation Ir
Ir: The most relevant feature image Ip
SFFCM
Ip: The preseg mented image
Figure 1. Overview of the proposed segmentation framework.
Ifi
is thei
th image of Gabor features.2.3. Fire Forest Image Application
Recently, wildfires devasted millions of hectares over the world. The lack of information about the current state and the dynamic evolution of fire plays a central role in the accidents. Nowadays the demand increases for remote monitoring of this natural disaster [2,18–20]. For that, artificial visual control is a new area that has gained interest. In literature, many techniques have been developed mainly for wildfire image processing [4,8,21,22]. In real applications, for smoke and fire, there is a different useful information: area, location, direction, etc. Because the forest environment suffers from many perception field drawbacks (uncontrollable and sudden changes in environmental conditions, calibration problems, non-rigid fire-model, etc.), this study involves many advanced computer vision techniques in 2D [13] and extends them to the 3D domain [23]. Our project is divided into different research interests: image segmentation, semantic fire and smoke detection, and flame direction estimation.In this work we developed a color image segmentation technique as a part of mentioned tasks. The goal of the proposed method is to improve the segmentation performance of wildfire noisy images and to reduce the clustering computational complexity.
Figure 1.Overview of the proposed segmentation framework. I f i is the ithimage of Gabor features.
2.3. Fire Forest Image Application
Recently, wildfires devasted millions of hectares over the world. The lack of information about the current state and the dynamic evolution of fire plays a central role in the accidents. Nowadays the demand increases for remote monitoring of this natural disaster [2,18–20]. For that, artificial visual control is a new area that has gained interest. In literature, many techniques have been developed mainly for wildfire image processing [4,8,21,22]. In real applications, for smoke and fire, there is a different useful information: area, location, direction, etc. Because the forest environment suffers from many perception field drawbacks (uncontrollable and sudden changes in environmental conditions, calibration problems, non-rigid fire-model, etc.), this study involves many advanced computer vision techniques in 2D [13] and extends them to the 3D domain [23]. Our project is divided into different research interests: image segmentation, semantic fire and smoke detection, and flame direction estimation. In this work we developed a color image segmentation technique as a part of mentioned tasks. The goal of the proposed method is to improve the segmentation performance of wildfire noisy images and to reduce the clustering computational complexity.
Sensors 2020, 20, 6429 4 of 27
3. Methodology
The developed method is based on two principal tasks: - The Pre-segmentation, also called the Superpixels Extraction, - The Clustering of firstly extracted superpixels
The framework of our proposed algorithm is shown in Figure1. 3.1. Superpixels Based on Gabor Filtering and Morphological Operations 3.1.1. Superpixels Extraction: An Overview
Superpixels extraction, called also pre-segmentation, is the subdivision of the input image into a number of regions. Each region is a collection of pixels with homogenous characteristics. This procedure is always used for image classification and labeling. Compared to neighboring window-based methods, it is able to provide more representative local spatial information [9].
As given by [24], superpixel algorithms are classified into two principal categories:
Graph-based methods: each pixel is considered as a node in a graph. Similarities between neighboring pixels are defined as edge weights. Superpixels extraction minimizes a cost function defined over the graph. This category includes a large variety of developed methods: Normalized Cuts (NC), Homogeneous Superpixels (HS), Superpixels via Pseudo-Boolean Optimization (PB), and Entropy Rate Superpixels (ERS) [25,26].
Clustering-based methods: all image pixels are iteratively grouped until satisfying some convergence criteria. As given by [27], the most popular techniques are Simple Linear Iterative Clustering named (SLIC), Watersheds Transform (WT), Quick Shift (QS), and Turbo Pixel (TP). More details and evaluation of 15 superpixel algorithms are given in [24]. All mentioned approaches are usually considered as over-segmentation algorithms to improve the final segmentation. Referring to [9,27], in our work, we use the implementation of WT for the superpixels extraction. In the last part of experiments (Section5.2), the SLIC is also implemented.
3.1.2. Gabor Filters and Their Characteristics
Image filtering based on Gabor filters is a procedure widely used for the extraction of spatially localized spectral features. The frequency and orientation representation of Gabor filters are similar to human visual system, and they have been found vital features that can be used for image segmentation [16,28]. In our project, the processed images of fire forest combine many complexities due to the higher intensity’s variation and the texture geometrical diversity. To cope with complex image regions, we use a bank of filters as a multi-scale features extractor.
The Gabor filter is obtained by a Gaussian kernel function modulated by multiplying a sinusoidal plane wave. As shown in Figure2, combining a 2D sinusoid with a Gaussian function results in a 2D Gabor filter.
Sensors 2020, 20, x FOR PEER REVIEW 4 of 28
3. Methodology
The developed method is based on two principal tasks: - The Pre-segmentation, also called the Superpixels Extraction, - The Clustering of firstly extracted superpixels
The framework of our proposed algorithm is shown in Figure 1.
3.1. Superpixels Based on Gabor Filtering and Morphological Operations
3.1.1. Superpixels Extraction: An Overview
Superpixels extraction, called also pre-segmentation, is the subdivision of the input image into a number of regions. Each region is a collection of pixels with homogenous characteristics. This procedure is always used for image classification and labeling. Compared to neighboring window-based methods, it is able to provide more representative local spatial information [9].
As given by [24], superpixel algorithms are classified into two principal categories:
Graph-based methods: each pixel is considered as a node in a graph. Similarities between
neighboring pixels are defined as edge weights. Superpixels extraction minimizes a cost function defined over the graph. This category includes a large variety of developed methods: Normalized Cuts (NC), Homogeneous Superpixels (HS), Superpixels via Pseudo-Boolean Optimization (PB), and Entropy Rate Superpixels (ERS) [25,26].
Clustering-based methods: all image pixels are iteratively grouped until satisfying some
convergence criteria. As given by [27], the most popular techniques are Simple Linear Iterative Clustering named (SLIC), Watersheds Transform (WT), Quick Shift (QS), and Turbo Pixel (TP). More details and evaluation of 15 superpixel algorithms are given in [24]. All mentioned approaches are usually considered as over-segmentation algorithms to improve the final segmentation. Referring to [9,27], in our work, we use the implementation of WT for the superpixels extraction. In the last part of experiments (Section 5.2), the SLIC is also implemented.
3.1.2. Gabor Filters and Their Characteristics
Image filtering based on Gabor filters is a procedure widely used for the extraction of spatially localized spectral features. The frequency and orientation representation of Gabor filters are similar to human visual system, and they have been found vital features that can be used for image segmentation [16,28]. In our project, the processed images of fire forest combine many complexities due to the higher intensity’s variation and the texture geometrical diversity. To cope with complex image regions, we use a bank of filters as a multi-scale features extractor.
The Gabor filter is obtained by a Gaussian kernel function modulated by multiplying a sinusoidal plane wave. As shown in Figure 2, combining a 2D sinusoid with a Gaussian function results in a 2D Gabor filter.
=
X
Y Y X Y X
x
Figure 2. Spatial localization in 2D sinusoid (left row), Gaussian function (middle row), and corresponding 2D Gabor filter (right row).
Gabor features are extracted by the convolution of the original image
I x y
( , )
and the impulse response of the 2-D Gabor filterg x y
( , )
:Figure 2. Spatial localization in 2D sinusoid (left row), Gaussian function (middle row), and corresponding 2D Gabor filter (right row).
Sensors 2020, 20, 6429 5 of 27
Gabor features are extracted by the convolution of the original image I(x, y)and the impulse response of the 2-D Gabor filter g(x, y):
G(x, y) =I(x, y)⊗ g(x, y) (1) x and y are the spatial coordinates of the plane.
The Gabor kernel generating g(x, y)is defined as follows:
As we have shown in [29], in the spatial domain, the 2-D Gabor function is formulated by: gλ,θ,ϕ(x, y) =e− x02+γ2 y02 2σ2 cos((2π x0/λ) +ϕ)) (2) where x0=x cosθ+y sinθ y0 =−x sinθ+y cosθ
σ is the standard deviation of the Gaussian factor that determines the size of the receptive field. The parameterλ is the wavelength and F=1/λ the spatial frequency of the cosine factor. They are,
respectively, called the preferred wavelength and preferred spatial frequency of the Gabor function. The ratioσ/λ determines the spatial frequency bandwidth and the number of parallel excitatory and inhibitory stripe zones that can be observed in the receptive field (see Figure3).
Sensors 2020, 20, x FOR PEER REVIEW 5 of 28
( , )
( , )
( , )
G x y
=
I x y
⊗
g x y
(1)
x and y are the spatial coordinates of the plane.
The Gabor kernel generating
g x y
( , )
is defined as follows:
As we have shown in
[29]
,
in the spatial domain, the 2-D Gabor function is formulated by:
2 2 2 2 ' ' 2 , ,( , )
cos((2
'/ )
))
x yg
x y
e
x
+ −=
+
γ σ λ θ ϕπ
λ ϕ
(2)
where
'
cos
sin
'
sin
cos
x
x
y
y
x
y
=
+
= −
+
θ
θ
θ
θ
σ
is the standard deviation of the Gaussian factor that determines the size of the receptive field. The
parameter
λ is the wavelength and
F
=
1
/
λ
the spatial frequency of the cosine factor. They are,
respectively, called the preferred wavelength and preferred spatial frequency of the Gabor function.
The ratio
σ λ
/
determines the spatial frequency bandwidth and the number of parallel excitatory
and inhibitory stripe zones that can be observed in the receptive field (see Figure 3).
Figure 3. Example of the receptive field of the 2D-Gabor filter.
γ is a constant, called the spatial aspect ratio, that determines the ellipticity of the receptive
field.
θ
represents the preferred orientation of the normal of the parallel stripes of a Gabor function,
ϕ
is the phase offset which defines the symmetry of Gabor filter.
As an example, with a different range of frequencies
f
k2
(
k
1 2 3
{ , , }
)
k
=
=
and
orientations
.
(
{0,1,...,7}
)
8
ll
l
π
θ
=
=
, the convolution generates a Gabor feature matrix given by:
(
)
0 0 0 , 0( , ) . . .
( , )
.
. . .
.
.
. . .
.
.
. . .
.
( ,
) . . .
(
,
)
l k N f M M Nr x y
r x y
G
r x y
r x
y
θ
=
(3)
The set of 3 spatial frequencies and 8 equidistant orientations is applied. Each Gabor kernel size
is proportional to the wavelength value. The replication padding is used to reduce boundary artifacts.
For each specific pair of frequency and orientation
(
f
k,
θ
l)
, the feature image size is
(
M N
×
)
.
In our work, only the magnitude
r x , y
(
i j)
is considered.
r x , y
(
i j)
gives the intensity
variations near the object boundaries (see Figure 4).
Figure 3.Example of the receptive field of the 2D-Gabor filter.
γ is a constant, called the spatial aspect ratio, that determines the ellipticity of the receptive field. θ represents the preferred orientation of the normal of the parallel stripes of a Gabor function, ϕ is the phase offset which defines the symmetry of Gabor filter.
As an example, with a different range of frequencies fk = √ 2 k (k={1, 2, 3})and orientations θl=l· π 8
(l={0, 1,. . . , 7}), the convolution generates a Gabor feature matrix given by:
G( fl,θk)= r(x0, y0) · · · r(xN, y0) .. . . .. ... r(x0, yM) · · · r(xM, yN) (3)
The set of 3 spatial frequencies and 8 equidistant orientations is applied. Each Gabor kernel size is proportional to the wavelength value. The replication padding is used to reduce boundary artifacts. For each specific pair of frequency and orientation(fk,θl), the feature image size is(M × N).
In our work, only the magnitude rxi, yj
is considered. rxi, yj
gives the intensity variations near the object boundaries (see Figure4).
The Gabor features are processed with L2 normalization technique. The L2 norm is performed by: g(x, y) = kG(x, y)k
maxkG(x, y)k (4)
Sensors 2020, 20, 6429 6 of 27
Sensors 2020, 20, x FOR PEER REVIEW 6 of 28
Figure 4. Example of object boundaries extraction using Gabor filters of
(
f
1,
3)
. First row: original images. Second row: image of boundaries.The Gabor features are processed with L2 normalization technique. The L2 norm is performed
by;
( )
( )
( )
G x, y
g x, y
max G x, y
=
(4)
g
is the normalized Gabor feature image.
3.1.3. Gabor Feature Reduction Based on PCA
High dimension data are extremely complex to process due to the inconsistences in the features
which increase the computation time [30,31]. In our work, we only focus on the variation of frequency
and orientation parameters of Gabor filters. In Figure 5, we present a convolution results of a
synthetic image of sinusoids of different orientations, frequencies and magnitudes by Gabor filters of
different orientations and frequencies.
f 1 2 3 1 f f2 f3 Co nv ol ut io n
Figure 5. Convolution outputs of a synthetic image of sinusoids with various properties (orientations,
frequencies and magnitudes) by Gabor filters of different frequencies and orientations
(
f
k,
l)
.Figure 4.Example of object boundaries extraction using Gabor filters of(f1,θ3). First row: original images. Second row: image of boundaries.
3.1.3. Gabor Feature Reduction Based on PCA
High dimension data are extremely complex to process due to the inconsistences in the features which increase the computation time [30,31]. In our work, we only focus on the variation of frequency and orientation parameters of Gabor filters. In Figure5, we present a convolution results of a synthetic image of sinusoids of different orientations, frequencies and magnitudes by Gabor filters of different orientations and frequencies.
Sensors 2020, 20, x FOR PEER REVIEW 6 of 28
Figure 4. Example of object boundaries extraction using Gabor filters of
(
f
1,
θ
3)
. First row: original images. Second row: image of boundaries.The Gabor features are processed with L2 normalization technique. The L2 norm is performed by;
( )
( )
( )
{
}
G x, y g x, y max G x, y = (4)g
is the normalized Gabor feature image. 3.1.3. Gabor Feature Reduction Based on PCAHigh dimension data are extremely complex to process due to the inconsistences in the features which increase the computation time [30,31]. In our work, we only focus on the variation of frequency and orientation parameters of Gabor filters. In Figure 5, we present a convolution results of a synthetic image of sinusoids of different orientations, frequencies and magnitudes by Gabor filters of different orientations and frequencies.
f 1 θ θ2 θ3 1 f f2 f3 Con vo lu tion
Figure 5. Convolution outputs of a synthetic image of sinusoids with various properties (orientations, frequencies and magnitudes) by Gabor filters of different frequencies and orientations
(
f
k,
θ
l)
. Figure 5.Convolution outputs of a synthetic image of sinusoids with various properties (orientations, frequencies and magnitudes) by Gabor filters of different frequencies and orientations(fk,θl).Both the mentioned parameters (frequencies and orientations) generate a large feature dimension
(K × L). As mentioned above, a set of K=3 different frequencies and L=8 orientations are considered producing 24 features for each position of the filter. This is not performant because of the redundancy of features due to correlation of the overlapping filters. Moreover, as illustrated in Figure5, by comparing the convolution results we notice a higher sensibility of filter parametrization. Many researchers
Sensors 2020, 20, 6429 7 of 27
propose the use of a small bank of filters [30–32]. In this work, the problem of redundancy was addressed. Because its performance compared to other dimensionality reducer methods [32], we have used the PCA retaining only the most representative response of 24 outputs. It will be considered as the input of the superpixesls extraction stage (see Figure1).
3.1.4. Pre-segmentation Based on Gabor-WT
The WT produces a set of basins starting with a local minimal of a gradient image and searching lines between adjacent local minima that separate catchment watersheds. As given by [33], this is a relatively fast algorithm used for images with high resolution.
For noisy image segmentation, to fulfill both regional consistency and boundary keeping simultaneously, become more and more difficult. As shown by Figure6, the MMGR-WT, introduced in [9], causes an over-segmentation or under-segmentation because it is sensitive to added noise.
Sensors 2020, 20, x FOR PEER REVIEW 7 of 28
Both the mentioned parameters (frequencies and orientations) generate a large feature
dimension
(
K L
×
)
. As mentioned above, a set of
K
3
=
different frequencies and
L
8
=
orientations are considered producing 24 features for each position of the filter. This is not performant
because of the redundancy of features due to correlation of the overlapping filters. Moreover, as
illustrated in Figure 5, by comparing the convolution results we notice a higher sensibility of filter
parametrization. Many researchers propose the use of a small bank of filters [30–32]. In this work, the
problem of redundancy was addressed. Because its performance compared to other dimensionality
reducer methods [32], we have used the PCA retaining only the most representative response of 24
outputs. It will be considered as the input of the superpixesls extraction stage (see Figure 1).
3.1.4. Pre-segmentation Based on Gabor-WT
The WT produces a set of basins starting with a local minimal of a gradient image and searching
lines between adjacent local minima that separate catchment watersheds. As given by [33], this is a
relatively fast algorithm used for images with high resolution.
For noisy image segmentation, to fulfill both regional consistency and boundary keeping
simultaneously, become more and more difficult. As shown by Figure 6, the MMGR-WT, introduced
in [9],
causes an
over-segmentation or under-segmentation
because it is sensitive to added noise.
‘4_15_s’ ‘18_29_s’ ‘5_13_s’ ‘18_10_s’
Figure 6. MMGR-WT robustness test. First row: original images from the MSRC-dataset. Second row:
Superpixels extraction results with original images. 3rd, 4th, and 5th rows: the obtained results for corrupted images by (5%, 10%, 15%) salt and pepper noise.
Moreover, these techniques greatly depend on the accurate extraction of region boundaries. The
superpixels extraction performance of these methods deteriorates when the processed regions are
textured or are of high varying intensities (see Figure 7).
Figure 6.MMGR-WT robustness test. First row: original images from the MSRC-dataset. Second row: Superpixels extraction results with original images. 3rd, 4th, and 5th rows: the obtained results for corrupted images by (5%, 10%, 15%) salt and pepper noise.
Moreover, these techniques greatly depend on the accurate extraction of region boundaries. The superpixels extraction performance of these methods deteriorates when the processed regions are textured or are of high varying intensities (see Figure7).
Sensors 2020, 20, 6429 8 of 27
Sensors 2020, 20, x FOR PEER REVIEW 8 of 28
’14-24_s’
’3_12_s’
’3_29_s’ ’1_2_s’ ’2_16_s’
Figure 7. MMGR-WT superpixels extraction: test on uniform and textured regions. First row: original images from the MSRC_dataset. Second row: obtained results.
As a summary of all the superpixels extraction given by Figures 6 and 7, the MMGR-WT results exhibit major limits, namely the poor boundary keeping and superpixels consistency. This is clearly noticed with noisy images and ones of textured regions (grass, trees, sand, etc.). In our work, it should be noted that Fire Forest images suffer from all the general drawbacks (noise, higher textured regions, environmental conditions, etc.). In literature, many algorithms have been introduced to avoid such issues. Major methods tend to modify the gradient output of original image. In this paper, a 2-D Gabor filtering stage is used for the enhancement of the boundaries of regions for better superpixels extraction.
3.2. Fuzzy Superpixels Clustering
3.2.1. Overview
A clustering divides data objects into homogeneous groups and performs a high similarity within a cluster (called compactness). Data partitioning is made according to a membership degree, in the range (0,1), which is proportional to the distance between the data and each cluster center. The partitioning result depends on the final centroid location [29]. The fuzzy oriented methods are based on the mentioned aspects and have been successfully used. For many an application, the traditional FCM clustering algorithm, firstly introduced by Bezdek, has depicted a higher performance. It is widely used for image segmentation. As an unsupervised clustering method, FCM does not need any prior knowledge about the image.
Let
X
=
{
x x
1, ,...,
2x
n}
be a color image andnc
be the number of clusters. Eachi
th imagepixel belongs to the
j
th cluster with a fuzzy membership degree denoted byu
ij according to its distance from the cluster centerv
j. FCM can yield a good segmentation result by minimizing thefollowing objective function:
2
n nc m FCM ij i j i j
J
μ
x v
= ==
−
(5)where
u
ij andv
j are given as follows;1 2 1 1
m nc i j ij k i j
x v
u
x v
− − =
−
=
−
(6) 1 1 n m ij j j j n m ij j u x v u = = =
(7)and
m
is the degree of fuzziness.Figure 7.MMGR-WT superpixels extraction: test on uniform and textured regions. First row: original images from the MSRC_dataset. Second row: obtained results.
As a summary of all the superpixels extraction given by Figures6and7, the MMGR-WT results exhibit major limits, namely the poor boundary keeping and superpixels consistency. This is clearly noticed with noisy images and ones of textured regions (grass, trees, sand, etc.). In our work, it should be noted that Fire Forest images suffer from all the general drawbacks (noise, higher textured regions, environmental conditions, etc.). In literature, many algorithms have been introduced to avoid such issues. Major methods tend to modify the gradient output of original image. In this paper, a 2-D Gabor filtering stage is used for the enhancement of the boundaries of regions for better superpixels extraction. 3.2. Fuzzy Superpixels Clustering
3.2.1. Overview
A clustering divides data objects into homogeneous groups and performs a high similarity within a cluster (called compactness). Data partitioning is made according to a membership degree, in the range (0,1), which is proportional to the distance between the data and each cluster center. The partitioning result depends on the final centroid location [29]. The fuzzy oriented methods are based on the mentioned aspects and have been successfully used. For many an application, the traditional FCM clustering algorithm, firstly introduced by Bezdek, has depicted a higher performance. It is widely used for image segmentation. As an unsupervised clustering method, FCM does not need any prior knowledge about the image.
Let X={x1, x2, . . . , xn} be a color image and nc be the number of clusters. Each ithimage pixel belongs to the jthcluster with a fuzzy membership degree denoted by ui jaccording to its distance from
the cluster center vj. FCM can yield a good segmentation result by minimizing the following objective
function: JFCM= n X i= nc X j= µm i jkxi− vjk 2 (5) where ui jand vjare given as follows:
ui j= nc X k=1 kxi− vjk kxi− vjk !m−12 −1 (6) vj= Pn j=1umi jxj Pn j=1umi j (7) and m is the degree of fuzziness.
Sensors 2020, 20, 6429 9 of 27
Algorithm 1.Traditional FCM algorithm.
1: Input:X of n data to be clustered, the number of clusters nc, the convergence testε > 0 (or the max number of iteration), randomly cluster centers v(t=0), the fuzzifier m> 1
2: Output:clustered data (pixel groups map) 3: Begin
4: Step 1.Compute the membership matrix U by using Equation (6) 5: Step 2.Update the cluster centers v(t+1)with Equation (7)
6: Step 3.Test if kv(t+1)− v(t)k< ε, execute step 4; otherwise, t=t+1 and go to step 1 7: Step 4.Output the pixels group map
8: End
In literature, a large variety of modified versions of the FCM clustering algorithm have been proposed. In 2003, Zhang developed a new extension called KFCM by introducing the “Kernel method”. Later, in 2012, Zanaty et al. included a spatial information in the objective function of KFCM [34]. From 2005 to 2013 Pal et al. developed a possibilistic fuzzy clustering method called PFCM [35]. Another modification of FCM was proposed in 2015 by Zheng et al. [36] named the Generalized and Hierarchical FCM (GFCM), (HFCM). In 2017, in order to remove the information redundancy, Gu et al. proposed a novel version of FCM called SL-FCM [37]. Most of the mentioned methods are still time-consuming and unable to provide the desired segmentation accuracy. As mentioned above, Lei et al. developed the SFFCM algorithm. The modification is also based on the integration of the spatial information [9].
3.2.2. The Proposed Clustering Method
Further to the time-consuming, FCM describes an image in terms of fuzzy classes. It only depends on global features. As given by Figure8, we developed a Gabor-PCA superpixels-based method to extract the most representative local spatial information. By this, the input data to be clustered include only the subsegment levels. In our work, the proposed segmentation method has three goals. The first is to reach a higher robustness to added noise with the multiscale processing based on Gabor Filters. The second goal is the improvement of the segmentation accuracy by incorporating local features. The third, is reducing the computational complexity and time consuming by minimizing the size of data to be clustered.
In this paper, the SFFCM algorithm, firstly proposed by [9], is adopted. Adding the spatial information, the problem of fuzzily partitioning into nc clusters becomes formulated as the minimization of the objective function given by:
Jm= ns X i=1 nc X j=1 Siumi jkMedi− vjk2 (8)
With Medithe mean value of color pixels within the corresponding region Riof ith superpixel
image given by:
Medi = 1 Si X p∈Ri xp (9) where:
ns is the number of superpixels, 1 ≤ i ≤ ns the color level, and Sithe number of pixels with color
xpin Ri. The new objective function incorporates the histogram information by the level’s frequencies
given by Si. Thereby, each color pixel in original image is replaced by the mean value Mediof the
Sensors 2020, 20, 6429 10 of 27
Sensors 2020, 20, x FOR PEER REVIEW 10 of 28
Gabor filtering PCA
Superpixels extraction
SFFCM clustering
Histogram of or iginal image
Histogram after presegmentation
Gabor Feature image (Ir)
Segmented i mage Presegmented i mage (I p)
Ori gi nal i mage
Figure 8. Example of the “End to End” segmentation pipeline with our proposed method.
In this paper, the SFFCM algorithm, firstly proposed by [9], is adopted. Adding the spatial
information, the problem of fuzzily partitioning into
nc
clusters becomes formulated as the
minimization of the objective function given by:
2 1 1 ns nc m m i ij i j i j
J
S u
Med
v
= ==
−
(8)
With
Med
ithe mean value of color pixels within the corresponding region
R
iof
i
thsuperpixel image given by:
1
i p p Ri iMed
x
S
=
(9)
where:
ns
is the number of superpixels,
1
i
ns
the color level, and
S
ithe number of pixels
with color
x
pin
R
i. The new objective function incorporates the histogram information by the
level’s frequencies given by
S
i. Thereby, each color pixel in original image is replaced by the mean
value
Med
iof the region for which was assigned. The “Med-image” is called the pre-segmented
image (see Figure 8).
New SFFCM objective function generates two novel formulation memberships (
u
ij) and
centroid functions (
v
j) as follows:
2 1 2 1 1 m i j ij nc m i k k
Med
v
u
Med
v
− − − − =−
=
−
(10)
Figure 8.Example of the “End to End” segmentation pipeline with our proposed method.
New SFFCM objective function generates two novel formulation memberships (ui j) and centroid
functions (vj) as follows: ui j= kMedi − vjk −2 m−1 nc P k=1 kMedi − vkk −2 m−1 (10) vj= Pns i=1umi j P p∈Rixp Pns i=1Siumi j (11) In Algorithm 2, we show the pseudo-code of the Spatial Fast Fuzzy C-means clustering method (SFFCM).
Algorithm 2.SFFCM Algorithm.
1: Input: S={S1, . . . , Sns} number of pixels with color corresponding to the presegmented regions R={R1, . . . , Rns}, Med={Med1, . . . , Medns} the mean values of superpixels levels (equation. . . ), the number of clusters nc, the convergence testε > 0 (or the max number of iteration), randomly cluster centers v(t=0), the fuzzifier m> 1
2: Output:clustered data (pixel groups map) 3: Begin
4: Step 1.Compute the membership matrix U by using Equation (10) 5: Step 2.Update the cluster centers v(t+1)with Equation (11)
6: Step 3.Test if kv(t+1)− v(t)k< ε, execute step 4; otherwise, t=t+1 and go to step 1 7: Step 4.Output the pixels group map
Sensors 2020, 20, 6429 11 of 27
3.3. Evaluation Criteria
In the last decade, several metrics have been applied to evaluate the segmentation methods [38]. The major ones focus on segmentation accuracy, superpixels compactness, regularity, coherence, and efficiency. In [24], Wang et al. divided the set of metrics into three groups: segmentation quality evaluation, superpixels quality, and the efficiency measure based on runtime. In this work, two metrics categories are considered.
a. Segmentation accuracy
To test the clustering performance, we use two metrics given in [9]. The first measures the Equality Degree (ED) between Clustered Pixels (CP) and Ground truth Prediction (GP). The second measures the Segmentation Accuracy (SA) based on the sum of correctly classified pixels. Both metrics are, respectively, given by:
ED= nc X k=1 CPk∩ GPk CPk∪ GPk (12) SA= nc X k=1 CPk∩ GPk Pc j=1GPj (13) where, CPkis the set of pixels assigned to kthcluster and GPkthe set of pixels belonging to the same
class k of Ground Truth (GT). nc denotes the number of clusters.
CPk∩ GPk: the comprised of the labeled pixels AND the ground truth of the kthcluster.
CPk∪ GPk: the comprised of all pixels found in either the prediction OR the ground truth of the
kthcluster.
b. Sensitivity and Specificity
These measures are based on region overlapping. Here, two aspects are considered: the matching direction and the corresponding criteria. For the sensitivity measure, the matching direction is defined as a ground truth to segmentation result directional correspondence and vice versa for the recall measure. Sensitivity (SEN) and Specificity (SPE) are formulated as follows:
SEN(CP, GP) = TP(CP, GP)
TP(CP, GP) + FN(CP, GP) (14)
SPE(CP, GP) = TN(CP, GP)
TN(CP, GP) + FP(CP, GP) (15)
where:
TP(CP, GP)—True Positives: intersection between segmentation and ground truth
TN(CP, GP)—True Negatives: part of the image beyond the intersection mentioned above FP(CP, GP)—False Positives: segmented parts not overlapping the ground truth
FN(CP, GP)—False negatives: missed parts of the ground truth
As given by the Equations (14) and (15), the quantitative evaluation based on Sensitivity (SEN) and Specificity (SPE) were performed between the (GT) and the clustering result. (SEN) was the percentage of Region of Interest (ROI) recognized by the segmentation method. (SPE) was the percentage of non-ROI recognized by the segmentation method.
Measures based on (SEN) and (SPE) are commonly used for the semantic segmentation. In our work, the mentioned metrics are applied to evaluate the clustering performance for supervised topics where the number of classes and region contents are known.
Sensors 2020, 20, 6429 12 of 27
For real images, cluster frequencies are unbalanced. Mentioned metrics are not appropriate for evaluating because they are biased by the dominant classes. To avoid this, we have conducted the evaluation per-class. The obtained results are averaged over the total number of classes.
For multiclass, Sensitivity and Specificity are called Average True Positive Rate (Av_TPR) and Average True Negative Rate (Av_TNR) and given by:
Av_TPR= Pnc k=1TPk Pnc k=1(TPk+FNk) (16) Av_TNR= Pnc k=1TNk Pnc k=1(TNk+FPk) (17) 4. Experimental Results Experimental Setting
For all the experiments discussed below, the particular parameters for the compared methods are summarized in Table1. Only Gabor filtering parametrization is detailed in [29].
Table 1.Parameters of different applied methods.
Method Pre-segmentation Classification
SFFCM [9] Min, Max radius:(r1, r2) = (1, 10) m=2,ε=10−3 Gabor-SLIC
Number of desired pixels: sk=500 Weighting factor: sm=50 Threshold for region merging: ss=1
m=2,ε=10−3 Gabor-WT Structured Element SE: a disk
Radius: r=5 m=2,ε=10−3
In the first experiments, the tested images are synthetic with natural textures of Smoke, Fire, Sky, Sand, and Grass. For the second, we have tested on six real images from real scene of fire forest images. For the limited length of paper, in the last experiments, we only demonstrate the robustness and segmentation performance of our proposed method on a subset of twenty images from BSDS500 and MRSC datasets.
5. Application on Fire Forest Images
5.1. Results on Synthetic Images
At the first part of evaluation, we test the proposed method with the WT and the SFFCM algorithm on a set of six synthetic images shown by Figure9a. For each class, (Fire, Smoke, Grass, Sand, Sky), the selected region is chosen from a random location in the original corresponding texture. All of the used synthetic images are with regions of regular boundaries. This is more suitable to manually generate the desired segmentation (see Figure9b).
In this experiment, two types of noise are considered: Gaussian and Salt and Pepper. The robustness of each method is tested with four different densities of each kind of mentioned noises (10%, 20%, 30%, 40%). The quantitative segmentation results on the different blurred images achieved by using our developed method with WT and the SFFCM proposed by Lei 2019 [9]. Each experiment is repeated 10 times. All the obtained results are depicted in the boxplots in Figures10and11, reporting both ED and SA metrics values given by Equations (12) and (13). The graphs of boxplots arranged similarly as the map of images (SI1–SI6) given by Figure9(e.g., the top left boxplot corresponds to the results on image (SI1)).
Sensors 2020, 20, 6429 13 of 27
Sensors 2020, 20, x FOR PEER REVIEW 13 of 28
Table 1. Parameters of different applied methods.
Method Pre-segmentation Classification
SFFCM [9]
Min, Max radius:
( )
r r
1,
2=
(1,10)
m
=
2
,
ε
=
10
−3Gabor-SLIC
Number of desired pixels:
s
k=
500
Weighting factor:
s
m=
50
Threshold for region merging:
s
s=
1
2
m
=
,
ε
=
10
−3Gabor-WT
Structured Element SE: a disk
Radius:
r
=
5
m
=
2
,
3
10
ε
=
−In the first experiments, the tested images are synthetic with natural textures of Smoke, Fire, Sky,
Sand, and Grass. For the second, we have tested on six real images from real scene of fire forest
images. For the limited length of paper, in the last experiments, we only demonstrate the robustness
and segmentation performance of our proposed method on a subset of twenty images from BSDS500
and MRSC datasets.
5. Application on Fire Forest Images
5.1. Results on Synthetic Images
At the first part of evaluation, we test the proposed method with the WT and the SFFCM
algorithm on a set of six synthetic images shown by Figure 9a. For each class, (Fire, Smoke, Grass,
Sand, Sky), the selected region is chosen from a random location in the original corresponding
texture. All of the used synthetic images are with regions of regular boundaries. This is more suitable
to manually generate the desired segmentation (see Figure 9b).
SI2 SI1 SI4 SI3 SI6 SI5
nc=2
nc=3
nc=4
(a)
Sensors 2020, 20, x FOR PEER REVIEW 14 of 28
GT1 GT2 GT3 GT4 GT5 GT6
(b)
Figure 9. Synthetic test images of
(
256 256
×
)
pixels with manually created (GT). (a) Images with
different regions of real contents. (b) The corresponding desired segmentation (GT).
In this experiment, two types of noise are considered: Gaussian and Salt and Pepper. The
robustness of each method is tested with four different densities of each kind of mentioned noises
(10%, 20%, 30%, 40%). The quantitative segmentation results on the different blurred images achieved
by using our developed method with WT and the SFFCM proposed by Lei 2019 [9]. Each experiment
is repeated 10 times. All the obtained results are depicted in the boxplots in Figures 10 and 11,
reporting both ED and SA metrics values given by Equations (12) and (13). The graphs of boxplots
arranged similarly as the map of images (SI1–SI6) given by Figure 9 (e.g., the top left boxplot
corresponds to the results on image (SI1)).
(a) Boxplots of
Equality Degree (
ED) metric obtained by 10 experiments
Figure 9. Synthetic test images of(256 × 256)pixels with manually created (GT). (a) Images with different regions of real contents. (b) The corresponding desired segmentation (GT).
In Figures10and11, the lower and the upper bounds of each boxplot represent the first and third quartiles of the distribution, respectively. The mean values of used metrics (ED, SA) are represented by a black solid line and outliers are displayed as black diamonds. We observe that there is a greater variability of the SFFCM results compared to our proposed method. Moreover, the boxplots pertaining to the proposed method results present the lowest statistical dispersion in terms of box height and number of outliers, thus implying a lower standard deviation compared to the SFFCM method. Therefore, the use of the novel method allows for considerably robust and accurate segmentation results.
Sensors 2020, 20, 6429 14 of 27
5.2. Results on Real Images
In addition to synthetic images, we shall evaluate the performance of our method on natural images. We apply the proposed method to the images from real fire forest sequences to examine the segmentation performance of our approach. The test images, given by Figure12, are with different
regions (fire, forest, smoke, cloud, grass, etc.). We shall assess the segmentation accuracy according to the visual inspection because no ground truths are available.
Sensors 2020, 20, x FOR PEER REVIEW 14 of 28
GT1 GT2 GT3 GT4 GT5 GT6 (b)
Figure 9. Synthetic test images of
(
256 256×)
pixels with manually created (GT). (a) Images with different regions of real contents. (b) The corresponding desired segmentation (GT).In this experiment, two types of noise are considered: Gaussian and Salt and Pepper. The robustness of each method is tested with four different densities of each kind of mentioned noises (10%, 20%, 30%, 40%). The quantitative segmentation results on the different blurred images achieved by using our developed method with WT and the SFFCM proposed by Lei 2019 [9]. Each experiment is repeated 10 times. All the obtained results are depicted in the boxplots in Figures 10 and 11, reporting both ED and SA metrics values given by Equations (12) and (13). The graphs of boxplots arranged similarly as the map of images (SI1–SI6) given by Figure 9 (e.g., the top left boxplot corresponds to the results on image (SI1)).
(a) Boxplots of
Equality Degree (
ED) metric obtained by 10 experimentsSensors 2020, 20, x FOR PEER REVIEW 15 of 28
(b) Boxplots of
Segmentation Accuracy (
SA) metric obtained by 10 experiments Figure 10. Comparison of SFFCM and our proposed method (G-WT) robustness. Application on the set of synthetic images SI1-SI6 (Figure 9) corrupted by the (10%, 20%, 30%, 40%) Gaussian noise.(a) Boxplots of ED metric obtained by 10 experiments
Figure 10.Comparison of SFFCM and our proposed method (G-WT) robustness. Application on the set of synthetic images SI1-SI6 (Figure9) corrupted by the (10%, 20%, 30%, 40%) Gaussian noise.
Sensors 2020, 20, 6429 15 of 27
Sensors 2020, 20, x FOR PEER REVIEW 15 of 28
(b) Boxplots of Segmentation Accuracy (SA) metric obtained by 10 experiments
Figure 10. Comparison of SFFCM and our proposed method (G-WT) robustness. Application on the set of synthetic images SI1-SI6 (Figure 9) corrupted by the (10%, 20%, 30%, 40%) Gaussian noise.
(a) Boxplots of ED metric obtained by 10 experiments
Sensors 2020, 20, x FOR PEER REVIEW 16 of 28
(b) Boxplots of SA metric obtained by 10 experiments
Figure 11. Comparison of SFFCM and our proposed method (G-WT) robustness. Application on the set of synthetic images SI1–SI6 (Figure 9 corrupted by the (10%, 20%, 30%, 40%) Salt and Pepper noise. In Figures 10 and 11, the lower and the upper bounds of each boxplot represent the first and third quartiles of the distribution, respectively. The mean values of used metrics (ED, SA) are represented by a black solid line and outliers are displayed as black diamonds. We observe that there is a greater variability of the SFFCM results compared to our proposed method. Moreover, the boxplots pertaining to the proposed method results present the lowest statistical dispersion in terms of box height and number of outliers, thus implying a lower standard deviation compared to the SFFCM method. Therefore, the use of the novel method allows for considerably robust and accurate segmentation results.
5.2. Results on Real Images
In addition to synthetic images, we shall evaluate the performance of our method on natural images. We apply the proposed method to the images from real fire forest sequences to examine the segmentation performance of our approach. The test images, given by Figure 12, are with different regions (fire, forest, smoke, cloud, grass, etc.). We shall assess the segmentation accuracy according to the visual inspection because no ground truths are available.
Ima 1 Ima 2 Ima 3
Ima 4 Ima 5 Ima 6
Figure 12. A set of real test fire forest images [21].
Figure 11.Comparison of SFFCM and our proposed method (G-WT) robustness. Application on the set of synthetic images SI1–SI6 (Figure9corrupted by the (10%, 20%, 30%, 40%) Salt and Pepper noise.
Sensors 2020, 20, x FOR PEER REVIEW 16 of 28
(b) Boxplots of SA metric obtained by 10 experiments
Figure 11. Comparison of SFFCM and our proposed method (G-WT) robustness. Application on the
set of synthetic images SI1–SI6 (Figure 9 corrupted by the (10%, 20%, 30%, 40%) Salt and Pepper noise.
In Figures 10 and 11, the lower and the upper bounds of each boxplot represent the first and
third quartiles of the distribution, respectively. The mean values of used metrics (ED, SA) are
represented by a black solid line and outliers are displayed as black diamonds. We observe that there
is a greater variability of the SFFCM results compared to our proposed method. M
oreover, the
boxplots pertaining to the proposed method results present the lowest statistical dispersion in terms
of box height and number of outliers, thus implying a lower standard deviation compared to the
SFFCM method. Therefore, the use of the novel method allows for considerably robust and accurate
segmentation results.
5.2. Results on Real Images
In addition to synthetic images, we shall evaluate the performance of our method on natural
images. We apply the proposed method to the images from real fire forest sequences to examine the
segmentation performance of our approach. The test images, given by Figure 12, are with different
regions (fire, forest, smoke, cloud, grass, etc.). We shall assess the segmentation accuracy according
to the visual inspection because no ground truths are available.
Ima 1 Ima 2 Ima 3
Ima 4 Ima 5 Ima 6
Figure 12. A set of real test fire forest images [21].
The difficulty of real image segmentation can be attributed to two reasons. The first is that image
segmentation is a multiple solution problem. The number of clusters differs from a person to another.
The second is that an image is always complex because of added noise and regions nonuniformity.
In this study, in order to address the first mentioned difficulty, we have shared the real test
images with a group of 30 of our students in order to obtain their observations about the number of
different observed clusters. The obtained statistics are summarized in Figure 13.
For each image, only the first three decisions with higher percentages were considered. i.e., as
given by Figure 13, 53.1% of persons have considered that the image “Ima 2” is of
4
clusters, 21.9%
have considered that the mentioned image is only of 3 clusters, and 15.6% observed that “Ima 2” is
Sensors 2020, 20, 6429 16 of 27
The difficulty of real image segmentation can be attributed to two reasons. The first is that image segmentation is a multiple solution problem. The number of clusters differs from a person to another. The second is that an image is always complex because of added noise and regions nonuniformity.
In this study, in order to address the first mentioned difficulty, we have shared the real test images with a group of 30 of our students in order to obtain their observations about the number of different observed clusters. The obtained statistics are summarized in Figure13.
Figure 13.30 humans’ observations about the number of clusters of the real images given by Figure12.
For each image, only the first three decisions with higher percentages were considered. i.e., as given by Figure13, 53.1% of persons have considered that the image “Ima 2” is of 4 clusters, 21.9% have considered that the mentioned image is only of 3 clusters, and 15.6% observed that “Ima 2” is with 5 clusters. In our experiments, for mentioned image, we conduct the segmentation with 4, 3, and 5 clusters. All the obtained results are illustrated by Figures14–19.
Figures14–19show the segmentation results of the real images depicted by Figure12and corrupted by the salt and pepper. In this experiment, we compare the SFFCM and our proposed method for two versions. The first by using the WT and for the second, we have introduced the SLIC pre-segmentation technique. By a visual inspection, for three compared methods, we notice that the region partition is satisfying. When the noise density is added, a lower performance is achieved. This is mainly due to the fact that high density of noise affects the texture structures, leading to the input image color degradation. Added noise affects the pre-segmentation performance and yields a lower classification performance. This clearly noticed with the SFFCM algorithm compared to the proposed method with WT and SLIC.
As given by Figures14b,15b,16b,17b,18b and19b, for different corrupted images, the obtained
results with the proposed method using WT depicts that the different regions separation is more accurate than using the SLIC. For instance, we can see that the “fire” in Figure14b and the “smoke” in Figures15b,16b and17b are accurately segmented.
In summary, the segmentation results obtained by the proposed method, using WT or SLIC, are still more satisfying. This is due to the higher robustness of the multiresolution transform based on Gabor filters and the integration of the PCA in the pre-segmentation stage.
Sensors 2020, 20, 6429 17 of 27
Sensors 2020, 20, x FOR PEER REVIEW 18 of 28
Ima 1
nc= 5 nc= 6
nc= 4
(a)
(b)
Figure 14. Comparison of segmentation results of original image Ima 1 (a) and corrupted by a 10% salt and pepper noise (b) obtained by: SFFCM (the first row), the proposed method based on Simple Linear Iterative Clustering (SLIC) (the second row), and the proposed method based on Watersheds Transform (WT) (the third row).
Figure 14. Comparison of segmentation results of original image Ima 1 (a) and corrupted by a 10% salt and pepper noise (b) obtained by: SFFCM (the first row), the proposed method based on Simple Linear Iterative Clustering (SLIC) (the second row), and the proposed method based on Watersheds Transform (WT) (the third row).
Sensors 2020, 20, 6429 18 of 27
Sensors 2020, 20, x FOR PEER REVIEW 19 of 28
Ima 2
nc= 4 nc= 3 nc=5
(a)
(b)
Figure 15. Comparison of segmentation results of original image Ima 2 (a) and corrupted by a 10% salt and pepper noise, (b) obtained by: SFFCM (the first row), the proposed method based on SLIC (the second row), and the proposed method based on WT (the third row).
Figure 15. Comparison of segmentation results of original image Ima 2 (a) and corrupted by a 10% salt and pepper noise, (b) obtained by: SFFCM (the first row), the proposed method based on SLIC (the second row), and the proposed method based on WT (the third row).
Sensors 2020, 20, 6429 19 of 27
Sensors 2020, 20, x FOR PEER REVIEW 20 of 28
Ima 3
nc= 4
nc= 5
nc= 3
(a)
(b)
Figure 16. Comparison of segmentation results of original image Ima 3 (a) and corrupted by a 10% salt and pepper noise (b) obtained by: SFFCM (the first row), the proposed method based on SLIC (the second row), and the proposed method based on WT (the third row).
Figure 16. Comparison of segmentation results of original image Ima 3 (a) and corrupted by a 10% salt and pepper noise (b) obtained by: SFFCM (the first row), the proposed method based on SLIC (the second row), and the proposed method based on WT (the third row).
Sensors 2020, 20, 6429 20 of 27
Sensors 2020, 20, x FOR PEER REVIEW 21 of 28
Ima 4
nc= 5
nc= 4
nc= 6
(a)
(b)
Figure 17. Comparison of segmentation results of original image Ima 4 (a) and corrupted by a 10% salt and pepper noise. (b) obtained by: SFFCM (the first row), the proposed method based on SLIC (the second row), and the proposed method based on WT (the third row).
Figure 17. Comparison of segmentation results of original image Ima 4 (a) and corrupted by a 10% salt and pepper noise. (b) obtained by: SFFCM (the first row), the proposed method based on SLIC (the second row), and the proposed method based on WT (the third row).
Sensors 2020, 20, 6429 21 of 27
Sensors 2020, 20, x FOR PEER REVIEW 22 of 28
Ima 5
nc= 5
nc= 4
nc= 3
(a)
(b)
Figure 18. Comparison of segmentation results of original image Ima 5 (a) and corrupted by a 10% salt and pepper noise (b) obtained by: SFFCM (the first row), the proposed method based on SLIC (the second row), and the proposed method based on WT (the third row).
Figure 18. Comparison of segmentation results of original image Ima 5 (a) and corrupted by a 10% salt and pepper noise (b) obtained by: SFFCM (the first row), the proposed method based on SLIC (the second row), and the proposed method based on WT (the third row).
