A sun/shadow approach for the classification of hyperspectral data

HAL Id: hal-01385498

https://hal.archives-ouvertes.fr/hal-01385498

Submitted on 24 Oct 2016

HAL is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

A sun/shadow approach for the classification of hyperspectral data

G. Roussel, C. Ceamanos, X. Briottet, C. Weber

To cite this version:

G. Roussel, C. Ceamanos, X. Briottet, C. Weber. A sun/shadow approach for the classification of

hyperspectral data. 16th Onera-DLR Aerospace Symposium (ODAS 2016), Jun 2016, OBERPFAF-

FENHOFEN, Germany. �hal-01385498�

COMMUNICATION A CONGRES

A sun/shadow approach for the classification of hyperspectral data

G. Roussel, X. Ceamanos, X. Briottet, C. Weber (LIVE)

ODAS 2016

OBERPFAFFENHOFEN, ALLEMAGNE 21-23 juin 2016

TP 2016-623

A sun/shadow approach for the classification of hyperspectral data

G. Roussel, X. Ceamanos, and X. Briottet

ONERA, DOTA, 2 avenue Edouard Belin, 31055, Toulouse cedex 4, France

C. Weber

LIVE, 3 rue de l’Argonne, 67000 Strasbourg

Abstract

Shadows constitute a major issue for various remote sensing applications, from un- mixing and anomaly detection to image segmentation and classification. It is particularly problematic in urban areas, characterized by a high heterogeneity in terms of both relief, with the presence of numerous tall buildings, and materials, which implies that a single shadow can hide a wide variety of different objects. Shadow induces a strong weakening of the signal which can lead, in the context of a classification process, to the creation of a parasit shadow class for unsupervised cases, or to an increase of class variability for supervised ones. This paper introduces a new classification algorithm where sunlit and shadowed pixels are processed separately. Using a shadow mask, sunlit pixels are isolated and classified using a SVM or a Kmeans algorithm. Then, a centroid is processed for each classe and used by a spectral angle mapper (SAM) method to classify the shadowed pixels. This approach showed promissing results, especially in an unsupervised context.

1 Introduction

When shadows are omnipresent in an image, a de-shadowing preprocessing step is highly recommended in order to improve the classification results. Shadow processing is usually separated into two phases : shadow detection and shadow compensation. Adeline et al. [1]

proposed a very exhaustive taxonomy on shadow detection methods, which are classified in four categories including property-based, model-based, physics-based and machine learning methods. As for the shadow compensation phase, Li and al. [2] have identified two kinds of methods, those that focus on the intensity domain [3, 4, 5] and those working in the gradient domain [6]. However, these methods either rely on a prior manual selection of paired regions, are imprecise when the shadowed areas are composed of a wide variety of materials or tend to alter significantly the non-shadowed areas. Moreover, they have been widely tested on natural multispectral images and rarely on aerial hyperspectral data, where the distance between the sensor and the target can be very important and imply distorsion and absorption effects from the atmosphere and reflection effects from the surrounding 3D structure. Yet, a third categorie of shadow compensation method, more suited to aerial images, can be considered. Indeed, a few atmospheric correction methods, which aim to process a reflectance image through an inverse radiative transfer equation, take shadows into account. The ICARE [7] algorithm can correct both hard shadows, casted by tall objects, and vague shadows, induced by slope orientations, using a precise 3D model of the observed scene. However this algorithm is very expensive in terms of computation time and requires

1

a digital elevation model of the scene which is not always available. Then, Chen et al. [8]

introduced a faster and simpler method able to compute reflectance in shadow areas using only a radiance image and the atmospheric and viewing conditions, by approximating some of the radiative terms involved in the inversion.

Usually, shadow compensation and classification are two distinct processes. A shadow- free image is produced by the first step, and then processed by regular a classification algorithm, which can leads to misclassification when the shadowed areas aren’t perfectly compensated. In this paper, we propose a new hybrid method where sunlit and shadowed pixels are classified sequentially in order to overcome the flaws of the shadow compensation step. First, an hyperspectral adaptation of the Chen et al. [8] empirical method is intro- duced. Then our hybrid classification approache is detailed, followed by a discussion on the classification results.

2 Empirical reflectance processing method

This preprocessing step aim to compute for each pixel a reflectance value, unitless and wavelength-dependant, representing the effectiveness of a material in reflecting radiant en- ergy. The method we chose in this study is an empirical algorithm introduced by Chen et al. [8]. The main asset of this algorithme is its ability to take shadows into account, which implies that it can be considered as a proper shadow compensation method. Assuming a lambertian hypothesis and for given illumination conditions, the reflectance can be expressed as :

ρ = π × (R tot − R env − R atm )

(I _dir + I _dif + I _coup + I _{ref l} ) × τ _dir ^↑ (1) where R _tot is the radiance value received by the sensor. R _env is the portion of R _tot due to the neighborhood of the target pixel and R atm the one scattered by the atmosphere to the tagert withour interacting with the ground. I _dir , I _dif , I coup and I _{ref l} are portions of the overall irradiance striking the target pixel. I _dir stands for the photons coming directly from the top of atmosphere, I _dif for those which are scattered at least once by the atmosphere, I coup for those which make at least one round trip between the ground and the atmosphere and I _{ref l} for those which are reflected by the environment before hitting the ground. Finally, τ _dir ^↑ is the direct upward transmission. All these terms are pixel dependant except for I _dir , I _dif , R atm and τ _dir ^↑ which are constant for the whole scene. In the original algorithm, these constant terms were computed by the radiative transfert processing software 6S [9] but we chose to replace it by MODTRAN [10], more suited to hyperspectral data due to its better spectral resolution.

As a first step of the algorithm, a shadow mask is processed using an automatic shadow detection method introduced by Nagao et al. [11], which was pointed fot its efficiency by Adeline et al. [1]. The mask is obtained by applying an automatic histogram thresholding to the following combination of bands :

R _comb = 2R _{λ r} + R _{λ g} + R _{λ b} + 2R _{λ nir}

6 (2)

where R λ r , R λ g , R λ b and R λ nir are respectively the radiance values associated to the red, green, blue and near infrared channels of an image. In order to discriminate water from shadows, a final step has been added where the detected shadow pixels are thresholded a second time, only on the green channel which showed the best ability for this discrimination.

Shadowed and sunlit pixels are then processed separately. In sunlit areas, I _dir and I _dif constitute most of the irradiance received by a pixel. We consider then that the coupling and reflective terms can be neglected. In shadow areas, there is no direct irradiance and

2

Figure 1: Comparison between a shadow and a sunlit reflectance spectrum obtained with two correction algorithms : COCHISE and the empirical method

(a) Original luminance im- age

(b) Reflectance image ob- tained with the empirical method

Figure 2: Benefits of the empirical atmospheric correction method

the coupling term is also put aside. Therefore, the total irradiance is reduced to a diffuse and a reflective term. However in this case, because of the proximity of buildings, the diffuse irradiance is weighted by a skyview factor α _sky . Using a DEM of the downtown of Toulouse, France, we observed that the shadowed areas of this kind of environment are mainly composed of narrow streets and half-opened areas, and that skyview factors belong to [0.45, 0.65]. A mean value of 0.55 is then chosen. As for the reflected irradiance, it is processed as the sum of direct and diffuse irradiances weighted by an attenuation factor κ and a hypothetical mean reflectance of the scene ¯ ρ :

I _{ref l} = κ × (I _dir + I _dif ) × ρ ¯ (3)

Chen et al. [8] recommend to set κ at 0.2 for cluttered urban environment and 0 for suburban areas. As our data focus on the downtown of Toulouse, only the first value will be used. For ¯ ρ, a combination of several urban materials (concrete, asphalt, tile and vegetation) gathered from spectral libraries is used. At this point, only one radiative term remains, the environment radiance R env . Contrary to S, it is not a direct outpout of MODTRAN, si it needs to be computed iteratively as follows :

3

Figure 3: Sun/shadow classification

R env,t = τ _{dif f} ^↑ · I tot · ρ ˇ t

1 − ρ ˇ _t · S (4)

ρ t = π(R tot − R env,t − R atm )

I _tot ⁰ × τ _dir ^↑ (5)

where

ˇ ρ t+1 =

Z Z

(u,v)∈V (x,y)

ρ t (u, v) · du · dv (6)

is the mean reflectance spectrum associated to the neighborhood of the target for the iter- ation t + 1 and ˇ ρ ₀ = ¯ ρ. τ _{dif f} ^↑ , the diffuse upward transmission and S, the spherical albedo, are processed by MODTRAN. Usually, two iterations of the process suffice to reach the con- vergence. The graphic on figure 1 shows four spectra corresponding to two vegetation pixels (one in the shadow and the other in the sun light) and two atmospheric correction methods, the studied empirical reflectance processing method and COCHISE [12], an approach which doesn’t take shadows into account. For the empirical method, the reflectance spectra of the shadowed spectrum is slightly weaker than that of the sunlit spectrum, but the shape of a vegetation spectrum is still easily recognizable. This is not the case for the COCHISE approach, where the shadowed spectrum is strongly flattened. The benefits of the empiral method can also be noted by comparing a luminance image and a reflectance image on the figure 2. However, we can see that the shadow areas are not perfectly corrected, which is why they have to be taken into account in the classification process.

3 Sun/shadow classification method

In order to complement the empirical atmospheric correction method, we propose a solution implying a sequential process (cf. figure 3) where sunlit and shadow pixels are classified separately. Using the shadow mask described in section 2, the sunlit pixels are isolated and classified through a Kmeans or a SVM algorithm, with a number of classes C fixed by the

4

SVM SSVM KM SSKM OA 89.2% 95.1% 80.9% 94.7%

Kappa 0.86 0.93 0.75 0.93

Table 1: Overall accuracies and Kappa for the four classification methods

user. Then, the cendroids of each class are preserved and used as input for the classification of shadow pixels. In this case, a spectral angle mapper (SAM) classification has been chosen for its ability to focus on the shape of spectra, which remain globally similar for a pair of pixels associated to the same material, even if one is located in a sunlit area and the other in a shadow area. For each pixel p and each centroid µ _i , i ∈ [1, C], a SAM value is processed as follow :

SAM (p, µ) = cos ⁻¹













B

P

b=1

p(b)µ(b) s B

P

b=1

p(b) ² s B

P

b=1

µ(b) ²













(7)

where B is the number of bands of the image. The class chosen for pixel p is the one minimizing the spectral angle mapper between p and its representative centroid. It should be noted that for this method to be efficient, the same classes must be present in both sunlit and shadow areas.

4 Results

The classification algorithm is applied on a section of a dataset acquired during the Umbra campaign over the downtown of Toulouse (cf. figure 4a). Two Hyspex hyperspectral sensors have been used in order to cover a wavelengths interval starting from 0.4µm to 2.5µm, with a spatial resolution of 1.7m. The noisy bands located at the extremities of the spectrum or in the middle of water vapor absorption bands (centered on 0.94, 1.1, 1.38 and 1.87µm) have been removed, resulting in a set of 255 bands. Four classification approach have been applied to the image : a support vector machine algorithm (SVM), a K-means (KM), and our new method with a SVM (SS-SVM) and a K-means (SS-KM) for the sunlit pixels classification.

This latter is applied on the whole set of 255 bands but to process shadowed pixels, we used only the hundred first bands since after 0.9µm, shadowed areas are too noisy. Five classes have been considered : Vegetation (green), water (blue), tile (orange), asphalt (black) and gravel (white). The results on figure 4 show that the SVM and KM methods don’t work well in shadow areas, where most pixels are classified as asphalt. The two sun/shadow classifications on the other hand manage to discriminate water and vegetation pixels even in the shadow. The only issue remains in gravel and asphalt classes which still present strong confusions. The improvement induced by the sun/shadow approach is also observable in the increase of overall accuracies from simple SVM and K-means to sun/shadow SVM and K-means (cf. table 1).

5 Conclusion

A new sun/shadow approach have been introduced for the classification of hyperspectral data. This method involve several preprocessing steps, including an atmospheric correction algorithm and the creation of a shadow mask, before classifying sunlit and shadowed pixels

5

(a) RGB image (b) Shadow sub-areas (c) SVM

(d) Sun/shadow SVM (e) Kmeans (f) Sun/shadow Kmeans

Figure 4: Classification results obtained on four points of interest in the image

successively. Depending on the chosen algorithm for the classification of sunlit pixels, the method can either supervised (SVM) or unsupervised (K-means). In either cases, it appears that taking shadows into account in the classification process can strongly improve its accu- racy. Therefore, this approach is a good complement to shadow compensation techniques, especially in a hyperspectral context where these latter are still rare and perfectible.

6 Acknowledgment

The authors would like to acknowledge the ANR HYEP for the financial support of this work, the Onera institute for the provision of the hyperspectral data and the IGN institute for the coregistration of the two HYSPEX sensors.

References

[1] K.R.M. Adeline, M. Chen, X. Briottet, S.K. Pang, and N. Paparoditis, “Shadow de- tection in very high spatial resolution aerial images: A comparative study,” {ISPRS}

Journal of Photogrammetry and Remote Sensing, vol. 80, pp. 21 – 38, 2013.

[2] Huifang Li, Liangpei Zhang, and Huanfeng Shen, “An adaptive nonlocal regularized shadow removal method for aerial remote sensing images,” IEEE Transactions on Geo- science and Remote Sensing, vol. 52, no. 1, pp. 106–120, Jan 2014.

[3] F. Yamazaki, W. Liu, and M. Takasaki, “Characteristics of shadow and removal of its effects for remote sensing imagery,” in Geoscience and Remote Sensing Symposium,2009 IEEE International,IGARSS 2009, July 2009, vol. 4, pp. IV–426–IV–429.

[4] L. Lorenzi, F. Melgani, and G. Mercier, “A complete processing chain for shadow detection and reconstruction in vhr images,” IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 9, pp. 3440–3452, Sept 2012.

6

[5] E. Arbel and H. Hel-Or, “Texture-preserving shadow removal in color images containing curved surfaces,” in Computer Vision and Pattern Recognition, 2007. CVPR ’07. IEEE Conference on, June 2007, pp. 1–8.

[6] G. D. Finlayson, M. S. Drew, and C. Lu, “Entropy minimization for shadow removel,”

Int. J. Comput. Vis., vol. 85, no. 1, pp. 35–37, October 2009.

[7] S. Lacherade, C. Miesch, D. Boldo, X. Briottet, C. Valorge, and H. Le Men, “Icare: A physically-based model to correct atmospheric and geometric effects from high spatial and spectral remote sensing images over 3d urban areas,” Meteorology and Atmospheric Physics, vol. 102, no. 3-4, pp. 209–222, 2008.

[8] M. Chen, K.L.C. Seow, X. Briottet, and Sze Kim Pang, “Efficient empirical reflectance retrieval in urban environments,” Selected Topics in Applied Earth Observations and Remote Sensing, IEEE Journal of, vol. 6, no. 3, pp. 1596–1601, June 2013.

[9] E.F. Vermote, D. Tanre, J.L. Deuze, M. Herman, and J.-J. Morcette, “Second simulation of the satellite signal in the solar spectrum, 6s: an overview,” Geoscience and Remote Sensing, IEEE Transactions on, vol. 35, no. 3, pp. 675–686, May 1997.

[10] A. Berk, L. S. Bernstein, and D. C. Robertson, “Modtran : A moderate resolution model for lowtran 7,” Air Force Geophysics Laboratory Technical Report, 1989.

[11] M. Nagao, T. Matsuyama, and Y. Ikeda, “Region extraction and shape analysis in aerial photographs,” Computer Graphics and Image Processing, vol. 10, no. 3, pp. 195–223, 1979.

[12] C. Miesch, L. Poutier, V. Achard, X. Briottet, X. Lenot, and Y. Boucher, “Direct and inverse radiative transfer solutions for visible and near-infrared hyperspectral imagery,”

Geoscience and Remote Sensing, IEEE Transactions on, vol. 43, no. 7, pp. 1552–1562, July 2005.

7

Office National d'Études et de Recherches Aérospatiales 2 avenue Edouard Belin - BP 74025

31055 TOULOUSE Cedex 4 Tél. : +33 5 62 25 25 25

http://www.onera.fr