HAL Id: hal-00801968
https://hal.archives-ouvertes.fr/hal-00801968
Submitted on 18 Mar 2013
HAL
is a multi-disciplinary open access archive for the deposit and dissemination of sci- entific research documents, whether they are pub- lished or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.
L’archive ouverte pluridisciplinaire
HAL, estdestinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.
Contact less Hand Recognition using shape and texture features
Julien Doublet, Olivier Lepetit, Marinette Revenu
To cite this version:
Julien Doublet, Olivier Lepetit, Marinette Revenu. Contact less Hand Recognition using shape and texture features. 8th International Conference on Signal Processing, Nov 2006, Guilin, China. 4 p.,
�10.1109/ICOSP.2006.345881�. �hal-00801968�
Contact less Hand Recognition Using Shape and Texture Features
Julien Doublet1 Olivier Lepetit1 Marinette Revenu2
1: France Telecom, 42 rue des coutures, B.P. 6243, 14066 CAEN Cedex 4, France 2: GREYC – ENSICAEN, 6Bd Maréchal Juin, 14050 CAEN Cedex, France
E-mail: [email protected] Abstract: The hand recognition in biometric security was
developed successfully for authentication or identification. In this paper, we propose an original method of biometric recognition combining of information of color, texture and form.
First of all, the segmentation integrates the skin color components and a form model. Then, the authentication process amalgamates by convolution the geometrical characteristics of the fingers and the texture of the analyzed palm. The experimental results show an error recognition rate lower than 2% with a population of 16 people.
Key words: Hand recognition, Biometrics, fusion, skin detection, active shape model
I. INTRODUCTION
Biometrics plays an increasingly important part in authentication and identification systems. The processes of biometric recognition allow the recognition of individual based on the physical or behavioral characteristics. Various technologies were developed such as: fingerprints, iris, face, voice, signature and hand. This last method is based on a study of hand shape and palm texture. It has many advantages compared to other technologies. Firstly, the capture device is less expensive than that for iris recognition, the hand characteristics are more numerous than those of fingerprints and they can be specified with weak resolution images. Moreover, this system is well accepted by users and the hand leaves few traces contrary to fingerprints.
The architecture of biometric systems is organized in 4 stages:
the data acquisition, the signal treatment for data improvement and the segmentation of analyzed modality, the characteristics extraction and the comparison with one or more references. In our system, stage 2 corresponds to hand detection in a complex image. The methods of Skin Blob Tracking, active contours, Mean Shift or condensation [1] are usually used in man-machine interactions systems. These processes however have two major constraints incompatible with a biometric recognition. They provide an approximate detection of each finger and they require a sequence of images. To increase the segmentation quality, we propose a method combining information of skin color and hand shape. The third stage consists of a hand form and palm characterization. Many works were completed these last years. The hand shape can be described by the palm width and the shape of fingers. Each finger is defined classically by its length, a whole of widths and the curve of its end. Initially, the palmprint was characterized by its lines and points features [2]
by using a method close to the fingerprints recognition. Other systems use the global information of the hand palm [3-5] and they present better performances in recognition. To lead to more reliable systems, it is possible to amalgamate the decisions obtained starting from the palm and the hand shape [6].
In this work, we propose a method of contact less hand recognition. Section 2 introduces the hand detection process.
Section 3 describes the hand shape and palmprint features
extractions as well as the fusion method of these characteristics.
Sections 4 and 5 present the experimental results and give the conclusions, respectively.
II. HAND SEGMENTATION IN COMPLEX IMAGES In biometric recognition system, a precise and fast segmentation process must be elaborate. In this section, we briefly describe our hand detection method based on a combination of skin color modelling and shape model [7].
II.1 Skin color modelling
Contrary to models based on Bayes classifiers or Gaussian Mixtures Models [8], our skin color is modelled by machine learning. To obtain a good compromise between the execution speed and the precision of detection, we use a neural network (NN). The NN entries are composed by three neurons, one for each color component of pixels in RGB domain. The NN output is the probability that a pixel is a skin pixel. The network learning, from a skin and no skin pixels database in RGB domain, allows the skin color modelling. In parallel, a Principal Components Analysis [9] on the skin pixels database establishes a specific color space to the skin tone.
II.2 Skin segmentation
In the detection phase, the NN calculates the probability that each pixel is a skin pixel. This process builds the probabilities map (fig. 1). To obtain a segmentation close to real time, a multi-resolution process builds the probabilities map. A thresholding and a contrast reinforcement are applied to obtain a cleaned map.
Fig. 1 Probabilities map computation II.3 Hand segmentation by active shape model
The segmentation by skin color cannot carry out in a robust way the task of hand detection. A specific active shape model is defined to solve this problem and to solve the two major difficulties in active shape model [10]. Its problems are the contour initialization which must be close to the real form and the model convergence in detection phase.
Classically, the form to detect is defined by a set of points:
the landmarks. In training phase, the average form and the contour variations are computed by Principal Components Analysis on an annotated hands database by these landmarks. In
segmentation phase, the contour is initialized by the characteristic points of the hand: the five fingertip points and the four points located in the valley between two adjacent fingers.
These points are calculated from the probabilities map by contour analysis. Next, two other points are automatically added close to wrist from these points. The others landmarks defining with more precision the hand shape are disposed between those.
Thus, the model X is defined by the 11 initial points and N intermediate points between those. We obtain X by
(
[0], , [11+ ×5×2−1])
= X X N
X K where X[i] is the ith
landmark. After the initialization phase, the model is deformed.
To control the problem of model divergence which does not follow the real hand contours, a weight is applied to deformations to limit the shape constraints [7]. So that the gradient only uses the hand contours and limits thus the possibilities of form divergence, it is computed in skin color space by Di Zenzo algorithm [11]. Then, this gradient is balanced by the coefficient of the probabilities map pixels. The experiments show that a good compromise between the execution time and the detection precision is obtained by fixing N at 12. The complete detection process is illustrated on Figure 2.
(a) (b)
(c) (d)
Fig. 2 Complete process (a) original image (b) skin detection (c) initial hand shape (d) final hand shape
III. FEATURES EXTRACTION
This section defines the palm features and the hand shape characteristics as well as the fusion method for a powerful biometric recognition. In section 3.1, the palm extraction and the texture specification by a Gabor filter are described. The hand shape features are defined from its contours. Finally, the fusion of all the characteristics by a convolution process is detailed in section 3.2.
III.1 Palmprint and shape features computation
After hand detection, it is necessary to extract the acquired palm independently of the distance between the hand and the capture device. This extraction is based on hand dimension and the method of palms extraction described in [3]. In this article, two values are fixed: the distance between the points O1 and O2 and the palm size ||A1A2|| (Fig. 3). These values, constants in traditional recognition systems, are defined here according to hand size. They are determined from the hand width, computed by the Euclidean distance between the points X[L1] and X[L2]
where L1 and L2 are some indexes fixed after experiments at 30 and 125. Thus, ||O1O2|| and ||A1A2|| are defined by:
||O1O2|| = α ||X[L1]X[L2]|| (1)
||A1A2|| = β ||X[L1]X[L2]|| (2)
Where α and β are the dimensions coefficients chosen at 1/10 and 2/3, respectively. Then the palm is resized with a fixed size M*M where M is fixed at 100.
Fig. 3 Schematic palm extraction with N=4
The extracted palm contains some principal lines which can be determined by a specific palm line extraction [2]. These lines are not unique to each individual, thus it is necessary to use the secondary hand lines. These finer lines, the wrinkles, cannot be extracted from the palm with images in weak resolution, so a global palm characterization is preferable.
Various methods can to obtain the palm characteristics:
wavelet analysis, Fourier transforms, Principal Component Analysis, Gabor filter... Thanks to its good performances in palmprint and iris recognition and with its specific qualities:
accurate time-frequency location and robustness with the contrast and luminosity variations, we used a Gabor filter. A variety of implementations of this filter exist. In [3], a 2D Gabor filter in the complex domain is used. To limit the computing time and the characteristics size, the filter in real domain describes in [12] is used:
+
×
+
−
= ϕ
π λ σ
γ '
2
² cos 2
'²
² exp '²
) ,
( x y x
y x
G (3)
θ η θ
ξ
θ η θ
ξ
cos ) ( sin ) ( '
sin ) ( cos ) ( '
−
−
−
=
−
−
−
=
y x
y
y x
x
Where the couple (ξ, η) defines the function center, θ controls the function orientation, σ is the standard deviation of the Gaussian factor, γ is the spatial aspect ratio fixed at 0.5, λ is the wavelength and φ is the phase. σ is defined by the constant ratio σ/λ=0.56 [12]. For more luminosity robustness, the filter is centred to the point (0, 0) using for a filter of size (2k+1)² the formula:
)² 1 2 (
) , ( )
, ( ) ,
( = − +
Ω
∑ ∑
=− =−k
j i G y
x G y x
k k i
k k
j (4)
Thus the palm features are obtained by the convolution result of the palm image I with this robust Gabor filter by:
Ω
=I*
C (5)
Where * is the convolution operator. To supplement the representation, the hand shape characteristics are extracted.
They are the widths and the lengths of the fingers. The lengths and the widths are approximated by the distance between the model points and they are defined by:
(
( [ ], [ ]), [ ])
]
[i =d m X H2i X H2i+2 X H2i+1
L (6)
(
[ ], [ ])
] ][
[i j d X H2 1 j X H2 1 j
l = i+ − i+ + (7)
Where d(a, b) is the Euclidean distance between the points a and b, m(a, b) returns the medium of the segment ||ab||, L[i] is the ith finger length, l[i][j] is the jth width of the ith finger and HI=i(N+1) is the index of the ith initial characteristic point of the hand (Fig. 4).
Fig. 4 Hand shape features with N=4 III.2 Texture and shape fusion
In biometric systems, three combination methods are used.
The combination can be carried out with the representation by gathering the extracted characteristics, at the comparison level or at the decisions level. Our fusion is based on the palm texture convolution with the hand shape features in order to add a geometrical factor to texture. The result of this convolution is binarised to limit the characteristics size and the computing times in comparison phase. The complete process is defined by:
)
* ) , ( ( ) ,
(x y b C x y H
S = (8)
Where b(x)=0 if x<0 and b(x)=1 if not, * is the convolution operator and H is a filter of size 5x5 corresponding to shape characteristics. It can be explained by:
∑ ∑
= =×
= 4
0 4
0 '[ ][ ] ) 1
, ( ' ) , (
i j H i j
y x H y x
H (9)
Where H'=
L[0] l[0][1] l[0][2] l[0][3] l[0][4]
L[1] l[1][1] l[1][2] l[1][3] l[1][4]
L[2] l[2][1] l[2][2] l[2][3] l[2][4]
L[3] l[3][1] l[3][2] l[3][3] l[3][4]
L[4] l[4][1] l[4][2] l[4][3] l[4][4]
The normalisation coefficient of H makes it possible to be robust the distance between the hand and the acquisition device.
Fusion by characteristics convolution increases the hand unicity.
It enables to distinguish people having hand shapes or palms very similar. The texture features alone and the combination of palm texture and hand geometry characteristics are illustrated on Figure 5.
(a) (b) (c)
Fig. 5 Features extraction (a) palm extraction (b) palm texture (c) fusion of texture and shape
IV. EXPERIMENTS
In this section, our experimental results are indicated to validate this convolutional approach. Firstly, the hand matching is indicated. Then, the hand database and the performance evaluation of the system are described.
IV.1 Hand matching
For the matching, the hand features S are assimilated with a matrix. A traditional comparison method of binary matrices is thus applied: the normalized Hamming distance. This distance is a pixel by pixel comparison and it gives a standardized answer between 0 and 1, 0 being the perfect correspondence. It is defined for two hands characteristics X and Y by:
²
) , ( ) , ( )
,
( 1 1
0 M
j i Y j i X Y
X D
M i
M
∑ ∑
= j= ⊕= (10)
Where ⊕ is XOR operator. The hand segmentation as well as the palm detection being not perfect, a tolerance in translation and rotation is applied to matching process. This flexible comparison is expressed for two hands characteristics X and Y by:
Dr(X,Y)=mins<S,t<T,a<ADO(R(T(X,s,t),a),Y) (11)
Where T(X, s, t) is the translation of image X horizontally by s and vertically by t and R(X, a) is the rotation of image X by an angle a. The constants S and T are fixed at 2 pixels and A is limited to 2° in order to limit the computation time.
IV.2 Performance evaluation
A specific database is formed to validate our approach. All images of the base were acquired by a webcam Philips ToUcam Pro 740K with the resolution 640x480 pixels. The base contains 160 images, some with hands having of the rings, coming from 16 people. 10 images are acquired for each individual of the base. Figure 6 shows four examples of palms extracted from our acquisitions.
To obtain the optimal coefficients and the best filter representing the hand shape, a set of parameters is used to validate our approach. Firstly, a collection of filter is created to determine the best recognition score for the palm alone. Each palm of the hands database is compared with all the others in order to determine this score. The recognition rate is the ratio
between the number of incorrect comparisons and the total number of comparisons. A comparison is incorrect if a user is accepted or rejected wrongly. Two rates characterize these errors:
the false acceptance rate (FAR) defined by the ratio between the number of people authenticated wrongly and the number of comparisons and the false rejection rate (FRR) defined by the ratio between the number of people rejected wrongly and the number of comparisons.
Fig. 6 Four extracted palm in our database
The parameters of tests for the Gabor filter are inspired by [12].
The filter orientation is tested according to eight values θ=22.5°, θ=45°, …, θ=180°, three values are used for the space frequency λ=5.47, λ=8.20 and λ=10.93, (ξ, η) and the phase φ are fixed at (0,0) and 0 respectively, while the standard deviation σ is given according to the ratio σ/λ=0 56. The best comparison rate presented in Figure 7 is obtained with the parameters θ=157.5°, λ=10.93 and σ = 6.12. With these coefficients, the EER (Equal Error Rate) when FAR=FRR indicates an error of 2.25%. The Gabor filter in complex domain presented in [3] shows similar results on our database with optimal parameters. To decrease the recognition errors, we presented a new fusion method of the hand shape and palm texture. After experiments, the optimal filter defining the shape is only composed of the fingers widths.
Thus, the optimal filter H is defined by:
∑ ∑
= = +× +
= 4
0 4
0 [][ 1]
) 1 1 , ( ) , (
i j l i j
y x l y x
H (12)
The global system performances are well increased between the palmprint recognition and the global hand recognition (Fig. 7).
Indeed, the EER is equal to 1.85% and one secured system can be defined with FAR=10 -3 % and FRR=2.2%. The complete process of recognition (segmentation, characteristics extraction and matching) is carried out in less than 1 second on Pentium M with 1.60GHz.
Fig. 7 Hand recognition
V. CONCLUSION
In this paper, we present a new method of biometric hand recognition for a contact less system. First of all, the hand segmentation is explained. It is carried out thanks to an integration of the skin color components and a shape model.
Then, the authentication process by fusion is described. It is based on a combination by convolution of the hand geometrical features and the palm texture. The palm characteristics are computed by a Gabor filter in the real domain allowing a compact representation. The hand shape features are extracted from the model. The complete process is validated after experiments. It shows an error rate of 1.85% with an execution time lower than 1 second.
In order to manage space rotation of the hand, an invariant method with the perspective and the shear will have to be required. Moreover, the test database will have to be diversified and supplemented to confirm our approaches. These two tracks are the subject currently of our work.
REFERENCES
[1] Y. Zhu, "Hand Detection in an Active Vision System", Thesis, 2003.
[2] N. Duta, A. K. Jain, K. V. Mardia, "Matching of palmprints", Pattern Recognition Letters, Volume 23, Issue 4, 2001.
[3] W. Kong, D. Zhang, W. Li, "Palmprint feature extraction using 2-D Gabor filters", Pattern Recognition, Volume 36, Issue 10, 2003.
[4] G. Lu, D. Zhang, K. Wang, "Palmprint recognition using eigenpalms features", Pattern Recognition, Volume 24, pp.
1473-1477, 2003.
[5] X. Q. Wu, K. Q. Wang, D. Zhang, "Wavelet Based Palmprint Recognition", Conference on Machine Learning and Cybernetics, Beijing, 2002.
[6] A. Kumar, C. M. Wong, C. Shen, A. K. Jain, "Personal Verification using palmprint and hand geometry biometric", Conference on Audio-Guildford, 2003.
[7] J. Doublet, O. Lepetit, M. Revenu: "Hand detection for contactless biometrics identification", Cognitive System with Interactive Sensors, Paris, 2006.
[8] L. Lucchese, S. Mitra, "Color image segmentation: A state of the art survey", Proc. of the Indian National Science Academy, 2001.
[9] L. I. Smith, "A tutorial on Principal Components Analysis", 2002.
[10] T.F. Cootes, C.J. Taylor, "Statistical models of appearance for computer vision", Technical report, University of Manchester, Wolfson Image Analysis Unit, Imaging Science and Biomedical Engineering, Manchester M13 9PT, United Kingdom, 1999.
[11] S. Di Zenzo, "A note on the gradient of a multi-image", Computer Vision, Graphics and Image Processing, 33(1), 1986.
[12] P. Kruizinga, N. Petkov, "Nonlinear Operator for Oriented Texture", IEEE Transactions on image processing, Volume 8, Issue 10, 1999.
