Publisher’s version / Version de l'éditeur:
Carleton Journal of Computer Science, 5, 2001
READ THESE TERMS AND CONDITIONS CAREFULLY BEFORE USING THIS WEBSITE.
https://nrc-publications.canada.ca/eng/copyright
Vous avez des questions? Nous pouvons vous aider. Pour communiquer directement avec un auteur, consultez la
première page de la revue dans laquelle son article a été publié afin de trouver ses coordonnées. Si vous n’arrivez
pas à les repérer, communiquez avec nous à PublicationsArchive-ArchivesPublications@nrc-cnrc.gc.ca.
Questions? Contact the NRC Publications Archive team at
PublicationsArchive-ArchivesPublications@nrc-cnrc.gc.ca. If you wish to email the authors directly, please see the
first page of the publication for their contact information.
NRC Publications Archive
Archives des publications du CNRC
This publication could be one of several versions: author’s original, accepted manuscript or the publisher’s version. /
La version de cette publication peut être l’une des suivantes : la version prépublication de l’auteur, la version
acceptée du manuscrit ou la version de l’éditeur.
Access and use of this website and the material on it are subject to the Terms and Conditions set forth at
Colour Correlation Window Matching using Colour Spaces
Miles, A.; Roth, Gerhard
https://publications-cnrc.canada.ca/fra/droits
L’accès à ce site Web et l’utilisation de son contenu sont assujettis aux conditions présentées dans le site
LISEZ CES CONDITIONS ATTENTIVEMENT AVANT D’UTILISER CE SITE WEB.
NRC Publications Record / Notice d'Archives des publications de CNRC:
https://nrc-publications.canada.ca/eng/view/object/?id=554c4a21-beda-4a23-a42e-d205c19dc856
https://publications-cnrc.canada.ca/fra/voir/objet/?id=554c4a21-beda-4a23-a42e-d205c19dc856
National Research
Council Canada
Institute for
Information Technology
Conseil national
de recherches Canada
Institut de technologie
de l'information
Correlation Window Matching using Colour Spaces *
Miles, A., Roth, G.
January 2001
* published in Carleton Journal of Computer Science, 5:12, January 16, 2001.
NRC 45867.
Copyright 2001 by
National Research Council of Canada
Permission is granted to quote short excerpts and to reproduce figures and tables from this report,
provided that the source of such material is fully acknowledged.
Using Colour Spa es
Andrew Miles
S hool of Computer S ien e Carleton University
Ottawa, Canada miless s. arleton. a
Gerhard Roth
Visual Information Te hnology Group National Resear h Coun il of Canada
Ottawa, Canada Gerhard.Rothnr . a
Abstra t
Consider two small re tangular win-dows, ea h of whi h is entered at a dif-ferent pixel in two di erent olor images. Correlationisthepro essof omputingthe similarity between these twowindows. If the amerarotation between the two im-ages is not large then if these two pixels representthe same physi al surfa e point thenthesimilarity,andhen ethe orrela-tion value, should be high. While orre-lation is a well known te hnique in om-putervision,therearemanypossible vari-ations. Themain variability omesin the type of orrelation algorithm, the olour spa ethatisused,andthesizeofthe win-dow. Thispaperdes ribesaset of experi-mentswhosegoalistoexploretheimpa t of hanging these three parameters. The methodology is rst to manually sele t a set of mat hing features betweentwo im-ages,whi hareknownasthegroundtruth. Then the di erent orrelation variations are ompared in terms of their ability to determinethe orre tmat hingfeatureout
ofallthepossibilities.
1 Introdu tion
A wellknownproblemin omputervision isthemat hingoffeaturesbetweentwo im-ages. Features may represent orners or edgesintheimage,andthemost ommon method to mat h these features is orre-lation. The orrelation fun tion returns thesimilaritybetweenthefeaturepointsof twodi erentimages. It has as inputtwo small re tangular windows that are en-tered on these feature points in both im-ages.
There are anumberof di erent orre-lation algorithms: Least Squares, Cross- orrelation, Histogram, Ordinal Measures andanA umulatedDi eren es. Itisalso possibletoperformthe orrelationon dif-ferent olor spa es, and the possibilities are the RGB, HSV, HLS and CIE olour spa es. Finally,there isalso variabilityin
With all these variations a number questions anbeasked:
Is there a \best" Correlation Method.
What e e t do the various olour spa eshaveontheCorrelation Meth-ods.
HowdotheCorrelationMethods be-havewhen hangingtheCorrelation Windowsize.
Thegoalofthis paperisto presentan experimentalmethodologyandasetof ex-periments whi h attempt to answer these questions.
2 Correlation Window
Methods
Firstwedes ribethedi erenttypesof or-relationmethods that are in ommon us-age. A orrelationwindowis aprede ned nn area around a single point. Usu-allynisapositiveoddintegerintherange of three to twenty [3℄. The totalnumber of pixels in the orrelation window is N, whi hisequaltonn.
The orrelation methods we have im-plemented are the Least Squares, His-togram, Bhat and Nayar's Ordinal Mea-sure[5℄,NormalizedCross-Correlation,as well as the A umulated Di eren es ap-proa hes.
Inall these asesthe inputto the or-relation fun tions are the two windows I
1 and I
2
and the output is value be-tween zero and one, where zero is a per-fe tmat h,andoneisaperfe tmismat h. More formally the orrelation fun tion is de ned as '(I
1 ;I
2
) 2 [0;1℄. We also as-sumethat theimage pixelsofthetwo im-ages are bounded, that is I (I);I (I) 2
[0;k℄, wherek istypi ally asmall integer. In this notation the index I ranges over ea hoftheNpixelsinthe orrelation win-dow.
We now des ribe ea h of the imple-mented orrelationmethodsinmoredetail.
2.1 The Least Squares
Method
The Least Squares Method omputes the similarity between the two windows by simply omputingthe sumof squares dif-feren ebetweenthepixels.
'= P N i=1 (I 1 (i) I 2 (i)) 2 k 2 N (1)
2.2 The Histogram Method
TheHistogramMethod rstpla esthe pix-els in the orrelation windows I
1 and I
2 intoj equallydistributedbins,whereea h bin ontainsthenumberofpixelsthat are intherangeofthatbin. Moreformally,ifa pixelp2[0;k℄,thenea hbinhasarangeof k=j onse utivepixels. The rstof j bins ounts the number of pixels whi h have valuesfromzerotok=j 1,these ondfrom k=jto2(k=j) 1andsoon. Thisisknows asahistogram,andj de nesthe granular-ityofthehistogram. Thehistogramvalues arethen omparedtoea hotherusingthe sumofthedi eren esperbin:
'= P j i=1 jH 1 (i) H 2 (i)j 2N (2)
2.3 The Ordinal Measures
OrdinalMeasuresarebasedonrelative or-deringofvaluesinanimagewindow. The relative ordering of image values in ea h
windowisrepresentedbyarank permuta-tion,whi his obtainedby sortingthe im-age values. Sin e the rank permutations areinvarianttomonotonetransformations oftheimagevalues,the oeÆ ientsare un-a e ted by nonlinear e e ts like gamma variationsbetweenimages[5℄.
2.4 The Normalized Cross Correlation
The Normalized Cross-Correlation of an image I
1 and I
2
having N data points is givenby: '= P N i=1 (I 1 (i) I 1 )(I 2 (i) I 2 ) 2N p (I 1 )(I 2 ) (3) Here I 1 = ( P N i=1 I 1
(i))=N is the the average pixel values in the rst window andI
2
isde ned similarly. Also, (I 1 )= q P N i=1 (I 1 (i)) 2 (I 1 ) 2
isthestandard de-viation of the rst image in the window, and(I
2
)isde nedsimilarly.
This advantage of this approa h over the least squares method is the ability to deal with an additive transformation broughtonforexample,byaglobal hange inthelighting.
2.5 The A umulated Di er-en es Correlation
The A umulated Di eren es is another Correlating Window method for image sub-regionsgivenby:
'= P N i=1 jI 1 (i) I 2 (i)j (4) 3 Colour Spa es
A olour spa e is a spe i ation of a 3D olour oordinatesystemandavisible sub-set inthe oordinatesystemwithin whi h all oloursin aparti ular olourgamutlie [4℄. Thereareanumberofdi erent olour spa esusedin pra ti e.
3.1 RGB (Red, Green, Blue)
The RGB (Red, Green, and Blue) olour spa e is the unit ube subset of the 3D Cartesian oordinatesystem. The individ-ual ontributions of ea h primary olour are added together to yield the RGB ColourModel[4℄.
3.2 HSV (Hue, Saturation, Value)
The HSV (Hue, Saturation, and Value) olour spa e is user-orientedbe auseit is basedonrepresenting olourintuitively us-ingtint,shade,andtone.
3.3 CIE (Commission
Internationale de
l'E larirage's)
The CIE (Commission Internationale de l'E larirage's) proposes a olour s heme based on the per eptual uniform olour system(UCS). The onversionfrom RGB to CIE's XYZ olour system o urs through the linear transformation as fol-lows: x= X X+Y +Z ;y= Y X+Y +Z (5)
X=0:619R+0:177G+0:204B (6) Y =0:299R+0:586G+0:115B (7) Z =0:56G+0:944B (8)
3.4 Other Colour Spa es
The normalized TSL (Tint, Saturation, and Luminan e) hrominan e-luminan e spa e,CMY(Cyan,Magenta,andYellow), and the YIQ (U.S. olour broad ast TV format)areother ommon olourspa es.
4 Correlation Testing
To test the various orrelation methods (Least Squares, Cross- orrelation, His-togram, Ordinal Measures and an A u-mulatedDi eren emethod)aswellastest theimpa t of using di erent olour mod-els(RGB,HSV,HLS,andCIE)weusethe followingmethod.
1. Sele t pairsimages (I 1
and I 2
) that haveasuÆ ient overlap[2℄. We as-sume that the overlapis at least 70 %oftheimagesize.
2. Find orrespondingfeaturepointsin ea h image (f
1 and f
2
) [3℄. These feature pointsare theground truth, theyareknowntobe orre t.
Weperformthe testingbytryingea h permutation of orrelation method and olour mat hing using the given image pairsandfeatures. We orrelateevery fea-ture ombination, whi h is a total of (f
1 x f
2
) possibilities. Only one of these is the orre tmat h,andthe orrelation su - eedsonlyifwe ndthe orre tlymat hing featurespairoutofthe(f
1 xf
2
)
possibili-5 Experimental Results
In Figure 1 are two images, ea h of size 768x512that weusein theexperiments. Ea h image has a white box at the pre-de ned features. The orrespdonen e be-tween the features (that is whi h of the rossesmat h)is sele tedbyhand,and is thegroundtruth. Wehaveusedthreepairs ofimagestoperformtheexperiments.
Figure 1: Two images whi h have 127 mat hingfeatures sele tedbyhand.
The results of the omparisons of the various orrelating window methods are depi tedinTable1.
This table shows that out of all the orrelation methods the Cross- orrelation method seemsto produ ethe best results in termsof ndingthe orre tmat h.
Thenextexperimentevaluatesthe im-pa t of using omponents of the olour
Method RGB(%) HSV (%) HLS (%) CIE (%) LeastSquares 70.8 52.0 0.8 66.9 Histogram 15.7 18.9 2.4 9.5 OrdinalMeasure 39.4 22.0 8.7 40.9 Norm. Cross- orr 78.0 61.4 0.8 78.0 A um. Di eren es 70.1 54.3 7.1 64.6
Table1: Comparisons ofCorrelatingWindowMethods (9x9) withvariousColourSpa es givenin theper entageof orre t orrelation's.
spa e. For example, is it feasible to use only one omponent, say the Hue, of the HSV olourspa e to a quireadequate re-sults? Theansweristabulated in table2. The experiment was run on image set 3 having size 768 x 512using a 9x 9 pixel orrelationwindow. The ross- orrelation methodwasusedinthetabulationofTable 2.
From table 1 we know that the `best' orrelationwas78%. Intable2weseethat the individual RGB omponents range from75.6to 77.2%. AlsotheHSVValue omponent gave good results. The CIE omponents faired similarly to the RGB omponents, notsurprisingsin eCIE isa linear ombinationofRGB.
Thislastexperiment ompares Correla-tionWindowmethodstoCorrelation Win-dow sizes using the RGB Colour Spa e. Onewouldexpe tthatthelargerthe Cor-relation Window size, the more a urate themat hing. This is indeed the ase,as anbeseenfromtable3.
Figure3isthegraphi alinterpretation of table 3. The Least Squares and the A umulatedDi eren esmethodhavethe highestnumberof orre t orrelation's us-ingtheRGB olourspa e;imageset3with 175 orrelation points. The Normalized Cross-CorrelationMethoddoesnotdowell for small Correlation Windows but gives betterresultswhenthe orrelationwindow sizeishigh.
6 Con lusions
This paper has presented an evaluation of di erent orrelation windows and dif-ferent olour spa es. The orrelation methods that were tested are the Least Squares, Cross- orrelation, Histogram, Ordinal Measures, and A umulated Dif-feren es. The olour spa es tested were RGB, HSV, HLS,and CIE. Wehave also performed these tests with di erent win-dow sizes (3x3, 5x5, 7x7 et .) as well as with di erent image sets. The image sets anhaveseveralinterestingdi eren es su h as translations, rotations, di erent overlap on gurations,anynumberof or-relationpoints,and ombinationsofthese. The testing onsisted of omparing CorrelatingWindowMethodswithvarious Colour Spa es, image sets with individ-ual Colour Components, and Correlation Window methods with Correlation Win-dowsizes. Sin ethegroundtruthinterms of orre t mat heswasknowna-prioriwe were ableto test how well ea h permuta-tion faired in termsof nding the orre t mat hes.
The Normalized Cross-Correlation method faired well for a `small' number of orrelating points using a large Corre-lation Window size. The Least Squares and A umulated Di eren es method performed well over all. The Histogram methodperformedpoorlyforsmall
Corre-Image PTS R G B H S V C I E
Set (%) (%) (%) (%) (%) (%) (%) (%) (%)
1 127 77.2 75.6 76.4 11.8 52.0 74.8 77.2 75.6 74.8 2 319 49.5 48.9 48.6 8.8 35.1 49.5 48.9 49.2 48.6 3 175 76.0 77.1 77.7 13.7 54.3 77.7 76.6 78.3 78.3 Table2: Comparisons ofCorrelatingWindowMethods (9x9) withvariousColourSpa es givenin theper entageof orre t orrelation's.
Correlation 3x3 5x5 7x7 9x9 11x11 13x13 15x15 Window Method (%) (%) (%) (%) (%) (%) (%) LeastSquares 52.0 70.9 81.1 89.7 92.6 94.3 94.3 Histogram 14.3 13.7 23.4 29.1 38.9 44.0 52.6 OrdinalMeasure 8.6 33.1 54.9 60.0 63.4 61.1 54.3 Norm. Cross- orr 22.9 53.7 68.0 78.3 82.9 86.3 89.1 A um. Di eren es 49.1 71.4 83.4 91.4 92.6 94.3 90.9
Table 3: Correlation Window methods ompared to Correlation Window sizes using the RGB ColourSpa e. Theresultisgivenin per entageofa orre tmat ha ross175 Corre-lationWindowsofimageset3.
Figure 2: Correlation Window methods ompared to Correlation Windowsizes using the RGB ColourSpa e from dataof table 3. Series1=Least Squares,Series2= Histogram, Series3=OrdinalMeasures,Series4=NormalizedCross-Correlation,Series5= A umu-latedDi eren es.
astheCorrelationWindowsizein reased. The RGB and CIE olour spa e per-formed well overall, their similar perfor-man e an be attributed to the fa t that theyarelinear ombinationsofea hother. The HSV olour spa e performed poorly overallbuttheValue omponentonitsown performed very well. The Value ompo-nent ompiled resultsfavorable to that of the ombinedRGB olourspa e. Improve-mentstothemodel anbemadeheresin e the ombinationofthevarious olourspa e omponentswasignored. Itmaybe possi-bleto ombinevarious omponentsa ross olourspa esinsu hawaythat theywill improvetheresulting orrelation.
Referen es
[1℄ Mokhtarian, Farzin and Suomela, Riku, Robust Image Corner Dete -tionThroughCurvature S aleSpa e, IEEETrans.onPatternAnalysisand Ma hine Intel., Vol. 20, No. 12, De- ember1998.
[2℄ Roth,Gerhard,Registeringtwo over-lapping range images, 2nd Int. Conf. on Re ent Adv. in 3D Imaging and Modelling, Ottawa,Canada, pp. 302-311,1999.
[3℄ Tru o,EmanueleandVerri, Alessan-dro, Introdu tory Te hniques for 3D ComputerVision,Prenti eHall,1998. [4℄ Foley, VanDam, Feiner, and Hughes, Computer Graphi s: Prin iples and Pra ti e, Addison-Wesley Sys. Prog. Series, 2nd editionin C,reprinted in 1995.
[5℄ Bhat,DinkarandNayar,Shree, Ordi-nal Measures for Visual Corrsespon-den e, Departmentof Computer S i-en e,ColumbiaU.,N.Y.,pp.584-598,
[6℄ Wu, Haiyuan, Chen, Qian, and Ya hida, Masahiko, Fa e Dete tion From Color Images Using a Fuzzy Pattern Mat hing Method, IEEE Trans. on Pattern Analysis and Ma- hineIntel.,Vol.21,No.6,June1999. [7℄ Terrillon, Jean-Christophe and Aka-matsu, Shigeru, Comparative Per-forman e of Di erent Chrominan e Spa es for Color Segmentation and Dete tion of Human Fa es in Com-plexS eneImages,Vis.Int.99,T rois-Rivieres,Canada,pp.19-21,1999. [8℄ S herer, S., Werth, P., and Pinz, A.,
TheDis riminatoryPowerofOrdinal Measures TowardsaNewCoeÆ ient, ComputerGraphi sandVision,Graz UniversityofTe hnology.
