HANDBOOK OF
COMPUTER VISION AND APPLICATIONS
Volume 2
Signal Processing and Pattern Recognition
ACADEMIC PRESS
Bernd Jähne Horst Haußecker Peter Geißler
2
1 1
Handbook of Computer Vision and Applications
Volume 2
Signal Processing and
Pattern Recognition
Handbook of Computer Vision and Applications
Volume 2
Signal Processing and Pattern Recognition
Editors Bernd Jähne
Interdisciplinary Center for Scientific Computing University of Heidelberg, Heidelberg, Germany
and
Scripps Institution of Oceanography University of California, San Diego
Horst Haußecker Peter Geißler
Interdisciplinary Center for Scientific Computing University of Heidelberg, Heidelberg, Germany
ACADEMIC PRESS
San Diego London Boston New York Sydney Tokyo Toronto
All rights reserved.
No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher.
The appearance of code at the bottom of the first page of a chapter in this book indicates the Publisher’s consent that copies of the chapter may be made for personal or internal use of specific clients. This consent is given on the con- dition, however, that the copier pay the stated per-copy fee through the Copy- right Clearance Center, Inc. (222 Rosewood Drive, Danvers, Massachusetts 01923), for copying beyond that permitted by Sections 107 or 108 of the U.S.
Copyright Law. This consent does not extend to other kinds of copying, such as copying for general distribution, for advertising or promotional purposes, for creating new collective works, or for resale. Copy fees for pre-1999 chap- ters are as shown on the title pages; if no fee code appears on the title page, the copy fee is the same as for current chapters. ISBN 0-12-379770-5/$30.00
ACADEMIC PRESS
A Division of Harcourt Brace & Company
525 B Street, Suite 1900, San Diego, CA 92101-4495 http://www.apnet.com
ACADEMIC PRESS
24-28 Oval Road, London NW1 7DX, UK http://www.hbuk.co.uk/ap/
Library of Congress Cataloging-In-Publication Data
Handbook of computer vision and applications / edited by Bernd Jähne, Horst Haussecker, Peter Geissler.
p. cm.
Includes bibliographical references and indexes.
Contents: v. 1. Sensors and imaging — v. 2. Signal processing and pattern recognition — v. 3. Systems and applications.
ISBN 0–12–379770–5 (set). — ISBN 0–12–379771-3 (v. 1) ISBN 0–12–379772–1 (v. 2). — ISBN 0–12–379773-X (v. 3) 1. Computer vision — Handbooks, manuals. etc. I. Jähne, Bernd 1953– . II. Haussecker, Horst, 1968– . III. Geissler, Peter, 1966– . TA1634.H36 1999
006.307 — dc21 98–42541
CIP
Printed in the United States of America 99 00 01 02 03 DS 9 8 7 6 5 4 3 2 1
Preface xi
Contributors xiii
1 Introduction 1
B. Jähne
1.1 Signal processing for computer vision . . . . 2
1.2 Pattern recognition for computer vision . . . . 3
1.3 Computational complexity and fast algorithms . . . . 4
1.4 Performance evaluation of algorithms . . . . 5
1.5 References . . . . 6
I Signal Representation 2 Continuous and Digital Signals 9 B. Jähne 2.1 Introduction . . . . 10
2.2 Continuous signals. . . . 10
2.3 Discrete signals . . . . 13
2.4 Relation between continuous and discrete signals . . . . 23
2.5 Quantization. . . . 30
2.6 References . . . . 34
3 Spatial and Fourier Domain 35 B. Jähne 3.1 Vector spaces and unitary transforms . . . . 35
3.2 Continuous Fourier transform (FT) . . . . 41
3.3 The discrete Fourier transform (DFT) . . . . 51
3.4 Fast Fourier transform algorithms (FFT) . . . . 57
3.5 References . . . . 66
4 Multiresolutional Signal Representation 67 B. Jähne 4.1 Scale in signal processing . . . . 67
4.2 Scale filters. . . . 70
4.3 Scale space and diffusion. . . . 76
4.4 Multigrid representations . . . . 84
4.5 References . . . . 90
v
II Elementary Spatial Processing
5 Neighborhood Operators 93
B. Jähne
5.1 Introduction . . . . 94
5.2 Basics . . . . 94
5.3 Linear shift-invariant filters . . . . 98
5.4 Recursive filters. . . . 106
5.5 Classes of nonlinear filters. . . . 113
5.6 Efficient neighborhood operations. . . . 116
5.7 References . . . . 124
6 Principles of Filter Design 125 B. Jähne, H. Scharr, and S. Körkel 6.1 Introduction . . . . 125
6.2 Filter design criteria . . . . 126
6.3 Windowing techniques . . . . 128
6.4 Filter cascading . . . . 132
6.5 Filter design as an optimization problem . . . . 133
6.6 Design of steerable filters and filter families . . . . 143
6.7 References . . . . 151
7 Local Averaging 153 B. Jähne 7.1 Introduction . . . . 153
7.2 Basic features . . . . 154
7.3 Box filters. . . . 158
7.4 Binomial filters . . . . 163
7.5 Cascaded averaging . . . . 167
7.6 Weighted averaging . . . . 173
7.7 References . . . . 174
8 Interpolation 175 B. Jähne 8.1 Introduction . . . . 175
8.2 Basics . . . . 176
8.3 Interpolation in Fourier space. . . . 180
8.4 Polynomial interpolation . . . . 182
8.5 Spline-based interpolation . . . . 187
8.6 Optimized interpolation . . . . 190
8.7 References . . . . 192
9 Image Warping 193 B. Jähne 9.1 Introduction . . . . 193
9.2 Forward and inverse mapping . . . . 194
9.3 Basic geometric transforms . . . . 195
9.4 Fast algorithms for geometric transforms . . . . 199
9.5 References . . . . 206
III Feature Estimation
10 Local Structure 209
B. Jähne
10.1 Introduction . . . . 210
10.2 Properties of simple neighborhoods . . . . 210
10.3 Edge detection by first-order derivatives. . . . 213
10.4 Edge detection by zero crossings . . . . 223
10.5 Edges in multichannel images. . . . 226
10.6 First-order tensor representation . . . . 227
10.7 References . . . . 238
11 Principles for Automatic Scale Selection 239 T. Lindeberg 11.1 Introduction . . . . 240
11.2 Multiscale differential image geometry . . . . 240
11.3 A general scale-selection principle. . . . 247
11.4 Feature detection with automatic scale selection . . . . 251
11.5 Feature localization with automatic scale selection . . . . 262
11.6 Stereo matching with automatic scale selection . . . . 265
11.7 Summary and conclusions . . . . 269
11.8 References . . . . 270
12 Texture Analysis 275 T. Wagner 12.1 Importance of texture. . . . 276
12.2 Feature sets for texture analysis . . . . 278
12.3 Assessment of textural features . . . . 299
12.4 Automatic design of texture analysis systems . . . . 306
12.5 References . . . . 307
13 Motion 309 H. Haußecker and H. Spies 13.1 Introduction . . . . 310
13.2 Basics: flow and correspondence. . . . 312
13.3 Optical flow-based motion estimation . . . . 321
13.4 Quadrature filter techniques . . . . 345
13.5 Correlation and matching . . . . 353
13.6 Modeling of flow fields . . . . 356
13.7 Confidence measures and error propagation . . . . 369
13.8 Comparative analysis . . . . 373
13.9 References . . . . 392
14 Bayesian Multiscale Differential Optical Flow 397 E. P. Simoncelli 14.1 Introduction . . . . 397
14.2 Differential formulation . . . . 398
14.3 Uncertainty model . . . . 400
14.4 Coarse-to-fine estimation. . . . 404
14.5 Implementation issues . . . . 410
14.6 Examples . . . . 414
14.7 Conclusion . . . . 419
14.8 References . . . . 420
15 Nonlinear Diffusion Filtering 423 J. Weickert
15.1 Introduction . . . . 424
15.2 Filter design . . . . 425
15.3 Continuous theory . . . . 433
15.4 Algorithmic details. . . . 436
15.5 Discrete theory . . . . 439
15.6 Parameter selection . . . . 441
15.7 Generalizations . . . . 444
15.8 Summary . . . . 446
15.9 References . . . . 446
16 Variational Methods 451 C. Schnörr 16.1 Introduction . . . . 451
16.2 Processing of two- and three-dimensional images. . . . 455
16.3 Processing of vector-valued images . . . . 471
16.4 Processing of image sequences . . . . 476
16.5 References . . . . 481
17 Stereopsis - Geometrical and Global Aspects 485 H. A. Mallot 17.1 Introduction . . . . 485
17.2 Stereo geometry . . . . 487
17.3 Global stereopsis . . . . 499
17.4 References . . . . 502
18 Stereo Terrain Reconstruction by Dynamic Programming 505 G. Gimel’farb 18.1 Introduction . . . . 505
18.2 Statistical decisions in terrain reconstruction . . . . 509
18.3 Probability models of epipolar profiles . . . . 514
18.4 Dynamic programming reconstruction . . . . 520
18.5 Experimental results. . . . 524
18.6 References . . . . 528
19 Reflectance-Based Shape Recovery 531 R. Klette, R. Kozera, and K. Schlüns 19.1 Introduction . . . . 532
19.2 Reflection and gradients . . . . 539
19.3 Three light sources . . . . 552
19.4 Two light sources. . . . 559
19.5 Theoretical framework for shape from shading . . . . 571
19.6 Shape from shading . . . . 574
19.7 Concluding remarks . . . . 586
19.8 References . . . . 587
20 Depth-from-Focus 591 P. Geißler and T. Dierig 20.1 Introduction . . . . 592
20.2 Basic concepts. . . . 593
20.3 Principles of depth-from-focus algorithms . . . . 595
20.4 Multiple-view depth-from-focus . . . . 596
20.5 Dual-view depth-from-focus. . . . 601
20.6 Single-view depth-from-focus . . . . 608
20.7 References . . . . 622
IV Object Analysis, Classification, Modeling, Visualization 21 Morphological Operators 627 P. Soille 21.1 Introduction . . . . 628
21.2 Basics . . . . 629
21.3 Morphological operators . . . . 637
21.4 Efficient computation of morphological operators . . . . 659
21.5 Morphological image processing . . . . 664
21.6 References . . . . 678
22 Fuzzy Image Processing 683 H. Haußecker and H. R. Tizhoosh 22.1 Introduction . . . . 684
22.2 Why fuzzy image processing?. . . . 691
22.3 Fuzzy image understanding . . . . 692
22.4 Fuzzy image processing systems. . . . 699
22.5 Theoretical components of fuzzy image processing . . . . 702
22.6 Selected application examples . . . . 714
22.7 Conclusions . . . . 721
22.8 References . . . . 722
23 Neural Net Computing for Image Processing 729 A. Meyer-Bäse 23.1 Introduction . . . . 729
23.2 Multilayer perceptron (MLP) . . . . 730
23.3 Self-organizing neural networks . . . . 736
23.4 Radial-basis neural networks (RBNN) . . . . 740
23.5 Transformation radial-basis networks (TRBNN) . . . . 743
23.6 Hopfield neural networks . . . . 747
23.7 References . . . . 751
24 Graph Theoretical Concepts for Computer Vision 753 D. Willersinn et al. 24.1 Introduction . . . . 754
24.2 Basic definitions . . . . 754
24.3 Graph representation of two-dimensional digital images . . . 760
24.4 Voronoi diagrams and Delaunay graphs . . . . 762
24.5 Matching . . . . 775
24.6 Graph grammars . . . . 780
24.7 References . . . . 786
25 Shape Reconstruction from Volumetric Data 791 R. Eils and K. Sätzler 25.1 Introduction . . . . 791
25.2 Incremental approach. . . . 794
25.3 Three-dimensional shape reconstruction from contour lines. 797
25.4 Volumetric shape reconstruction . . . . 802
25.5 Summary . . . . 811
25.6 References . . . . 813
26 Probabilistic Modeling in Computer Vision 817 J. Hornegger, D. Paulus, and H. Niemann 26.1 Introduction . . . . 817
26.2 Why probabilistic models? . . . . 819
26.3 Object recognition: classification and regression . . . . 821
26.4 Parametric families of model densities . . . . 826
26.5 Automatic model generation . . . . 844
26.6 Practical issues . . . . 850
26.7 Summary, conclusions, and discussion. . . . 852
26.8 References . . . . 852
27 Knowledge-Based Interpretation of Images 855 H. Niemann 27.1 Introduction . . . . 855
27.2 Model of the task domain . . . . 859
27.3 Interpretation by optimization . . . . 864
27.4 Control by graph search . . . . 865
27.5 Control by combinatorial optimization. . . . 868
27.6 Judgment function. . . . 870
27.7 Extensions and remarks . . . . 872
27.8 References . . . . 872
28 Visualization of Volume Data 875 J. Hesser and C. Poliwoda 28.1 Selected visualization techniques . . . . 876
28.2 Basic concepts and notation for visualization . . . . 880
28.3 Surface rendering algorithms and OpenGL . . . . 881
28.4 Volume rendering . . . . 884
28.5 The graphics library VGL. . . . 890
28.6 How to use volume rendering. . . . 898
28.7 Volume rendering . . . . 901
28.8 Acknowledgments . . . . 905
28.9 References . . . . 905
29 Databases for Microscopes and Microscopical Images 907 N. Salmon, S. Lindek, and E. H. K. Stelzer 29.1 Introduction . . . . 908
29.2 Towards a better system for information management . . . . 909
29.3 From flat files to database systems . . . . 911
29.4 Database structure and content . . . . 912
29.5 Database system requirements . . . . 917
29.6 Data flow—how it looks in practice . . . . 918
29.7 Future prospects . . . . 921
29.8 References . . . . 925
Index 927
What this handbook is about
This handbook offers a fresh approach to computer vision. The whole vision process from image formation to measuring, recognition, or re- acting is regarded as an integral process. Computer vision is under- stood as the host of techniques to acquire, process, analyze, and un- derstand complex higher-dimensional data from our environment for scientific and technical exploration.
In this sense the handbook takes into account the interdisciplinary nature of computer vision with its links to virtually all natural sciences and attempts to bridge two important gaps. The first is between mod- ern physical sciences and the many novel techniques to acquire images.
The second is between basic research and applications. When a reader with a background in one of the fields related to computer vision feels he has learned something from one of the many other facets of com- puter vision, the handbook will have fulfilled its purpose.
The handbook comprises three volumes. The first volume, Sensors and Imaging, covers image formation and acquisition. The second vol- ume, Signal Processing and Pattern Recognition, focuses on processing of the spatial and spatiotemporal signal acquired by imaging sensors.
The third volume, Systems and Applications, describes how computer vision is integrated into systems and applications.
Prerequisites
It is assumed that the reader is familiar with elementary mathematical concepts commonly used in computer vision and in many other areas of natural sciences and technical disciplines. This includes the basics of set theory, matrix algebra, differential and integral equations, com- plex numbers, Fourier transform, probability, random variables, and graphing. Wherever possible, mathematical topics are described intu- itively. In this respect it is very helpful that complex mathematical relations can often be visualized intuitively by images. For a more for-
xi
mal treatment of the corresponding subject including proofs, suitable references are given.
How to use this handbook
The handbook has been designed to cover the different needs of its readership. First, it is suitable for sequential reading. In this way the reader gets an up-to-date account of the state of computer vision. It is presented in a way that makes it accessible for readers with different backgrounds. Second, the reader can look up specific topics of inter- est. The individual chapters are written in a self-consistent way with extensive cross-referencing to other chapters of the handbook and ex- ternal references. The CD that accompanies each volume of the hand- book contains the complete text of the handbook in the Adobe Acrobat portable document file format (PDF). This format can be read on all major platforms. Free Acrobat reader version 3.01 for all major com- puting platforms is included on the CDs. The texts are hyperlinked in multiple ways. Thus the reader can collect the information of interest with ease. Third, the reader can delve more deeply into a subject with the material on the CDs. They contain additional reference material, interactive software components, code examples, image material, and references to sources on the Internet. For more details see the readme file on the CDs.
Acknowledgments
Writing a handbook on computer vision with this breadth of topics is a major undertaking that can succeed only in a coordinated effort that involves many co-workers. Thus the editors would like to thank first all contributors who were willing to participate in this effort. Their cooperation with the constrained time schedule made it possible that the three-volume handbook could be published in such a short period following the call for contributions in December 1997. The editors are deeply grateful for the dedicated and professional work of the staff at AEON Verlag & Studio who did most of the editorial work. We also express our sincere thanks to Academic Press for the opportunity to write this handbook and for all professional advice.
Last but not least, we encourage the reader to send us any hints on errors, omissions, typing errors, or any other shortcomings of the handbook. Actual information about the handbook can be found at the editors homepagehttp://klimt.iwr.uni-heidelberg.de.
Heidelberg, Germany and La Jolla, California, December 1998 Bernd Jähne, Horst Haußecker, Peter Geißler
Etienne Bertin received the PhD degree in mathematics from Université Joseph Fourier in 1994. From 1990 to 1995 he worked on various topics in image analysis and computational geometry. Since 1995, he has been an as- sistant professor at the Université Pierre Mendès France in the Laboratoire de statistique et d’analyses de don- nées; he works on stochastic geometry.
Dr. Etienne Bertin
Laboratoire de Statistique et d’analyse de donnés Université Pierre Mendès, Grenoble, France bertin@labsad.upmf-grenoble.fr
Anke Meyer-Bäse received her M. S. and the PhD in elec- trical engineering from the Darmstadt Institute of Tech- nology in 1990 and 1995, respectively. From 1995 to 1996 she was a postdoctoral fellow with the Federal Insti- tute of Neurobiology, Magdeburg, Germany. Since 1996 she was a visiting assistant professor with the Dept. of Electrical Engineering, University of Florida, Gainesville, USA. She received the Max-Kade award in Neuroengineer- ing in 1996 and the Lise-Meitner prize in 1997. Her re- search interests include neural networks, image process- ing, biomedicine, speech recognition, and theory of non- linear systems.
Dr. Anke Meyer-Bäse, Dept. of Electrical Engineering and Computer Science, University of Florida, 454 New Engineering Building 33, Center Drive
PO Box 116130, Gainesville, FL 32611-6130, U.S.,anke@alpha.ee.ufl.edu Tobias Dierig graduated in 1997 from the University of Heidelberg with a master degree in physics and is now pursuing his PhD at the Interdisciplinary Center for Sci- entific Computing at Heidelberg university. He is con- cerned mainly with depth from focus algorithms, image fusion, and industrial applications of computer vision within the OpenEye project.
Tobias Dierig, Forschungsgruppe Bildverarbeitung, IWR Universität Heidelberg, Im Neuenheimer Feld 368 D-69120 Heidelberg, Germany
Tobias.Dierig@iwr.uni-heidelberg.de http://klimt.iwr.uni-heidelberg.de/˜tdierig
xiii
Roland Eils studied mathematics and computer science in Aachen, where he received his diploma in 1990. After a two year stay in Indonesia for language studies he joint the Graduiertenkolleg “Modeling and Scientific Comput- ing in Mathematics and Sciences” at the Interdisciplinary Center for Scientific Computing (IWR), University of Hei- delberg, where he received his doctoral degree in 1995.
Since 1996 he has been leading the biocomputing group, Structures in Molecular Biology. His research interests include computer vision, in particular computational ge- ometry, and application of image processing techniques in science and biotechnology.
Dr. Roland Eils, Biocomputing-Gruppe, IWR, Universität Heidelberg Im Neuenheimer Feld 368, D-69120 Heidelberg, Germany
eils@iwr.uni-heidelberg.de
http://www.iwr.uni-heidelberg.de/iwr/bioinf
Peter Geißler studied physics in Heidelberg. He received his diploma and doctoral degree from Heidelberg Uni- versity in 1994 and 1998, respectively. His research in- terests include computer vision, especially depth-from- focus, adaptive filtering, and flow visualization as well as the application of image processing in physical sciences and oceanography.
Dr. Peter Geißler
Forschungsgruppe Bildverarbeitung, IWR
Universität Heidelberg, Im Neuenheimer Feld 368 D-69120 Heidelberg, Germany
Peter.Geissler@iwr.uni-heidelberg.de http://klimt.iwr.uni-heidelberg.de
Georgy Gimel’farb received his PhD degree from the Ukrainian Academy of Sciences in 1969 and his Doctor of Science (the habilitation) degree from the Higher Certify- ing Commission of the USSR in 1991. In 1962, he began working in the Pattern Recognition, Robotics, and Image Recognition Departments of the Institute of Cybernetics (Ukraine). In 1994–1997 he was an invited researcher in Hungary, the USA, Germany, and France. Since 1997, he has been a senior lecturer in computer vision and digital TV at the University of Auckland, New Zealand. His re- search interests include analysis of multiband space and aerial images, computational stereo, and image texture analysis.
Dr. Georgy Gimel’farb, Centre for Image Technology and Robotics, Department of Computer Science, Tamaki Campus
The University of Auckland, Private Bag 92019, Auckland 1, New Zealand g.gimelfarb@auckland.ac.nz,http://www.tcs.auckland.ac.nz/˜georgy
Horst Haußecker studied physics in Heidelberg. He re- ceived his diploma in physics and his doctoral degree from Heidelberg University in 1994 and 1996, respec- tively. He was visiting scientist at the Scripps Institution of Oceanography in 1994. Currently he is conducting research in the image processing research group at the Interdisciplinary Center for Scientific Computing (IWR), where he also lectures on optical flow computation. His research interests include computer vision, especially image sequence analysis, infrared thermography, and fuzzy-image processing, as well as the application of im- age processing in physical sciences and oceanography.
Dr. Horst Haußecker, Forschungsgruppe Bildverarbeitung, IWR Universität Heidelberg, Im Neuenheimer Feld 368, D-69120 Heidelberg Horst.Haussecker@iwr.uni-heidelberg.de
http://klimt.iwr.uni-heidelberg.de
Jürgen Hesser is assistant professor at the Lehrstuhl für Informatik V, University of Mannheim, Germany. He heads the groups on computer graphics, bioinformat- ics, and optimization. His research interests are real- time volume rendering, computer architectures, compu- tational chemistry, and evolutionary algorithms. In addi- tion, he is co-founder of Volume Graphics GmbH, Heidel- berg. Hesser received his PhD and his diploma in physics at the University of Heidelberg, Germany.
Jürgen Hesser, Lehrstuhl für Informatik V Universität Mannheim
B6, 26, D-68131 Mannheim, Germany jhesser@rumms.uni-mannheim.de,
Joachim Hornegger graduated in 1992 and received his PhD degree in computer science in 1996 from the Uni- versität Erlangen-Nürnberg, Germany, for his work on statistical object recognition. Joachim Hornegger was research and teaching associate at Universität Erlangen- Nürnberg, a visiting scientist at the Technion, Israel, and at the Massachusetts Institute of Technology, U.S. He is currently a research scholar and teaching associate at Stanford University, U.S. Joachim Hornegger is the author of 30 technical papers in computer vision and speech processing and three books. His research inter- ests include 3-D computer vision, 3-D object recognition, and statistical meth- ods applied to image analysis problems.
Dr. Joachim Hornegger, Stanford University, Robotics Laboratory Gates Building 1A, Stanford, CA 94305-9010, U.S.
jh@robotics.stanford.edu,http://www.robotics.stanford.edu/˜jh
Bernd Jähne studied physics in Saarbrücken and Hei- delberg. He received his diploma, doctoral degree, and habilitation degree from Heidelberg University in 1977, 1980, and 1985, respectively, and a habilitation de- gree in applied computer science from the University of Hamburg-Harburg in 1992. Since 1988 he has been a Ma- rine Research Physicist at Scripps Institution of Oceanog- raphy, University of California, and, since 1994, he has been professor of physics at the Interdisciplinary Center of Scientific Computing. He leads the research group on image processing. His research interests include com- puter vision, especially filter design and image sequence analysis, the application of image processing techniques in science and industry, and small-scale air-sea interaction processes.
Prof. Dr. Bernd Jähne, Forschungsgruppe Bildverarbeitung, IWR Universität Heidelberg, Im Neuenheimer Feld 368, D-69120 Heidelberg Bernd.Jaehne@iwr.uni-heidelberg.de
http://klimt.iwr.uni-heidelberg.de
Reinhard Klette studied mathematics at Halle University, received his master degree and doctor of natural science degree in mathematics at Jena University, became a do- cent in computer science, and was a professor of com- puter vision at Berlin Technical University. Since June 1996 he has been professor of information technology in the Department of Computer Science at the University of Auckland. His research interests include theoretical and applied topics in image processing, pattern recogni- tion, image analysis, and image understanding. He has published books about image processing and shape reconstruction and was chairman of several international conferences and workshops on computer vision. Recently, his research interests have been directed at 3-D biomedical image analysis with digital geometry and computational geometry as major subjects.
Prof. Dr. Reinhard Klette, Centre for Image Technology and Robotics, Computer Science Department, Tamaki Campus
The Auckland University, Private Bag 92019, Auckland, New Zealand r.klette@auckland.ac.nz,http://citr.auckland.ac.nz/˜rklette
Christoph Klauck received his diploma in computer sci- ence and mathematics from the University of Kaiser- slautern, Germany, in 1990. From 1990 to 1994 he worked as research scientist at the German Research Center for Artificial Intelligence Inc. (DFKI GmbH) at Kaiserslautern. In 1994 he finished his dissertation in computer science. Since then he has been involved in the IRIS project at the University of Bremen (Artificial Intelligence Group). His primary research interests in- clude graph grammars and rewriting systems in general, knowledge representation, and ontologies.
Prof. Dr. Christoph Klauck, Dep. of Electrical Eng. and Computer Science University of Hamburg (FH), Berliner Tor 3, D-20099 Hamburg, Germany cklauck@t-online.de,http://fbi010.informatik.fh-hamburg.de/˜klauck
Stefan Körkel is member of the research groups for nu- merics and optimization of Prof. Bock and Prof. Reinelt at the Interdisciplinary Center for Scientific Computing at the University of Heidelberg, Germany. He studied mathematics in Heidelberg. Currently he is pursuing his PhD in nonlinear and mixed integer optimization meth- ods. His research interests include filter optimization as well as nonlinear optimum experimental design.
Stefan Körkel
Interdisciplinary Center for Scientific Computing Im Neuenheimer Feld 368, 69120 Heidelberg Stefan.Koerkel@IWR.Uni-Heidelberg.de http://www.iwr.uni-heidelberg.de/˜Stefan.Koerkel/
Ryszard Kozera received his M.Sc. degree in pure mathe- matics in 1985 from Warsaw University, Poland, his PhD degree in computer science in 1991 from Flinders Uni- versity, Australia, and finally his PhD degree in mathe- matics in 1992 from Warsaw University, Poland. He is currently employed as a senior lecturer at the University of Western Australia. Between July 1995 and February 1997, Dr. Kozera was at the Technical University of Berlin and at Warsaw University as an Alexander von Humboldt Foundation research fellow. His current research inter- ests include applied mathematics with special emphasis on partial differential equations, computer vision, and numerical analysis.
Dr. Ryszard Kozera, Department of Computer Science, The University of West- ern Australia, Nedlands, WA 6907, Australia,ryszard@cs.uwa.edu.au
http://www.cs.uwa.edu.au/people/info/ryszard.html
Tony Lindeberg received his M.Sc. degree in engineer- ing physics and applied mathematics from KTH (Royal Institute of Technology), Stockholm, Sweden in 1987, and his PhD degree in computing science in 1991. He is currently an associate professor at the Department of Numerical Analysis and Computing Science at KTH.
His main research interests are in computer vision and relate to multiscale representations, focus-of-attention, and shape. He has contributed to the foundations of continuous and discrete scale-space theory, as well as to the application of these theories to computer vision problems. Specifically, he has developed principles for automatic scale selection, methodologies for extracting salient image struc- tures, and theories for multiscale shape estimation. He is author of the book
“Scale-Space Theory in Computer Vision.”
Tony Lindeberg, Department of Numerical Analysis and Computing Science KTH, S-100 44 Stockholm, Sweden.
tony@nada.kth.se,http://www.nada.kth.se/˜tony
Steffen Lindek studied physics at the RWTH Aachen, Ger- many, the EPF Lausanne, Switzerland, and the Univer- sity of Heidelberg, Germany. He did his diploma and PhD theses in the Light Microscopy Group at the Euro- pean Molecular Biology Laboratory (EMBL), Heidelberg, Germany, developing high-resolution light-microscopy techniques. Since December 1996 he has been a post- doctoral fellow with the BioImage project at EMBL. He currently works on the design and implementation of the image database, and he is responsible for the administra- tion of EMBL’s contribution to the project.
Dr. Steffen Lindek, European Molecular Biology Laboratory (EMBL) Postfach 10 22 09, D-69120 Heidelberg, Germany
lindek@EMBL-Heidelberg.de
Hanspeter A. Mallot studied biology and mathematics at the University of Mainz where he also received his doc- toral degree in 1986. He was a postdoctoral fellow at the Massachusetts Institute of Technology in 1986/87 and held research positions at Mainz University and the Ruhr-Universität-Bochum. In 1993, he joined the Max- Planck-Institut für biologische Kybernetik in Tübingen.
In 1996/97, he was a fellow at the Institute of Advanced Studies in Berlin. His research interests include the per- ception of shape and space in humans and machines, cognitive maps, as well as neural network models of the cerebral cortex.
Dr. Hanspeter A. Mallot, Max-Planck-Institut für biologische Kybernetik Spemannstr. 38, 72076 Tübingen, Germany
Hanspeter.Mallot@tuebingen.mpg.de http://www.kyb.tuebingen.mpg.de/bu/
Heinrich Niemann obtained the degree of Dipl.-Ing. in electrical engineering and Dr.-Ing. at Technical Univer- sity Hannover in 1966 and 1969, respectively. From 1967 to 1972 he was with Fraunhofer Institut für In- formationsverarbeitung in Technik und Biologie, Karls- ruhe. Since 1975 he has been professor of computer sci- ence at the University of Erlangen-Nürnberg and since 1988 he has also served as head of the research group, Knowledge Processing, at the Bavarian Research Institute for Knowledge-Based Systems (FORWISS). His fields of research are speech and image understanding and the application of artificial intelligence techniques in these fields. He is the author or co-author of 6 books and approximately 250 jour- nal and conference contributions.
Prof. Dr.-Ing. H. Niemann, Lehrstuhl für Mustererkennung (Informatik 5) Universität Erlangen-Nürnberg, Martensstraße 3, 91058 Erlangen, Germany niemann@informatik.uni-erlangen.de
http://www5.informatik.uni-erlangen.de
Dietrich Paulus received a bachelor degree in computer science at the University of Western Ontario, London, Canada (1983). He graduated (1987) and received his PhD degree (1991) from the University of Erlangen- Nürnberg, Germany. He is currently a senior researcher (Akademischer Rat) in the field of image pattern recog- nition and teaches courses in computer vision and ap- plied programming for image processing. Together with J. Hornegger, he has recently written a book on pattern recognition and image processing in C++.
Dr. Dietrich Paulus, Lehrstuhl für Mustererkennung
Universität Erlangen-Nürnberg, Martensstr. 3, 91058 Erlangen, Germany paulus@informatik.uni-erlangen.de
http://www5.informatik.uni-erlangen.de
Christoph Poliwoda is PhD student at the Lehrstuhl für Informatik V, University of Mannheim, and leader of the development section of Volume Graphics GmbH. His re- search interests are real-time volume and polygon ray- tracing, 3-D image processing, 3-D segmentation, com- puter architectures and parallel computing. Poliwoda received his diploma in physics at the University of Hei- delberg, Germany.
Christoph Poliwoda Lehrstuhl für Informatik V Universität Mannheim
B6, 26, D-68131 Mannheim, Germany
poliwoda@mp-sun1.informatik.uni-mannheim.de Nicholas J. Salmon received the master of engineering degree from the Department of Electrical and Electronic Engineering at Bath University, England, in 1990. Then he worked as a software development engineer for Mar- coni Radar Systems Ltd., England, helping to create a vastly parallel signal-processing machine for radar appli- cations. Since 1992 he has worked as software engineer in the Light Microscopy Group at the European Molecu- lar Biology Laboratory, Germany, where he is concerned with creating innovative software systems for the con- trol of confocal microscopes, and image processing.
Nicholas J. Salmon, Light Microscopy Group, European Molecular Biology Laboratory (EMBL) Postfach 10 22 09, D-69120 Heidelberg, Germany salmon@EMBL-Heidelberg.de,
Kurt Sätzler studied physics at the University of Hei- delberg, where he received his diploma in 1995. Since then he has been working as a PhD student at the Max- Planck-Institute of Medical Research in Heidelberg. His research interests are mainly computational geometry applied to problems in biomedicine, architecture and computer graphics, image processing and tilted view mi- croscopy.
Kurt Sätzler, IWR, Universität Heidelberg Im Neuenheimer Feld 368, D-69120 Heidelberg Max-Planck-Institute for Medical Research, Department of Cell Physiologyor Jahnstr. 29, D-69120 Heidelberg, Germany
Kurt.Saetzler@iwr.uni-heidelberg.de
Hanno Scharr studied physics at the University of Hei- delberg, Germany and did his diploma thesis on tex- ture analysis at the Interdisciplinary Center for Scien- tific Computing in Heidelberg. Currently, he is pursu- ing his PhD on motion estimation. His research interests include filter optimization and motion estimation in dis- crete time series ofn-D images.
Hanno Scharr
Interdisciplinary Center for Scientific Computing Im Neuenheimer Feld 368, 69120 Heidelberg, Germany Hanno.Scharr@iwr.uni-heidelberg.de
http://klimt.iwr.uni-heidelberg.de/˜hscharr/
Karsten Schlüns studied computer science in Berlin. He received his diploma and doctoral degree from the Tech- nical University of Berlin in 1991 and 1996. From 1991 to 1996 he was research assistant in the Computer Vision Group, Technical University of Berlin, and from 1997 to 1998 he was a postdoctoral research fellow in com- puting and information technology, University of Auck- land. Since 1998 he has been a scientist in the image processing group at the Institute of Pathology, Univer- sity Hospital Charité in Berlin. His research interests include pattern recognition and computer vision, espe- cially three-dimensional shape recovery, performance analysis of reconstruc- tion algorithms, and teaching of computer vision.
Dr. Karsten Schlüns, Institute of Pathology,
University Hospital Charité, Schumannstr. 20/21, D-10098 Berlin, Germany Karsten.Schluens@charite.de,http://amba.charite.de/˜ksch
Christoph Schnörr received the master degree in electri- cal engineering in 1987, the doctoral degree in computer science in 1991, both from the University of Karlsruhe (TH), and the habilitation degree in Computer Science in 1998 from the University of Hamburg, Germany. From 1987–1992, he worked at the Fraunhofer Institute for In- formation and Data Processing (IITB) in Karlsruhe in the field of image sequence analysis. In 1992 he joined the Cognitive Systems group, Department of Computer Sci- ence, University of Hamburg, where he became an assis- tant professor in 1995. He received an award for his work on image segmentation from the German Association for Pattern Recognition (DAGM) in 1996. Since October 1998, he has been a full professor at the University of Mannheim, Germany, where he heads the Com- puter Vision, Graphics, and Pattern Recognition Group. His research interests include pattern recognition, machine vision, and related aspects of computer graphics, machine learning, and applied mathematics.
Prof. Dr. Christoph Schnörr, University of Mannheim
Dept. of Math. & Computer Science, D-68131 Mannheim, Germany schnoerr@ti.uni-mannheim.de,http://www.ti.uni-mannheim.de
Eero Simoncelli started his higher education with a bach- elor’s degree in physics from Harvard University, went to Cambridge University on a fellowship to study mathe- matics for a year and a half, and then returned to the USA to pursue a doctorate in Electrical Engineering and Com- puter Science at MIT. He received his PhD in 1993, and joined the faculty of the Computer and Information Sci- ence Department at the University of Pennsylvania that same year. In September of 1996, he joined the faculty of the Center for Neural Science and the Courant Insti- tute of Mathematical Sciences at New York University. He received an NSF Faculty Early Career Development (CA- REER) grant in September 1996, for teaching and research in “Visual Informa- tion Processing”, and a Sloan Research Fellowship in February 1998.
Dr. Eero Simoncelli, 4 Washington Place, RM 809, New York, NY 10003-6603 eero.simoncelli@nyu.edu,http://www.cns.nyu.edu/˜eero
Pierre Soille received the engineering degree from the Université catholique de Louvain, Belgium, in 1988. He gained the doctorate degree in 1992 at the same univer- sity and in collaboration with the Centre de Morphologie Mathématique of the Ecole des Mines de Paris. He then pursued research on image analysis at the CSIRO Math- ematical and Information Sciences Division, Sydney, the Centre de Morphologie Mathématique of the Ecole des Mines de Paris, and the Abteilung Mustererkennung of the Fraunhofer-Institut IPK, Berlin. During the period 1995-1998 he was lecturer and research scientist at the Ecole des Mines d’Alès and EERIE, Nîmes, France. Now he is a senior research scientist at the Silsoe Research Institute, England. He worked on many ap-
plied projects, taught tutorials during international conferences, co-organized the second International Symposium on Mathematical Morphology, wrote and edited three books, and contributed to over 50 scientific publications.
Prof. Pierre Soille, Silsoe Research Institute, Wrest Park Silsoe, Bedfordshire, MK45 4HS, United Kingdom Pierre.Soille@bbsrc.ac.uk,http://www.bbsrc.ac.uk
Hagen Spies graduated in January 1998 from the Univer- sity of Heidelberg with a master degree in physics. He also received an MS in computing and information tech- nology from the University of Dundee, Scotland in 1995.
In 1998/1999 he spent one year as a visiting scientist at the University of Western Ontario, Canada. Currently he works as a researcher at the Interdisciplinary Center for Scientific Computing at the University of Heidelberg. His interests concern the measurement of optical and range flow and their use in scientific applications.
Hagen Spies, Forschungsgruppe Bildverarbeitung, IWR Universität Heidelberg, Im Neuenheimer Feld 368
D-69120 Heidelberg, Germany,Hagen.Spies@iwr.uni-heidelberg.de http://klimt.iwr.uni-heidelberg.de/˜hspies
E. H. K. Stelzer studied physics in Frankfurt am Main and in Heidelberg, Germany. During his Diploma thesis at the Max-Planck-Institut für Biophysik he worked on the physical chemistry of phospholipid vesicles, which he characterized by photon correlation spectroscopy. Since 1983 he has worked at the European Molecular Biol- ogy Laboratory (EMBL). He has contributed extensively to the development of confocal fluorescence microscopy and its application in life sciences. His group works on the development and application of high-resolution techniques in light microscopy, video microscopy, con- focal microscopy, optical tweezers, single particle analy- sis, and the documentation of relevant parameters with biological data.
Prof. Dr. E. H. K. Stelzer, Light Microscopy Group,
European Molecular Biology Laboratory (EMBL), Postfach 10 22 09 D-69120 Heidelberg, Germany,stelzer@EMBL-Heidelberg.de,
Hamid R. Tizhoosh received the M.S. degree in electrical engineering from University of Technology, Aachen, Ger- many, in 1995. From 1993 to 1996, he worked at Man- agement of Intelligent Technologies Ltd. (MIT GmbH), Aachen, Germany, in the area of industrial image pro- cessing. He is currently a PhD candidate, Dept. of Tech- nical Computer Science of Otto-von-Guericke-University, Magdeburg, Germany. His research encompasses fuzzy logic and computer vision. His recent research efforts include medical and fuzzy image processing. He is cur- rently involved in the European Union project INFOCUS, and is researching enhancement of medical images in radiation therapy.
H. R. Tizhoosh, University of Magdeburg (IPE)
P.O. Box 4120, D-39016 Magdeburg, Germany tizhoosh@ipe.et.uni-magdeburg.de
http://pmt05.et.uni-magdeburg.de/˜hamid/
Thomas Wagner received a diploma degree in physics in 1991 from the University of Erlangen, Germany. In 1995, he finished his PhD in computer science with an applied image processing topic at the Fraunhofer Institute for In- tegrated Circuits in Erlangen. Since 1992, Dr. Wagner has been working on industrial image processing problems at the Fraunhofer Institute, from 1994 to 1997 as group manager of the intelligent systems group. Projects in his research team belong to the fields of object recogni- tion, surface inspection, and access control. In 1996, he received the “Hans-Zehetmair-Habilitationsförderpreis.”
He is now working on automatic solutions for the design of industrial image processing systems.
Dr.-Ing. Thomas Wagner, Fraunhofer Institut für Intregrierte Schaltungen Am Weichselgarten 3, D-91058 Erlangen, Germany
wag@iis.fhg.de,http://www.iis.fhg.de
Joachim Weickert obtained a M.Sc. in industrial math- ematics in 1991 and a PhD in mathematics in 1996, both from Kaiserslautern University, Germany. After re- ceiving the PhD degree, he worked as post-doctoral re- searcher at the Image Sciences Institute of Utrecht Uni- versity, The Netherlands. In April 1997 he joined the computer vision group of the Department of Computer Science at Copenhagen University. His current research interests include all aspects of partial differential equa- tions and scale-space theory in image analysis. He was awarded the Wacker Memorial Prize and authored the book “Anisotropic Diffusion in Image Processing.”
Dr. Joachim Weickert, Department of Computer Science, University of Copen- hagen, Universitetsparken 1, DK-2100 Copenhagen, Denmark
joachim@diku.dk,http://www.diku.dk/users/joachim/
Dieter Willersinn received his diploma in electrical en- gineering from Technical University Darmstadt in 1988.
From 1988 to 1992 he was with Vitronic Image Process- ing Systems in Wiesbaden, working on industrial appli- cations of robot vision and quality control. He then took a research position at the Technical University in Vienna, Austria, from which he received his PhD degree in 1995.
In 1995, he joined the Fraunhofer Institute for Informa- tion and Data Processing (IITB) in Karlsruhe, where he initially worked on obstacle detection for driver assis- tance applications. Since 1997, Dr. Willersinn has been the head of the group, Assessment of Computer Vision Systems, Department for Recognition and Diagnosis Systems.
Dr. Dieter Willersinn, Fraunhofer Institut IITB, Fraunhoferstr. 1 D-76131 Karlsruhe, Germany,wil@iitb.fhg.de
Bernd Jähne
Interdisziplinäres Zentrum für Wissenschaftliches Rechnen (IWR) Universität Heidelberg, Germany
1.1 Signal processing for computer vision . . . . 2 1.2 Pattern recognition for computer vision . . . . 3 1.3 Computational complexity and fast algorithms . . . . 4 1.4 Performance evaluation of algorithms . . . . 5 1.5 References . . . . 6 The second volume of the Handbook on Computer Vision and Ap- plications deals with signal processing and pattern recognition. The signals processed in computer vision originate from the radiance of an object that is collected by an optical system (Volume 1, Chapter5). The irradiance received by a single photosensor or a 2-D array of photosen- sors through the optical system is converted into an electrical signal and finally into arrays of digital numbers (Volume 2, Chapter2). The whole chain of image formation from the illumination and interaction of radiation with the object of interest up to the arrays of digital num- bers stored in the computer is the topic of Volume 1 of this handbook (subtitled Sensors and Imaging).
This volume deals with the processing of the signals generated by imaging sensors and this introduction covers four general topics. Sec- tion1.1discusses in which aspects the processing of higher-dimension- al signals differs from the processing of 1-D time series. We also elab- orate on the task of signal processing for computer vision. Pattern recognition (Section1.2) plays a central role in computer vision because it uses the features extracted by lowlevel signal processing to classify and recognize objects.
Given the vast amount of data generated by imaging sensors the question of the computational complexity and of efficient algorithms is of utmost importance (Section1.3). Finally, the performance evaluation of computer vision algorithms (Section1.4) is a subject that has been neglected in the past. Consequently, a vast number of algorithms exist for which the performance characteristics are not sufficiently known.
Handbook of Computer Vision and Applications 1 Copyright © 1999 by Academic Press
Volume 2 All rights of reproduction in any form reserved.
Signal Processing and Pattern Recognition ISBN 0–12–379772-1/$30.00
This constitutes a major obstacle for progress of applications using computer vision techniques.
1.1 Signal processing for computer vision
One-dimensional linear signal processing and system theory is a stan- dard topic in electrical engineering and is covered by many standard textbooks, for example, [1,2]. There is a clear trend that the classical signal processing community is moving into multidimensional signals, as indicated, for example, by the new annual international IEEE confer- ence on image processing (ICIP). This can also be seen from some re- cently published handbooks on this subject. The digital signal process- ing handbook by Madisetti and Williams [3] includes several chapters that deal with image processing. Likewise the transforms and applica- tions handbook by Poularikas [4] is not restricted to one-dimensional transforms.
There are, however, only a few monographs that treat signal pro- cessing specifically for computer vision and image processing. The monograph of Lim [5] deals with 2-D signal and image processing and tries to transfer the classical techniques for the analysis of time series to 2-D spatial data. Granlund and Knutsson [6] were the first to publish a monograph on signal processing for computer vision and elaborate on a number of novel ideas such as tensorial image processing and nor- malized convolution that did not have their origin in classical signal processing.
Time series are 1-D, signals in computer vision are of higher di- mension. They are not restricted to digital images, that is, 2-D spatial signals (Chapter2). Volumetric sampling, image sequences and hyper- spectral imaging all result in 3-D signals, a combination of any of these techniques in even higher-dimensional signals.
How much more complex does signal processing become with in- creasing dimension? First, there is the explosion in the number of data points. Already a medium resolution volumetric image with 5123vox- els requires 128 MB if one voxel carries just one byte. Storage of even higher-dimensional data at comparable resolution is thus beyond the capabilities of today’s computers. Moreover, many applications require the handling of a huge number of images. This is also why appropriate databases including images are of importance. An example is discussed in Chapter29.
Higher dimensional signals pose another problem. While we do not have difficulty in grasping 2-D data, it is already significantly more de- manding to visualize 3-D data because the human visual system is built only to see surfaces in 3-D but not volumetric 3-D data. The more di- mensions are processed, the more important it is that computer graph-
ics and computer vision come closer together. This is why this volume includes a contribution on visualization of volume data (Chapter28).
The elementary framework for lowlevel signal processing for com- puter vision is worked out in part II of this volume. Of central impor- tance are neighborhood operations (Chapter5). Chapter6focuses on the design of filters optimized for a certain purpose. Other subjects of elementary spatial processing include fast algorithms for local averag- ing (Chapter7), accurate and fast interpolation (Chapter8), and image warping (Chapter9) for subpixel-accurate signal processing.
The basic goal of signal processing in computer vision is the extrac- tion of “suitable features” for subsequent processing to recognize and classify objects. But what is a suitable feature? This is still less well de- fined than in other applications of signal processing. Certainly a math- ematically well-defined description of local structure as discussed in Chapter10is an important basis. The selection of the proper scale for image processing has recently come into the focus of attention (Chap- ter 11). As signals processed in computer vision come from dynam- ical 3-D scenes, important features also include motion (Chapters13 and 14) and various techniques to infer the depth in scenes includ- ing stereo (Chapters17and18), shape from shading and photometric stereo (Chapter19), and depth from focus (Chapter20).
There is little doubt that nonlinear techniques are crucial for fea- ture extraction in computer vision. However, compared to linear filter techniques, these techniques are still in their infancy. There is also no single nonlinear technique but there are a host of such techniques often specifically adapted to a certain purpose [7]. In this volume, a rather general class of nonlinear filters by combination of linear convolution and nonlinear point operations (Chapter10), and nonlinear diffusion filtering (Chapter15) are discussed.
1.2 Pattern recognition for computer vision
In principle, pattern classification is nothing complex. Take some ap- propriate features and partition the feature space into classes. Why is it then so difficult for a computer vision system to recognize objects?
The basic trouble is related to the fact that the dimensionality of the in- put space is so large. In principle, it would be possible to use the image itself as the input for a classification task, but no real-world classifi- cation technique—be it statistical, neuronal, or fuzzy—would be able to handle such high-dimensional feature spaces. Therefore, the need arises to extract features and to use them for classification.
Unfortunately, techniques for feature selection have widely been ne- glected in computer vision. They have not been developed to the same degree of sophistication as classification where it is meanwhile well un-
