Contents
PART I Signals and Systems
1
Fourier Series, Fourier Transforms, and the DFT W. Kenneth Jenkins2
Ordinary Linear Differential and Difference Equations B.P. Lathi3
Finite Wordlength Effects Bruce W. BomarPART II Signal Representation and Quantization 4
On Multidimensional Sampling Ton Kalker5
Analog-to-Digital Conversion Architectures Stephen Kosonocky and Peter Xiao6
Quantization of Discrete Time Signals Ravi P. RamachandranPART III Fast Algorithms and Structures
7
Fast Fourier Transforms: A Tutorial Review and a State of the Art P. Duhamel and M.Vetterli
8
Fast Convolution and Filtering Ivan W. Selesnick and C. Sidney Burrus9
Complexity Theory of Transforms in Signal Processing Ephraim Feig10
Fast Matrix Computations Andrew E. Yagle11
Digital Filtering Lina J. Karam, James H. McClellan, Ivan W. Selesnick, and C. Sidney BurrusPART V Statistical Signal Processing 12
Overview of Statistical Signal Processing Charles W. Therrien13
Signal Detection and Classification Alfred Hero14
Spectrum Estimation and Modeling Petar M. Djuri´c and Steven M. Kay15
Estimation Theory and Algorithms: From Gauss to Wiener to Kalman Jerry M. Mendel16
Validation, Testing, and Noise Modeling Jitendra K. Tugnait17
Cyclostationary Signal Analysis Georgios B. GiannakisPART VI Adaptive Filtering 18
Introduction to Adaptive Filters Scott C. Douglas19
Convergence Issues in the LMS Adaptive Filter Scott C. Douglas and Markus Rupp20
Robustness Issues in Adaptive Filtering Ali H. Sayed and Markus Rupp21
Recursive Least-Squares Adaptive Filters Ali H. Sayed and Thomas Kailath22
Transform Domain Adaptive Filtering W. Kenneth Jenkins and Daniel F. Marshall23
Adaptive IIR Filters Geoffrey A. Williamson24
Adaptive Filters for Blind Equalization Zhi DingPART VII Inverse Problems and Signal Reconstruction 25
Signal Recovery from Partial Information Christine Podilchuk26
Algorithms for Computed Tomography Gabor T. Herman27
Robust Speech Processing as an Inverse Problem Richard J. Mammone and Xiaoyu Zhang28
Inverse Problems, Statistical Mechanics and Simulated Annealing K. Venkatesh Prasad29
Image Recovery Using the EM Algorithm Jun Zhang and Aggelos K. Katsaggelos30
Inverse Problems in Array Processing Kevin R. Farrell31
Channel Equalization as a Regularized Inverse Problem John F. Doherty32
Inverse Problems in Microphone Arrays A.C. Surendran33
Synthetic Aperture Radar Algorithms Clay Stewart and Vic Larson34
Iterative Image Restoration Algorithms Aggelos K. KatsaggelosPART VIII Time Frequency and Multirate Signal Processing
35
Wavelets and Filter Banks Cormac Herley36
Filter Bank Design Joseph Arrowood, Tami Randolph, and Mark J.T. Smith37
Time-Varying Analysis-Synthesis Filter Banks Iraj Sodagar38
Lapped Transforms Ricardo L. de QueirozPART IX Digital Audio Communications 39
Auditory Psychophysics for Coding Applications Joseph L. Hall40
MPEG Digital Audio Coding Standards Peter Noll41
Digital Audio Coding: Dolby AC-3 Grant A. Davidson42
The Perceptual Audio Coder (PAC) Deepen Sinha, James D. Johnston, Sean Dorward, and Schuyler R. Quackenbush43
Sony Systems Kenzo Akagiri, M.Katakura, H. Yamauchi, E. Saito, M. Kohut, Masayuki Nishiguchi, and K. TsutsuiPART X Speech Processing
44
Speech Production Models and Their Digital Implementations M. Mohan Sondhi and Juergen Schroeter45
Speech Coding Richard V. Cox46
Text-to-Speech Synthesis Richard Sproat and Joseph Olive47
Speech Recognition by Machine Lawrence R. Rabiner and B. H. Juang48
Speaker Verification Sadaoki Furui and Aaron E. Rosenberg49
DSP Implementations of Speech Processing Kurt Baudendistel50
Software Tools for Speech Research and Development John ShorePART XI Image and Video Processing
51
Image Processing Fundamentals Ian T. Young, Jan J. Gerbrands, and Lucas J. van Vliet52
Still Image Compression Tor A. Ramstad53
Image and Video Restoration A. Murat Tekalp54
Video Scanning Format Conversion and Motion Estimation Gerard de Haan55
Video Sequence Compression Osama Al-Shaykh, Ralph Neff, David Taubman, and Avideh Zakhor56
Digital Television Kou-Hu Tzou57
Stereoscopic Image Processing Reginald L. Lagendijk, Ruggero E.H. Franich, and Emile A.Hendriks
58
A Survey of Image Processing Software and Image Databases Stanley J. Reeves59
VLSI Architectures for Image Communications P. Pirsch and W. GehrkePART XII Sensor Array Processing
60
Complex Random Variables and Stochastic Processes Daniel R. Fuhrmann61
Beamforming Techniques for Spatial Filtering Barry Van Veen and Kevin M. Buckley62
Subspace-Based Direction Finding Methods Egemen Gonen and Jerry M. Mendel63
ESPRIT and Closed-Form 2-D Angle Estimation with Planar Arrays Martin Haardt, Michael D. Zoltowski, Cherian P. Mathews, and Javier Ramos64
A Unified Instrumental Variable Approach to Direction Finding in Colored Noise Fields P. Stoica, M. Viberg, M. Wong, and Q. Wu65
Electromagnetic Vector-Sensor Array Processing Arye Nehorai and Eytan Paldi66
Subspace Tracking R.D. DeGroat, E.M. Dowling, and D.A. Linebarger67
Detection: Determining the Number of Sources Douglas B. Williams68
Array Processing for Mobile Communications A. Paulraj and C. B. Papadias69
Beamforming with Correlated Arrivals in Mobile Communications Victor A.N. Barroso and Jos´e M.F. Moura70
Space-Time Adaptive Processing for Airborne Surveillance Radar Hong WangPART XIII Nonlinear and Fractal Signal Processing 71
Chaotic Signals and Signal Processing Alan V. Oppenheim and Kevin M. Cuomo72
Nonlinear Maps Steven H. Isabelle and Gregory W. Wornell73
Fractal Signals Gregory W. Wornell74
Morphological Signal and Image Processing Petros Maragos75
Signal Processing and Communication with Solitons Andrew C. Singer76
Higher-Order Spectral Analysis Athina P. PetropuluPART XIV DSP Software and Hardware
77
Introduction to the TMS320 Family of Digital Signal Processors Panos Papamichalis78
Rapid Design and Prototyping of DSP Systems T. Egolf, M. Pettigrew, J. Debardelaben, R.Hezar, S. Famorzadeh, A. Kavipurapu, M. Khan, Lan-Rong Dung, K. Balemarthy, N. Desai, Yong-kyu Jung, and V. Madisetti
To our families
Preface
Digital Signal Processing (DSP) is concerned with the theoretical and practical aspects of representing information bearing signals in digital form and with using computers or special purpose digital hardware either to extract that information or to transform the signals in useful ways. Areas where digital signal processing has made a significant impact include telecommunications, man-machine communications, computer engineering, multimedia applications, medical technology, radar and sonar, seismic data analysis, and remote sensing, to name just a few.
During the first fifteen years of its existence, the field of DSP saw advancements in the basic theory of discrete-time signals and processing tools. This work included such topics as fast algorithms, A/D and D/A conversion, and digital filter design. The past fifteen years has seen an ever quickening growth of DSP in application areas such as speech and acoustics, video, radar, and telecommunications.
Much of this interest in using DSP has been spurred on by developments in computer hardware and microprocessors. Digital Signal Processing Handbook CRCnetBASE is an attempt to capture the entire range of DSP: from theory to applications — from algorithms to hardware.
Given the widespread use of DSP, a need developed for an authoritative reference, written by some of the top experts in the world. This need was to provide information on both theoretical and practical issues suitable for a broad audience — ranging from professionals in electrical engineering, computer science, and related engineering fields, to managers involved in design and marketing, and to graduate students and scholars in the field. Given the large number of excellent introductory texts in DSP, it was also important to focus on topics useful to the engineer or scholar without overemphasizing those aspects that are already widely accessible. In short, we wished to create a resource that was relevant to the needs of the engineering community and that will keep them up-to-date in the DSP field.
A task of this magnitude was only possible through the cooperation of many of the foremost DSP researchers and practitioners. This collaboration, over the past three years, has resulted in a CD-ROM containing a comprehensive range of DSP topics presented with a clarity of vision and a depth of coverage that is expected to inform, educate, and fascinate the reader. Indeed, many of the articles, written by leaders in their fields, embody unique visions and perceptions that enable a quick, yet thorough, exposure to knowledge garnered over years of development.
As with other CRC Press handbooks, we have attempted to provide a balance between essential information, background material, technical details, and introduction to relevant standards and software. The Handbook pays equal attention to theory, practice, and application areas. Digital Signal Processing Handbook CRCnetBASE can be used in a number of ways. Most users will look up a topic of interest by using the powerful search engine and then viewing the applicable chapters. As such, each chapter has been written to stand alone and give an overview of its subject matter while providing key references for those interested in learning more. Digital Signal Processing Handbook CRCnetBASE can also be used as a reference book for graduate classes, or as supporting material for continuing education courses in the DSP area. Industrial organizations may wish to provide the CD-ROM with their products to enhance their value by providing a standard and up-to-date reference source.
We have been very impressed with the quality of this work, which is due entirely to the contributions of all the authors, and we would like to thank them all. The Advisory Board was instrumental in helping to choose subjects and leaders for all the sections. Being experts in their fields, the section leaders provided the vision and fleshed out the contents for their sections.
Finally, the authors produced the necessary content for this work. To them fell the challenging task of writing for such a broad audience, and they excelled at their jobs.
In addition to these technical contributors, we wish to thank a number of outstanding individuals whose administrative skills made this project possible. Without the outstanding organizational skills of Elaine M. Gibson, this handbook may never have been finished. Not only did Elaine manage the paperwork, but she had the unenviable task of reminding authors about deadlines and pushing them to finish. We also thank a number of individuals associated with the CRC Press Handbook Series over a period of time, especially Joel Claypool, Dick Dorf, Kristen Maus, Jerry Papke, Ron Powers, Suzanne Lassandro, and Carol Whitehead.
We welcome you to this handbook, and hope you find it worth your interest.
Vijay K. Madisetti and Douglas B. Williams Center for Signal and Image Processing School of Electrical and Computer Engineering Georgia Institute of Technology
Atlanta, Georgia
Editors
Vijay K. Madisetti is an Associate Professor in the School of Electrical and Computer Engineering at Georgia Institute of Technology in Atlanta. He teaches undergraduate and graduate courses in signal processing and computer engineering, and is affiliated with the Center for Signal and Image Processing (CSIP) and the Microelectronics Research Center (MiRC) on campus. He received his B.
Tech (honors) from the Indian Institute of Technology (IIT), Kharagpur, in 1984, and his Ph.D. from the University of California at Berkeley, in 1989, in electrical engineering and computer sciences.
Dr. Madisetti is active professionally in the area of signal processing, having served as an Associate Editor of the IEEE Transactions on Circuits and Systems II, the International Journal in Computer Simulation, and the Journal of VLSI Signal Processing. He has authored, co-authored, or edited six books in the areas of signal processing and computer engineering, including VLSI Digital Signal Processors (IEEE Press, 1995), Quick-Turnaround ASIC Design in VHDL (Kluwer, 1996), and a CD- ROM tutorial on VHDL (IEEE Standards Press, 1997). He serves as the IEEE Press Signal Processing Society liaison, and is counselor to Georgia Tech’s IEEE Student Chapter, which is one of the largest in the world with over 600 members in 1996. Currently, he is serving as the Technical Director of DARPA’s RASSP Education and Facilitation program, a multi-university/industry effort to develop a new digital systems design education curriculum.
Dr. Madisetti is a frequent consultant to industry and the U.S. government, and also serves as the President and CEO of VP Technologies, Inc., Marietta, GA., a corporation that specializes in rapid prototyping, virtual prototyping, and design of embedded digital systems. Dr. Madis- etti’s home page URL is at http://www.ee.gatech.edu/users/215/index.html, and he can be reached at vkm@ee.gatech.edu.
Editors
Douglas B. Williams received the B.S.E.E. degree (summa cum laude), the M.S. degree, and the Ph.D. degree, in electrical and computer engineering from Rice University, Houston, Texas in 1984, 1987, and 1989, respectively. In 1989, he joined the faculty of the School of Electrical and Computer Engineering at the Georgia Institute of Technology, Atlanta, Georgia, where he is currently an Associate Professor. There he is also affiliated with the Center for Signal and Image Processing (CSIP) and teaches courses in signal processing and telecommunications.
Dr. Williams has served as an Associate Editor of the IEEE Transactions on Signal Processing and was on the conference committee for the 1996 International Conference on Acoustics, Speech, and Signal Processing that was held in Atlanta. He is currently the faculty counselor for Georgia Tech’s student chapter of the IEEE Signal Processing Society. He is a member of the Tau Beta Pi, Eta Kappa Nu, and Phi Beta Kappa honor societies.
Dr. Williams’s current research interests are in statistical signal processing with emphasis on radar signal processing, communications systems, and chaotic time-series analysis. More information on his activities may be found on his home page at http://dogbert.ee.gatech.edu/users/276. He can also be reached at dbw@ee.gatech.edu.
I
Signals and Systems
Vijay K. Madisetti
Georgia Institute of Technology
Douglas B. Williams
Georgia Institute of Technology
1 Fourier Series, Fourier Transforms, and the DFT W. Kenneth Jenkins
Introduction•Fourier Series Representation of Continuous Time Periodic Signals•The Classical Fourier Transform for Continuous Time Signals•The Discrete Time Fourier Transform•The Discrete Fourier Transform•Family Tree of Fourier Transforms•Selected Applications of Fourier Methods•Summary
2 Ordinary Linear Differential and Difference Equations B.P. Lathi
Differential Equations•Difference Equations
3 Finite Wordlength Effects Bruce W. Bomar
Introduction•Number Representation•Fixed-Point Quantization Errors•Floating-Point Quan- tization Errors•Roundoff Noise•Limit Cycles•Overflow Oscillations•Coefficient Quantization Error•Realization Considerations
T
HE STUDY OF “SIGNALS AND SYSTEMS” has formed a cornerstone for the development of digital signal processing and is crucial for all of the topics discussed in this Handbook. While the reader is assumed to be familiar with the basics of signals and systems, a small portion is reviewed in this chapter with an emphasis on the transition from continuous time to discrete time.The reader wishing more background may find in it any of the many fine textbooks in this area, for example [1]-[6].
In the chapter “Fourier Series, Fourier Transforms, and the DFT” by W. Kenneth Jenkins, many important Fourier transform concepts in continuous and discrete time are presented. The discrete Fourier transform (DFT), which forms the backbone of modern digital signal processing as its most common signal analysis tool, is also described, together with an introduction to the fast Fourier transform algorithms.
In “Ordinary Linear Differential and Difference Equations”, the author, B.P. Lathi, presents a detailed tutorial of differential and difference equations and their solutions. Because these equations are the most common structures for both implementing and modelling systems, this background is necessary for the understanding of many of the later topics in this Handbook. Of particular interest are a number of solved examples that illustrate the solutions to these formulations.
While most software based on workstations and PCs is executed in single or double precision arithmetic, practical realizations for some high throughput DSP applications must be implemented in fixed point arithmetic. These low cost implementations are still of interest to a wide community in the consumer electronics arena. The chapter “Finite Wordlength Effects” by Bruce W. Bomar describes basic number representations, fixed and floating point errors, roundoff noise, and practical considerations for realizations of digital signal processing applications, with a special emphasis on filtering.
References
[1] Jackson, L.B.,Signals, Systems, and Transforms, Addison-Wesley, Reading, MA, 1991.
[2] Kamen, E.W. and Heck, B.S.,Fundamentals of Signals and Systems Using MATLAB, Prentice-Hall, Upper Saddle River, NJ, 1997.
[3] Oppenheim, A.V. and Willsky, A.S., with Nawab, S.H.,Signals and Systems, 2nd Ed., Prentice-Hall, Upper Saddle River, NJ, 1997.
[4] Strum, R.D. and Kirk, D.E.,Contemporary Linear Systems Using MATLAB, PWS Publishing, Boston, MA, 1994.
[5] Proakis, J.G. and Manolakis, D.G.,Introduction to Digital Signal Processing, Macmillan, New York;
Collier Macmillan, London, 1988.
[6] Oppenheim, A.V. and Schafer, R.W.,Discrete Time Signal Processing, Prentice-Hall, Englewood Cliffs, NJ, 1989.
1
Fourier Series, Fourier Transforms, and the DFT
W. Kenneth Jenkins
University of Illinois, Urbana-Champaign
1.1 Introduction
1.2 Fourier Series Representation of Continuous Time Periodic Signals
Exponential Fourier Series•The Trigonometric Fourier Series
•Convergence of the Fourier Series
1.3 The Classical Fourier Transform for Continuous Time Signals
Properties of the Continuous Time Fourier Transform • Fourier Spectrum of the Continuous Time Sampling Model• Fourier Transform of Periodic Continuous Time Signals•The Generalized Complex Fourier Transform
1.4 The Discrete Time Fourier Transform
Properties of the Discrete Time Fourier Transform•Relation- ship between the Continuous and Discrete Time Spectra 1.5 The Discrete Fourier Transform
Properties of the Discrete Fourier Series•Fourier Block Pro- cessing in Real-Time Filtering Applications • Fast Fourier Transform Algorithms
1.6 Family Tree of Fourier Transforms 1.7 Selected Applications of Fourier Methods
Fast Fourier Transform in Spectral Analysis•Finite Impulse Response Digital Filter Design•Fourier Analysis of Ideal and Practical Digital-to-Analog Conversion
1.8 Summary References
1.1 Introduction
Fourier methods are commonly used for signal analysis and system design in modern telecommu- nications, radar, and image processing systems. Classical Fourier methods such as the Fourier series and the Fourier integral are used for continuous time (CT) signals and systems, i.e., systems in which a characteristic signal,s(t), is defined at all values ofton the continuum−∞< t <∞. A more recently developed set of Fourier methods, including the discrete time Fourier transform (DTFT) and the discrete Fourier transform (DFT), are extensions of basic Fourier concepts that apply to discrete time (DT) signals. A characteristic DT signal,s[n], is defined only for values ofnwherenis an integer in the range−∞< n <∞. The following discussion presents basic concepts and outlines important properties for both the CT and DT classes of Fourier methods, with a particular emphasis on the relationships between these two classes. The class of DT Fourier methods is particularly useful
as a basis for digital signal processing (DSP) because it extends the theory of classical Fourier analysis to DT signals and leads to many effective algorithms that can be directly implemented on general computers or special purpose DSP devices.
The relationship between the CT and the DT domains is characterized by the operations of sampling and reconstruction. Ifsa(t)denotes a signals(t)that has been uniformly sampled everyT seconds, then the mathematical representation ofsa(t)is given by
sa(t)= X∞
n=−∞
s(t)δ(t−nT ) (1.1)
whereδ(t)is a CT impulse function defined to be zero for allt 6= 0, undefined att =0, and has unit area when integrated fromt = −∞tot = +∞. Because the only places at which the product s(t)δ(t−nT )is not identically equal to zero are at the sampling instances,s(t)in (1.1) can be replaced withs(nT )without changing the overall meaning of the expression. Hence, an alternate expression forsa(t)that is often useful in Fourier analysis is given by
sa(t)= X∞
n=−∞
s(nT )δ(t−nT ) (1.2)
The CT sampling modelsa(t)consists of a sequence of CT impulse functions uniformly spaced at intervals ofTseconds and weighted by the values of the signals(t)at the sampling instants, as depicted in Fig.1.1. Note thatsa(t)is not defined at the sampling instants because the CT impulse function itself is not defined att =0. However, the values ofs(t)at the sampling instants are imbedded as
“area under the curve” ofsa(t), and as such represent a useful mathematical model of the sampling process. In the DT domain the sampling model is simply the sequence defined by taking the values ofs(t)at the sampling instants, i.e.,
s[n] =s(t)|t=nT (1.3)
In contrast tosa(t), which is not defined at the sampling instants,s[n]is well defined at the sampling instants, as illustrated in Fig.1.2. Thus, it is now clear thatsa(t)ands[n]are different but equivalent models of the sampling process in the CT and DT domains, respectively. They are both useful for signal analysis in their corresponding domains. Their equivalence is established by the fact that they have equal spectra in the Fourier domain, and that the underlying CT signal from whichsa(t)and s[n]are derived can be recovered from either sampling representation, provided a sufficiently large sampling rate is used in the sampling operation (see below).
1.2 Fourier Series Representation of Continuous Time Periodic Signals
It is convenient to begin this discussion with the classical Fourier series representation of a periodic time domain signal, and then derive the Fourier integral from this representation by finding the limit of the Fourier coefficient representation as the period goes to infinity. The conditions under which a periodic signals(t)can be expanded in a Fourier series are known as the Dirichet conditions. They require that in each periods(t)has a finite number of discontinuities, a finite number of maxima and minima, and thats(t)satisfies the following absolute convergence criterion [1]:
Z T /2
−T /2
|s(t)|dt <∞ (1.4)
It is assumed in the following discussion that these basic conditions are satisfied by all functions that will be represented by a Fourier series.
FIGURE 1.1: CT model of a sampled CT signal.
FIGURE 1.2: DT model of a sampled CT signal.
1.2.1 Exponential Fourier Series
If a CT signals(t)is periodic with a periodT, then the classical complex Fourier series representation ofs(t)is given by
s(t)= X∞ n=−∞
anejnω0t (1.5a)
whereω0=2π/T, and where theanare the complex Fourier coefficients given by an=(1/T )
Z T /2
−T /2
s(t)e−jnω0tdt (1.5b)
It is well known that for every value oft wheres(t)is continuous, the right-hand side of (1.5a) converges tos(t). At values oft wheres(t)has a finite jump discontinuity, the right-hand side of (1.5a) converges to the average ofs(t −)ands(t+), wheres(t−)≡lim→0s(t−)ands(t+)≡ lim→0s(t+).
For example, the Fourier series expansion of the sawtooth waveform illustrated in Fig.1.3is char- acterized byT =2π,ω0 =1,a0=0, andan =a−n =Acos(nπ)/(jnπ)forn=1,2, . . .,. The coefficients of the exponential Fourier series represented by (1.5b) can be interpreted as the spec- tral representation ofs(t), because thean-th coefficient represents the contribution of the (nω0)-th frequency to the total signals(t). Because theanare complex valued, the Fourier domain represen-
tation has both a magnitude and a phase spectrum. For example, the magnitude of theanis plotted in Fig.1.4for the sawtooth waveform of Fig.1.3. The fact that thean constitute a discrete set is consistent with the fact that a periodic signal has a “line spectrum,” i.e., the spectrum contains only integer multiples of the fundamental frequencyω0. Therefore, the equation pair given by (1.5a) and (1.5b) can be interpreted as a transform pair that is similar to the CT Fourier transform for periodic signals. This leads to the observation that the classical Fourier series can be interpreted as a special transform that provides a one-to-one invertible mapping between the discrete-spectral domain and the CT domain. The next section shows how the periodicity constraint can be removed to produce the more general classical CT Fourier transform, which applies equally well to periodic and aperiodic time domain waveforms.
FIGURE 1.3: Periodic CT signal used in Fourier series example.
FIGURE 1.4: Magnitude of the Fourier coefficients for example of Figure1.3.
1.2.2 The Trigonometric Fourier Series
Although Fourier series expansions exist for complex periodic signals, and Fourier theory can be generalized to the case of complex signals, the theory and results are more easily expressed for real- valued signals. The following discussion assumes that the signals(t)is real-valued for the sake of simplifying the discussion. However, all results are valid for complex signals, although the details of the theory will become somewhat more complicated.
For real-valued signalss(t), it is possible to manipulate the complex exponential form of the Fourier series into a trigonometric form that containssin(ω0t)andcos(ω0t)terms with corresponding real-
valued coefficients [1]. The trigonometric form of the Fourier series for a real-valued signals(t)is given by
s(t)= X∞ n=0
bncos(nω0t)+ X∞ n=1
cnsin(nω0t) (1.6a)
whereω0=2π/T. Thebnandcnare real-valued Fourier coefficients determined by
FIGURE 1.5: Periodic CT signal used in Fourier series example 2.
FIGURE 1.6: Fourier coefficients for example of Figure1.5.
b0 = (1/T ) Z T /2
−T /2
s(t) dt bn = (2/T )
Z T /2
−T /2
s(t)cos(nω0t) dt, n=1,2, . . . , (1.6b) cn = (2/T )
Z T /2
−T /2
s(t)sin(nω0t) dt, n=1,2, . . . ,
An arbitrary real-valued signals(t)can be expressed as a sum of even and odd components,s(t)= seven(t)+sodd(t), where seven(t) = seven(−t) andsodd(t) = −sodd(−t), and whereseven(t) = [s(t)+s(−t)]/2andsodd(t) = [s(t)−s(−t)]/2. For the trigonometric Fourier series, it can be shown thatseven(t)is represented by the (even) cosine terms in the infinite series,sodd(t)is represented by the (odd) sine terms, andb0is the DC level of the signal. Therefore, if it can be determined by inspection that a signal has DC level, or if it is even or odd, then the correct form of the trigonometric
series can be chosen to simplify the analysis. For example, it is easily seen that the signal shown in Fig.1.5is an even signal with a zero DC level. Therefore it can be accurately represented by the cosine series withbn=2Asin(πn/2)/(πn/2), n=1,2, . . . ,as illustrated in Fig.1.6. In contrast, note that the sawtooth waveform used in the previous example is an odd signal with zero DC level; thus, it can be completely specified by the sine terms of the trigonometric series. This result can be demonstrated by pairing each positive frequency component from the exponential series with its conjugate partner, i.e.,cn=sin(nω0t)=anejnω0t+a−ne−jnω0t, whereby it is found thatcn=2Acos(nπ)/(nπ)for this example. In general it is found thatan =(bn−jcn)/2forn=1,2, . . . , a0=b0, anda−n=a∗n. The trigonometric Fourier series is common in the signal processing literature because it replaces complex coefficients with real ones and often results in a simpler and more intuitive interpretation of the results.
1.2.3 Convergence of the Fourier Series
The Fourier series representation of a periodic signal is an approximation that exhibits mean squared convergence to the true signal. Ifs(t)is a periodic signal of periodT, ands0(t)denotes the Fourier series approximation ofs(t), thens(t)ands0(t)are equal in the mean square sense if
MSE =
Z T /2
−T /2
|s(t)−s(t)0|2dt=0 (1.7) Even with (1.7) satisfied, mean square error (MSE) convergence does not mean thats(t) =s0(t) at every value oft. In particular, it is known that at values oft, wheres(t)is discontinuous, the Fourier series converges to the average of the limiting values to the left and right of the discontinuity.
For example, ift0is a point of discontinuity, then s0(t0) = [s(t0−)+s(t0+)]/2, wheres(t0−)and s(t0+)were defined previously. (Note that at points of continuity, this condition is also satisfied by the definition of continuity.) Because the Dirichet conditions require thats(t)have at most a finite number of points of discontinuity in one period, the setSt, defined as all values oft within one period wheres(t)6=s0(t), contains a finite number of points, andSt is a set of measure zero in the formal mathematical sense. Therefore,s(t)and its Fourier series expansions0(t)are equal almost everywhere, ands(t)can be considered identical tos0(t)for the analysis of most practical engineering problems.
Convergence almost everywhere is satisfied only in the limit as an infinite number of terms are included in the Fourier series expansion. If the infinite series expansion of the Fourier series is truncated to a finite number of terms, as it must be in practical applications, then the approximation will exhibit an oscillatory behavior around the discontinuity, known as the Gibbs phenomenon [1].
LetsN0 (t)denote a truncated Fourier series approximation ofs(t), where only the terms in (1.5a) fromn= −N ton=Nare included if the complex Fourier series representation is used, or where only the terms in (1.6a) fromn=0ton=Nare included if the trigonometric form of the Fourier series is used. It is well known that in the vicinity of a discontinuity att0the Gibbs phenomenon causessN0 (t)to be a poor approximation tos(t). The peak magnitude of the Gibbs oscillation is 13%
of the size of the jump discontinuitys(t0−)−s(t0+)regardless of the number of terms used in the approximation. AsNincreases, the region that contains the oscillation becomes more concentrated in the neighborhood of the discontinuity, until, in the limit asN approaches infinity, the Gibbs oscillation is squeezed into a single point of mismatch att0.
Ifs0(t)is replaced bys0N(t)in (1.7), it is important to understand the behavior of the error MSEN as a function ofN, where
MSEN = Z T /2
−T /2
|s(t)−sN0 (t)|2dt (1.8)
An important property of the Fourier series is that the exponential basis functionsejnω0t(orsin(nω0t) andcos(nω0t)for the trigonometric form) for n = 0,±1,±2, . . .(orn = 0,1,2, . . . for the trigonometric form) constitute an orthonormal set, i.e.,tnk =1forn=k, andtnk =0forn6=k, where
tnk=(1/T ) Z T /2
−T /2
(e−jnω0t)(ejkω0t) dt (1.9) As terms are added to the Fourier series expansion, the orthogonality of the basis functions guarantees that the error decreases in the mean square sense, i.e., that MSENmonotonically decreases asNis increased. Therefore, a practitioner can proceed with the confidence that when applying Fourier series analysis more terms are always better than fewer in terms of the accuracy of the signal representations.
1.3 The Classical Fourier Transform for Continuous Time Signals
The periodicity constraint imposed on the Fourier series representation can be removed by taking the limits of (1.5a) and (1.5b) as the periodT is increased to infinity. Some mathematical preliminaries are required so that the results will be well defined after the limit is taken. It is convenient to remove the(1/T )factor in front of the integral by multiplying (1.5b) through byT, and then replacing T anbyan0 in both (1.5a) and (1.5b). Becauseω 0=2π/T, asT increases to infinity,ω0becomes infinitesimally small, a condition that is denoted by replacingω0with1ω. The factor(1/T )in (1.5a) becomes(1ω/2π). With these algebraic manipulations and changes in notation (1.5a) and (1.5b) take on the following form prior to taking the limit:
s(t) = (1/2π) X∞
n=−∞
an0ejn1ωt1ω (1.10a)
an0 = Z T /2
−T /2
s(t)e−jn1ωtdt (1.10b)
The final step in obtaining the CT Fourier transform is to take the limit of both (1.10a) and (1.10b) asT → ∞. In the limit the infinite summation in (1.10a) becomes an integral,1ωbecomesdω, n1ωbecomesω, andan0 becomes the CT Fourier transform ofs(t), denoted byS(jω). The result is summarized by the following transform pair, which is known throughout most of the engineering literature as the classical CT Fourier transform (CTFT):
s(t) = (1/2π) Z ∞
−∞S(jω)ejωtdω (1.11a)
S(jω) = Z ∞
−∞s(t)e−jωtdt (1.11b)
Often (1.11a\) is called the Fourier integral and (1.11b) is simply called the Fourier transform. The relationshipS(jω)=F{s(t)}denotes the Fourier transformation ofs(t), whereF{·}is a symbolic notation for the Fourier transform operator, and whereωbecomes the continuous frequency variable after the periodicity constraint is removed. A transform pairs(t) ↔ S(jω)represents a one-to- one invertible mapping as long ass(t)satisfies conditions which guarantee that the Fourier integral converges.
From (1.11a) it is easily seen thatF{δ(t−t 0)} =e−jωt0, and from (1.11b) thatF −1{2πδ(ω− ω0)} =ejω0t, so thatδ(t−t0)↔e−jωt0 andejω0t ↔2πδ(ω−ω0)are valid Fourier transform
pairs. Using these relationships it is easy to establish the Fourier transforms ofcos(ω0t)andsin(ω0t), as well as many other useful waveforms that are encountered in common signal analysis problems.
A number of such transforms are shown in Table1.1.
The CTFT is useful in the analysis and design of CT systems, i.e., systems that process CT signals.
Fourier analysis is particularly applicable to the design of CT filters which are characterized by Fourier magnitude and phase spectra, i.e., by|H (jω)|and argH (jω), whereH (jω)is commonly called the frequency response of the filter. For example, an ideal transmission channel is one which passes a signal without distorting it. The signal may be scaled by a real constantAand delayed by a fixed time incrementt0, implying that the impulse response of an ideal channel isAδ(t −t0), and its corresponding frequency response isAe−jωt0. Hence, the frequency response of an ideal channel is specified by constant amplitude for all frequencies, and a phase characteristic which is linear function given byωt0.
1.3.1 Properties of the Continuous Time Fourier Transform
The CTFT has many properties that make it useful for the analysis and design of linear CT systems.
Some of the more useful properties are stated below. A more complete list of the CTFT properties is given in Table1.2. Proofs of these properties can be found in [2] and [3]. In the following discus- sionF{·}denotes the Fourier transform operation,F−1{·}denotes the inverse Fourier transform operation, and∗denotes the convolution operation defined as
f1(t)∗f2(t)= Z ∞
−∞f1(t−τ)f2(τ) dτ
1. Linearity (superposition):F{af1(t)+bf2(t)} =aF{f1(t)} +bF{f2(t)}
(aandb, complex constants)
2. Time shifting:F{f (t−t0)} =e−jωt0F{f (t)}
3. Frequency shifting:ejω0tf (t)=F−1{F (j (ω−ω0))}
4. Time domain convolution:F{f1(t)∗f2(t)} =F{f1(t)}F{f2(t)}
5. Frequency domain convolution:F{f1(t)f2(t)} =(1/2π)F{f1(t)} ∗F{f2(t)}
6. Time differentiation:−jωF (jω)=F{d(f (t))/dt}
7. Time integration:F{Rt
−∞f (τ) dτ} =(1/jω)F (jω)+πF (0)δ(ω)
The above properties are particularly useful in CT system analysis and design, especially when the system characteristics are easily specified in the frequency domain, as in linear filtering. Note that properties 1, 6, and 7 are useful for solving differential or integral equations. Property 4 provides the basis for many signal processing algorithms because many systems can be specified directly by their impulse or frequency response. Property 3 is particularly useful in analyzing communication systems in which different modulation formats are commonly used to shift spectral energy to frequency bands that are appropriate for the application.
1.3.2 Fourier Spectrum of the Continuous Time Sampling Model
Because the CT sampling modelsa(t), given in (1.1), is in its own right a CT signal, it is appropriate to apply the CTFT to obtain an expression for the spectrum of the sampled signal:
F{sa(t)} =F ( ∞
X
n=−∞
s(t)δ(t−nT ) )
= X∞
n=−∞
s(nT )e−jωT n (1.12) Because the expression on the right-hand side of (1.12) is a function ofejωTit is customary to denote the transform asF (ejωT)=F{sa(t)}. Later in the chapter this result is compared to the result of
TABLE 1.1 Some Basic CTFT Pairs
Fourier Series Coefficients Signal Fourier Transform (if periodic)
+∞X k=−∞
akejkω0t 2π +∞X
k=−∞
akδ(ωkω0) ak
ejω0t 2πδ(ω+ω0) a1=1
ak=0, otherwise cosω0t π[δ(ω−ω0)+δ(ω+ω0)] a1=a−1=12
ak=0, otherwise
sinω0t πj[δ(ω−ω0)−δ(ω+ω0)] a1= −a−1=21j ak=0, otherwise
x(t)=1 2πδ(ω) a0=1, ak=0, k6=0
has this Fourier series representation for any choice ofT0>0
Periodic square wave
x(t)=
1, |t|< T1
0, T1<|t| ≤T02
+∞X k=−∞
2 sinkω0T1
k δ(ωkω0) ω0T1 π sinc
kω0T1 π
=sinkω0T1 kπ and
x(t+T0)=x(t) +∞X
n=−∞
δ(t−nT ) 2π
T +∞X
k= −∞δ
ω−2πk T
ak= 1
T for allk x(t)=
1, |t|< T1
0, |t|> T1 2T1sinc ωT1
π
=2 sinωωT1 —
W π sinc
Wt π
=sinWt
πt X(ω)=
( 1, |ω|< W
0, |ω|> W —
δ(t) 1 —
u(t) 1
jω+πδ(ω) —
δ(t−t0) ejωt0 —
e−atu(t),Re{a}>0 1
a+jω —
te−atu(t),Re{a}>0 1
(a+jω)2 —
tn−1 (n−1)!e−atu(t),
Re{a}>0
1
(a+jω)n —
TABLE 1.2 Properties of the CTFT Name IfFf (t)=F (jω), then
Definition f (jω)= Z∞
−∞f (t)ejωtdt f (t)= 1
2π Z∞
−∞F (jω)ejωtdω Superposition F[af1(t)+bf2(t)] =aF1(jω)+bF2(jω) Simplification if:
(a)f (t)is even F (jω)=2 Z∞
0 f (t)cosωt dt (b)f (t)is odd F (jω)=2j
Z∞
0 f (t)sinωt dt Negativet Ff (−t)=F∗(jω) Scaling:
(a) Time Ff (at)= 1
|a|F jω
a
(b) Magnitude Faf (t)=aF (jω) Differentiation F
dn dtnf (t)
=(jω)nF (jω)
Integration FZt
−∞f (x) dx
=jω1 F (jω)+πF (0)δ(ω)
Time shifting Ff (t−a)=F (jω)ejωa Modulation Ff (t)ejω0t=F[j (ω−ω0)]
{Ff (t)cosω0t=12F[j (ω−ω0)] +F[j (ω+ω0)]}
{Ff (t)sinω0t=12j[F[j (ω−ω0)] −F[j (ω+ω0)]}
Time convolution F−1[F1(jω)F2(jω)] = Z∞
−∞f1(τ)f2(τ)f2(tτ) dτ Frequency convolution F[f1(t)f2(t)] = 1
2π Z∞
−∞F1(jλ)F2[j (ωλ)]dλ
operating on the DT sampling model, namelys[n], with the DT Fourier transform to illustrate that the two sampling models have the same spectrum.
1.3.3 Fourier Transform of Periodic Continuous Time Signals
We saw earlier that a periodic CT signal can be expressed in terms of its Fourier series. The CTFT can then be applied to the Fourier series representation ofs(t)to produce a mathematical expression for the “line spectrum” characteristic of periodic signals.
F{s(t)} =F ( ∞
X
n=−∞
anejnω0t )
=2π X∞ n=−∞
anδ(ω−nω0) (1.13) The spectrum is shown pictorially in Fig.1.7. Note the similarity between the spectral representation of Fig.1.7and the plot of the Fourier coefficients in Fig.1.4, which was heuristically interpreted as a “line spectrum”. Figures1.4and1.7are different but equivalent representations of the Fourier
spectrum. Note that Fig.1.4is a DT representation of the spectrum, while Fig.1.7is a CT model of the same spectrum.
FIGURE 1.7: Spectrum of the Fourier series representation ofs(t).
1.3.4 The Generalized Complex Fourier Transform
The CTFT characterized by (1.11a) and (1.11b) can be generalized by considering the variablejω to be the special case ofu =σ +jωwithσ =0, writing (1.11a) in terms ofu, and interpretingu as a complex frequency variable. The resulting complex Fourier transform pair is given by (1.14a) and (1.14b)
s(t) = (1/2πj)
Z σ+j∞
σ−j∞ S(u)ejutdu (1.14a) S(u) =
Z ∞
−∞s(t)e−jutdt (1.14b)
The set of all values ofufor which the integral of (1.14b) converges is called the region of convergence (ROC). Because the transformS(u)is defined only for values of uwithin the ROC, the path of integration in (1.14a) must be defined byσ so that the entire path lies within the ROC. In some literature this transform pair is called the bilateral Laplace transform because it is the same result obtained by including both the negative and positive portions of the time axis in the classical Laplace transform integral. [Note that in (1.14a) the complex frequency variable was denoted byurather than by the more commons, in order to avoid confusion with earlier uses ofs(·)as signal notation.]
The complex Fourier transform (bilateral Laplace transform) is not often used in solving practical problems, but its significance lies in the fact that it is the most general form that represents the point at which Fourier and Laplace transform concepts become the same. Identifying this connection reinforces the notion that Fourier and Laplace transform concepts are similar because they are derived by placing different constraints on the same general form.
1.4 The Discrete Time Fourier Transform
The discrete time Fourier transform (DTFT) can be obtained by using the DT sampling model and considering the relationship obtained in (1.12) to be the definition of the DTFT. LettingT =1so that the sampling period is removed from the equations and the frequency variable is replaced with
a normalized frequencyω0=ωT, the DTFT pair is defined in (1.15a). Note that in order to simplify notation it is not customary to distinguish betweenωandω0, but rather to rely on the context of the discussion to determine whetherωrefers to the normalized(T =1)or the unnormalized(T 6=1) frequency variable.
S(ejω0) = X∞ n=−∞
s[n]e−jω0n (1.15a)
s[n] = (1/2π) Z π
−πS(ejω0)ejnω0dω0 (1.15b) The spectrum S(ejω0)is periodic in ω0 with period2π. The fundamental period in the range
−π < ω0≤π, sometimes referred to as the baseband, is the useful frequency range of the DT system because frequency components in this range can be represented unambiguously in sampled form (without aliasing error). In much of the signal processing literature the explicit primed notation is omitted from the frequency variable. However, the explicit primed notation will be used throughout this section because the potential exists for confusion when so many related Fourier concepts are discussed within the same framework.
By comparing (1.12) and (1.15a), and noting thatω 0=ωT, it is established that
F{sa(t)} = DTFT{s[n]} (1.16) wheres[n] =s(t)t=nT. This demonstrates that the spectrum ofsa(t), as calculated by the CT Fourier transform is identical to the spectrum ofs[n]as calculated by the DTFT. Therefore, althoughsa(t) ands[n]are quite different sampling models, they are equivalent in the sense that they have the same Fourier domain representation.
A list of common DTFT pairs is presented in Table1.3. Just as the CT Fourier transform is useful in CT signal system analysis and design, the DTFT is equally useful in the same capacity for DT systems. It is indeed fortuitous that Fourier transform theory can be extended in this way to apply to DT systems.
In the same way that the CT Fourier transform was found to be a special case of the complex Fourier transform (or bilateral Laplace transform), the DTFT is a special case of the bilateralz-transform withz=ejω0t. The more general bilateralz-transform is given by
S(z) = X∞
n=−∞
s[n]z−n (1.17a)
s[n] = (1/2πj) Z
CS(z)zn−1dz (1.17b)
where C is a counterclockwise contour of integration which is a closed path completely contained within the region of convergence ofS(z). Recall that the DTFT was obtained by taking the CT Fourier transform of the CT sampling model represented bysa(t). Similarly, the bilateralz-transform results by taking the bilateral Laplace transform ofsa(t). If the lower limit on the summation of (1.17a) is taken to ben=0, then (1.17a) and (1.17b) become the one-sidedz-transform, which is the DT equivalent of the one-sided LT for CT signals. The hierarchical relationship among these various concepts for DT systems is discussed later in this chapter, where it will be shown that the family structure of the DT family tree is identical to that of the CT family. For every CT transform in the CT world there is an analogous DT transform in the DT world, and vice versa.
