This book precipitated from my ``spectral modeling'' course which has been offered at the Center for Computer Research in Music and Acoustics (CCRMA) since 1984. The course originally evolved as a dissemination vehicle for spectral-oriented signal-processing research in computer music, aimed at beginning graduate students in computer music and engineering programs et al. Over the years it has become more of a tour of fundamentals in spectral audio signal processing, with occasional mention and citation of prior and ongoing related research. In principle, the only prerequisites are the first two books in the music signal processing series [264,263].

The focus of this book is on spectral modeling applied to audio signals. More completely, the principal tasks are spectral analysis, modeling, and resynthesis (and/or effects). We analyze sound in terms of spectral models primarily because this is what the human brain does. We may synthesize/modify sound in terms of spectral models for the same reason.

The primary tool for audio spectral modeling is the short-time Fourier transform (STFT). The applications we will consider lie in the fields of audio signal processing and musical sound synthesis and effects.

The reader should already be familiar with the Fourier transform and elementary digital signal processing. One source of this background material is [264]. Some familiarity with digital filtering and associated linear systems theory, e.g., on the level of [263], is also assumed.

There is a notable absence in this book of emphasis on audio coding of spectral representations. While audio coding is closely related, there are other books which cover this topic in detail (e.g., [273,16,159]). On the other hand, comparatively few works address applications of spectral modeling in areas other than audio compression. This book attempts to help fill that gap.


Thanks to Bill Putnam, former teaching assistant, for creating initial versions of many of the figures in this book and their associated Matlab scripts. Also, many of the figures pertaining to sinusoidal modeling10.4) were contributed to the class by Music Ph.D. student Xavier Serra [246]. Thanks are also due to later teaching assistants Scott Levine [149] and Harvey Thornburg who developed lectures for the class that have been incorporated into the ongoing class material (and this book, particularly in Chapter 11). Teaching assistants Yi-Wen Liu and Pamornpol (Tak) Jinachitra contributed significantly to the laboratory exercises and theory problems, and Yi-Wen also developed the software used in §4.5.2.

Thanks to visiting scholar Mototsugu Abe, and graduate students Steven Backer, Edgar Berdahl, Nicholas Bryan, Ryan Cassidy, Roy Fejgin, Patty Huang, Arvindh Krishnaswamy, Gautham Mysore, Colin Raffel, Ryan Said, Joelle Skaf, Kurt Werner, Matthew Wright, and Peter von Wrycza for logging and reporting errata in earlier draft versions of this book. Thanks also to Andrew Best and Bill Schottstaedt for especially helpful and detailed errata reporting and feedback. Thanks to Kyle Spratt for bringing references [52,285,63] to my attention, which greatly informed the historical summary in Appendix G.

Book Series Overview

This book is the fourth in my music signal processing series, after [264], [263], and [266]. The books can be loosely summarized by the following ``design goals'':

    All about the DFT formula and its constituents, with frequent references to audio applications.

    A gentle introduction to the analysis and implementation of digital filters, with particular emphasis on audio applications.

    Efficient computational physical models for delay effects and virtual acoustic musical instruments.

    Analysis, processing, and synthesis of audio signals in terms of spectral representations computed using a Fast Fourier Transform (FFT).

Figure 1: Schematic of interdependencies in the music signal processing book series, along with some closely related topics.

Figure 1 illustrates the dependencies. A solid line indicates a strong dependence, while a dotted line indicates a weaker (optional) dependence. The student is expected to pick up elementary physics [100] and programming skills [260,45] elsewhere. In all books, the main chapters contain approximately what is covered in class, while the appendices provide both elementary background material and additional advanced topics.


Like any relatively large work, this book is sure to have some typos and other errors. Please report any suspected errata to the author via email1.1 so that they can be fixed right away in the online version and later in the next printing (or revision) of the hardcopy version.

The author and publisher make no warranties, express or implied, regarding the contents of this book.

Next Section:
Introduction and Overview