Introduction and Overview

The ear is a kind of Fourier analyzer. That is, sound is spread out along the inner ear according to frequency, much like a prism separates light into various colors. As a result, hearing in the brain is based on a kind of ``short term spectrum analysis'' of sound. This is useful for a variety reasons:

• Perhaps most important, when the frequency content of one sound is different from that of another sound, and the two sounds are mixed (added) together, the sounds are largely separated out by the hearing process. This allows us to mentally ``unmix'' the sounds, enabling us to focus on one sound in the mix, excluding all others. This is hard to do by computer, and it remains an active research topic in the field of Music Information Retrieval (MIR).

• The formant resonances that distinguish the vowels of speech are separated in the auditory nerve, thereby facilitating vowel recognition by the brain.

• Periodic sounds are more audible than random sounds in the same frequency band. Throughout the animal kingdom, this fact provides a basis for various ``calls'' that can be heard above the ambient environmental noise.

• Transient sounds such as breaking twigs or rustling leaves can be recognized and distinguished based on spectral profile.

• Last but not least, we are able to appreciate tonal music!

As another example of the utility of spectrum analysis, the fields of chemistry, physics, astronomy, and cosmology were all advanced profoundly by the study of light spectra. To cite just one of many, many examples, the ``red shift'' (downward Doppler frequency-shift) of light coming from stars led Edwin Hubble (in 1929) to conclude that the Universe was expanding according to the Big Bang theory of cosmology (the farther apart two stars are, the faster they are racing away from each other).

In summary, spectrum analysis provides a wealth of information about signals that can be used for detection, classification, and discrimination tasks. Since hearing is based on a spectral decomposition, spectrum analysis provides an important foundation for many audio signal processing applications.

---

Subsections

• Organization
• Overview
  • Elementary Spectrum Analysis
  • The Short-Time Fourier Transform (STFT) and Time-Frequency Displays
  • Short-Time Analysis, Modification, and Resynthesis
  • STFT Applications
  • Multirate Polyphase and Wavelet Filter Banks
  • Appendices

---

Previous: Book Series Overview
Next: Organization

Order a Hardcopy of Spectral Audio Signal Processing

---

About the Author: Julius Orion Smith III
Julius Smith's background is in electrical engineering (BS Rice 1975, PhD Stanford 1983). He is presently Professor of Music and Associate Professor (by courtesy) of Electrical Engineering at Stanford's Center for Computer Research in Music and Acoustics (CCRMA), teaching courses and pursuing research related to signal processing applied to music and audio systems. See http://ccrma.stanford.edu/~jos/ for details.