Time-Frequency Displays

The preceding chapters have been concerned with the spectrum analysis of sinusoids and noise at a particular point in time (or a single spectrum for all time). This chapter introduces the Short-Time Fourier Transform (STFT)--a time-ordered sequence of spectral estimates, each using a finite-length analysis window. The STFT is used to compute the classic spectrogram, used extensively for speech and audio signals in general [116,50,183,147,209,64,71]. Finally, we point to methods for making spectrograms correspond better to audio perception, so that what you see is what you hear, to a greater extent. In particular, a loudness spectrogram based on a psychoacoustic model of time-varying loudness perception is described.

---

Subsections

• The Short-Time Fourier Transform
  • Mathematical Definition of the STFT
  • Practical Computation of the STFT
  • Summary of STFT Computation Using the FFT
  • Two Dual Interpretations of the STFT
  • STFT in Matlab
  
  
• Classic Spectrograms
  • Spectrogram of Speech
  
  
• Audio Spectrograms
  • Auditory Filter Banks
  • Loudness Spectrogram
  • Loudness Spectrogram Examples
    • Multiresolution STFT
    • Excitation Pattern
    • Nonuniform Spectral Resampling
    • Specific Loudness
    • Spectrograms Compared
    • Instantaneous, Short-Term, and Long-Term Loudness
  
  
• Summary

---

Order a Hardcopy of Spectral Audio Signal Processing

Previous: The Panning Problem
Next: The Short-Time Fourier Transform

---

written by Julius Orion Smith III
Julius Smith's background is in electrical engineering (BS Rice 1975, PhD Stanford 1983). He is presently Professor of Music and Associate Professor (by courtesy) of Electrical Engineering at Stanford's Center for Computer Research in Music and Acoustics (CCRMA), teaching courses and pursuing research related to signal processing applied to music and audio systems. See http://ccrma.stanford.edu/~jos/ for details.