SPECTRAL AUDIO SIGNAL PROCESSING

JULIUS O. SMITH III
Center for Computer Research in Music and Acoustics (CCRMA)

• Preface
  • Acknowledgments
  • Book Series Overview
  • Errata
  
  
• Introduction and Overview
  • Organization
  • Overview
    • Elementary Spectrum Analysis
    • The Short-Time Fourier Transform (STFT) and
      Time-Frequency Displays
    • Short-Time Analysis, Modification, and Resynthesis
    • STFT Applications
    • Multirate Polyphase and Wavelet Filter Banks
    • Appendices
  
  
• Fourier Transforms and Theorems
  • Discrete Time Fourier Transform
  • Fourier Transform (FT) and Inverse
    • Existence of the Fourier Transform
  • Fourier Theorems for the DTFT
    • Linearity of the DTFT
    • Time Reversal
    • Symmetry of the DTFT for Real Signals
      • DTFT of Real Signals
      • Real Even (or Odd) Signals
    • Shift Theorem
    • Convolution Theorem
    • Correlation Theorem
    • Autocorrelation
    • Power Theorem
    • Stretch Operator
    • Repeat (Scaling) Operator
    • Stretch/Repeat (Scaling) Theorem
    • Downsampling and Aliasing
      • Proof of Aliasing Theorem
    • Differentiation Theorem Dual
  • Continuous-Time Fourier Theorems
    • Scaling Theorem
    • Spectral Roll-Off
  • Spectral Interpolation
    • Ideal Spectral Interpolation
    • Interpolating a DFT
    • Zero Padding in the Time Domain
      • Practical Zero Padding
      • Zero-Padding to the Next Higher Power of 2
      • Zero-Padding for Interpolating Spectral Displays
      • Zero-Padding for Interpolating Spectral Peaks
    • Zero-Phase Zero Padding
      • Matlab/Octave fftshift utility
      • Index Ranges for Zero-Phase Zero-Padding
      • Summary
  
  
• Spectrum Analysis Windows
  • The Rectangular Window
    • Side Lobes
    • Summary
  • Generalized Hamming Window Family
    • Hann or Hanning or Raised Cosine
    • Matlab for the Hann Window
    • Hamming Window
    • Matlab for the Hamming Window
    • Summary of Generalized Hamming Windows
    • The MLT Sine Window
  • Blackman-Harris Window Family
    • Blackman Window Family
    • Classic Blackman
    • Matlab for the Classic Blackman Window
    • Three-Term Blackman-Harris Window
    • Frequency-Domain Blackman-Harris
    • Power-of-Cosine Window Family
  • Spectrum Analysis of an Oboe Tone
    • Rectangular-Windowed Oboe Recording
    • Hamming-Windowed Oboe Recording
    • Blackman-Windowed Oboe Recording
    • Conclusions
  • Bartlett (``Triangular'') Window
    • Matlab for the Bartlett Window:
  • Poisson Window
  • Hann-Poisson Window
    • Matlab for the Hann-Poisson Window
  • Slepian or DPSS Window
    • Matlab for the DPSS Window
  • Kaiser Window
    • Kaiser Window Beta Parameter
    • Kaiser Windows and Transforms
    • Minimum Frequency Separation vs. Window Length
    • Kaiser and DPSS Windows Compared
  • Dolph-Chebyshev Window
    • Matlab for the Dolph-Chebyshev Window
    • Example Chebyshev Windows and Transforms
    • Chebyshev and Hamming Windows Compared
    • Dolph-Chebyshev Window Theory
      • Chebyshev Polynomials
      • Dolph-Chebyshev Window Definition
      • Dolph-Chebyshev Window Main-Lobe Width
      • Dolph-Chebyshev Window Length Computation
  • Gaussian Window and Transform
    • Matlab for the Gaussian Window
    • Gaussian Window and Transform
    • Exact Discrete Gaussian Window
  • Optimized Windows
    • Optimal Windows for Audio Coding
    • General Rule
  • Optimal Window Design by linprog
    • Linear Programming (LP)
    • LP Formulation of Chebyshev Window Design
    • Symmetric Window Constraint
    • Positive Window-Sample Constraint
    • DC Constraint
    • Sidelobe Specification
    • LP Standard Form
    • Remez Exchange Algorithm
      • Convergence of Remez Exchange
    • Monotonicity Constraint
    • L-Infinity Norm of Derivative Objective
    • L-One Norm of Derivative Objective
    • Summary
  
  
• FIR Digital Filter Design
  • The Ideal Lowpass Filter
  • Lowpass Filter Design Specifications
    • Ideal Lowpass Filter Revisited
  • Least-Squares FIR Filter Design
    • Examples
  • Frequency-Sampling FIR Filter Design
  • Window Method for FIR Filter Design
    • Matlab Support for the Window Method
    • Bandpass Filter Design Example
      • Under the Hood of kaiserord
      • Comparison to the Optimal Chebyshev FIR Bandpass Filter
  • Hilbert Transform Design Example
    • Primer on Hilbert Transform Theory
      • Hilbert Transform
    • Preparing the Desired Impulse Response
      • Matlab, Continued
      • Kaiser Window
      • Hilbert Transformer by the Window Method
    • More General FIR Filter Design
    • Comparison to Optimal Chebyshev FIR Filter
    • Conclusions
  • Generalized Window Method
  • Minimum-Phase Filter Design
  • Minimum-Phase and Causal Cepstra
  • Optimal FIR Digital Filter Design
    • Lp norms
      • Special Cases
      • Filter Design using Lp Norms
    • Optimal Chebyshev FIR Filters
    • Least-Squares Linear-Phase FIR Filter Design
      • Geometric Interpretation of Least Squares
      • Matlab Support for Least-Squares FIR Filter Design
    • Chebyshev FIR Design via Linear Programming
    • More General Real FIR Filters
    • Nonlinear-Phase FIR Filter Design
      • Problem Formulation
    • Matlab for General FIR Filter Design
    • Second-Order Cone Problems
      • Resources
    • Nonlinear Optimization in Matlab
  
  
• Spectrum Analysis of Sinusoids
  • Spectrum of a Sinusoid
  • Spectrum of Sampled Complex Sinusoid
  • Spectrum of a Windowed Sinusoid
  • Effect of Windowing
    • Frequency Resolution
      • Two Cosines (``In-Phase'' Case)
      • One Sine and One Cosine ``Phase Quadrature'' Case
  • Resolving Sinusoids
    • Other Definitions of Main Lobe Width
    • Simple Sufficient Condition for Peak Resolution
    • Periodic Signals
    • Tighter Bounds for Minimum Window Length
    • Summary
  • Sinusoidal Peak Interpolation
    • Quadratic Interpolation of Spectral Peaks
      • Phase Interpolation at a Peak
      • Matlab for Parabolic Peak Interpolation
    • Bias of Parabolic Peak Interpolation
  • Optimal Peak-Finding in the Spectrum
    • Minimum Zero-Padding for High-Frequency Peaks
    • Minimum Zero-Padding for Low-Frequency Peaks
    • Matlab for Computing Minimum Zero-Padding Factors
    • Least Squares Sinusoidal Parameter Estimation
      • Sinusoidal Amplitude Estimation
      • Sinusoidal Amplitude and Phase Estimation
      • Sinusoidal Frequency Estimation
    • Maximum Likelihood Sinusoid Estimation
    • Likelihood Function
      • Multiple Sinusoids in Additive Gaussian White Noise
      • Non-White Noise
    • Generality of Maximum Likelihood Least Squares
  
  
• Spectrum Analysis of Noise
  • Introduction to Noise
    • Why Analyze Noise?
    • What is Noise?
  • Spectral Characteristics of Noise
  • White Noise
    • Testing for White Noise
  • Sample Autocorrelation
  • Sample Power Spectral Density
  • Biased Sample Autocorrelation
  • Smoothed Power Spectral Density
  • Cyclic Autocorrelation
  • Practical Bottom Line
  • Why an Impulse is Not White Noise
  • The Periodogram
    • Matlab for the Periodogram
  • Welch's Method
    • Welch Autocorrelation Estimate
    • Resolution versus Stability
  • Welch's Method with Windows
    • Matlab for Welch's Method
  • Filtered White Noise
    • Example: FIR-Filtered White Noise
    • Example: Synthesis of 1/F Noise (Pink Noise)
    • Example: Pink Noise Analysis
  • Processing Gain
  • The Panning Problem
  
  
• Time-Frequency Displays
  • The Short-Time Fourier Transform
    • Mathematical Definition of the STFT
    • Practical Computation of the STFT
    • Summary of STFT Computation Using FFTs
    • Two Dual Interpretations of the STFT
    • The STFT as a Time-Frequency Distribution
    • STFT in Matlab
      • Notes
  • Classic Spectrograms
    • Spectrogram of Speech
  • Audio Spectrograms
    • Auditory Filter Banks
    • Loudness Spectrogram
      • A Note on Hop Size
    • Examples
      • Multiresolution STFT
      • Excitation Pattern
      • Nonuniform Spectral Resampling
      • Auditory Filter Shapes
      • Specific Loudness
      • Spectrograms Compared
      • Instantaneous, Short-Term, and Long-Term Loudness
  • Summary
  
  
• Overlap-Add STFT Processing
  • Convolution of Short Signals
    • Cyclic FFT Convolution
    • Acyclic FFT Convolution
      • Acyclic Convolution in Matlab
      • Pictorial View of Acyclic Convolution
    • Acyclic FFT Convolution in Matlab
    • FFT versus Direct Convolution
      • Audio FIR Filters
      • Example 1: Low-Pass Filtering by FFT Convolution
      • Example 2: Time Domain Aliasing
  • Convolving with Long Signals
    • Overlap-Add Decomposition
    • COLA Examples
    • STFT of COLA Decomposition
    • Acyclic Convolution
    • Example of Overlap-Add Convolution
    • Summary of Overlap-Add FFT Processing
  • Dual of Constant Overlap-Add
    • Poisson Summation Formula
    • Frequency-Domain COLA Constraints
      • Strong COLA
    • PSF Dual and Graphical Equalizers
    • PSF and Weighted Overlap Add
    • Example COLA Windows for WOLA
  • Overlap-Save Method
  • Time Varying OLA Modifications
    • Time-Varying STFT Modifications
    • Length L FIR Frame Filters
  • Weighted Overlap Add
    • WOLA Processing Steps
      • Choice of WOLA Window
  • Review of Zero Padding
  
  
• Filter Bank View of the STFT
  • Dual Views of the STFT
    • Overlap-Add View of the STFT
    • Filter Bank View of the STFT
    • FBS and Perfect Reconstruction
  • STFT Filter Bank
    • Computational Examples in Matlab
  • The DFT Filter Bank
    • The Running-Sum Lowpass Filter
    • Modulation by a Complex Sinusoid
    • Making a Bandpass Filter from a Lowpass Filter
    • Uniform Running-Sum Filter Banks
      • System Diagram of the Running-Sum Filter Bank
      • DFT Filter Bank
      • Inverse DFT and the DFT Filter Bank Sum
  • FBS Window Constraints for R=1
  • Nyquist(N) Windows
  • Duality of COLA and Nyquist Conditions
    • Specific Windows
    • The Nyquist Property on the Unit Circle
  • Portnoff Windows
  • Downsampled STFT Filter Banks
    • Downsampled STFT Filter Bank
      • Filter Bank Reconstruction
    • Downsampling with Anti-Aliasing
      • Properly Anti-Aliasing Window Transforms
      • Hop Sizes for WOLA
    • Constant-Overlap-Add (COLA) Cases
      • Hamming Overlap-Add Example
      • Periodic-Hamming OLA from Poisson Summation Formula
      • Kaiser Overlap-Add Example
  • STFT with Modifications
    • FBS Fixed Modifications
    • Time Varying Modifications in FBS
      • Points to Note
  • STFT Summary and Conclusions
  
  
• Applications of the STFT
  • Estimation from Spectral Peaks
    • Useful Preprocessing
    • Getting Closer to Maximum Likelihood
    • References on Estimation
  • Cross-Synthesis
  • Spectral Envelope Extraction
    • Cepstral Windowing
    • Linear Prediction Spectral Envelope
      • Linear Prediction is Peak Sensitive
      • Linear Prediction Methods
      • Computation of Linear Prediction Coefficients
      • Linear Prediction Order Selection
      • Summary of LP Spectral Envelopes
    • Spectral Envelope Examples
      • Signal Synthesis
      • Spectral Envelope by the Cepstral Windowing Method
      • Spectral Envelope by Linear Prediction
      • Linear Prediction in Matlab and Octave
  • Spectral Modeling Synthesis
    • Additive Synthesis (Early Sinusoidal Modeling)
    • Additive Synthesis Analysis
      • Following Spectral Peaks
      • Sinusoidal Peak Finding
      • Tracking Sinusoidal Peaks in a Sequence of FFTs
    • Sines + Noise Modeling
      • Sines+Noise Analysis
      • S+N Synthesis
      • Sines+Noise Summary
    • Sines + Noise + Transients Models
      • Sines+Noise+Transients Time-Frequency Maps
      • Sines+Noise+Transients Noise Model
    • S+N+T Sound Examples
  • Time Scale Modification
    • TSM and S+N+T
    • TSM by Resampling STFTs Across Time
    • Phase Continuation
    • TSM Examples
      • Phase-Continued STFT TSM
      • Relative-Phase-Preserving STFT TSM
      • SOLA-FS TSM
    • Further Reading
  • Gaussian Windowed Chirps (Chirplets)
    • Chirplet Fourier Transform
    • Modulated Gaussian-Windowed Chirp
    • Identifying Chirp Rate
      • Chirplet Frequency-Rate Estimation
      • Simulation Results
  • FFT Filter Banks
    • Audio Filter Banks
    • Basic Idea
    • Summing STFT Bins
    • Inverse Transforming STFT Bin Groups
    • Improving the Channel Filters
    • Fast Octave Filter Banks
    • Spectral Rotation of Real Signals
    • Improving the Octave Band Filters
    • Aliasing on Downsampling
    • Restricting Aliasing to Stop-Bands
      • Tightening the IFFTs
    • Real Filter Bank Example
    • Optimal Band Filters
  • FFT Filter-Bank Summary and Fourier Duality with OLA
  • Pointers to Sound Examples
  
  
• Multirate Filter Banks
  • Upsampling and Downsampling
    • Upsampling (Stretch) Operator
    • Downsampling (Decimation) Operator
      • Example: Downsampling by 2
      • Example: Upsampling by 2
    • Filtering and Downsampling
  • Polyphase Decomposition
    • Two-Channel Case
    • N-Channel Polyphase Decomposition
    • Type II Polyphase Decomposition
    • Filtering and Downsampling, Revisited
    • Multirate Noble Identities
  • Critically Sampled PR Filter Banks
    • Two-Channel Critically Sampled Filter Banks
    • Amplitude-Complementary 2-Channel Filter Bank
    • Haar Example
    • Polyphase Decomposition of Haar Example
    • Quadrature Mirror Filters (QMF)
    • Linear Phase Quadrature Mirror Filter Banks
    • Conjugate Quadrature Filters (CQF)
    • Orthogonal Two-Channel Filter Banks
  • Perfect Reconstruction Filter Banks
    • Simple Examples of Perfect Reconstruction
    • Sliding Polyphase Filter Bank
    • Hopping Polyphase Filter Bank
    • Sufficient Condition for Perfect Reconstruction (PR)
    • Necessary and Sufficient Conditions for PR
    • Polyphase View of the STFT
    • Polyphase View of the Overlap-Add STFT
    • Polyphase View of the Weighted-Overlap-Add STFT
  • Paraunitary Filter Banks
    • Lossless Filters
    • Lossless Filter Examples
    • Properties of Paraunitary Filter Banks
    • Examples
  • Filter Banks Equivalent to STFTs
    • Polyphase Analysis of Portnoff STFT
  • MPEG Filter Banks
    • Pseudo-QMF Cosine Modulation Filter Bank
    • PR Cosine Modulated Filter Banks
    • MPEG Layer III Filter Bank
  • Review of STFT Filterbanks
    • STFT, Rectangular Window, No Overlap
    • STFT, Rectangular Window, 50% Overlap
    • STFT, Triangular Window, 50% Overlap
    • STFT, Hamming Window, 75% Overlap
    • STFT, Kaiser Window, Beta=10, 90% Overlap
    • Sliding FFT (Maximum Overlap), Any Window, Zero-Padded by 5
  • Wavelet Filter Banks
    • Geometric Signal Theory
      • Natural Basis
      • Normalized DFT Basis for
      • Normalized Fourier Transform Basis
      • Normalized DTFT Basis
      • Normalized STFT Basis
      • Continuous Wavelet Transform
      • Discrete Wavelet Transform
      • Discrete Wavelet Filterbank
      • Dyadic Filter Banks
      • Dyadic Filter Bank Design
      • Generalized STFT
  • Further Reading
  • Conclusions
  
  
• Summary and Conclusions
• Notation
  • Frequency and Time
  • Signal Notation
  • Fourier Transform Notation
  
  
• Continuous Fourier Theorems
  • Radians versus Cycles
  • Differentiation Theorem
  • Differentiation Theorem Dual
  • Scaling Theorem
  • Shift Theorem
  • Modulation Theorem (Shift Theorem Dual)
  • Convolution Theorem
  • Flip Theorems
  • Power Theorem
  • The Continuous-Time Impulse
  • Gaussian Pulse
  • Rectangular Pulse
  • Sinc Impulse
  • Impulse Trains
  • Poisson Summation Formula
  • Sampling Theory
  • The Uncertainty Principle
    • Second Moments
    • Time-Limited Signals
    • Time-Bandwidth Products Unbounded Above
  • Relation of Smoothness to Roll-Off Rate
  
  
• Statistical Signal Processing
  • Stochastic Processes
    • Probability Distribution
    • Independent Events
    • Random Variable
    • Stochastic Process
    • Stationary Stochastic Process
    • Expected Value
    • Mean
    • Sample Mean
    • Variance
    • Sample Variance
  • Correlation Analysis
    • Cross-Correlation
    • Cross-Power Spectral Density
    • Autocorrelation
    • Sample Autocorrelation
    • Power Spectral Density
    • Sample Power Spectral Density
  • White Noise
    • Making White Noise with Dice
    • Independent Implies Uncorrelated
    • Estimator Variance
      • Sample-Mean Variance
      • Sample-Variance Variance
  
  
• Gaussian Function Properties
  • Gaussian Window and Transform
  • Gaussians Closed under Multiplication
    • Product of Two Gaussian PDFs
  • Gaussians Closed under Convolution
  • Fitting a Gaussian to Data
  • Infinite Flatness at Infinity
  • Integral of a Complex Gaussian
    • Area Under a Real Gaussian
  • Gaussian Integral with Complex Offset
  • Fourier Transform of Complex Gaussian
    • Alternate Proof
  • Why Gaussian?
    • Central Limit Theorem
    • Iterated Convolutions
    • Binomial Distribution
  • Gaussian Probability Density Function
  • Maximum Entropy Property
    • Entropy of a Probability Distribution
    • Example: Random Bit String
    • Maximum Entropy Distributions
      • Uniform Distribution
      • Exponential Distribution
      • Gaussian Distribution
  • Gaussian Moments
    • Gaussian Mean
    • Gaussian Variance
    • Higher Order Moments Revisited
    • Moment Theorem
    • Gaussian Characteristic Function
    • Gaussian Central Moments
  • Sums of Gaussian Random Variables
  
  
• Bilinear Audio Frequency Warping
  • The Bark Frequency Scale
  • The Bilinear Transform
  • Optimal Bilinear Bark Warping
    • Computing
    • Optimal Frequency Warpings
    • Bark Relative Bandwidth Mapping Error
    • Error Significance
    • Arctangent Approximations for
  • Application to Audio Filter Design
    • Filter Design Example
  • Equivalent Rectangular Bandwidth
    • ERB Relative Bandwidth Mapping Error
    • Arctangent Approximations for , ERB Case
  • Directions for Improvements
  • Summary
  
  
• Examples in Matlab and Octave
  • Matlab for Spectrum Analysis Windows
    • Blackman Window Example
    • Matlab listing: dpssw.m
  • Interpolating Spectral Peaks
    • Matlab listing: findpeaks.m
    • Matlab listing: maxr.m
    • Matlab listing: qint.m
    • Matlab listing: zpfmin.m
    • Matlab listing: oboeanal.m
  • Matlab for Computing Spectrograms
    • Matlab listing: myspectrogram.m
    • Matlab listing: invmyspectrogram.m
    • Matlab listing: testmyspectrogram.m
  • Matlab for Unwrapping Spectral Phase
    • Matlab listing: unwrap.m
  • Non-Parametric Frequency Warping
  • Fundamental Frequency Estimation
    • Test Program for F0 Estimation
  
  
• Spectral Audio Modeling History
  • Daniel Bernoulli's Modal Decomposition
  • The Telharmonium
  • Early Additive Synthesis in Film Making
  • The Hammond Organ
  • Dudley's Channel Vocoder
    • Speech Synthesis Examples
  • Voder
  • Phase Vocoder
    • FFT Implementation of the Phase Vocoder
  • Additive Synthesis
    • Inverse FFT Synthesis
    • Chirplet Synthesis
    • Nonparametric Spectral Peak Modeling
    • Efficient Specialized Methods
      • Wavetable Synthesis
      • Group-Additive Synthesis
    • Further Reading, Additive Synthesis
  • Frequency Modulation (FM) Synthesis
    • FM Harmonic Amplitudes
    • FM Brass
    • FM Voice
      • Further Reading about FM Synthesis
  • Phase Vocoder Sinusoidal Modeling
    • Computing Vocoder Parameters
      • Frequency Envelopes
      • Envelope Compression
      • Vocoder-Based Additive-Synthesis Limitations
    • Further Reading on Vocoders
  • Spectral Modeling Synthesis
    • Short-Term Fourier Analysis/Modification/Resynth.
    • Sinusoidal Modeling Systems
    • Inverse FFT Synthesis
    • Sines+Noise Synthesis
    • Multiresolution Sinusoidal Modeling
    • Transient Models
    • Time-Frequency Reassignment
    • Perceptual Audio Compression
    • Further Reading
  • Perceptual audio coding
  • Future Prospects
  • Summary
  
  
• The PARSHL Program
  • Choice of Hop Size
  • Filling the FFT Input Buffer (Step 2)
  • Peak Detection (Steps 3 and 4)
  • Peak Matching (Step 5)
  • Parameter Modifications (Step 6)
  • Synthesis (Step 7)
  • Magnitude-only Analysis/Synthesis
  • Preprocessing
  • Applications
  • Conclusions
  • Acknowledgments
  • Software Listing
  
  
• Bibliography
• Index for this Document
• About this document ...

Sign in