Cross Synthesis

Cross-synthesis is generally concerned with impressing the spectral envelope of one sound on the flattened spectrum of another. Let's call the first signal the ``modulating'' signal, and the other the ``carrier'' signal. A classic example is for the modulator to be voice and the carrier to be any spectrally rich sound such as wind, rain, creaking noises, flute, or other musical instrument sound. Commercial ``vocoders'' used as musical instruments consist of a keyboard synthesizer (the carrier sounds) with a microphone for picking up the voice of the performer (to extract the modulation envelope). An example of cross-synthesis in computer music is ``talking wind.''

Cross-synthesis may be summarized as consisting of the following steps:

1. Perform a Short-Time Fourier Transform (STFT) of both the modulator and carrier signals (§6.1).
2. Compute the spectral envelope of each time-frame (see next section below).
3. Divide the spectrum of each carrier frame by its own envelope, thereby flattening it.
4. Multiply the flattened spectral frame by the envelope of the corresponding modulator frame, thereby replacing the carrier's envelope by the modulator's envelope.

The next section discusses methods for spectral envelope estimation.

---

Order a Hardcopy of Spectral Audio Signal Processing

Previous: Applications of the STFT
Next: Spectral Envelope Extraction

---

written by Julius Orion Smith III
Julius Smith's background is in electrical engineering (BS Rice 1975, PhD Stanford 1983). He is presently Professor of Music and Associate Professor (by courtesy) of Electrical Engineering at Stanford's Center for Computer Research in Music and Acoustics (CCRMA), teaching courses and pursuing research related to signal processing applied to music and audio systems. See http://ccrma.stanford.edu/~jos/ for details.