Time-Scale Modification of Sinusoidal Models

Sinusoidal models naturally support time-scale modification (TSM) (and frequency-scaling, its Fourier dual), because the original signal is replaced by oscillator amplitude and frequency envelopes which are easily time-scaled without causing unnatural artifacts. When amplitude or frequency envelopes are rescaled in time, the oscillators are allowed to run continuously under them, thereby avoiding artifacts. Similarly, in sines-plus-noise synthesis, the time-varying noise-filter is a time-frequency envelope that can be smoothly rescaled along either dimension.

When time-stretching, say, a sines+noise model, transients are ``smeared out'' over time. In sines+noise+transients models, transients may be time-shifted instead. However, while this sounds very good when it works, it can be difficult to detect and preserve each and every transient in the signal. In fact, it can become difficult to define exactly what is meant by the term ``transient''. For example, an abrupt cello note onset is naturally defined as a starting transient for the note. If the note onset is then played in a more and more legato style, when is there no longer a starting transient? One idea is to define ``transientness'' as a variable between 0 and 1, so that the degree of TSM ``stretchiness'' for that time interval can be multiplied by (1-transientness).

Audio demonstrations of TSM and frequency-scaling based on the sines+noise+transients model of Scott Levine may be found online at
http://ccrma.stanford.edu/~jos/pdf/SMS.pdf .

See §9.9 for additional details.

---

Previous: Perceptual Audio Compression
Next: Further Reading

Order a Hardcopy of Spectral Audio Signal Processing

---

About the Author: Julius Orion Smith III
Julius Smith's background is in electrical engineering (BS Rice 1975, PhD Stanford 1983). He is presently Professor of Music and Associate Professor (by courtesy) of Electrical Engineering at Stanford's Center for Computer Research in Music and Acoustics (CCRMA), teaching courses and pursuing research related to signal processing applied to music and audio systems. See http://ccrma.stanford.edu/~jos/ for details.