Coding a software synthesizer

(Sorry, this post is a bit long)

Hi,

About a week ago I decided to create a software synthesizer, as a
learning experience, and because it seems to be a hard enough problem
to be rewarding to work on. I have years of programming experience so
the "software" part is not a problem. So far, so good right?

Well, the problems are three, or four if you count the fact I never
went to college (= not formally introduced to [complex]analysis):

1) I know very little about synthesizers.
2) I know very little about DSP.
3) I am not a musician, either (although I am trying to learn how to
play piano, and that's where I got this idea)

So far I've managed learn bits and pieces, enough to create a silly
little program that understands MIDI and can output PCM WAV,
synthesizing sounds with a non-bandlimited sawtooth oscillator and a
low-pass filter I found on the net. It sounds horrible, but at least
it's a horrible creation of my own! 

(Don't be fooled by the tricky words in the above paragraph. I learned
the meaning of the word band-limited a few days ago, and the only
reason my program has a low pass filter is because a musician friend
told me a synth needs one!).

I don't really know what to do now though, and this is why I'm writing
this. I have read a few chapters from the the book at dspguide.com.
I've been thinking about ordering Richard Lyons book recommended in
this newsgroup. I have read Wikipedia, talked with DSP-people in
#musicdsp at EFNet, familiarized myself with a few synthesis concepts
(additive, subtractive etc). I have downloaded papers by Julius Smith
from his website, papers that sound useful but which I don't
understand,, and I've browsed an online course about music programming
at McGill Universities webpage. 

But I still can't tell the important bits of knowledge from the
unimportant ones, and my knowledge is very fragmented at the moment. I
have learnt lots of tricky words and can combine them to create very
stupid questions ("What windowing functions does the Minimoog use?"),
but that's hardly useful. :-)

Here are my questions:

1) How much DSP-and-synthesis-knowledge do I need to create a software
synthesizer powerful enough to sound like Kraftwerk, Laserdance etc
(As an ignorant beginner, I here assume they use _simple_ synthesizers
which are fairly easy to implement by those interested in DSP, but if
they have some qualities which makes them very hard to implement, even
for the DSP-gurus in here, I appologize!)

2) What's the best way to acquire said knowledge with my specific goal
in mind? My goal is to create a software synthesizer I understand how
it works and why. Are there any good books especially about DSP and
synths, or should I get a general book about DSP (such as Lyons) and
read about synthesis elsewhere? If so, any recommendations?

Any pointers in the right direction would be really nice!

Best regards,
Bj�rn

http://musicdsp.org/ has sample code and a forum.  

Jeff

Bjorn Edstrom <be@bjrn.se> writes:

> Here are my questions:
> 
> 1) How much DSP-and-synthesis-knowledge do I need to create a software
> synthesizer powerful enough to sound like Kraftwerk, Laserdance etc
> (As an ignorant beginner, I here assume they use _simple_ synthesizers
> which are fairly easy to implement by those interested in DSP, but if
> they have some qualities which makes them very hard to implement, even
> for the DSP-gurus in here, I appologize!)

Start simple.  Once you understand that (and it doesn't work the way
you like it), try understanding why it doesn't work the way you like
it.

> 2) What's the best way to acquire said knowledge with my specific goal
> in mind? My goal is to create a software synthesizer I understand how
> it works and why. Are there any good books especially about DSP and
> synths, or should I get a general book about DSP (such as Lyons) and
> read about synthesis elsewhere? If so, any recommendations?

You're starting it the right way: try doing something.  You probably
won't get very far, at least initially, but it sounds like you're
starting to ask the right questions,

Rick's book is an excellent backgrounder to DSP; I don't think there's
much in it about Music for DSP.

Julius Smith at Stanford has some links from his home page that might
be useful:

http://ccrma.stanford.edu/~jos/

Specifically the course page "Signal Processing Methods in Musical
Acoustics" might be of interest.

http://ccrma.stanford.edu/courses/420/Welcome.html

Ask more questions! :-)

Ciao,

Peter K.

-- 
"And he sees the vision splendid 
of the sunlit plains extended
And at night the wondrous glory of the everlasting stars."

 

On 31 Dec 2005 22:22:15 -0500, p.kootsookos@remove.ieee.org (Peter K.)
wrote:

>You're starting it the right way: try doing something.  You probably
>won't get very far, at least initially, but it sounds like you're
>starting to ask the right questions,
>
>Rick's book is an excellent backgrounder to DSP; I don't think there's
>much in it about Music for DSP.

Thanks for your reply!

I have searched around the group archive and found a few references to
books with DSP /and/ audio content that seems interesting. Don't know
if the extra info is useful for creating synths though, I'm beginning
to feel there's so much more to DSP and audio than synthesis.

A Digital Signal Processing Primer
Ken Steiglitz
0805316841

Introduction to Signal Processing
Sophocles Orfanidis
0132091720
(above book gets bonus points because the author has a really cool
name!)

The two above and Richard Lyons book all sound good though, so this
will be a tricky decision!

>Julius Smith at Stanford has some links from his home page that might
>be useful:
>
>http://ccrma.stanford.edu/~jos/
>
>Specifically the course page "Signal Processing Methods in Musical
>Acoustics" might be of interest.
>
>http://ccrma.stanford.edu/courses/420/Welcome.html

Ahh, interesting. Appears the article assumes prior knowledge of DSP
though (per the preface). I feel I have some reading to do. :-) 

In the meantime, here are some more specific questions so I can keep
myself busy coding while the book(s) arrives (oh, the joy of not
having a technical enough bookstore nearby): Please excuse my sloppy
use of the word "filter".

1) I assume the typical real world synthesizer doesn't have any
knowledge of what will happen in the future, and can do it's
wave-modifying-tricks by using past values only? Or are there
instances where you _need_ knowledge of the future (such as future
samples from an oscillator) to synthesize a sound correctly? If this
is the case, I assume some sounds cannot be synthesized in real-time
and must be "played back" from pre-genereted chunks of data? (I mean
this theoretically, in practice it's probably a good idea to
pregenerate data sometimes for performance reasons).

2) If 1 is true (don't need knowledge from the future) I'd imagine
this would lead to the nice property that all wave-modifying-code only
modifies one sample at a time, and are therefore stackable, as in 
	newsample = filter1(params1,filter2(params2,oldsample)); ?
I also assume this would mean synthesis in general consists of lots of
small building blocks (lowpass, allpass, oscillators, other things)
combined in interesting ways?

3) I read somewhere most synthesizers are modular and above all
parallell, because parameters to filters can change with time as the
output (for example, a filtered oscillator can change by updating it's
parameters with values from another oscillator). Okay, here's the
important part:

Can a software synthesizer then render a complete instrument playing
in one go, assuming the parameters for the "instruments" are known?
For example, say I connect a MIDI keyboard to the synth, then I can
have a render loop that writes WAV-data to a file:

// some kind of driver loop, don't pay to much attention to this, I am
// only trying to capture the "feel" of the program here
while Playing:
    check for new midi event and add to queue if one exists
    sample = 0 // amplitude
    for each midi_event in queue:
        if midi_action == noteon:
            init instrument (this is probably complex)
            modify sample accordingly
            modify instrument accordingly
        else if midi_event == noteoff:
            reset instrument
            modify sample accordingly
            remove action from queue
    // will append a number to the end of a PCM WAV file
    output(sample)

Or is it just wishful thinking being able to "drive" a whole synth in
one loop, doing one sample at a time in a linear fashion? Above all,
does software synths work like this generally, or do they work more
like trackers, that is you have pregenerated samples you modifiy and
add into a "main channel"?

In the Synthesis ToolKit, you do pretty much everything with the
tick() function, and one tick equals one sample (you can then have one
seconds of CD sounds in 44100 ticks). This is what I've been getting
at for the last three questions: Can you build a complete synth with a
general assortment of functions and use them on a sample-by-sample
basis?

(Phew)

Okay, here are some easier, less verbose technical questions I can't
find the answers to. 

4) What's a good internal representation for waves if my target output
is 44.1 khz? I currently use -32000 to +32000 but I'm thinking of
changing this to double precision floats between -1.0 and 1.0. Is this
a good idea? "Mathematintuitively", pi to -pi seems to be a pure
choice too. :-)

5) Okay, this must be a common one, but what are the common methods
for bandlimiting common waveforms (sawtooth, square etc)? Intuitively
I want to use a low pass filter on my naive oscillators but I hear
this won't work at 22 khz :-) I have read mostly everything here:
http://musicdsp.org/archive.php?classid=1 but I still can't tell the
"schoolbook ways" from the more advanced techniques.

6) Othar than bandlimited oscillators, what does a synth really need?
I know of lowpass, hipass, bandpass, allpass, but that's pretty  much
it, unfortunately. I have read about ADSR-envelopes, but I assume they
are created by combining more basic filters/functions?

Thanks for insight!

Bj�rn

Bjorn Edstrom <be@bjrn.se> writes:

> On 31 Dec 2005 22:22:15 -0500, p.kootsookos@remove.ieee.org (Peter K.)
> wrote:
> 
> Thanks for your reply!
> 

No worries.

> I have searched around the group archive and found a few references to
> books with DSP /and/ audio content that seems interesting. Don't know
> if the extra info is useful for creating synths though, I'm beginning
> to feel there's so much more to DSP and audio than synthesis.
> 
> A Digital Signal Processing Primer
> Ken Steiglitz
> 0805316841
> 
> Introduction to Signal Processing
> Sophocles Orfanidis
> 0132091720
> (above book gets bonus points because the author has a really cool
> name!)
> 
> The two above and Richard Lyons book all sound good though, so this
> will be a tricky decision!

I vote for Rick's book.  I don't gain any financial benefit from it,
but Rick posts here quite a bit and I did a minor amount of reviewing
the manuscript for the second edition.

> Ahh, interesting. Appears the article assumes prior knowledge of DSP
> though (per the preface). I feel I have some reading to do. :-) 

:-) Probably.

> In the meantime, here are some more specific questions so I can keep
> myself busy coding while the book(s) arrives (oh, the joy of not
> having a technical enough bookstore nearby): Please excuse my sloppy
> use of the word "filter".
> 
> 1) I assume the typical real world synthesizer doesn't have any
> knowledge of what will happen in the future, and can do it's
> wave-modifying-tricks by using past values only? Or are there
> instances where you _need_ knowledge of the future (such as future
> samples from an oscillator) to synthesize a sound correctly? If this
> is the case, I assume some sounds cannot be synthesized in real-time
> and must be "played back" from pre-genereted chunks of data? (I mean
> this theoretically, in practice it's probably a good idea to
> pregenerate data sometimes for performance reasons).

Sure. You won't be able to implement something that requires future
values, unless you can count on an appropriate time-delay.

> 2) If 1 is true (don't need knowledge from the future) I'd imagine
> this would lead to the nice property that all wave-modifying-code only
> modifies one sample at a time, and are therefore stackable, as in 
> 	newsample = filter1(params1,filter2(params2,oldsample)); ?

Yes, cascades of filters are often a good way to implement higher
order filters by, for example, cascading several second order
sub-systems.

However, sometimes doing more than one sample at a time is better
computationally. Do a google search for "overlap-add" or
"overlap-save".

> I also assume this would mean synthesis in general consists of lots of
> small building blocks (lowpass, allpass, oscillators, other things)
> combined in interesting ways?

It's generally a good idea to build complicated things up out of
simple building blocks, and you can do it this way.

> 3) I read somewhere most synthesizers are modular and above all
> parallell, because parameters to filters can change with time as the
> output (for example, a filtered oscillator can change by updating it's
> parameters with values from another oscillator). Okay, here's the
> important part:
> 
> Can a software synthesizer then render a complete instrument playing
> in one go, assuming the parameters for the "instruments" are known?

Yes, I believe so.

> For example, say I connect a MIDI keyboard to the synth, then I can
> have a render loop that writes WAV-data to a file:
> 
> // some kind of driver loop, don't pay to much attention to this, I am
> // only trying to capture the "feel" of the program here
> while Playing:
>     check for new midi event and add to queue if one exists
>     sample = 0 // amplitude
>     for each midi_event in queue:
>         if midi_action == noteon:
>             init instrument (this is probably complex)
>             modify sample accordingly
>             modify instrument accordingly
>         else if midi_event == noteoff:
>             reset instrument
>             modify sample accordingly
>             remove action from queue
>     // will append a number to the end of a PCM WAV file
>     output(sample)
> 
> Or is it just wishful thinking being able to "drive" a whole synth in
> one loop, doing one sample at a time in a linear fashion? 

Like I said, you might be able to do that on fast enough processor,
but it might be easier to use something like overlap-add/save.

> Above all,
> does software synths work like this generally, or do they work more
> like trackers, that is you have pregenerated samples you modifiy and
> add into a "main channel"?

I really couldn't say.

> In the Synthesis ToolKit, you do pretty much everything with the
> tick() function, and one tick equals one sample (you can then have one
> seconds of CD sounds in 44100 ticks). This is what I've been getting
> at for the last three questions: Can you build a complete synth with a
> general assortment of functions and use them on a sample-by-sample
> basis?

You probably can.

Caveat: I haven't done what you're asking about, but I do have some
experience at DSP on PCs.

> Okay, here are some easier, less verbose technical questions I can't
> find the answers to. 
> 
> 4) What's a good internal representation for waves if my target output
> is 44.1 khz? I currently use -32000 to +32000 but I'm thinking of
> changing this to double precision floats between -1.0 and 1.0. Is this
> a good idea? "Mathematintuitively", pi to -pi seems to be a pure
> choice too. :-)

It depends on your implementation platform and technique... 

> 5) Okay, this must be a common one, but what are the common methods
> for bandlimiting common waveforms (sawtooth, square etc)?
> Intuitively I want to use a low pass filter on my naive oscillators
> but I hear this won't work at 22 khz :-) I have read mostly
> everything here: http://musicdsp.org/archive.php?classid=1 but I
> still can't tell the "schoolbook ways" from the more advanced
> techniques.

Why won't it work at 22kHz?

> 6) Othar than bandlimited oscillators, what does a synth really need?
> I know of lowpass, hipass, bandpass, allpass, but that's pretty  much
> it, unfortunately. 

Have a look at the wavetable notes by rb-j (a poster here) on the
musicdsp.org size you posted.

> I have read about ADSR-envelopes, but I assume they
> are created by combining more basic filters/functions?

ADSR is more a time-domain description of what the waveform looks
like. AFAIK it's an older way to obtain different synth effects.

Ciao,

Peter K.

Peter K. wrote:

> However, sometimes doing more than one sample at a time is better
> computationally. Do a google search for "overlap-add" or
> "overlap-save".

I'm a bit confused by this statement. Do you mean that the OP might
want to consider doing their audio generation and/or
processing/filtering in the frequency domain, rather than the time
domain?

Cheers,

Ross-c

"Ross Clement (Email address invalid - do not use)" <clemenr@wmin.ac.uk> writes:

> I'm a bit confused by this statement. Do you mean that the OP might
> want to consider doing their audio generation and/or
> processing/filtering in the frequency domain, rather than the time
> domain?

Yes. It depends on how complex the processing needs to be.  For simple
stuff, the time domain is OK, but for more complex filtering /
generation, they might want to investigate the frequency domain.

Ciao,

Peter K.

-- 
"And he sees the vision splendid 
of the sunlit plains extended
And at night the wondrous glory of the everlasting stars."

 

Ross Clement (Email address invalid - do not use) wrote:
> Peter K. wrote:
> 
> 
>>However, sometimes doing more than one sample at a time is better
>>computationally. Do a google search for "overlap-add" or
>>"overlap-save".
> 
> 
> I'm a bit confused by this statement. Do you mean that the OP might
> want to consider doing their audio generation and/or
> processing/filtering in the frequency domain, rather than the time
> domain?

Building a sound up from its constituent frequency components as a good 
a way as any to to "compose" a timbre. Mixing oscillators together is a 
well established synthesis method. Witness the star-wheel Hammond organ.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

Thanks. I can see how this form of synthesis would work for linear
combinations and filtering of audio, including full additive synthesis.
But, wouldn't it be difficult/expensive if you wanted to create a
waveform by the frequency or ring modulation of complex timbres?

Cheers,

Ross-c

Bjorn Edstrom wrote:

> Please excuse my sloppy use of the word "filter".

By itself, that term tends to be understood as "linear filter", i.e., 
an FIR or IIR process. "System" or "process" are general terms.

> 1) I assume the typical real world synthesizer doesn't have any
> knowledge of what will happen in the future, and can do it's
> wave-modifying-tricks by using past values only?

The overall synth will be causal, but subsystems can still use 
lookahead. It is a matter of description: Consider a summing node 
with inputs from subsystems A and B where A uses n samples lookahead. 
Then the implementation will *delay* the input to B by n samples and 
accept the resulting summing output delay. This output delay is 
called the processing latency of the total system. Too long latency 
makes a synthesizer unfit for live play.

> Or are there instances where you _need_ knowledge of the future
> (such as future samples from an oscillator) to synthesize a sound
> correctly?

For example, finding signal values between sampling instants by 
interpolation is a fairly common operation that usually requires 
samples on both sides of the target instant. Those samples may come 
from a table like in the wavetable and sample-playback synthesis 
methods; otherwise the overall system is forced causal as I described 
above.

> 2) If 1 is true (don't need knowledge from the future) I'd
> imagine this would lead to the nice property that all wave-
> modifying-code only modifies one sample at a time, and are
> therefore stackable, as in 
>      newsample = filter1(params1,filter2(params2,oldsample)); ?

Systems based on the DFT or a similar transform are inherently block-
based. You are right for systems that work wholly in the time domain, 
but they too often act on blocks for performance reasons. There is a 
tradeoff between data locality on one hand, function-call overhead or 
(with inlining) code locality on the other hand.

> I also assume this would mean synthesis in general consists of
> lots of small building blocks (lowpass, allpass, oscillators,
> other things) combined in interesting ways?

Yes. Play with a graphical synthesis language like Csound 
(http://www.csounds.com) or pure data (http://puredata.org).

> 3)
> Or is it just wishful thinking being able to "drive" a whole
> synth in one loop, doing one sample at a time in a linear
> fashion?

That is possible. I presume your concern here is control latency, 
i.e., the difference between an event's time of origin and the 
instant the synth consumes it? The synth's total latency is the sum 
of its control and processing latencies, and small non-minimal 
latency is benign in many uses. So you might gain performance by 
processing and outputting either fixed-length blocks, parsing events 
in between; or, more complex but more accurate, blocks that reach up 
either to the next event or to a set maximum length. But I think you 
need not worry about those options until later.

> 4) What's a good internal representation for waves if my target
> output is 44.1 khz? I currently use -32000 to +32000 but I'm
> thinking of changing this to double precision floats between
> -1.0 and 1.0.

Floating point is easier to use than fixed point, and [-1, 1] is a 
good nominal range. Note that internal signals may legitimately 
exceed it.

> 5)
> I have read mostly everything here: 
> http://musicdsp.org/archive.php?classid=1 but I still can't
> tell the "schoolbook ways" from the more advanced techniques.

I would say that of the techniques mentioned there, sinusoidal 
oscillators, wavetable synthesis and oversampling naive oscillators 
("Arbitrary-shaped...") are basic.

> 6) Othar than bandlimited oscillators, what does a synth really
> need? I know of lowpass, hipass, bandpass, allpass, but that's
> pretty  much it, unfortunately.

Lowpass, highpass, bandpass, envelopes are essential; notch and comb 
filters, LFO's, noise oscillators are nice to have. Allpass is 
probably unneeded. Oscillator sync and ring modulation are cool 
goodies.

> I have read about ADSR-envelopes, but I assume they are created
> by combining more basic filters/functions? 

An envelope generator is a kind of oscillator whose waveform is a 
piecewise polynomial and/or exponential function. You can generate an 
exponential piece recursively with what is basically a onepole 
lowpass filter.


Martin

-- 
Teach a man to make fire, and he will be
warm for a day. Set a man on fire, and he
will be warm for the rest of his life.
--John A. Hrastar