Source separation of harmonic sounds when f0 and harmonics are given??

I have a collection of sound files (mono, 44100, 16bit) where two
harmonic musical instruments are playing together, e.g. cello and
flute.

For each frame of the sound (4096 samples, hop=1024), I have the f0's
and the harmonic frequencies (not their amplitudes) already known for
both sounds.

Can anybody recommend me of a good way to separate the sounds, i.e.
produce separate wavs for both instruments?

Since there inevitably will be overlapping frequencies, there's no way
to separate these sounds in the frequency domain without considerable
ambiguity.

Check out: Klapuri. "Number Theoretical Means of Resolving a Mixture
of Several Harmonic Sounds". Proceedings of the European Signal
Processing Conference EUSIPCO, 1998. to get some ideas.

--smb

Are the notes they playing "harmonically related"  i.e. is the fo of the
flute a mutliple of the fo of the cello?

If they are not related, some adaptive filter approaches may be possible.

-Shawn Steenhagen
Applied Signal Processing


"Arie" <allen@alum.cs.huji.ac.il> wrote in message
news:64a73096.0407031424.2581bcfa@posting.google.com...
> I have a collection of sound files (mono, 44100, 16bit) where two
> harmonic musical instruments are playing together, e.g. cello and
> flute.
>
> For each frame of the sound (4096 samples, hop=1024), I have the f0's
> and the harmonic frequencies (not their amplitudes) already known for
> both sounds.
>
> Can anybody recommend me of a good way to separate the sounds, i.e.
> produce separate wavs for both instruments?

Stephan:
Thanks for the article, but it deals with ways to find notes in a
sinusoid mixture. I already have the approximate harmonic frequencies
of the notes (although not their exact amplitudes) in the duet musical
pieces, and I am interested in ways to remove the frequencies of one
note while retaining the other (source separation while the f0 and
harmonics are already known).

Shawn:
Some of the notes are harmonically related, but most are not.
I would be happy if you could tell me more about these adaptive filter
approaches you mentioned.

Thanks,
Arie.

"Shawn Steenhagen" <shawn.steenhagen@NOSPAMappliedsignalprocessing.com> wrote in message news:<ccf3rk$gnc$1@grandcanyon.binc.net>...
> Are the notes they playing "harmonically related"  i.e. is the fo of the
> flute a mutliple of the fo of the cello?
> 
> If they are not related, some adaptive filter approaches may be possible.
> 
> -Shawn Steenhagen
> Applied Signal Processing
> 
> 
> "Arie" <allen@alum.cs.huji.ac.il> wrote in message
> news:64a73096.0407031424.2581bcfa@posting.google.com...
> > I have a collection of sound files (mono, 44100, 16bit) where two
> > harmonic musical instruments are playing together, e.g. cello and
> > flute.
> >
> > For each frame of the sound (4096 samples, hop=1024), I have the f0's
> > and the harmonic frequencies (not their amplitudes) already known for
> > both sounds.
> >
> > Can anybody recommend me of a good way to separate the sounds, i.e.
> > produce separate wavs for both instruments?

In article <64a73096.0407031424.2581bcfa@posting.google.com>,
Arie <allen@alum.cs.huji.ac.il> wrote:
(snip)
>For each frame of the sound (4096 samples, hop=1024), I have the f0's
>and the harmonic frequencies (not their amplitudes) already known for
>both sounds.
(sinp)

Gosh, how did you do that?

-- Mark

As I said, this is difficult to do because even if the sounds are not
harmonically related there will most certainly be overlapping
harmonics, which makes a solution ambiguous if you're working in the
frequency domain.

--smb

Check out the posts/thread on "60Hz Hum", it disusses the adaptive filter
approach.  Basically your input (or reference) signal to the adaptive filter
are the sinusoids of the known frequencies, the "desired signal" is the
recorded sound, the output of the adaptive filter will adapt to best match
the amplitude and phase of each harmonic component in the desired
signal.that appears in the reference signal.  As in the 60Hz hum thread, I
recomend two-tap quadrature adaptive filters, one for each known frequency.
There's an excellent book by "Quo" on adaptive filter applications (don't
remember the name of the book off hand).

-Shawn

"Arie" <allen@alum.cs.huji.ac.il> wrote in message
news:64a73096.0407070855.150f4da1@posting.google.com...
> Stephan:
> Thanks for the article, but it deals with ways to find notes in a
> sinusoid mixture. I already have the approximate harmonic frequencies
> of the notes (although not their exact amplitudes) in the duet musical
> pieces, and I am interested in ways to remove the frequencies of one
> note while retaining the other (source separation while the f0 and
> harmonics are already known).
>
> Shawn:
> Some of the notes are harmonically related, but most are not.
> I would be happy if you could tell me more about these adaptive filter
> approaches you mentioned.
>
> Thanks,
> Arie.
>
> "Shawn Steenhagen" <shawn.steenhagen@NOSPAMappliedsignalprocessing.com>
wrote in message news:<ccf3rk$gnc$1@grandcanyon.binc.net>...
> > Are the notes they playing "harmonically related"  i.e. is the fo of the
> > flute a mutliple of the fo of the cello?
> >
> > If they are not related, some adaptive filter approaches may be
possible.
> >
> > -Shawn Steenhagen
> > Applied Signal Processing
> >
> >
> > "Arie" <allen@alum.cs.huji.ac.il> wrote in message
> > news:64a73096.0407031424.2581bcfa@posting.google.com...
> > > I have a collection of sound files (mono, 44100, 16bit) where two
> > > harmonic musical instruments are playing together, e.g. cello and
> > > flute.
> > >
> > > For each frame of the sound (4096 samples, hop=1024), I have the f0's
> > > and the harmonic frequencies (not their amplitudes) already known for
> > > both sounds.
> > >
> > > Can anybody recommend me of a good way to separate the sounds, i.e.
> > > produce separate wavs for both instruments?

I am working with Chunghsin Yeh of Ircam, who is responsible for the
multiple-f0 estimation.

His next article: "A new score function for joint evaluation of
multiple F0  hypotheses" is going to be presented in Dafx'04.

Arie.

Mark Carroll <markc@chiark.greenend.org.uk> wrote in message news:<W+E*YfXoq@news.chiark.greenend.org.uk>...
> In article <64a73096.0407031424.2581bcfa@posting.google.com>,
> Arie <allen@alum.cs.huji.ac.il> wrote:
> (snip)
> >For each frame of the sound (4096 samples, hop=1024), I have the f0's
> >and the harmonic frequencies (not their amplitudes) already known for
> >both sounds.
> (sinp)
> 
> Gosh, how did you do that?
> 
> -- Mark

So, your part is removing/splitting the sounds and you have no clue
how to go about it?

--smb


allen@alum.cs.huji.ac.il (Arie) wrote in message news:<64a73096.0407101635.327c6036@posting.google.com>...
> I am working with Chunghsin Yeh of Ircam, who is responsible for the
> multiple-f0 estimation.
> 
> His next article: "A new score function for joint evaluation of
> multiple F0  hypotheses" is going to be presented in Dafx'04.
> 
> Arie.

allen@alum.cs.huji.ac.il (Arie) wrote in message news:<64a73096.0407031424.2581bcfa@posting.google.com>...
> I have a collection of sound files (mono, 44100, 16bit) where two
> harmonic musical instruments are playing together, e.g. cello and
> flute.
> 
> For each frame of the sound (4096 samples, hop=1024), I have the f0's
> and the harmonic frequencies (not their amplitudes) already known for
> both sounds.
> 
> Can anybody recommend me of a good way to separate the sounds, i.e.
> produce separate wavs for both instruments?

What is the context of your question? Is your job to find out whether 
such a sepeartion can be done at all, or do you need such a routine 
as a "black box" to do your real job, like generate score transcriptions 
from recorded music?

The difference is cruical. I don't know whether these things can be 
done at all. If they can, adapive filters would most likely play 
a significant part in the solution. You have already received advice 
to learn more about adaptive filters, and that's an advice worth 
following. 

The second possible job description is a bit more delicate. If your 
assignment to find some tool to do this separation is a sidestep 
to an "higher" objective, you are asking the right questions but to 
the wrong people. If it in your job description is assumed such 
separation techniques are available, you should ask your supervisor
or boss what knowledge he has about where to find these techniques. 
You can confidently assume he has such knowledge, if he has committed 
you to *use* the techniques, as opposed to *develop* them. 

So, in the case you need these tools to be able do the real job of 
your project, ask your supervisor where to find them. The professional 
project initiator only writes project descriptions that are based on 
tools he knows to be available and have verified to work. It follows,
of course, that he knows where and how to find them.

Rune