On Apr 8, 12:52 pm, Andor <andor.bari...@gmail.com> wrote:
> Ron N. wrote:
> > On Apr 8, 7:43 am, Andor wrote:
>
> > > On 8 Apr., 10:01, Steve Underwood <ste...@dis.org> wrote:
>
> > > > Al Clark wrote:
> > > > > "VelociChicken" <b...@yahoob.com> wrote in
> > > > >news:0GmJj.25$Wf3.4@newsfe16.ams2:
>
> > > > >>>>> Hi,
>
> > > > >>>>> I've always been interested in DSP methods to decompose a piece of
> > > > >>>>> music into discrete instruments. I came across this new tool that's
> > > > >>>>> pretty fascinating:
>
> > > > >>>>>http://www.celemony.com/cms/index.php?id=dna&L=0
>
> > > > >>>>> How do you think something like this works? Considering an acoustic
> > > > >>>>> guitar chord and all its intermixed, varying and slightly non-integer
> > > > >>>>> harmonics, how would one go about separating such a signal into
> > > > >>>>> individual
> > > > >>>>> notes that actually sound pretty clean?
> > > > >>>> of course i don't know exactly how they did it. i doubt that they are
> > > > >>>> assuming any non-integer harmonics. if it were non-real-time, i could
> > > > >>>> imaging processing the audio with a set of cascaded, adjustable comb-
> > > > >>>> filters (where the notches are at the harmonic frequencies) until they
> > > > >>>> get something that is residual. dealing with the attacks is another
> > > > >>>> problem. dealing with harmonics of two different notes that land on
> > > > >>>> top of each other is another problem.
>
> > > > >>>> lot'sa problems. dunno how they dealt with them all. but, from what
> > > > >>>> i have seen in Electronic Musician, i think this is no bogus product.
> > > > >>>> i think it's real. i'm looking forward to finding their booth at the
> > > > >>>> AES show in October (they better the hell be there).
> > > > >>> I'd agree with your assessment of "lot'sa problems" if I hadn't heard
> > > > >>> the short talk by the guy behind it. He makes it sound like something
> > > > >>> clicked in his mind, and made him realise that something which was not
> > > > >>> rigorous was going to fail with sufficient rarity for it to useful. He
> > > > >>> makes it sound like the magic trick was one key insight.
> > > > >> Well, you've got to start somewhere. I've been looking at Continuous
> > > > >> Wavelet's, and as the data is pre-processed I guess you could do a lot
> > > > >> in the analysis stage. How about a CWT at note resolution pitches? It
> > > > >> would give you the fundamentals, and a nice hi-res spectrum.
> > > > >> If it's designed mainly for one instrument, it could use different
> > > > >> number of notes played at once to figure out the harmonics of each
> > > > >> through-out the piece. But like someone above said, there's a lot of
> > > > >> potential breakages, and it completely depends on the source material.
> > > > >> VC
>
> > > > > I have never tried to solve this problem, but it seems to me a cepstrum
> > > > > approach might be useful.
>
> > > > > A cepstrum (spectrum spelled sideways), is a forward FFT of an FFT power
> > > > > spectrum. The bins represents harmonics of the original FFT.
>
> > > > > It's been a long time since I looked at cepstrum, but I know that they have
> > > > > been used to help isolate components in mechanical systems like gear boxes.
> > > > > I think their might be some old Bruel & Kjaer application notes on this
> > > > > idea.
>
> > > > Cepstrums are widely used in speech analysis for isolating components,
> > > > too. Thats not a million kilometres away from music analysis, so
> > > > cepstral analysis might well be involved.
>
> > > If I were to tackle this problem, I would start off with an STFT
> > > decomposition (spectrogram) of the signal (alternatively, use wavlet
> > > decomposition).
>
> > > Assume that the signal consists of a piano (or guitar) chord, possibly
> > > with delayed starting points of each separate note. Then each note
> > > generates multiple "stripes" of harmonics in the spectrogram, spaced
> > > by the fundamental frequency of that note. Group the "stripes" into
> > > notes (starting positions might help). For a given fundamental
> > > "stripe", only stripes that lie on multiples of the fundamental
> > > frequency belong to that note.
>
> > > Once the harmonics are grouped notes, filter out the harmonics of the
> > > signal. To do this, programm a bunch of linear-phase FIR bandpass
> > > filters, one filter for each detected harmonic. The bandpass frequency
> > > is given by the offset of the stripe. The filters are time-varying,
> > > switching on at the beginning of the stripe and switching off at the
> > > end of the stripe.
>
> > > There already exists a plug-in that does exactly what I described in
> > > the second paragraph (http://www.algorithmix.com/en/renovator.htm), so
> > > the novelty lies in grouping the harmonics into notes. That doesn't
> > > look like too hard a problem.
>
> > > Once the notes are seperated, use existing pitch-shifting and time-
> > > stretching algorithms on the single notes.
>
> > > Easy.
>
> > > :-)
>
> > In Western music, one property of the most common chords is
> > that the harmonics of several notes in these chords overlap.
> > If you group a harmonic into one note, then you will "hollow
> > out" the sound of one or more other notes which include that
> > same harmonic. To avoid this, your scheme might have to also
> > potentially factor each harmonic into fractions apportioned
> > among different groups.
>
> That should be possible. One can interpolate the appropriate amplitude
> from the neighbouring harmonics.
Assuming that you have un-mixed these neighboring harmonics
first, or at the same time, and that you have a model of the
harmonic envelope (or formant) which may be varying with time
as well as several other parameters (which instrument, which
string, dynamics, & etc.). Where do you get these models?
> > In perfect Western equally tempered tuning, the harmonics are
> > at slightly different frequencies, perhaps beyond the time
> > frequency resolution of a sub-note length STFT. And real
> > instruments may not be tuned this precisely. So even if
> > you could use some sort of pattern recognition to factor
> > each harmonic into fractional portions, you would need a
> > filter that could slice those harmonics very thinly.
>
> Agreed. But it is doable. One would have to also consider that for
> example piano notes don't necessarily have integer multiple harmonics
> of the fundamental. Don't know how relevant that effect is, but it
> could be taken into account (simply guess at what frequency the
> harmonic should lie, based on the fundamental, then measure the exact
> harmonic frequency).
The problem is more that the algorithm may need to measure
and then separate an extremely narrowband mix of several
harmonics (one for each note in a chord) that may differ
by only a small fraction of a percent in frequency (a few
cents), if at all. Not sure what would happen if you put
the wrong harmonic (at almost the right frequency) with
a given note. If the note has vibrato, or a bend, this
would certain cause weird artifacts.
> > For some instruments, you might be able to filter out some
> > fractional harmonic content by phase locking a quadrature
> > filter to the fundamental; but for other instruments (piano,
> > for instance), the relationship of a note to its harmonics
> > may not be an exact integer ratio. An interesting problem.
>
> Indeed, an interesting problem.
>
> Regards,
> Andor
IMHO. YMMV.
--
rhn A.T nicholson d.0.t C-o-M