http://createdigitalmusic.com/2009/10/the-speaking-piano-and-transforming-audio-to-midi/ -- Randy Yates Digital Signal Labs http://www.digitalsignallabs.com
interesting application of dsp to music/audio
Started by ●June 6, 2015
Reply by ●June 6, 20152015-06-06
Randy Yates <yates@digitalsignallabs.com> writes:> http://createdigitalmusic.com/2009/10/the-speaking-piano-and-transforming-audio-to-midi/This is like something right out of "The Outer Limits"! WEIRD!!! -- Randy Yates Digital Signal Labs http://www.digitalsignallabs.com
Reply by ●June 6, 20152015-06-06
Randy Yates wrote:> http://createdigitalmusic.com/2009/10/the-speaking-piano-and-transforming-audio-to-midi/ >"...the original is almost certainly mixed in." Compare that to this: https://www.youtube.com/watch?v=_qODtwAU1LI -- Les Cargill
Reply by ●June 7, 20152015-06-07
On Sat, 06 Jun 2015 12:34:07 -0500, Les Cargill <lcargill99@comcast.com> wrote:>Randy Yates wrote: >> http://createdigitalmusic.com/2009/10/the-speaking-piano-and-transforming-audio-to-midi/ >> > >"...the original is almost certainly mixed in."I'm not so sure about that. Where the artist talks about "pixels" I think he is referring to pixels of a spectrogram. You can see from the closeups of the piano action that multiple keys are being activated at the same time, just like pixels of a spectrogram. For each spectrum (column of the spectrogram) he would have used some algorithm to assign peak frequencies (linear) to keys (log), and peak amplitudes to key velocity or force. I guess this is like a vocoder whose outputs play mechanical piano keys... might not do such a bad job of emulating a human voice.> >Compare that to this: >https://www.youtube.com/watch?v=_qODtwAU1LIHere you can see that the artist is using a mic (for the vocoder) and playing the keyboard at the same time. I think he is using the vocoder "normally" to re-synthesize the sound of his voice, then using the keyboard sampler to capture that and play it at different pitches. Best regards, Bob Masta DAQARTA v7.60 Data AcQuisition And Real-Time Analysis www.daqarta.com Scope, Spectrum, Spectrogram, Sound Level Meter Frequency Counter, Pitch Track, Pitch-to-MIDI FREE Signal Generator, DaqMusiq generator Science with your sound card!
Reply by ●June 8, 20152015-06-08
Les Cargill <lcargill99@comcast.com> writes:> Randy Yates wrote: >> http://createdigitalmusic.com/2009/10/the-speaking-piano-and-transforming-audio-to-midi/ >> > > "...the original is almost certainly mixed in."Hi Les, That was a comment. I don't think it is true.> Compare that to this: > https://www.youtube.com/watch?v=_qODtwAU1LIIt seems to me that this is a somewhat different process. In vocoder you have two inputs, the modulating signal (e.g., voice) and the signal being modulated (e.g., guitae). Both can be modulated themselves, e.g., the guitar. Here you have only one input. I guess you could say the piano's "response" is an input, but I just don't think of it the same way as a regular vocoder. -- Randy Yates Digital Signal Labs http://www.digitalsignallabs.com
Reply by ●June 11, 20152015-06-11
On 6/6/15 1:34 PM, Les Cargill wrote:> Randy Yates wrote: >> http://createdigitalmusic.com/2009/10/the-speaking-piano-and-transforming-audio-to-midi/very interesting. thanks Randy.> > "...the original is almost certainly mixed in."who is saying that? i don't think that is the case.> > Compare that to this: > https://www.youtube.com/watch?v=_qODtwAU1LIthis is a nice (and typical) vocoder synth. might be a channel vocoder (filter bank thingie) but i think it might be doing an LPC (autocorrelation and Levison-Durbin and decomposition to lattice filter coefficients), but i dunno. it might be a straight-forward phase vocoder where a vocal spectrum becomes the spectral envelope for the synthesized note. but the speaking piano thing is an interesting curiosity. using a piano note to be the transducer for a particular frequency band. i wonder if they compensate for harmonics so when a certain piano key is hit, they account for the harmonics of that key in the octave (12th) and 19th and 24th and 29th and 31st semitones above and reduce the hits on those keys by some amount. because the filter bank in a channel vocoder is not a bank of comb filters that includes harmonics, there might be some assumption that the piano note is essentially just its fundamental. dunno if they're doing that, but it would be simpler if they are. BTW, i was surprised (and take issue) with the one engineer saying that "rendering in fairly high resolution" is obtainable "only ... with a mechanical piano." dunno where that guy comes to that conclusion. -- r b-j rbj@audioimagination.com "Imagination is more important than knowledge."
Reply by ●June 11, 20152015-06-11
robert bristow-johnson wrote:> On 6/6/15 1:34 PM, Les Cargill wrote: >> Randy Yates wrote: >>> http://createdigitalmusic.com/2009/10/the-speaking-piano-and-transforming-audio-to-midi/ >>> > > very interesting. thanks Randy. > >> >> "...the original is almost certainly mixed in." > > who is saying that? i don't think that is the case.The person who wrote the article at the link. " Edit: Listening again, the short answer to how you can hear so much of the voice through the piano seems to be, you can�t; the original is almost certainly mixed in. It�s nonetheless an interesting effect, and I�d like to hear the piano on its own. "> >> >> Compare that to this: >> https://www.youtube.com/watch?v=_qODtwAU1LI > > this is a nice (and typical) vocoder synth. might be a channel vocoder > (filter bank thingie) but i think it might be doing an LPC > (autocorrelation and Levison-Durbin and decomposition to lattice filter > coefficients), but i dunno. it might be a straight-forward phase > vocoder where a vocal spectrum becomes the spectral envelope for the > synthesized note. >No telling, really.> > but the speaking piano thing is an interesting curiosity.it is.> using a piano > note to be the transducer for a particular frequency band. i wonder if > they compensate for harmonics so when a certain piano key is hit, they > account for the harmonics of that key in the octave (12th) and 19th and > 24th and 29th and 31st semitones above and reduce the hits on those keys > by some amount. >This would also vary from piano to piano.> because the filter bank in a channel vocoder is not a bank of comb > filters that includes harmonics, there might be some assumption that the > piano note is essentially just its fundamental. dunno if they're doing > that, but it would be simpler if they are. > > BTW, i was surprised (and take issue) with the one engineer saying that > "rendering in fairly high resolution" is obtainable "only ... with a > mechanical piano." dunno where that guy comes to that conclusion. >That is pretty strange thing to say - mechanical pianos have a lot of latency at least. -- Les Cargill
Reply by ●June 12, 20152015-06-12
Les Cargill <lcargill99@comcast.com> writes:> robert bristow-johnson wrote: >> On 6/6/15 1:34 PM, Les Cargill wrote: >>> Randy Yates wrote: >>>> http://createdigitalmusic.com/2009/10/the-speaking-piano-and-transforming-audio-to-midi/ >>>> >> >> very interesting. thanks Randy. >> >>> >>> "...the original is almost certainly mixed in." >> >> who is saying that? i don't think that is the case. > > > The person who wrote the article at the link. > > " Edit: Listening again, the short answer to how you can hear so much > of the voice through the piano seems to be, you can’t; the original is > almost certainly mixed in. It’s nonetheless an interesting effect, and > I’d like to hear the piano on its own. "Les, Wasn't that just a comment from a reader? -- Randy Yates Digital Signal Labs http://www.digitalsignallabs.com
Reply by ●June 12, 20152015-06-12
Randy Yates <yates@digitalsignallabs.com> writes:> Les Cargill <lcargill99@comcast.com> writes: > >> robert bristow-johnson wrote: >>> On 6/6/15 1:34 PM, Les Cargill wrote: >>>> Randy Yates wrote: >>>>> http://createdigitalmusic.com/2009/10/the-speaking-piano-and-transforming-audio-to-midi/ >>>>> >>> >>> very interesting. thanks Randy. >>> >>>> >>>> "...the original is almost certainly mixed in." >>> >>> who is saying that? i don't think that is the case. >> >> >> The person who wrote the article at the link. >> >> " Edit: Listening again, the short answer to how you can hear so much >> of the voice through the piano seems to be, you can’t; the original is >> almost certainly mixed in. It’s nonetheless an interesting effect, and >> I’d like to hear the piano on its own. " > > Les, > > Wasn't that just a comment from a reader?I see now it is not. This guy is just a writer. I don't think he really knows what's going on in the algorithm - just judging by ear (which could be wrong). -- Randy Yates Digital Signal Labs http://www.digitalsignallabs.com
Reply by ●June 12, 20152015-06-12
On 12/06/2015 02:24, Les Cargill wrote:> robert bristow-johnson wrote: >> On 6/6/15 1:34 PM, Les Cargill wrote: >>> Randy Yates wrote: >>>> http://createdigitalmusic.com/2009/10/the-speaking-piano-and-transforming-audio-to-midi/ >>>> >>>> >> >> very interesting. thanks Randy. >> >>> >>> "...the original is almost certainly mixed in." >> >> who is saying that? i don't think that is the case. > > > The person who wrote the article at the link. > > " Edit: Listening again, the short answer to how you can hear so much of > the voice through the piano seems to be, you can�t; the original is > almost certainly mixed in. It�s nonetheless an interesting effect, and > I�d like to hear the piano on its own. " >Using my musician's ear (and also because this example has been around for a while and pretty well known in computer music circles), I can assert that I most definitely do not hear any original mixed in - it's not. The piano is on its own. Sibilants are converted into clusters of (mostly) high notes, very familiar to anyone used to such things in various forms of contemporary music. The rest of the voice is represented "in the usual way" through the recognisability of the distinct formant regions of different vowels, and their voice-like manner of evolving over time. Our ears have been trained subliminally and biologically over millennia to be very sensitive to such things; much as certain combinations of shadows and shapes, in clouds, on the moon, in toast, may ineluctably suggest a human face. Richard Dobson
