On Sep 4, 9:19 am, Frnak McKenney <fr...@far.from.the.madding.crowd.com> wrote:> ... the tried-and-true basic Fourier Trap > looks like it handles "stationary" rodents fairly well, but I'm > still learning the fundamentals of capturing the more active, > "non-stationary" variety.there's also the Short-Term (or is it Short-Time?) Fourier Transform: +inf X(f, tau) = integral{ x(t) * w((t-tau)/W) * e^(-j*2*pi*f*t) dt} -inf where w((t-tau)/W) is some window of width W, centered about time tau. it's pretty close to wavelets, but the width, W, of the window does not normally shrink as |f| gets larger.> It's not so much that I believe that > understanding the basics of Wavelet Theory will keep me from making > silly implementation errors as a hope that it will improve my odds > of catching and correcting them after the fact.dunno how to help. i hadn't found wavelets to be super useful in audio yet. not to say that they are not useful for someone else. r b-j
Trying to follow the math behind wavelets
Started by ●August 22, 2008
Reply by ●September 4, 20082008-09-04
Reply by ●September 4, 20082008-09-04
On 1 Sep 2008 13:30:18 GMT, Martin Eisenberg <martin.eisenberg@udo.edu> wrote:> Frnak McKenney wrote: > >> Assumption: The CWT(g(),psi(),t,s) is, in some measure, related >> to the Fourier "frequency" spectrum of f(). That is, for aTypo alert: The 'f()' above should have been changed to a 'g()' when I decided to use 'f' for frequency.>> fixed g() and psi(), if the frequency spectrum of g() contains >> some frequency f0 at time t0, then for some 'scale' s0 the >> results of the CWT(g(),psi(),t0,s0) has some sort of "peak". > > It has power there but that doesn't say anything about the waveform > carrying it."Power". As in "magnitude squared". Okay... the signal g() has amplitude-hence-magnitude, and a wavelet has amplitude-hence -magnitude, so even though my head hurts when I try to visualize it, I can accept the idea that g()*psi() -- or, more precisely, the sum/integral of g()*psi() -- represents "power". --snip--> No, it should be a waveform of constant power.My father was an electrical engineer (and a long-time steam- engine-eer wannabe, but that's another story <grin!>). He introduced me to the Simpson (analog) VOM at an early age, and ever since then I've had trouble visualizing things I couldn't measure. <grin!> Another question, or at least request for confirmation. Suppose I have the (infinite) CWT scale-time (y-x) plane laid out in front of me, and further assume that I can stretch out a pair of more-than- infinitely-extensible hands to squeeze it from the top and bottom (Ack!) into a horizontal line. If my chosen wavelets form an orthonormal basis (that is, they chop up functions in such a way that they can be exactly reconstructed), don't I get back my original function? Okay, that seems fairly trivial. But... suppose I squeeze from the right and left? Well, if I've left in any unbounded thingies, such as sine-waves-extended-to-forever, some or all of the result may blow up in my face. But ignoring that messy (but very real) possibility, it "feels" like I wind up with a set of values representing the "scales" at which my original function had "power" at any point in time. What would you call it? A... "scale spectrum"? Some of the values might be very small, but if the original function (before being so rudely wavelet-julienned) had any activity at that scale at any time, the value would be greater than zero. "Power footprints". Don't get me wrong... I don't see any immediate use for this set of values; I'm just trying to double-check my internal model by chasing down its consequences and exposing them for comment. (And, before you ask, no, I don't feel like squeezing on/along _either_ diagonal just now. <grin!>)>> But this says -- assuming I haven't dropped a radical or index >> or something -- that, at scale s1, and for a function g() of >> "constant frequency", the CWT _varies_. It's _not_ a nice, neat >> constant "horizontal" value running along the time axis. > > And neither would the output of a DFT channel be, but the variation > would be all in the phase and none in the magnitude. The same is true > of the HP example which uses a complex wavelet (with a one-sided > spectrum). Your real Haar wavelet gives real output that has only one > dimension available to vary in, but its amplitude and waveshape > (hence effective value) are still constant for input with the same > properties. > >> I realize that my question may be somewhat confusing -- it's >> certainly confused <grin!> -- and a detailed breakdown might >> just pass over my head at this point, but it might help me get >> my bearings a bit if you can comment on what the "shape" of the >> CWT of a simple, single-frequency sin() function should "look >> like". > > You're filtering a sinusoid so a sinusoid will come out -- real if > the wavelet is real, complex otherwise. In the latter case, the > *scalogram* will show what you want it to (and that's what fig. 11 in > the HP paper is), just like a spectrogram would (but not the DFT > itself).Okay. My imagination just hit "overload". That's not a request for further clarification, it's an acknowledgement that I need to sit back for a few days and let the dust of my fragmented models settle before I'll be able to handle any further input. But I do appreciate your feedback. Meanwhile, in TheRealWorld(tm), it appears that Murphy -- long recognized as the patron saint of Data Processing -- has decided to take a hand in the American Presidential Election process. "May we live in interesting times", indeed. <grin!> Frank -- If you cannot -- in the long run -- tell everyone what you have been doing, your doing has been worthless. -- Erwin Schrodinger -- Frank McKenney, McKenney Associates Richmond, Virginia / (804) 320-4887 Munged E-mail: frank uscore mckenney ayut mined spring dawt cahm (y'all)
Reply by ●September 4, 20082008-09-04
On Sep 4, 11:54�am, robert bristow-johnson <r...@audioimagination.com> wrote:> On Sep 4, 9:19 am, Frnak McKenney > > <fr...@far.from.the.madding.crowd.com> wrote: > > ... the tried-and-true basic Fourier Trap > > looks like it handles "stationary" rodents fairly well, but I'm > > still learning the fundamentals of capturing the more active, > > "non-stationary" variety. > > there's also the Short-Term (or is it Short-Time?) Fourier Transform: > > � � � � � � � � +inf > �X(f, tau) = integral{ x(t) * w((t-tau)/W) * e^(-j*2*pi*f*t) dt} > � � � � � � � � -inf > > where w((t-tau)/W) is some window of width W, centered about time > tau. > > it's pretty close to wavelets, but the width, W, of the window does > not normally shrink as |f| gets larger. > > > �It's not so much that I believe that > > understanding the basics of Wavelet Theory will keep me from making > > silly implementation errors as a hope that it will improve my odds > > of catching and correcting them after the fact. > > dunno how to help. �i hadn't found wavelets to be super useful in > audio yet. �not to say that they are not useful for someone else. > > r b-jHello Robert and others, If you let the window be a gaussian function, then the short time fourier transform becomes a gabor transform. The following link shows an application of the gabor transform and the reconstruction of missing samples in audio. Its not wavelets, but is getting close. http://www.eecs.harvard.edu/~patrick/publications/wolfe_icassp_05_submit.pdf Clay
Reply by ●September 4, 20082008-09-04
On Aug 22, 10:30�am, Frnak McKenney <fr...@far.from.the.madding.crowd.com> wrote:> Hi. �I'm trying to understand wavelet theory, and the CWT in > particular, and I'm running into some rough spots. �If anyone has a > few minutes to comment, or point me at a better place to ask these > questions, it would be greatly appreciated. > > After scanning a few 'web pages and online articles, it appeared > that wavelets would be a good way of decomposing a complex, coded > data stream like the (demodulated) audio from NIST's WWV 10MHz > transmitter. �Given a collection of WAV files and the signal > description from: > > � NIST Time and Frequency Services, Pub. 432 > �http://tf.nist.gov/timefreq/general/pdf/1383.pdf > > all I'd have to do would be feed the sampled signal int A Wavelet > and, Hey! �Presto! �I'd have a 2D array telling me the exact pattern > of the 100Hz "subcarrier" carrying the digital timecode. <grin!> > > Then I started reading the math in more detail, and trying to > "unpack" the notation into small steps that I could understand. > > Here's the definition of the CWT that I found easiest to follow > (sorry about the notation): > > � For a given wavelet psi() and function f(), the CWT of "tau" (an > � offset value in the domain of f()) and "s" (a scaling factor) is > � given by: > > � CWT(psi, f, tau, s) = �(1/sqrt(abs(s))) * > > � � � � Integral(-inf,+inf, du, > > � � � � � � �( f(u) * Conjugate( psi( (u-tau)/s ) ) ) � � ) > > (I hope that's clear.) > > First question: �The CWT for any specific value of "s", that is, for > any single "horizontal" evaluation, this formula looks a lot like > the formula for a cross-correlation between f() and psi(). �If so, > this would mean that the CWT could be described in terms of multiple > cross-correlations between a given signal and one's chosen "mother" > wavelet. > > This was puzzling, since none of the wavelet papers I have scanned > mentioned this way of looking at wavelets. �Is it simply considered > so obvious that no-one bothers mentioning it? �Or did I lose a > symbol or something along the way? > > Second question: �I can "see" how the CWT transforms a time-based > signal into a time-scale plane, but I'm having trouble seeing how > one goes from that to a more useful -- to me, anyway <grin!> -- > time-frequency plane. �What I've read suggests that the relationship > between 'scale' and 'frequency' depends on the specific choice of > the wavelet function psi(), but other than using the Matlab magic > 'scal2freq' function it's not clear to me how I would go about doing > it in practice. > > Third question: �The theory I've read so far offers ways of deciding > whether or not a given possibility for a wavelet function psi() is > acceptable or not, and there seem to be a _lot_ <grin!> of > acceptable functions. �However, one that would seem like an obvious > choice never seems to be mentioned: �a sine-singlet, e.g. > > � � f(t) = { sin(t) for 0 <= t < 2*%pi, and 0 otherwise } > > There may well be a simple reason why this is an unsuitable -- or > perhaps just un-useful -- wavelet function. �I just haven't stumbled > onto the reason yet. > > Ah, well. �That should pretty well expose the level of my ignorance > in this area. �I found a few hints by searching the fora atwww.DSPRelated.com, but they're hard to read; this may be related, > perhaps, to my attempting to view them in 24-point type. �<grin!> > > Any hints, clues, or re-routings will be appreciated. > > Frank McKenney > -- > � �"A foolish consistency is the hobgoblin of little minds, > � � Adored by little statesmen and philosophers and divines." > � � � � � � � � � � � � � � �-- Ralph Waldo Emerson > -- > Frank McKenney, McKenney Associates > Richmond, Virginia / (804) 320-4887 > Munged E-mail: frank uscore mckenney ayut mined spring dawt cahm (y'all)Hello Frank, You may also wish to look at using a Gabor-Wigner transform. This can give a good time-frequency localization estimate. Clay
Reply by ●September 5, 20082008-09-05
Frnak McKenney wrote:> English is a versatile language with many variations in > spelling and pronunciation (see below <grin!>).I've harvested some of your sigs, by the way.> So... you're saying that, over time and with great effort, I > might eventually be able to claim comprehension of... > zilch(^-2)??Baselines constantly slipping away as they do in scalefree systems like the Real World, each new achievement has to be more singular.> I think I am in general agreement with the author's concern over > classifying student abilities based on limited data; on the > other hand, his comment that "those who do not have mathematical > ability have no choice but to go into worthless subjects" > concerns me a bit.I concur. It didn't say that back when I downloaded it, but anyway, it's immaterial to the main content I recommended the book for. Martin -- No wonder that illegitimate children are commonly the greatest minds; they are the result of an hour full of wit, marital ones often spring from boredom. --Theodor Gottlieb von Hippel
Reply by ●September 5, 20082008-09-05
Frnak McKenney wrote:>>> Assumption: The CWT(g(),psi(),t,s) is, in some measure, >>> related to the Fourier "frequency" spectrum of f(). That is, >>> for a fixed g() and psi(), if the frequency spectrum of g() >>> contains some frequency f0 at time t0, then for some 'scale' >>> s0 the results of the CWT(g(),psi(),t0,s0) has some sort of >>> "peak". >> >> It has power there but that doesn't say anything about the >> waveform carrying it. > > "Power". As in "magnitude squared". Okay... the signal g() > has amplitude-hence-magnitude, and a wavelet has amplitude-hence > -magnitude, so even though my head hurts when I try to visualize > it, I can accept the idea that g()*psi() -- or, more precisely, > the sum/integral of g()*psi() -- represents "power".Argh no, erase that thought ;) It's nothing to do with the mechanics of computation! The signal has some spectrotemporal distribution of power as a matter of fact, and the scalogram is designed to present that in a recognizable way (one way among others, of course).> Another question, or at least request for confirmation. Suppose > I have the (infinite) CWT scale-time (y-x) plane laid out in > front of me, and further assume that I can stretch out a pair of > more-than- infinitely-extensible hands to squeeze it from the > top and bottom (Ack!) into a horizontal line. If my chosen > wavelets form an orthonormal basis (that is, they chop up > functions in such a way that they can be exactly reconstructed), > don't I get back my original function?You can imagine it that way, noting that the synthesis wavelet (reconstruction filterbank) gets thrown in too.> Okay, that seems fairly trivial. But... suppose I squeeze from > the right and left? Well, if I've left in any unbounded > thingies, such as sine-waves-extended-to-forever, some or all of > the result may blow up in my face. But ignoring that messy (but > very real) possibility, it "feels" like I wind up with a set of > values representing the "scales" at which my original function > had "power" at any point in time. > > What would you call it? A... "scale spectrum"?Good question; I don't know if there's an established term. But you could also describe it without reference to wavelets as a Fourier spectrum aggregated over log-spaced intervals.> But I do appreciate your feedback.Don't mention it.> Meanwhile, in TheRealWorld(tm), it appears that Murphy -- long > recognized as the patron saint of Data Processing -- has decided > to take a hand in the American Presidential Election process. > "May we live in interesting times", indeed. <grin!>I'm not American, you know. Does that mean you consider the election decided with McCain's nomination? -- The more you can say with a language, the less you can say about the language. --Kathy Yellick
Reply by ●September 6, 20082008-09-06
On Thu, 4 Sep 2008 08:54:00 -0700 (PDT), robert bristow-johnson <rbj@audioimagination.com> wrote:> On Sep 4, 9:19 am, Frnak McKenney ><fr...@far.from.the.madding.crowd.com> wrote: >> ... the tried-and-true basic Fourier Trap >> looks like it handles "stationary" rodents fairly well, but I'm >> still learning the fundamentals of capturing the more active, >> "non-stationary" variety. > > there's also the Short-Term (or is it Short-Time?) Fourier Transform: > > +inf > X(f, tau) = integral{ x(t) * w((t-tau)/W) * e^(-j*2*pi*f*t) dt} > -inf > > where w((t-tau)/W) is some window of width W, centered about time > tau. > > it's pretty close to wavelets, but the width, W, of the window does > not normally shrink as |f| gets larger.Right. The STFT is frequently mentioned in discussions of wavelets; the main diffrerences I see mentioned are the STFT's domain (frequency-time) and fixed "window width" vs. wavelets domain (scale-time) and varying "window width".>> It's not so much that I believe that >> understanding the basics of Wavelet Theory will keep me from making >> silly implementation errors as a hope that it will improve my odds >> of catching and correcting them after the fact. > > dunno how to help. i hadn't found wavelets to be super useful in > audio yet. not to say that they are not useful for someone else.Hubbard mentions a couple of audio applications (any confusion in the descriptions is probably mine): 1) De-noising, by using wavelets to differentiate (e.g.) white noise with "activity" over a wide set of scales from "signal" which is assumed to have "activity" at only a few scales. 2) More broadly, separating stuff-we-want "signals" from stuff-we- don't-want "noise" based on the scales at which "activity" is present. 3) Signal compression, based on the time-honored compression principle of "ignore what isn't changing". Not changing, that is, from a "wavelet point of view". <grin!> Me, I'm still trying to understand the rules governing the selection of an appropriately-sized... er, -scaled wavelet-screwdriver for pounding in a given set of signal-nails. <grin!> Frank -- Ultimate office automation: networked coffee. -- Frank McKenney, McKenney Associates Richmond, Virginia / (804) 320-4887 Munged E-mail: frank uscore mckenney ayut mined spring dawt cahm (y'all)
Reply by ●September 6, 20082008-09-06
On Thu, 4 Sep 2008 13:38:16 -0700 (PDT), clay@claysturner.com <clay@claysturner.com> wrote:> On Aug 22, 10:30�am, Frnak McKenney ><fr...@far.from.the.madding.crowd.com> wrote: >> Hi. �I'm trying to understand wavelet theory, and the CWT in >> particular, and I'm running into some rough spots. �If anyone has a >> few minutes to comment, or point me at a better place to ask these >> questions, it would be greatly appreciated. >> >> After scanning a few 'web pages and online articles, it appeared >> that wavelets would be a good way of decomposing a complex, coded >> data stream like the (demodulated) audio from NIST's WWV 10MHz >> transmitter.--snip-- I may have been a bit optimistic. <grin!>> You may also wish to look at using a Gabor-Wigner transform. This can > give a good time-frequency localization estimate.Clay, Thanks for the suggestion. I'll file it for future reference, but for the moment I think I'll stick to trying to understand exactly what "wavelets" do -- and don't -- accomplish. I'm having enough trouble with that. <grin!> Frank -- "I am only one, but still I am one. I cannot do everything, but still I can do something" --Helen Keller -- Frank McKenney, McKenney Associates Richmond, Virginia / (804) 320-4887 Munged E-mail: frank uscore mckenney ayut mined spring dawt cahm (y'all)
Reply by ●September 6, 20082008-09-06
On 5 Sep 2008 14:32:22 GMT, Martin Eisenberg <martin.eisenberg@udo.edu> wrote:> Frnak McKenney wrote: >> English is a versatile language with many variations in >> spelling and pronunciation (see below <grin!>). > > I've harvested some of your sigs, by the way.Please feel free. Most are second-hand. <grin!> Frank -- There are many definitions of what art is, but what I am convinced art is not is self-expression. If I have an experience, it is not important because it is mine. It is important because it's worth writing about for other people, worth sharing with other people. That is what gives it validity. -- W. H. Auden -- Frank McKenney, McKenney Associates Richmond, Virginia / (804) 320-4887 Munged E-mail: frank uscore mckenney ayut mined spring dawt cahm (y'all)
Reply by ●September 6, 20082008-09-06
Hi, Martin. If I seem a bit more confused than usual it's because I'm alternating writing my reply with a periodic background sampling process: "The Effects of Extended Hurricane-Based Rainfall on Mostly-Impermeable Cinder Block Basement Walls". The good news is that my sampling device (a.k.a. "ShopVac") has a 16 gallon buffer. On 5 Sep 2008 15:33:20 GMT, Martin Eisenberg <martin.eisenberg@udo.edu> wrote:> Frnak McKenney wrote: > >>>> Assumption: The CWT(g(),psi(),t,s) is, in some measure, >>>> related to the Fourier "frequency" spectrum of f(). That is, >>>> for a fixed g() and psi(), if the frequency spectrum of g() >>>> contains some frequency f0 at time t0, then for some 'scale' >>>> s0 the results of the CWT(g(),psi(),t0,s0) has some sort of >>>> "peak". >>> >>> It has power there but that doesn't say anything about the >>> waveform carrying it. >> >> "Power". As in "magnitude squared". Okay... the signal g() >> has amplitude-hence-magnitude, and a wavelet has amplitude-hence >> -magnitude, so even though my head hurts when I try to visualize >> it, I can accept the idea that g()*psi() -- or, more precisely, >> the sum/integral of g()*psi() -- represents "power". > > Argh no, erase that thought ;) It's nothing to do with the mechanics > of computation! The signal has some spectrotemporal distribution of > power as a matter of fact, and the scalogram is designed to present > that in a recognizable way (one way among others, of course).Okay. I misunderstood. I thought I had a handle on what the range (output) of a CWT(scale,time) was, and my initial reading of your reply threw me a bit. Let me try to put my current "model" into words and see if it makes sense: The CWT(f(),phi(),s,tau) maps a single function/signal f() into a _family_ of function/signals. Each member of this (possibly infinite) family is created by repeatedly taking, at each point of f(), an inner product of f() with a scaled copy of the wavelet function psi() which has been translated to that point. So the "range" of the CWT(), the "surface" the CWT describes above the scale-time plane, is the same as the "range" of the signal(s), that is, amplitude.>> Another question, or at least request for confirmation. Suppose >> I have the (infinite) CWT scale-time (y-x) plane laid out in >> front of me, and further assume that I can stretch out a pair of >> more-than- infinitely-extensible hands to squeeze it from the >> top and bottom (Ack!) into a horizontal line. If my chosen >> wavelets form an orthonormal basis (that is, they chop up >> functions in such a way that they can be exactly reconstructed), >> don't I get back my original function? > > You can imagine it that way, noting that the synthesis wavelet > (reconstruction filterbank) gets thrown in too.Oh. Right. I forgot that the "chopping up" involved changing the shape of the signal "pieces" in a specific fashion (dependent on the choice of wavelet), and therefore one would have to "undo" those changes in order to put the "pieces" back together properly. Hey! That actually made sense! (And who said miracles were out of fashion? <grin!>>> Okay, that seems fairly trivial. But... suppose I squeeze from >> the right and left? Well, if I've left in any unbounded >> thingies, such as sine-waves-extended-to-forever, some or all of >> the result may blow up in my face. But ignoring that messy (but >> very real) possibility, it "feels" like I wind up with a set of >> values representing the "scales" at which my original function >> had "power" at any point in time. >> >> What would you call it? A... "scale spectrum"? > > Good question; I don't know if there's an established term. But you > could also describe it without reference to wavelets as a Fourier > spectrum aggregated over log-spaced intervals.After listening to your restatement, I think I just re-invented the Constant-Q Transform in "scale" clothing. <grin!> But the important thing (to me) is that I'm "pushing things around" and seeing results that match what my "model" predicts. So now I go back and re-read Hubbard and the five or six papers that seemed to make the most sense. If I'm lucky, a few more bits will make sense this time around.>> Meanwhile, in TheRealWorld(tm), it appears that Murphy -- long >> recognized as the patron saint of Data Processing -- has decided >> to take a hand in the American Presidential Election process. >> "May we live in interesting times", indeed. <grin!> > > I'm not American, you know. Does that mean you consider the election > decided with McCain's nomination?Not in the least. But let me explain, or at least try to. <grin!> Every four years we hold our Presidential elections. As a part of the process each candidate must undergo a long series of trials to show that they are Worthy: Trial by Insult, Trial by Shaming, Trial by Prolonged Examination in Minute Detail (a.k.a. Trial by Media), Trial of Ideological Purity, Trial of Historical Consistency, Trial by Slings and Arrows of Outrageous Fortune, ... well, you get the idea. Points are deducted each time a candidate fails to stoically endure one of their trials; points are added when they can not merely endure a trial better/longer than an opponent, but make it appear as if the results are unimportant to them. Until a week ago (only one week?) it appeared that we would be witness to the trials of four fairly similar candidates. The three "knowns" were all legislators (two old-timers and one newcomer), and most of us assumed that McCain's choice for VP would be yet another Washington "insider". McCain's selection of Governor Sarah Palin (Alaska) as his running mate has put new life into the process. She's not well known, so the media can spend hours discussing her without getting bored. She's female, as governor she has experience none of the other three have, she's conservative, she hunts and fishes, and she has successfully fought the oil companies, so the political pundits have plenty of material for speculating on (e.g.) what percentage of Senator Clinton's former supporters will cross over to vote for her. She's attractive and personable enough, and can speak coherent sentences, so she's likely to do well on talk shows. Finally, her personality and views have enough "edges" for easy caricature, which makes her a prime candidate for (e.g.) Saturday Night Live. <grin!> Even the people who don't like her are talking about her. As I say, "interesting times". What looked to be a fairly predictable (hence boring) see-saw battle between the parties for the last few independent voters has suddenly shifted into unexpected dimensions, and this is likely to increase the voter turnout on both sides. And since that puts the outcome of the election even more in doubt, people (voters) are paying even more attention to the process. <grin!> Anyway, that's my not-so-humble opinion. Hope y'all can enjoy it from afar. Frank -- The Democrats are the party of government activism, the party that says government can make you richer, smarter, taller, and get the chickweed out of your lawn. Republicans are the party that says government doesn't work, and then get elected and prove it. -- P. J. O'Rourke -- Frank McKenney, McKenney Associates Richmond, Virginia / (804) 320-4887 Munged E-mail: frank uscore mckenney ayut mined spring dawt cahm (y'all)
