Rune Allnor wrote:> On 7 Jul, 22:41, Jerry Avins <j...@ieee.org> wrote: >> Rune Allnor wrote: >>> On 7 Jul, 20:14, "maxplanck" <erik.bo...@comcast.net> wrote: >>>> Thanks very much Steve! >>>> The section of your book that you're referring to demonstrates numerically >>>> that correlation, when implemented as it is in the DFT, produces the >>>> desired results, i.e. what we want the output of the DFT to be. This is >>>> enough proof for me to believe that it works. >>> Wrong. That's smoke and mirrors that might look reasonable >>> up front, but only obfuscates in the long run. Both the >>> the DFT/IDFT pair and correlation is based on the inner >>> product, but that's where the similarity ends. > ... >>> Get a good book on DSP and look for the matrix W in the >>> chapter that discusses the DFT. >> Rune, >> >> What is the correlation between a sinusoid and an arbitrary function? > > It's the inner product between the sine and the function. > Introducing the term 'correlation' is a red herring if the > main objective is to study the DFT.Finding the inner product is a method; I'm after the meaning. Let me put the matter differently: since basis functions are all orthogonal, the integral over a whole number of periods of the product any one of them and any number of the others is necessarily zero. (Definition of orthogonality.) If the arbitrary term of the integrand includes some amount of the multiplying basis function, then the amount by the which result differs from zero tells us how much of the basis is "contained" in the arbitrary term of the integrand. As I see it, that's the same as saying it is a measure of how well the two terms in the integrand are correlated. Does that make any sense to you? Jerry -- Engineering is the art of making what you want from things you can get. �����������������������������������������������������������������������
Can you recommend a good explanation of the proof of the Fourier Transform?
Started by ●July 1, 2008
Reply by ●July 8, 20082008-07-08
Reply by ●July 8, 20082008-07-08
On Jul 7, 2:45�pm, Oli Charlesworth <ca...@olifilth.co.uk> wrote:> On Jul 7, 8:14 pm, "maxplanck" <erik.bo...@comcast.net> wrote: > > > I'm curious about a more rigorous proof of why the operation of > > correlating a signal with a basis function, then multiplying by an > > appropriate scaling factor, produces (after conversion to polar > > coordinates) the magnitude & initial phase of that basis function's > > presence in the signal. > > Because the basis functions are orthogonal.that's not always the case. it depends on how one defines the inner product in the metric space (Hilbert space) you are considering. consider constructing a general polynomial outa two different sets of basis functions. one set are just the power functions, x^n and the other set are Tchebyshev polynomials. there can be two different Hilbert spaces (with identical elements but different inner products) defined, one where different ordered power functions are orthogonal and the other with different ordered Tchebyshev polynomials that are orthogonal. both sets are basis functions in either space (because you can construct a arbitrary element of the space out of summing the members of either set of basis functions), but, for a given space, not both are orthogonal. orthogonal basis sets are a subset of basis sets. likewise, orthonormal are a subset of orthogonal.> �With any orthogonal > transform, correlating against the basis functions and then > multiplying the coefficients with the conjugate basis functions, > you're always going to get back to where you started (save for edge > cases, discontinuities and other mathematical niggles).that's true. hey Max, while what you wrote may be true (because of the qualification of "that I'm interested in"), it's not true that "the FT is the heart of just about every DSP technique" of interest to audio and music synthesis/processing. almost anything done in the time domain (which is still most analysis or synthesis processes) does not use the FT or DFT or FFT. r b-j
Reply by ●July 8, 20082008-07-08
Thanks for all of the replies guys. What everyone is saying makes sense, I am just looking for a more rigorous proof. In a rigorous proof I will find the "why" that I'm looking for. Many people say "pick up a math book," but can you recommend a particularly clear/good one that addresses this topic? Having a clear, well written explanation makes such a huge difference, and I think the odds are that I will get a poor explanation if I just choose randomly from all of the math texts that address this topic. rbj: i know that currently most audio synthesis and processing is done in the time domain, but if the frequency/phase domain analysis algorithms get good enough (or if they're currently good enough, then if they get implemented more), then I think that frequency domain techniques are the future (already being implemented to some extent (at least that's what i've heard) in Celemony's "Melodyne", which has made a big splash in the music production world in the past few years). Of course many things will always be done in the time domain, but spectral techniques potentially offer many new possibilities for sound design and editing
Reply by ●July 9, 20082008-07-09
On 8 Jul, 18:03, Jerry Avins <j...@ieee.org> wrote:> Rune Allnor wrote: > > On 7 Jul, 22:41, Jerry Avins <j...@ieee.org> wrote: > >> Rune Allnor wrote: > >>> On 7 Jul, 20:14, "maxplanck" <erik.bo...@comcast.net> wrote: > >>>> Thanks very much Steve! � > >>>> The section of your book that you're referring to demonstrates numerically > >>>> that correlation, when implemented as it is in the DFT, produces the > >>>> desired results, i.e. what we want the output of the DFT to be. �This is > >>>> enough proof for me to believe that it works. > >>> Wrong. That's smoke and mirrors that might look reasonable > >>> up front, but only obfuscates in the long run. Both the > >>> the DFT/IDFT pair and correlation is based on the inner > >>> product, but that's where the similarity ends. > > ... > >>> Get a good book on DSP and look for the matrix W in the > >>> chapter that discusses the DFT. > >> Rune, > > >> What is the correlation between a sinusoid and an arbitrary function? > > > It's the inner product between the sine and the function. > > Introducing the term 'correlation' is a red herring if the > > main objective is to study the DFT. > > Finding the inner product is a method;The inner product is a concept as good as any, along with 'product', 'sum', 'square root' etc.> I'm after the meaning.What's the meaning of 'difference'?> Let me put > the matter differently: since basis functions are all orthogonal,Who said they are?> the > integral over a whole number of periods of the product any one of them > and any number of the others is necessarily zero. (Definition of > orthogonality.)This is thhe definition of orthogonality. But a set of basis functions needs not be orthogonal.> If the arbitrary term of the integrand includes some > amount of the multiplying basis function, then the amount by the which > result differs from zero tells us how much of the basis is "contained" > in the arbitrary term of the integrand.I don't understand what you mean. The concept of 'basis' is the same if you talk about functions as when you talk about vectors in 2D. If you have an arbitrary vector x then the inner product c=<x,b> is computed by prijecting x onto b, and is thus a measure of 'similarity' between x and b. One can project x onto several basis vectors simultaneously, by collecting the basis vector in a matrix B: c =Bx Now c is a vector which expresses x in terms of the basis B. Trivial high-school linear algebra, generalizes straight-forward to the DFT. The inner product is stuff I fisrt encountered at age 16, it is hardly revolutionary, nor controversial.> As I see it, that's the same as > saying it is a measure of how well the two terms in the integrand are > correlated.'Correlation' is a term that is based on the inner product, not the other way around. What is the more fundamental concept to you: 'wheel' or 'bicycle'? Rune
Reply by ●July 9, 20082008-07-09
Rune Allnor wrote:> On 8 Jul, 18:03, Jerry Avins <j...@ieee.org> wrote: >> Rune Allnor wrote: >>> On 7 Jul, 22:41, Jerry Avins <j...@ieee.org> wrote: >>>> Rune Allnor wrote: >>>>> On 7 Jul, 20:14, "maxplanck" <erik.bo...@comcast.net> wrote: >>>>>> Thanks very much Steve! >>>>>> The section of your book that you're referring to demonstrates numerically >>>>>> that correlation, when implemented as it is in the DFT, produces the >>>>>> desired results, i.e. what we want the output of the DFT to be. This is >>>>>> enough proof for me to believe that it works. >>>>> Wrong. That's smoke and mirrors that might look reasonable >>>>> up front, but only obfuscates in the long run. Both the >>>>> the DFT/IDFT pair and correlation is based on the inner >>>>> product, but that's where the similarity ends. >>> ... >>>>> Get a good book on DSP and look for the matrix W in the >>>>> chapter that discusses the DFT. >>>> Rune, >>>> What is the correlation between a sinusoid and an arbitrary function? >>> It's the inner product between the sine and the function. >>> Introducing the term 'correlation' is a red herring if the >>> main objective is to study the DFT. >> Finding the inner product is a method; > > The inner product is a concept as good as any, along with > 'product', 'sum', 'square root' etc. > >> I'm after the meaning. > > What's the meaning of 'difference'? > >> Let me put >> the matter differently: since basis functions are all orthogonal, > > Who said they are? > >> the >> integral over a whole number of periods of the product any one of them >> and any number of the others is necessarily zero. (Definition of >> orthogonality.) > > This is thhe definition of orthogonality. But a set of basis > functions needs not be orthogonal. > >> If the arbitrary term of the integrand includes some >> amount of the multiplying basis function, then the amount by the which >> result differs from zero tells us how much of the basis is "contained" >> in the arbitrary term of the integrand. > > I don't understand what you mean. The concept of 'basis' is > the same if you talk about functions as when you talk about > vectors in 2D. If you have an arbitrary vector x then the > inner product c=<x,b> is computed by prijecting x onto b, > and is thus a measure of 'similarity' between x and b. > One can project x onto several basis vectors simultaneously, > by collecting the basis vector in a matrix B: > > c =Bx > > Now c is a vector which expresses x in terms of the > basis B. Trivial high-school linear algebra, generalizes > straight-forward to the DFT. > > The inner product is stuff I fisrt encountered at age 16, > it is hardly revolutionary, nor controversial. > >> As I see it, that's the same as >> saying it is a measure of how well the two terms in the integrand are >> correlated. > > 'Correlation' is a term that is based on the inner product, > not the other way around. What is the more fundamental concept > to you: 'wheel' or 'bicycle'?I think we're expressing different views of the same concept. Some people think that a sine is not a fundamental concept, but the sum of two exponentials whose exponents differ in sign. I don't, but recognize that it can be expressed that way. I also think that elementary trigonometry is more easily learned from my perspective. Max is seeking knowledge. I don't think that "inner product" is the smoother road. Jerry -- Engineering is the art of making what you want from things you can get. �����������������������������������������������������������������������
Reply by ●July 9, 20082008-07-09
On Jul 9, 4:18�am, Rune Allnor <all...@tele.ntnu.no> wrote:> Now c is a vector which expresses x in terms of the > basis B. Trivial high-school linear algebra, generalizes > straight-forward to the DFT. > > The inner product is stuff I fisrt encountered at age 16, > it is hardly revolutionary, nor controversial.I think the issue is the DFT dot product explanation works for you because it clicks with your background, not because it is easy or can be taught in grade school. An FIR filter is just a dot product too, and probably that view is very meaningful and insightful to you also. But if you talk with an experience analog engineer, someone who excelled in linear algebra class, he still doesn�t want to hear about dot products or linear transforms or optimum least square approximations, he wants to hear about a �bank of sinc filters� analogy. You have to find an analogy that is consistent with a persons background and build on it.
Reply by ●July 9, 20082008-07-09
On Jul 9, 4:18=A0am, Rune Allnor <all...@tele.ntnu.no> wrote:> Now c is a vector which expresses x in terms of the > basis B. Trivial high-school linear algebra, generalizes > straight-forward to the DFT. > > The inner product is stuff I fisrt encountered at age 16, > it is hardly revolutionary, nor controversial.I think the issue is the DFT dot product explanation works for you because it clicks with your background, not because it is easy or can be taught in grade school. An FIR filter is just a dot product too, and probably that view is very meaningful and insightful to you also. But if you talk with an experience analog engineer, someone who excelled in linear algebra class, he still doesn=92t want to hear about dot products or linear transforms or optimum least square approximations, he wants to hear about a =93bank of sinc filters=94 analogy. You have to find an analogy that is consistent with a persons background and build on it.
Reply by ●July 9, 20082008-07-09
On Jul 9, 4:18=A0am, Rune Allnor <all...@tele.ntnu.no> wrote:> Now c is a vector which expresses x in terms of the > basis B. Trivial high-school linear algebra, generalizes > straight-forward to the DFT. > > The inner product is stuff I fisrt encountered at age 16, > it is hardly revolutionary, nor controversial.I think the issue is the DFT dot product explanation works for you because it clicks with your background, not because it is easy or can be taught in grade school. An FIR filter is just a dot product too, and probably that view is very meaningful and insightful to you also. But if you talk with an experience analog engineer, someone who excelled in linear algebra class, he still doesn=92t want to hear about dot products or linear transforms or optimum least square approximations, he wants to hear about a =93bank of sinc filters=94 analogy. You have to find an analogy that is consistent with a persons background and build on it.
Reply by ●July 10, 20082008-07-10
On 10 Jul, 00:31, steve <bungalow_st...@yahoo.com> wrote:> On Jul 9, 4:18�am, Rune Allnor <all...@tele.ntnu.no> wrote: > > > Now c is a vector which expresses x in terms of the > > basis B. Trivial high-school linear algebra, generalizes > > straight-forward to the DFT. > > > The inner product is stuff I fisrt encountered at age 16, > > it is hardly revolutionary, nor controversial. > > I think the issue is the DFT dot product explanation works for you > because it clicks with your background, not because it is easy or can > be taught in grade school.That explanation works because it is accurate as well as easy. An EE practitioner is likely to have encountered the dot product in grade school or college at some point.> An FIR filter is just a dot product too, > and probably that view is very meaningful and insightful to you also.It is.> But if you talk with an experience analog engineer, someone who > excelled in linear algebra class, �he still doesn�t want to hear about > dot products or linear transforms or optimum least square > approximations, he wants to hear about a �bank of sinc filters� > analogy. You have to find an analogy that is consistent with a persons > background and build on it.The key phrase you use is 'wants to hear.' Not 'needs to hear' or 'benefits from hearing'. I know this is considered by many to be controversial, but my view is, and has always been, that analog electronics and DSP are separate diciplines that should be approached separately, each on its own terms. Rune
Reply by ●July 10, 20082008-07-10
Rune Allnor wrote:> On 10 Jul, 00:31, steve <bungalow_st...@yahoo.com> wrote:...>> But if you talk with an experience analog engineer, someone who >> excelled in linear algebra class, he still doesn�t want to hear about >> dot products or linear transforms or optimum least square >> approximations, he wants to hear about a �bank of sinc filters� >> analogy. You have to find an analogy that is consistent with a persons >> background and build on it. > > The key phrase you use is 'wants to hear.' Not 'needs to hear' or > 'benefits from hearing'. I know this is considered by many to be > controversial, but my view is, and has always been, that analog > electronics and DSP are separate diciplines that should be > approached separately, each on its own terms.Do you feel it is a general rule that insights learned from one activity ought not be used to enlighten another, or do you confine the rule to signal processing? Jerry -- Engineering is the art of making what you want from things you can get. �����������������������������������������������������������������������
