DSPRelated.com
Forums

spectral estimation question

Started by Randy Yates April 2, 2006
Mark skrev:
> Randy Yates wrote: > > Jerry Avins <jya@ieee.org> writes: > > > > > Ron N. wrote: > > >> Randy Yates wrote: > > >> > > >>>OK, here's one of those "This should be simple..." questions. > > >>> > > >>>We know that many (all?) methods of spectral estimation of > > >>>discrete-time data rely fundamentally on the DFT (the N-point DFT). > > >>> > > >>>However, the output of a specific bin of the DFT can be viewed as a > > >>>bandpass filter. This DFT bandpass filter is not particularly "good," > > >>>i.e., it has high side lobes and poor stop-band rejection. > > >>> > > > There has been a lot of discussion about the assumed finite time > duration of the input time domain data fundamentally implying a limited > resolution in the output in the frequency domain. Also if we assume > the input data is repetitive we can improve the output frequency > resolution. OK Good. Yes we can improve the resolution of the DFT by > making each bin narrower. > > But I was hoping to see some discussion about improving the SHAPE > FACTOR of the DFT bin filters, i.e. making the skirts steeper for a > given passband bandwidth. I think that was what Randy was getting at > with his original question. > > The SHAPE FACTOR of the DFT bins is fixed. But if we were making > individual filters instead of using the DFT, we could use "better" > filters ...here better does not just mean narrower, but instead > better means with a better SHAPE FACTOR.
I don't understand this. I might agree with the bandpass filter as a useful *analogy* for the DFT bin, but I am not comfortable with a discussion about the DFT that is based on the properties of the analogy and not the DFT. The properties of the DFT are well-known. The DFT is an abstract tool that has been around for a couple of centuries, at least since the time of Gauss. Being an abstract tool, the DFT should be discussed in abstract terms. Once the terms "finite data records" and "DFT" appear in the same sentence, the term "window function" usually appears in the next. If people have a problem with that, the only possible solution is to avoid bringing the DFT into the discussion in the first place. As for power spectrum analysis, that can be achieved by using wavelets or (sub)octave-band filters, as lots of people do in practical applications. It might be very interesting to discuss the merits of different filters in that setting. as far as I am concerned, it is futile to discus filters as a method to improve on the DFT. Rune
Mark wrote:
> > >> Randy Yates wrote: > > >> > > >>>OK, here's one of those "This should be simple..." questions. > > >>> > > >>>We know that many (all?) methods of spectral estimation of > > >>>discrete-time data rely fundamentally on the DFT (the N-point DFT). > > >>> > > >>>However, the output of a specific bin of the DFT can be viewed as a > > >>>bandpass filter. This DFT bandpass filter is not particularly "good," > > >>>i.e., it has high side lobes and poor stop-band rejection. > > >>> > > > There has been a lot of discussion about the assumed finite time > duration of the input time domain data fundamentally implying a limited > resolution in the output in the frequency domain. Also if we assume > the input data is repetitive we can improve the output frequency > resolution. OK Good. Yes we can improve the resolution of the DFT by > making each bin narrower. > > But I was hoping to see some discussion about improving the SHAPE > FACTOR of the DFT bin filters, i.e. making the skirts steeper for a > given passband bandwidth. I think that was what Randy was getting at > with his original question. > > The SHAPE FACTOR of the DFT bins is fixed. But if we were making > individual filters instead of using the DFT, we could use "better" > filters ...here better does not just mean narrower, but instead > better means with a better SHAPE FACTOR.
Here is where one of the DFT's properties comes in handy. The filters constituting a DFT filter bank are such that when used as basis functions they are all completely orthogonal. That means that although the output from a single filter might be considered poor in terms of passband, the output from multiple bins or filters isn't aliased, and thus conveys more information. Thus the DFT allows a clean interpolation of some sort of continuous spectrum between the bins (how well depends on things such as how close any lowpass pre-filter is to a bandlimiting process, windowing, quantization error, etc.) Depending on how many bins, you might be able to interpret the shape factor of a composite filter as extremely narrow. This is because all N bins of a DFT are information preserving, unlike some arbitrary combination of filters with a better shape factor. With a filter bank of "better" filters, you might not have enough and be underdetermined, which means that the process would be lossy, thus making it possible to construct signals which might cause some sort of problem in the interpretation of the results. Or you might have too many and be not only overdetermined, but much less efficient in the quality of the results vs. the compute power consumed. IMHO. YMMV. -- rhn A.T nicholson d.0.t C-o-M
"Randy Yates" <yates@ieee.org> wrote in message 
news:m3fykvom5y.fsf@ieee.org...
> OK, here's one of those "This should be simple..." questions. > > We know that many (all?) methods of spectral estimation of > discrete-time data rely fundamentally on the DFT (the N-point DFT). > > However, the output of a specific bin of the DFT can be viewed as a > bandpass filter. This DFT bandpass filter is not particularly "good," > i.e., it has high side lobes and poor stop-band rejection. > > So, how do we ever get a good spectral estimate using such inherently > poor filters? > > Now I'm pretty sure someone's going to bring up windowing, but in my > view windowing is a technique to tradeoff resolution and out-of-band > rejection due to the finite-extent of the data. The problem with DFT > lobes is separate. I think...
I think someone touched on it but here's a perspective that we haven't addressed all that much: Given a sinusoid. Given a finite-length Fourier Transform. Then: The rectangular window gives the best least squares estimate for the sinusoid. In other words, it is a matched filter for a sinusoid. There are a number of ways to come at this to show it's the case. Here's one: The "filter" / window is the same length as the sinusoid. The Fourier Transform kernel frequency is the same as the frequency of the sinusoid at one point in frequency. Then you have a matched filter you're convolving with in doing the Fourier Transform. So, in that sense it is the optimum filter and has the perfect shape factor for detecting the sinusoid in noise. This is often the spectral analysis objective. Fred
Fred Marshall skrev:
> "Randy Yates" <yates@ieee.org> wrote in message > news:m3fykvom5y.fsf@ieee.org... > > OK, here's one of those "This should be simple..." questions. > > > > We know that many (all?) methods of spectral estimation of > > discrete-time data rely fundamentally on the DFT (the N-point DFT). > > > > However, the output of a specific bin of the DFT can be viewed as a > > bandpass filter. This DFT bandpass filter is not particularly "good," > > i.e., it has high side lobes and poor stop-band rejection. > > > > So, how do we ever get a good spectral estimate using such inherently > > poor filters? > > > > Now I'm pretty sure someone's going to bring up windowing, but in my > > view windowing is a technique to tradeoff resolution and out-of-band > > rejection due to the finite-extent of the data. The problem with DFT > > lobes is separate. I think... > > I think someone touched on it but here's a perspective that we haven't > addressed all that much: > > Given a sinusoid. > Given a finite-length Fourier Transform. > Then: > The rectangular window gives the best least squares estimate for the > sinusoid. > In other words, it is a matched filter for a sinusoid. > > There are a number of ways to come at this to show it's the case. > > Here's one: > > The "filter" / window is the same length as the sinusoid. The Fourier > Transform kernel frequency is the same as the frequency of the sinusoid at > one point in frequency. Then you have a matched filter you're convolving > with in doing the Fourier Transform.
And the Fourier coefficient becomes the output of that matched filter at its peak. I agree with this, in principle, but some people might object that the foutput of the filter ought to be a sequence, not a scalar. One way to avoid this objection could be to introduce the vector space, where the matched filter or correlation are viewed as mappings of the test signal onto some reference basis. In that case, we can use the familiar result from elementary geometry where we find that the inner product c = <a,b>, |a| = |b| = 1 reaches its maximum for a == b. While this avoids the objection about the filter producing a sequence as output, some people might object that this is a non-intuitive way of analyzing the DFT.
> So, in that sense it is the optimum filter and has the perfect shape factor > for detecting the sinusoid in noise. This is often the spectral analysis > objective.
Agreed. BTW, there was a very interesting paper on detecting sinusoids in noise in the October 2005 issue of the IEEE Journal of Oceanic Engineering. Don't remember authors or title. The technique seemed to require fairly long measurements, though. Rune
Rune Allnor wrote:
> Fred Marshall skrev: > > "Randy Yates" <yates@ieee.org> wrote in message > > news:m3fykvom5y.fsf@ieee.org... > > > OK, here's one of those "This should be simple..." questions. > > > > > > We know that many (all?) methods of spectral estimation of > > > discrete-time data rely fundamentally on the DFT (the N-point DFT). > > > > > > However, the output of a specific bin of the DFT can be viewed as a > > > bandpass filter. This DFT bandpass filter is not particularly "good," > > > i.e., it has high side lobes and poor stop-band rejection. > > > > > > So, how do we ever get a good spectral estimate using such inherently > > > poor filters? > > > > > > Now I'm pretty sure someone's going to bring up windowing, but in my > > > view windowing is a technique to tradeoff resolution and out-of-band > > > rejection due to the finite-extent of the data. The problem with DFT > > > lobes is separate. I think... > > > > I think someone touched on it but here's a perspective that we haven't > > addressed all that much: > > > > Given a sinusoid. > > Given a finite-length Fourier Transform. > > Then: > > The rectangular window gives the best least squares estimate for the > > sinusoid. > > In other words, it is a matched filter for a sinusoid. > > > > There are a number of ways to come at this to show it's the case. > > > > Here's one: > > > > The "filter" / window is the same length as the sinusoid. The Fourier > > Transform kernel frequency is the same as the frequency of the sinusoid at > > one point in frequency. Then you have a matched filter you're convolving > > with in doing the Fourier Transform. > > And the Fourier coefficient becomes the output of that matched filter > at its peak. I agree with this, in principle, but some people might > object that the foutput of the filter ought to be a sequence, not a > scalar.
If you view the FT coefficient as just a single value from a sliding FT or a vector of FT's each following a sliding window, then you do infer a sequence. I think that a sliding FT requires one to think of any single FT result as an incomplete basis transform of a signal larger than the FT window, unless you just zero pad one window. IMHO. YMMV. -- rhn A.T nicholson d.0.t C-o-M
On Sun, 02 Apr 2006 18:39:33 GMT, Randy Yates <yates@ieee.org> wrote:

>OK, here's one of those "This should be simple..." questions. > >We know that many (all?) methods of spectral estimation of >discrete-time data rely fundamentally on the DFT (the N-point DFT). > >However, the output of a specific bin of the DFT can be viewed as a >bandpass filter. This DFT bandpass filter is not particularly "good," >i.e., it has high side lobes and poor stop-band rejection. > >So, how do we ever get a good spectral estimate using such inherently >poor filters? > >Now I'm pretty sure someone's going to bring up windowing, but in my >view windowing is a technique to tradeoff resolution and out-of-band >rejection due to the finite-extent of the data. The problem with DFT >lobes is separate. I think... >-- >% Randy Yates % "With time with what you've learned,
Hi Randy, wow your post generated many responses. I didn't read then all. I don't know if it'll help you any, but I wondered if you checked the spectrum analysis "trick" on page 544 of my DSP book. That material seems, to me, to be related to your question. By the way, I recently found a typo on page 544. The "N" in the denominator of the exponent in Eq. (13-104) should be an "M". Good Luck, [-Rick-]
"Ron N." <rhnlogic@yahoo.com> wrote in message 
news:1144262147.792726.297520@u72g2000cwu.googlegroups.com...
> Rune Allnor wrote: >> Fred Marshall skrev: >> > "Randy Yates" <yates@ieee.org> wrote in message >> > news:m3fykvom5y.fsf@ieee.org... >> > > OK, here's one of those "This should be simple..." questions. >> > > >> > > We know that many (all?) methods of spectral estimation of >> > > discrete-time data rely fundamentally on the DFT (the N-point DFT). >> > > >> > > However, the output of a specific bin of the DFT can be viewed as a >> > > bandpass filter. This DFT bandpass filter is not particularly "good," >> > > i.e., it has high side lobes and poor stop-band rejection. >> > > >> > > So, how do we ever get a good spectral estimate using such inherently >> > > poor filters? >> > > >> > > Now I'm pretty sure someone's going to bring up windowing, but in my >> > > view windowing is a technique to tradeoff resolution and out-of-band >> > > rejection due to the finite-extent of the data. The problem with DFT >> > > lobes is separate. I think... >> > >> > I think someone touched on it but here's a perspective that we haven't >> > addressed all that much: >> > >> > Given a sinusoid. >> > Given a finite-length Fourier Transform. >> > Then: >> > The rectangular window gives the best least squares estimate for the >> > sinusoid. >> > In other words, it is a matched filter for a sinusoid. >> > >> > There are a number of ways to come at this to show it's the case. >> > >> > Here's one: >> > >> > The "filter" / window is the same length as the sinusoid. The Fourier >> > Transform kernel frequency is the same as the frequency of the sinusoid >> > at >> > one point in frequency. Then you have a matched filter you're >> > convolving >> > with in doing the Fourier Transform. >> >> And the Fourier coefficient becomes the output of that matched filter >> at its peak. I agree with this, in principle, but some people might >> object that the foutput of the filter ought to be a sequence, not a >> scalar. > > If you view the FT coefficient as just a single value from a > sliding FT or a vector of FT's each following a sliding window, > then you do infer a sequence. > > I think that a sliding FT requires one to think of any single FT > result as an incomplete basis transform of a signal larger than > the FT window, unless you just zero pad one window.
Just to be clear: When I say "Fourier Transform" I mean the continuous, infinite time FT. I think here you mean finite DFT or there wouldn't be any "sliding" involved. Also, I think it's reasonable to say that it doesn't matter if a finite DFT is on a signal that is limited to the finite temporal length of the transform or a signal that is longer. The effect is the same. Well, unless you do something like time sliding and frequency averaging. What do you mean by "incomplete basis transform"? I'm interested. Fred
Ron N. skrev:
> Rune Allnor wrote: > > Fred Marshall skrev: > > > "Randy Yates" <yates@ieee.org> wrote in message > > > news:m3fykvom5y.fsf@ieee.org... > > > > OK, here's one of those "This should be simple..." questions. > > > > > > > > We know that many (all?) methods of spectral estimation of > > > > discrete-time data rely fundamentally on the DFT (the N-point DFT). > > > > > > > > However, the output of a specific bin of the DFT can be viewed as a > > > > bandpass filter. This DFT bandpass filter is not particularly "good," > > > > i.e., it has high side lobes and poor stop-band rejection. > > > > > > > > So, how do we ever get a good spectral estimate using such inherently > > > > poor filters? > > > > > > > > Now I'm pretty sure someone's going to bring up windowing, but in my > > > > view windowing is a technique to tradeoff resolution and out-of-band > > > > rejection due to the finite-extent of the data. The problem with DFT > > > > lobes is separate. I think... > > > > > > I think someone touched on it but here's a perspective that we haven't > > > addressed all that much: > > > > > > Given a sinusoid. > > > Given a finite-length Fourier Transform. > > > Then: > > > The rectangular window gives the best least squares estimate for the > > > sinusoid. > > > In other words, it is a matched filter for a sinusoid. > > > > > > There are a number of ways to come at this to show it's the case. > > > > > > Here's one: > > > > > > The "filter" / window is the same length as the sinusoid. The Fourier > > > Transform kernel frequency is the same as the frequency of the sinusoid at > > > one point in frequency. Then you have a matched filter you're convolving > > > with in doing the Fourier Transform. > > > > And the Fourier coefficient becomes the output of that matched filter > > at its peak. I agree with this, in principle, but some people might > > object that the foutput of the filter ought to be a sequence, not a > > scalar. > > If you view the FT coefficient as just a single value from a > sliding FT or a vector of FT's each following a sliding window, > then you do infer a sequence.
That's not an FT. That's a filter.
> I think that a sliding FT requires one to think of any single FT > result as an incomplete basis transform of a signal larger than > the FT window, unless you just zero pad one window.
I don't understand what you mean. The FT is well-defined as the set of inner products between the signal and the sinusoidal basis functions. Both the signal and basis functions have the same support, i.e. if the signal is defined from -infinite to +infinite, so are the basis functions. If the signal is defined on some finite interval, so are the basis functions. Introducing some sliding functions only create a mess. For instance, how do you define phase if the sines slide along the time axis? Where is your time reference? Rune
Rune Allnor wrote:
> Ron N. skrev: > > Rune Allnor wrote: > > > Fred Marshall skrev: > > > > "Randy Yates" <yates@ieee.org> wrote in message > > > > news:m3fykvom5y.fsf@ieee.org... > > > > > OK, here's one of those "This should be simple..." questions. > > > > > > > > > > We know that many (all?) methods of spectral estimation of > > > > > discrete-time data rely fundamentally on the DFT (the N-point DFT). > > > > > > > > > > However, the output of a specific bin of the DFT can be viewed as a > > > > > bandpass filter. This DFT bandpass filter is not particularly "good," > > > > > i.e., it has high side lobes and poor stop-band rejection. > > > > > > > > > > So, how do we ever get a good spectral estimate using such inherently > > > > > poor filters? > > > > > > > > > > Now I'm pretty sure someone's going to bring up windowing, but in my > > > > > view windowing is a technique to tradeoff resolution and out-of-band > > > > > rejection due to the finite-extent of the data. The problem with DFT > > > > > lobes is separate. I think... > > > > > > > > I think someone touched on it but here's a perspective that we haven't > > > > addressed all that much: > > > > > > > > Given a sinusoid. > > > > Given a finite-length Fourier Transform. > > > > Then: > > > > The rectangular window gives the best least squares estimate for the > > > > sinusoid. > > > > In other words, it is a matched filter for a sinusoid. > > > > > > > > There are a number of ways to come at this to show it's the case. > > > > > > > > Here's one: > > > > > > > > The "filter" / window is the same length as the sinusoid. The Fourier > > > > Transform kernel frequency is the same as the frequency of the sinusoid at > > > > one point in frequency. Then you have a matched filter you're convolving > > > > with in doing the Fourier Transform. > > > > > > And the Fourier coefficient becomes the output of that matched filter > > > at its peak. I agree with this, in principle, but some people might > > > object that the foutput of the filter ought to be a sequence, not a > > > scalar. > > > > If you view the FT coefficient as just a single value from a > > sliding FT or a vector of FT's each following a sliding window, > > then you do infer a sequence. > > That's not an FT. That's a filter. > > > I think that a sliding FT requires one to think of any single FT > > result as an incomplete basis transform of a signal larger than > > the FT window, unless you just zero pad one window. > > I don't understand what you mean. The FT is well-defined as the > set of inner products between the signal and the sinusoidal > basis functions. Both the signal and basis functions have the > same support, i.e. if the signal is defined from -infinite to > +infinite, > so are the basis functions. If the signal is defined on some finite > interval, so are the basis functions. Introducing some sliding > functions only create a mess.
I was responding to the post above regarding the quote "finite length FT" and how to make it into a "sequence and not a scalar" (not my words, ask the previous poster what he meant). Assuming such a beast exists, then if the finite length FT is say 2 inches long, and you have a 4 inch signal, then it will take 2 of these FT's to produce enough results to reconstruct the 4 inch signal. But the two halves will also be orthogonal. So it might be reasonable to call these 2 FT's together as a complete basis set for some sort of transform for signals twice as long as the finite FT length. Extend for even longer signals and for DFT's. It's not only not a mess, but perhaps a reasonable way to view phase vocoder algorithms, which work better than using a single longer FT's for some applications.
> For instance, how do you define phase if the sines slide along the > time axis? Where is your time reference?
Same way you do in the time domain. Pick just one of the sliding sines and call it a zero reference. IMHO. YMMV. -- rhn A.T nicholson d.0.t C-o-M
Ron N. skrev:
> Rune Allnor wrote: > > Ron N. skrev: > > > Rune Allnor wrote: > > > > Fred Marshall skrev: > > > > > "Randy Yates" <yates@ieee.org> wrote in message > > > > > news:m3fykvom5y.fsf@ieee.org... > > > > > > OK, here's one of those "This should be simple..." questions. > > > > > > > > > > > > We know that many (all?) methods of spectral estimation of > > > > > > discrete-time data rely fundamentally on the DFT (the N-point DFT). > > > > > > > > > > > > However, the output of a specific bin of the DFT can be viewed as a > > > > > > bandpass filter. This DFT bandpass filter is not particularly "good," > > > > > > i.e., it has high side lobes and poor stop-band rejection. > > > > > > > > > > > > So, how do we ever get a good spectral estimate using such inherently > > > > > > poor filters? > > > > > > > > > > > > Now I'm pretty sure someone's going to bring up windowing, but in my > > > > > > view windowing is a technique to tradeoff resolution and out-of-band > > > > > > rejection due to the finite-extent of the data. The problem with DFT > > > > > > lobes is separate. I think... > > > > > > > > > > I think someone touched on it but here's a perspective that we haven't > > > > > addressed all that much: > > > > > > > > > > Given a sinusoid. > > > > > Given a finite-length Fourier Transform. > > > > > Then: > > > > > The rectangular window gives the best least squares estimate for the > > > > > sinusoid. > > > > > In other words, it is a matched filter for a sinusoid. > > > > > > > > > > There are a number of ways to come at this to show it's the case. > > > > > > > > > > Here's one: > > > > > > > > > > The "filter" / window is the same length as the sinusoid. The Fourier > > > > > Transform kernel frequency is the same as the frequency of the sinusoid at > > > > > one point in frequency. Then you have a matched filter you're convolving > > > > > with in doing the Fourier Transform. > > > > > > > > And the Fourier coefficient becomes the output of that matched filter > > > > at its peak. I agree with this, in principle, but some people might > > > > object that the foutput of the filter ought to be a sequence, not a > > > > scalar. > > > > > > If you view the FT coefficient as just a single value from a > > > sliding FT or a vector of FT's each following a sliding window, > > > then you do infer a sequence. > > > > That's not an FT. That's a filter. > > > > > I think that a sliding FT requires one to think of any single FT > > > result as an incomplete basis transform of a signal larger than > > > the FT window, unless you just zero pad one window. > > > > I don't understand what you mean. The FT is well-defined as the > > set of inner products between the signal and the sinusoidal > > basis functions. Both the signal and basis functions have the > > same support, i.e. if the signal is defined from -infinite to > > +infinite, > > so are the basis functions. If the signal is defined on some finite > > interval, so are the basis functions. Introducing some sliding > > functions only create a mess. > > I was responding to the post above regarding the quote > "finite length FT" and how to make it into a "sequence and > not a scalar"
I don't understand why somebody would want to start messing with the FT. If you go back to one of my first posts in this thread, you will find that I warned against imposing "intuitive" models on abstract concepts, since an intuitive model will only have a limited validity, and at some point will start to create more confsion than would be the case were it not applied. I think we are past that point now.
> (not my words, ask the previous poster what > he meant). Assuming such a beast exists, then if the finite > length FT is say 2 inches long, and you have a 4 inch signal, > then it will take 2 of these FT's to produce enough results > to reconstruct the 4 inch signal.
I know spatial analysis can be fun, but most people prefer to discuss FTs in time domain.
> But the two halves will also be orthogonal. > So it might be reasonable to call these > 2 FT's together as a complete basis set for some sort of > transform for signals twice as long as the finite FT length.
This is not at all reasonable. The FT of each signal defined on each 2 cm signal (I hope you orgive my prefernce of SI units...) will be a discrete wavenumber spectrum with wavenumbers 0,1*2pi/2cm, 2*2pi/2cm, 3*2pi/2cm... The FT of the concatenated sequences, totla length 4 cm, will contain coefficients at wavenumbers 0,1*2pi/4cm,2*2pi/4cm,3*2pi/4cm,... So the concatenated basis is far from complete.
> Extend for even longer signals and for DFT's.
I don't have the faintest clue what you mean.
> It's not only not a mess, but perhaps a reasonable way to > view phase vocoder algorithms, which work better than using > a single longer FT's for some applications.
I don't know how these types of algorithms work. As far as I am cocerned, this thread is about the DFT, not vocoders.
> > For instance, how do you define phase if the sines slide along the > > time axis? Where is your time reference? > > Same way you do in the time domain. Pick just one of the > sliding sines and call it a zero reference.
Could you elaborate, please? Rune