DSPRelated.com
Forums

Large FFT vs Many FFTs

Started by Edison April 5, 2007
On 8 Apr, 03:26, "dbd" <d...@ieee.org> wrote:
> On Apr 7, 4:00 pm, "Rune Allnor" <all...@tele.ntnu.no> wrote: > > > > > > > On 8 Apr, 00:01, Randy Yates <y...@ieee.org> wrote: > > > > "Rune Allnor" <all...@tele.ntnu.no> writes: > > > > [...] > > > > On 5 Apr, 12:44, "Edison" <bell...@btinternet.com> wrote: > > > > >> The requirements state that the resulting very narrow bin width > > > >> will reduce the effect of noise. > > > > [...] > > > > Note that Var(P[k]) does *not* depend on the number of > > > > samples N, meaning that the argument that "a large data > > > > sequence reduces noise" is plain wrong. > > > > Perhaps he meant that, for a given noise power spectral density, a > > > smaller bin width reduces the input noise power in that bin, which is > > > true. > > > > The correctness of his statement depends on whether the "noise" he > > > mentioned is input noise or estimation noise (i.e., the variance you > > > were speaking of). > > > The statement "reduce the effect of noise" implies to me that > > one expects a lower variance in the estimated periodogram. > > > Whatever was intended, the OP ought to be aware that the > > periodogram is a notoriosly poor estimator for the PSD. > > > Rune > > The expression: > > Var(P[k]) = P[k]^2. > > is true for bins containing Gaussian noise. It is not true for bins > containing a tone with energy considerably greater than the noise > energy in the bin. The OP has a task of measuring a base frequency > component and harmonic components, not measuring noise.
In the applications I have encountered in the past where "harmonic distrortion" has been an issue, the energy contained in the harmonics have been on the order of 1% - 3% of the total power. The OP is tasked with measuring the 5th harmonic, indicating to me that he is well into the area where the energy of the harmonics are no longer "greater than the noise energy in the bin." In other words, the task is very likely to decide "is this really an harmonic or is it a spurious noise peak?"
> By the way, the internet has made a change in book availability. The > 'old books' are often available used a reasonable cost. I just picked > up a copy of Tukey and Blackman's Dover printing of "The Measurement > of Power Spectra" for under $10. That's a lot more than the $1.85 > price in 1959 but still less than current texbook prices.
Almost half the books in my bookshelf are Dovers. The problem is the books that presently are out of print (like Kay's 1988 book or Scharf's 1991 book) which have not yet become re-published by Dover. Rune
"Rune Allnor" <allnor@tele.ntnu.no> writes:
> [...] > In the applications I have encountered in the past where > "harmonic distrortion" has been an issue, the energy > contained in the harmonics have been on the order of > 1% - 3% of the total power. The OP is tasked with measuring > the 5th harmonic, indicating to me that he is well into > the area where the energy of the harmonics are no longer > "greater than the noise energy in the bin."
The energy of a sinusoid can ALWAYS be made greater than the noise energy in a bin if you make the FFT large enough. Dale's concern that, in practice, the frequency stability of the sinusoid limits the size of the FFT you can use reliably is one that I hadn't heard before, but it makes a lot of sense. -- % Randy Yates % "How's life on earth? %% Fuquay-Varina, NC % ... What is it worth?" %%% 919-577-9882 % 'Mission (A World Record)', %%%% <yates@ieee.org> % *A New World Record*, ELO http://home.earthlink.net/~yatescr
On 8 Apr, 15:48, Randy Yates <y...@ieee.org> wrote:
> "Rune Allnor" <all...@tele.ntnu.no> writes: > > [...] > > In the applications I have encountered in the past where > > "harmonic distrortion" has been an issue, the energy > > contained in the harmonics have been on the order of > > 1% - 3% of the total power. The OP is tasked with measuring > > the 5th harmonic, indicating to me that he is well into > > the area where the energy of the harmonics are no longer > > "greater than the noise energy in the bin." > > The energy of a sinusoid can ALWAYS be made greater than > the noise energy in a bin if you make the FFT large enough.
Provided the sinusoidal is perfect. Which is not the case in any practical measurement.
> Dale's concern that, in practice, the frequency stability > of the sinusoid limits the size of the FFT you can use > reliably is one that I hadn't heard before, but it makes > a lot of sense.
It does, for the very reason that no sinusoidal measured in real-life data is perfect. To what extent Dale and I disagree, it is to whether the local SNR (near each sinusoidal harmonic) or the imperfect sine is the limitig factor. There is no way either of us can know; only the OP can process the data and have a look. They key is to know what to look for. Rune
On Apr 8, 4:24 am, "Rune Allnor" <all...@tele.ntnu.no> wrote:
> On 8 Apr, 03:26, "dbd" <d...@ieee.org> wrote: > > ... > > By the way, the internet has made a change in book availability. The > > 'old books' are often available used a reasonable cost. I just picked > > up a copy of Tukey and Blackman's Dover printing of "The Measurement > > of Power Spectra" for under $10. That's a lot more than the $1.85 > > price in 1959 but still less than current texbook prices. > > Almost half the books in my bookshelf are Dovers. The problem > is the books that presently are out of print (like Kay's 1988 book > or Scharf's 1991 book) which have not yet become re-published > by Dover. > > Rune
"Measurement of Power Spectra" is out of print but there were copies available for sale used. When I looked a year ago there weren't. So it is hit and miss, but there are booksellers' offerings aggregated on the web in a way they weren't before the web. If you mean Scharf's "Statistical Signal Processing", amazon.com lists 14 sources. But the prices aren't like Dover. Dale B. Dalrymple http://dbdimages.com
On 8 Apr, 20:38, "dbd" <d...@ieee.org> wrote:
> [books] available for sale used.
To paraphrase some madman from times long since past: "I would never buy a used book from somebody who might consider selling his copy." Rune
On Apr 5, 7:01 am, R.Lyons@_BOGUS_ieee.org (Rick Lyons) wrote:
> Of course, smaller-sized FFTs will NOT have the > "fine-grained" frequency sample spacing that is > provided by large-sized FFTs.
However, smaller-sized FFTs, if their window offsets are known, do contain finer-grained information about sinusoids that span multiple FFTs. This principle is the one used by phase vocoder analysis. If one knows the exact ratios between the frequencies of interest and the sampling rate and window offsets, then coherent averaging of multiple complex FFTs might help improve the S/N ratio of the result over incoherent averaging of just the bin magnitudes. IMHO. YMMV. -- rhn A.T nicholson d.0.t C-o-M
On Apr 8, 1:55 pm, "Rune Allnor" <all...@tele.ntnu.no> wrote:
> On 8 Apr, 20:38, "dbd" <d...@ieee.org> wrote: > > > [books] available for sale used. > > To paraphrase some madman from times long since past: > > "I would never buy a used book from somebody who might > consider selling his copy." > > Rune
Amazon.com reports one of the sources for "Statistical Signal Processing" as hardcover print to order. The traditional meaning of 'out of print' was that the publisher had no more stock and simply kept available no records of the location of unsold uncirculated books. Now there are information aggregators like Amazon collecting this information, collecting the rights to reprint and collecting locations of available circulated copies. Rune, what is your position on libraries? Would you never borrow a book from somebody who would consider lending his copy to you? Dale B. Dalrymple http://dbdimages.com
dbd wrote:
> On Apr 8, 1:55 pm, "Rune Allnor" <all...@tele.ntnu.no> wrote: >> On 8 Apr, 20:38, "dbd" <d...@ieee.org> wrote: >> >>> [books] available for sale used. >> To paraphrase some madman from times long since past: >> >> "I would never buy a used book from somebody who might >> consider selling his copy." >> >> Rune > > > Amazon.com reports one of the sources for "Statistical Signal > Processing" as hardcover print to order. The traditional meaning of > 'out of print' was that the publisher had no more stock and simply > kept available no records of the location of unsold uncirculated > books. Now there are information aggregators like Amazon collecting > this information, collecting the rights to reprint and collecting > locations of available circulated copies. > > Rune, what is your position on libraries? Would you never borrow a > book from somebody who would consider lending his copy to you?
I think I know what Rune means. Too often, someone selling his books selects for sale those he thinks are inferior, and he's probably right. I have bought and been satisfied with used books from people in various circumstances, including recently bereaved spouses. Libraries don't enter this picture. Jerry -- Engineering is the art of making what you want from things you can get. &macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;&macr;
On Apr 8, 6:15 pm, Jerry Avins <j...@ieee.org> wrote:
> dbd wrote: > > On Apr 8, 1:55 pm, "Rune Allnor" <all...@tele.ntnu.no> wrote: > >> On 8 Apr, 20:38, "dbd" <d...@ieee.org> wrote: > > >>> [books] available for sale used. > >> To paraphrase some madman from times long since past: > > >> "I would never buy a used book from somebody who might > >> consider selling his copy." > > >> Rune > > > Amazon.com reports one of the sources for "Statistical Signal > > Processing" as hardcover print to order. The traditional meaning of > > 'out of print' was that the publisher had no more stock and simply > > kept available no records of the location of unsold uncirculated > > books. Now there are information aggregators like Amazon collecting > > this information, collecting the rights to reprint and collecting > > locations of available circulated copies. > > > Rune, what is your position on libraries? Would you never borrow a > > book from somebody who would consider lending his copy to you? > > I think I know what Rune means. Too often, someone selling his books > selects for sale those he thinks are inferior, and he's probably right. > I have bought and been satisfied with used books from people in various > circumstances, including recently bereaved spouses. Libraries don't > enter this picture. > > Jerry > -- > Engineering is the art of making what you want from things you can get. > =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=
=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF= =AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF=AF Performing a search on the net for a specific author or book doesn't enter this picture either. After reading Rune's earlier post one could search amazon for "Scharf" and find "Statistical Signal Processing" if it is available. It was. If he had given the name of the book I could have narrowed the search. The book sorters and spouses can only make the books available, they don't select what you search for or buy. If I bolstered my arguments with references I left vague so I wouldn't have to spend time checking, I'd discourage discussion of topics that would lead people to follow up my references in detail. As the Great OZ said: "Pay no attention to the man behind the curtain". Dale B. Dalrymple http://dbdimages.com
On 9 Apr, 01:49, "dbd" <d...@ieee.org> wrote:
> On Apr 8, 1:55 pm, "Rune Allnor" <all...@tele.ntnu.no> wrote: > > > On 8 Apr, 20:38, "dbd" <d...@ieee.org> wrote: > > > > [books] available for sale used. > > > To paraphrase some madman from times long since past: > > > "I would never buy a used book from somebody who might > > consider selling his copy." > > > Rune > > Amazon.com reports one of the sources for "Statistical Signal > Processing" as hardcover print to order.
If so, it has changed since I bought my copy. At the time, the book was availabe as "new". I interpreted this as original publisher's binding, what I got was a print to order. I have no problem with that, as long as what is sold is what is advertised.
> Rune, what is your position on libraries? Would you never borrow a > book from somebody who would consider lending his copy to you?
A library is a library, and you know what (not) to expect. Where I used to work the academic libraries were poor; the best one was last reasonably up-to-date some time just before 1980. If you need an early classic you know where to look and maybe even find it. I wanted -- and still wants -- a compehensive selection of useful books availale to me when I need them. I have no other choise but to spend my own hard-earned $$ buying them myself. Rune