Hi DSP experts, I want to do spectral analysis of a vector. I want to divide the vector into equal length block, multiply each block to a window, then do FFT on each block, then at each frequency, add the magnitude from each block’s FFT magnutide spectrum, divide by number of blocks, to obtain the averaged magnitude spectrum. I have two questions: 1. Can the blocks overlap? 2. How can I get the averaged power spectrum? Get sum of the square of magnitude at each frequency then divide by number of block? Thank you for your input.
Reducing noise impact in FFT: averaged segment method
Started by ●October 5, 2010
Reply by ●October 5, 20102010-10-05
On Oct 5, 5:39�pm, "shinchan75034" <shinchan75034@n_o_s_p_a_m.gmail.com> wrote:> Hi DSP experts, > I want to do spectral analysis of a vector. I want to divide the vector > into equal length block, multiply each block to a window, then do FFT on > each block, then at each frequency, add the magnitude from each block�s > FFT magnutide spectrum, divide by number of blocks, to obtain the averaged > magnitude spectrum. > > I have two questions: > 1. Can the blocks overlap? > 2. How can I get the averaged power spectrum? Get sum of the square of > magnitude at each frequency then divide by number of block? > > Thank you for your input.Yes and yes.
Reply by ●October 5, 20102010-10-05
Reply by ●October 6, 20102010-10-06
On Oct 5, 2:39�pm, "shinchan75034" <shinchan75034@n_o_s_p_a_m.gmail.com> wrote:> Hi DSP experts, > I want to do spectral analysis of a vector. I want to divide the vector > into equal length block, multiply each block to a window, then do FFT on > each block, then at each frequency, add the magnitude from each block�s > FFT magnutide spectrum, divide by number of blocks, to obtain the averaged > magnitude spectrum. > > I have two questions: > 1. Can the blocks overlap? > 2. How can I get the averaged power spectrum? Get sum of the square of > magnitude at each frequency then divide by number of block? > > Thank you for your input.Rick Lyons recently reminded me of a paper on flattop windows that begins with a nice explanation of the issues you have asked about. Look at pages 5-21 of the paper at: http://www.rssd.esa.int/SP/LISAPATHFINDER/docs/Data_Analysis/GH_FFT.pdf The rest of the paper may also be useful to you. Dale B. Dalrymple There is a paper you can
Reply by ●October 6, 20102010-10-06
On Oct 5, 11:05�pm, dbd <d...@ieee.org> wrote:> On Oct 5, 2:39�pm, "shinchan75034" > > <shinchan75034@n_o_s_p_a_m.gmail.com> wrote: > > Hi DSP experts, > > I want to do spectral analysis of a vector. I want to divide the vector > > into equal length block, multiply each block to a window, then do FFT on > > each block, then at each frequency, add the magnitude from each block�s > > FFT magnutide spectrum, divide by number of blocks, to obtain the averaged > > magnitude spectrum. > > > I have two questions: > > 1. Can the blocks overlap? > > 2. How can I get the averaged power spectrum? Get sum of the square of > > magnitude at each frequency then divide by number of block? > > > Thank you for your input. > > Rick Lyons recently reminded me of a paper on flattop windows that > begins with a nice explanation of the issues you have asked about. > Look at pages 5-21 of the paper at: > > http://www.rssd.esa.int/SP/LISAPATHFINDER/docs/Data_Analysis/GH_FFT.pdf > > The rest of the paper may also be useful to you. > > Dale B. Dalrymple > There is a paper you canWow Dale, For the life of me I can't figure out why you would recommend a "measurement" flat-top window for the OP's application. A "measurement" flat-top produces a lot of smearing in the spectrum, and has the property that it gets rid of the scalloping loss in the measurement of an isolated sinusoid/sinusoidal component, where the smearing doesn't efect the measurement value, but certainly effects the computed spectrum (see page 38 of your recommended article). Doesn't sound like the OP wants that. Why did you pick a "measurement" flat-top window to suggest? Where you thinking of a "time doman" flat-top window (quite different)? Maybe you are just poking fun at the OP? Dirk
Reply by ●October 6, 20102010-10-06
On Oct 5, 11:05�pm, dbd <d...@ieee.org> wrote:> On Oct 5, 2:39�pm, "shinchan75034" > > <shinchan75034@n_o_s_p_a_m.gmail.com> wrote: > > Hi DSP experts, > > I want to do spectral analysis of a vector. I want to divide the vector > > into equal length block, multiply each block to a window, then do FFT on > > each block, then at each frequency, add the magnitude from each block�s > > FFT magnutide spectrum, divide by number of blocks, to obtain the averaged > > magnitude spectrum. > > > I have two questions: > > 1. Can the blocks overlap? > > 2. How can I get the averaged power spectrum? Get sum of the square of > > magnitude at each frequency then divide by number of block? > > > Thank you for your input. > > Rick Lyons recently reminded me of a paper on flattop windows that > begins with a nice explanation of the issues you have asked about. > Look at pages 5-21 of the paper at: > > http://www.rssd.esa.int/SP/LISAPATHFINDER/docs/Data_Analysis/GH_FFT.pdf > > The rest of the paper may also be useful to you. > > Dale B. Dalrymple > There is a paper you canWow Dale, For the life of me I can't figure out why you would recommend a "measurement" flat-top window for the OP's application. A "measurement" flat-top window produces a lot of smearing in the computed spectrum. It has the property that it gets rid of the scalloping loss in the measurement of an isolated sinusoid/sinusoidal component, where the smearing doesn't effect the measurement value, but certainly effects the computed spectrum (see page 38 of your recommended article). Doesn't sound like the OP wants that. Why did you suggest a "measurement" flat-top window? Were you thinking of a "time domain" flat-top window (quite different)? Were you are just poking fun at the OP? Dirk
Reply by ●October 6, 20102010-10-06
On Oct 5, 11:05�pm, dbd <d...@ieee.org> wrote:> On Oct 5, 2:39�pm, "shinchan75034" > > <shinchan75034@n_o_s_p_a_m.gmail.com> wrote: > > Hi DSP experts, > > I want to do spectral analysis of a vector. I want to divide the vector > > into equal length block, multiply each block to a window, then do FFT on > > each block, then at each frequency, add the magnitude from each block�s > > FFT magnutide spectrum, divide by number of blocks, to obtain the averaged > > magnitude spectrum. > > > I have two questions: > > 1. Can the blocks overlap? > > 2. How can I get the averaged power spectrum? Get sum of the square of > > magnitude at each frequency then divide by number of block? > > > Thank you for your input. > > Rick Lyons recently reminded me of a paper on flattop windows that > begins with a nice explanation of the issues you have asked about. > Look at pages 5-21 of the paper at: > > http://www.rssd.esa.int/SP/LISAPATHFINDER/docs/Data_Analysis/GH_FFT.pdf > > The rest of the paper may also be useful to you. > > Dale B. Dalrymple > There is a paper you canDale, You might want to steer the OP away from "measurement" flat-top windows for the OP's application. I don't think that part of the rest of the paper will be useful to the OP. Dirk
Reply by ●October 8, 20102010-10-08
On Oct 6, 11:03�am, Dirk Bell <bellda2...@cox.net> wrote:> On Oct 5, 11:05�pm, dbd <d...@ieee.org> wrote: > ... > > The rest of the paper may also be useful to you. > > > Dale B. Dalrymple > > Dale, > > You might want to steer the OP away from "measurement" flat-top > windows for the OP's application. I don't think that part of the rest > of the paper will be useful to the OP. > > DirkI might want to know more about the OP's signals, noise and applications before jumping to conclusions. Dale B. Dalrymple
Reply by ●October 9, 20102010-10-09
On Tue, 05 Oct 2010 16:39:33 -0500, "shinchan75034" <shinchan75034@n_o_s_p_a_m.gmail.com> wrote:>Hi DSP experts, >I want to do spectral analysis of a vector. I want to divide the vector >into equal length block, multiply each block to a window, then do FFT on >each block, then at each frequency, add the magnitude from each block’s >FFT magnutide spectrum, divide by number of blocks, to obtain the averaged >magnitude spectrum.Hi shinchan, When you perform an N-point FFT on real-world signals (contaminated with some amount of noise) and measure a single FFT bin's spectral magnitude you'll get some number, say M1. If you repeat that N-point FFT on a new block of N time samples and measure the same bin's spectral magnitude you'll get a new number, say M2. If you repeat this ten times you'll have ten magnitudes, M1, M2, M3,..., M9, and M10. Those values in the 10-sample M sequence will all be different. That is, the 10-sample M sequence will have some non-zero variance. (That's the inherent nature of FFTs. And by the way, increasing the FFT size does *NOT* reduce that variance.) So then you'll wonder, "Sheece. What in the heck is the "true" magnitude value of that single FFT bin?" What people do is average those ten M values to obtain a single "averaged magnitude" value that is statistically closer to the "true" FFT bin magnitude.>I have two questions: >1. Can the blocks overlap?Yes. In fact, if you're sure that you must window your time samples before performing FFTs, then overlapping the individual blocks time samples is a good thing to do. If you don't overlap the blocks of time samples, then the time samples at the beginning and end of each block are attenuated, due to multiplication by the window, and those time samples don't have much effect on your FFT results. You don't want that to happen. Sorry that I don't have a reference for you but, somewhere I read the writing of DSP guru fred harris and he said that a time overlap of up to 50% yields acceptable FFT results in reducing the variance when averaging multiple FFTs. Dale Dalrymple gave you a reference that discussed overlapped spectrum analysis. Another place to read about this process is under the heading of "Welch's Method" at: http://www.mathworks.com/help/toolbox/signal/f12-6587.html>2. How can I get the averaged power spectrum? Get sum of the square of >magnitude at each frequency then divide by number of block?If I understand your question, the answer is yes. As far as I know, averaging spectral magnitudes or averaging spectral powers (magnitudes squared) will both give you the variance reduction that you seek. Good Luck, [-Rick-]
Reply by ●October 9, 20102010-10-09
On Wed, 6 Oct 2010 10:49:37 -0700 (PDT), Dirk Bell <bellda2005@cox.net> wrote: [Snipped by Lyons]>> >> Rick Lyons recently reminded me of a paper on flattop windows that >> begins with a nice explanation of the issues you have asked about. >> Look at pages 5-21 of the paper at: >> >> http://www.rssd.esa.int/SP/LISAPATHFINDER/docs/Data_Analysis/GH_FFT.pdf >> >> The rest of the paper may also be useful to you. >> >> Dale B. Dalrymple >> There is a paper you can > >Wow Dale, > >For the life of me I can't figure out why you would recommend a >"measurement" flat-top window for the OP's application. A >"measurement" flat-top produces a lot of smearing in the spectrum, >and has the property that it gets rid of the scalloping loss in the >measurement of an isolated sinusoid/sinusoidal component, where the >smearing doesn't efect the measurement value, but certainly effects >the computed spectrum (see page 38 of your recommended article). >Doesn't sound like the OP wants that. > >Why did you pick a "measurement" flat-top window to suggest? > >Where you thinking of a "time doman" flat-top window (quite >different)? > >Maybe you are just poking fun at the OP? > >DirkHi Dirk, I don't think Dale was recommending that the OP used flat-top windowing. Rather, I think Dale was merely giving an easily obtained refeence that discussed "overlapped-in-time" spectrum analysis. Another place to read about "overlapped-in-time" spectrum analysis is under the heading of "Welch's Method" at: http://www.mathworks.com/help/toolbox/signal/f12-6587.html See Ya', [-Rick-]
