Hi, I am using Matlab to do ffts for some data collected in the lab of electrophysiology recordings. When I plot the power spectrums, I get huge peaks from 0-1 Hz. Are these peaks representative of the data or artifacts carried over/created from the fft? Any help would be appreciated. Thanks.

# FFT Questions

Started by ●March 31, 2004

Reply by ●March 31, 20042004-03-31

Maria wrote:> Hi, > > I am using Matlab to do ffts for some data collected in the lab of > electrophysiology recordings. When I plot the power spectrums, I get > huge peaks from 0-1 Hz. Are these peaks representative of the data or > artifacts carried over/created from the fft? > > Any help would be appreciated. > > Thanks.Probably the latter. A common technique is to remove the DC component from the buffer before the FFT. e.g. fx = fft( x - mean(x) );

Reply by ●March 31, 20042004-03-31

"Mark Borgerding" <mark@borgerding.net> wrote in message news:7KJac.100982$8G2.69531@fe3.columbus.rr.com...> Maria wrote: > > Hi, > > > > I am using Matlab to do ffts for some data collected in the lab of > > electrophysiology recordings. When I plot the power spectrums, I get > > huge peaks from 0-1 Hz. Are these peaks representative of the data or > > artifacts carried over/created from the fft? > > > > Any help would be appreciated. > > > > Thanks. > > Probably the latter. > > A common technique is to remove the DC component from the buffer before > the FFT. > > e.g. > > fx = fft( x - mean(x) ); >If you don't remove the mean then the result around 0Hz isn't an artifact it's representative of the data. Somewhat related, if you don't taper the ends of the temporal data sequence then there will be a strong periodic component at the lowest frequency but not at 0Hz. Example: the temporal sample is highly positive at the beginning and highly negative at the end (after removing the mean). The Discrete Fourier Transform / FFT treats the single sequence as if it's periodic. Let's say the temporal epoch is 1 second and the sample rate is 512Hz. So, there are 512 samples in that 1 second and the assumption is that it repeats. The repetition is at 1Hz which is the fundamental frequency of the Fourier Series represented in the DFT. So, a large 1Hz component will emerge from the sharp discontinuity that repeats at 1Hz. Tapering the ends of the temporal sequence (windowing) will remove the sharpness of the discontinuity and, thus, remove the artificial 1Hz component that's introduced by it. Fred

Reply by ●April 1, 20042004-04-01

Thanks for the info. Also curious, there are several windowing options in Matlab, would you recommend one? Also, my sampling rate is 2000Hz. Would it be better to do ffts with shorter time intervals (~1-5s) and thus (smaller amount of points in the fft) vs larger time intervals (~60s) and thus (larger amount of points in the fft)? Any help would be appreciated. THanks alot. Maria "Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in message news:<rc2dnTzW1p965_bdRVn-ug@centurytel.net>...> "Mark Borgerding" <mark@borgerding.net> wrote in message > news:7KJac.100982$8G2.69531@fe3.columbus.rr.com... > > Maria wrote: > > > Hi, > > > > > > I am using Matlab to do ffts for some data collected in the lab of > > > electrophysiology recordings. When I plot the power spectrums, I get > > > huge peaks from 0-1 Hz. Are these peaks representative of the data or > > > artifacts carried over/created from the fft? > > > > > > Any help would be appreciated. > > > > > > Thanks. > > > > Probably the latter. > > > > A common technique is to remove the DC component from the buffer before > > the FFT. > > > > e.g. > > > > fx = fft( x - mean(x) ); > > > > If you don't remove the mean then the result around 0Hz isn't an artifact > it's representative of the data. > > Somewhat related, if you don't taper the ends of the temporal data sequence > then there will be a strong periodic component at the lowest frequency but > not at 0Hz. > Example: the temporal sample is highly positive at the beginning and highly > negative at the end (after removing the mean). The Discrete Fourier > Transform / FFT treats the single sequence as if it's periodic. Let's say > the temporal epoch is 1 second and the sample rate is 512Hz. So, there are > 512 samples in that 1 second and the assumption is that it repeats. The > repetition is at 1Hz which is the fundamental frequency of the Fourier > Series represented in the DFT. > So, a large 1Hz component will emerge from the sharp discontinuity that > repeats at 1Hz. > > Tapering the ends of the temporal sequence (windowing) will remove the > sharpness of the discontinuity and, thus, remove the artificial 1Hz > component that's introduced by it. > > Fred

Reply by ●April 1, 20042004-04-01

"Maria" <marija616@hotmail.com> wrote in message news:38a072b0.0404011002.6e16dc08@posting.google.com...> Thanks for the info. Also curious, there are several windowing > options in Matlab, would you recommend one? Also, my sampling rate is > 2000Hz. Would it be better to do ffts with shorter time intervals > (~1-5s) and thus (smaller amount of points in the fft) vs larger time > intervals (~60s) and thus (larger amount of points in the fft)? >Maria, Your questions are both answered by considering what windows do in general. The time spans represent rectangular windows of different widths. Using one or another length amounts to convolving the frequency information with a sinc=sin(kx)/kx function whose width is inversely proportional to the time length. So, longer time, narrower sinc to "smear" the frequency data. Other windows have the nice property of reducing the sidelobes or "tails" of that sinc and the choice is a matter of what's easiest to use and what your requirements might be. Kaiser's window is probably a pretty good choice because you can trade off the critical characteristics in frequency. One critical trade in windows is the width of the main lobe of its frequency character vs. the height of the sidelobes. In general, the wider the main lobe, the smaller the sidelobes can be. As above, the longer the time data, the narrower the main lobe of the sinc. So, if you have lots of time data then the main lobe might be quite narrow anyway and your concern may shift to the sidelobes ("spectral leakage"). You don't have to window the entire temporal sequence either. You might keep the sequence intact for much of its span and only taper the edges. This brings the result closer to having a rectangular window but gets rid of a sharp discontinuity at the edges. So, a very simple window might look like: 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.0 .......... 1.0 0.9 0.8 ...0.1 0 A more sophisticated version of this would shape the edges according to a cosine so that the ends of the transition have zero derivative. And, you can make the transition any width you like up to having it transition up for 1/2 of the temporal record and down for the remaining 1/2 - then it would be more like the windows you can compute with Matlab. Just look at the magnitude of the FFT of the window to see what it does in frequency. No matter what, to get rid of the 1Hz component we talked about, you probably want the window to go to zero at the edges. Fred

Reply by ●April 1, 20042004-04-01

Maria wrote:> Thanks for the info. Also curious, there are several windowing > options in Matlab, would you recommend one?Depends on whether you want sideband rejection or narrow band. Both are good things to have, but you can have everything :) A rectangular window (i.e. no windowing at all) has very sharp transition, but only 13dB sideband rejection (if memory serves). For some processing, this is enough. Especially if several consecutives fft outputs are added coherently ... but I digress. The kaiser window allows fine tuning to trade off between transition width and sideband suppression. You can use freqz(w) to see the various windows. If you don't want to spend too many brain cycles deciding, the hamming window is a decent starting point. > Also, my sampling rate is > 2000Hz. Would it be better to do ffts with shorter time intervals > (~1-5s) and thus (smaller amount of points in the fft) vs larger time > intervals (~60s) and thus (larger amount of points in the fft)? Again it depends: Shorter windows give you more sensitivity to brief features/events. Longer windows give you better frequency resolution. Decide how long an "event of interest" lasts. Decide how precisely you need to measure frequencies (subject to transition bandwidth dictated by windowing type). The answer to these two questions will tell you what range of window lengths would be acceptable. Hope this helps. -- Mark Borgerding

Reply by ●April 1, 20042004-04-01

Mark Borgerding wrote:> Maria wrote: > >> Thanks for the info. Also curious, there are several windowing >> options in Matlab, would you recommend one? > > > Depends on whether you want sideband rejection or narrow band. Both are > good things to have, but you can have everything :) >errr .... you *can't* have everything. But you already knew that. (sorry, brain fade)

Reply by ●April 1, 20042004-04-01

On 31 Mar 2004 15:26:06 -0800, marija616@hotmail.com (Maria) wrote:>Hi, > >I am using Matlab to do ffts for some data collected in the lab of >electrophysiology recordings. When I plot the power spectrums, I get >huge peaks from 0-1 Hz. Are these peaks representative of the data or >artifacts carried over/created from the fft? > >Any help would be appreciated. > >Thanks.Hi Maria, No, those peaks at low frequencies are *not* an artifact of of the FFT algorithm. They indicate that your time-domain sequence is riding on a DC bias (that is, it means your FFT-ed sequence's average is not zero). You should subtract the sequence's average (a single value) from each sample in the sequence giving you a new sequence. Now take the FFT of that new sequence. As for which window to use, that's a tougher question. Windowing is generally used to reduce the spectral leakage that causes a strong (high amplitude) signal's spectral magnitude to cover over (swamp out) the spectral components of nearby weak signals. If a weak signal (in which you're interested) is 3-4 FFT frequency bins away from a strong signal's center frequency, use the Hamming window. If the weak signal (in which you're interested) is more than, say, 6 FFT frequency bins away from a strong signal's center frequency, use the Hanning window. Ah, there's much to learn about window functions, particularly the Kaiser and Chebyshev windows, which give you some control over their spectral leakage cahracteristics. Keep asking questions here Maria, Regards, [-Rick-]

Reply by ●April 1, 20042004-04-01

"Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in message news:<rc2dnTzW1p965_bdRVn-ug@centurytel.net>...> "Mark Borgerding" <mark@borgerding.net> wrote in message > news:7KJac.100982$8G2.69531@fe3.columbus.rr.com... > > Maria wrote: > > > Hi, > > > > > > I am using Matlab to do ffts for some data collected in the lab of > > > electrophysiology recordings. When I plot the power spectrums, I get > > > huge peaks from 0-1 Hz. Are these peaks representative of the data or > > > artifacts carried over/created from the fft? > > > > > > Any help would be appreciated. > > > > > > Thanks. > > > > Probably the latter. > > > > A common technique is to remove the DC component from the buffer before > > the FFT. > > > > e.g. > > > > fx = fft( x - mean(x) ); > > > > If you don't remove the mean then the result around 0Hz isn't an artifact > it's representative of the data. > > Somewhat related, if you don't taper the ends of the temporal data sequence > then there will be a strong periodic component at the lowest frequency but > not at 0Hz. > Example: the temporal sample is highly positive at the beginning and highly > negative at the end (after removing the mean). The Discrete Fourier > Transform / FFT treats the single sequence as if it's periodic. Let's say > the temporal epoch is 1 second and the sample rate is 512Hz. So, there are > 512 samples in that 1 second and the assumption is that it repeats. The > repetition is at 1Hz which is the fundamental frequency of the Fourier > Series represented in the DFT. > So, a large 1Hz component will emerge from the sharp discontinuity that > repeats at 1Hz. > > Tapering the ends of the temporal sequence (windowing) will remove the > sharpness of the discontinuity and, thus, remove the artificial 1Hz > component that's introduced by it. > > FredHi, When I window the data (hamming window), I lose amplitude on the power spectrum. Can i do anything for this? Also, is it better to take short FFTs (i.e. 1s intervals for 100s and then average them?) as opposed to taking one 100s segement? Any feedback/help is much appreciated. Thanks.

Reply by ●April 2, 20042004-04-02

"Maria" <marija616@hotmail.com> wrote in message news:38a072b0.0404011848.6f332046@posting.google.com...> > Hi, > > When I window the data (hamming window), I lose amplitude on the power > spectrum. Can i do anything for this? Also, is it better to take > short FFTs (i.e. 1s intervals for 100s and then average them?) as > opposed to taking one 100s segement? > > Any feedback/help is much appreciated. > > Thanks.Maria, Yes, when you apply the window you are reducing the amplitudes in the time domain - so there will be corresponding reduction in the frequency domain. A good question might be: why do you want to "do something" about this? Assuming you want to do something, I guess you could scale the time samples at the same time you window so that the integral of the window is "N" like this: A rectangular window is the one you don't need to apply at all, it's what you get when to take a finite-length sample of data. All of the values in the window function are 1.0. The integral of these, their sum, is N where N is the number of samples taken over the interval. So, if you use a Hamming window or any other, you can add up the window weights (let's call this sum "W") and divide by N to see the scaling. Usually W is less than N so the scaling is less than 1.0 If your objective is to hold the integral of the window to be N, then you can multiply all the weights or coefficients of the window by N/W. This amounts to amplifying the time record by N/W and then windowing it - or vice-versa. Your question about shorter or longer FFTs has been answered in a few ways already. Most of the answers said: "it depends". You may need to ponder those responses more carefully. In the mean time, let's take a few cases: Let's assume that you already know that taking 1 second chunks (just as an example) will limit the resolution to 1Hz - and that this is acceptable to you. Saying this eliminates the issue of resolution entirely - so we can set that aside to keep the discussion simpler. Let's also assume that you know that taking a 100 second chunk will provide 0.01Hz resolution - but you don't care whether it does or doesn't because 1Hz is already deemed OK. Now, there are a couple of other considerations: - is the signal of interest stationary? That is, does it vary or not vary in its spectrum? For example, an engine running at constant rpm is pretty stationary - it doesn't vary all that much. - is there much noise in the signal? If the signal is stationary, then long samples make sense. If it isn't, then long samples may not make sense - nor would averaging spectra of short samples over longer times make as much sense either. This comes before the question of noise. SO: you need to decide how long you would either sample or average based on how stable the signal is. When you decide this, then you have an important parameter to work with. If there is noise added to the signal then: - the long record will give you really fine spectral detail of signal plus noise. If the noise is white, then there will be a grassy floor in the spectrum. If the signal is reasonably strong relative to the noise, then it will rise above that floor. Most importantly, the long record will split white noise over more samples and the signal to noise of the spectrum will improve (well, unless the signal itself is white noise and that another matter altogether!) - the short record / averaged will give you less spectral detail of signal plus noise and the noise floor will be higher relative to the long record - even with averaging because it starts out higher. I'm assuming that you have to average the magnitudes here - or take the sqrt of the sum of the squares .... something like that. What averaging will do is to throw out the occasional spectral spike that's caused by noise - and that may be important to you. So here are the steps for you to take: 1) Decide if the signal spectrum changes and, if so, decide about how often it changes. 2) Pick a temporal epoch that won't be too perturbed by the changes. That is, an epoch that is maybe 1/2 the time it takes the signal to change "very much". This can be your analysis window length. By this approach, if the signal is very stable, the epoch can be long. 3) Examine your desire for spectral resolution. If it's coarser than the reciprocal of the epoch length, then you're OK. If not, then you may have to increase the length of the epoch - which counteracts the decision you made in step (2). See, it's a trade between objectives and signal characteristics..... 4) Examine the signal to noise situation. If the noise level is so high that better resolution is necessary to pull the signal spectrum out of the noise spectrum then you have to increase the length of the epoch. Same issue as step (2) but from a different motivation. IF the resolution is already good enough then you might average spectra in order to reduce the effect of occasional noise peaks. Summary: Signal spectrum is(below): Very stable <> Moderately stable <> Quickly changing Long epochs <> Moderately long <> Short epochs Very narrow (high) <> Moderately narrow <> Broad (low) Spectral resolution needed is (above): Accordingly, your decisions about numbers will be easy if both the spectral character and the resolution "match" to yield either long or short epochs. It remains easy if the signal is very stable and broad resolution is OK. It's toughest when the signal isn't stable and there's a need for high resolution. That's because they just don't match... it's physics. ****************** Regarding averaging of spectra from a series of temporal epochs: IF the spectral resolution can be met AND Signal to noise is: (below) LowSNR (high noise) <> MedSNR <> HighSNR (low noise) Long epochs Can't average Can't ave Can't / don't need? (e.g. one only) Medium epochs Can / should? Can / should? Could (e.g. a few) Short epochs Can / should? Can / should? Could (e.g. many) Recall that the spectral signal to noise ratio obtained with a long epoch is better than with a shorter one. However, there might still be the occasional outlying spectral peak from the noise. If this is a concern, given adequate resolution, then averaging a number of spectra might be good to do. The trade could be: "Can I see the signal at all?" vs. "Can I stand an occasional erroneous peak?" This gets right to the heart of the age old trade between false alert vs. false rest. If the sensitivity is set high, I will see everything and get some bad positive readings. If the sensitivity to set low, I will miss something but won't have as many bad positive readings. The averaging is a compromise between high resolution with some errors and low resolution with fewer errors. It tends to reduce the errors IF its resolution is acceptable otherwise. That is as far as arm-waving will get. Thereafter, you may need some math if that's what's important to you. There are whole books - and certainly whole chapters - written on spectral estimation. So, after my simple arm-waving discussion that's where you might look next. Fred