frequency time resolution

Hi all,

I am working on some event-related potentials (ERP) and I wanted to find
the power at different frequencies. I understand that I can perform a FFT
to obtain the PSD, however the problem i have is that the duration of
these signals are very short ~350ms. The signals are sampled at 2500Hz.

What i am confuse is:

1) When performing a FFT, the frequency resolution is defined by 1/350ms
~2.85Hz. Does this mean that i will lose the information for frequency
intervals smaller than this resolution?

2) I used the pwelch method in matlab to compute the PSD. What i used was
nfft = 2500, fs = 2500Hz. This gives me frequency intervals of 1Hz.
However, according to the f = 1/T resolution i would only have valid
information at every 2.85Hz. So my question is where does the points in
between each 2.85Hz come from? Is this interpolated??

3) Also the signals are sampled at 2500Hz this means there should be 2500
samples in 1 second. However, the ERPs are only 350ms, so does this mean
that i will only have 875 sample points in one second (350ms *2500) and
hence have some information loss?? 

4) Is it ok to use 350ms as long as the frequency resolution i want to
look at is >= 2.85Hz??

5) What methods can i use to look at this? Wavelets, STFT??

I am new to this and I appreciate all help. Thanks,

cherrie

On 8 Okt, 09:11, "cherrie" <cherriegel...@hotmail.co.uk> wrote:
> Hi all,
>
> I am working on some event-related potentials (ERP) and I wanted to find
> the power at different frequencies. I understand that I can perform a FFT
> to obtain the PSD, however the problem i have is that the duration of
> these signals are very short ~350ms. The signals are sampled at 2500Hz.
>
> What i am confuse is:
>
> 1) When performing a FFT, the frequency resolution is defined by 1/350ms
> ~2.85Hz. Does this mean that i will lose the information for frequency
> intervals smaller than this resolution?

Yes.

> 2) I used the pwelch method in matlab to compute the PSD. What i used was
> nfft = 2500, fs = 2500Hz. This gives me frequency intervals of 1Hz.
> However, according to the f = 1/T resolution i would only have valid
> information at every 2.85Hz. So my question is where does the points in
> between each 2.85Hz come from?

They were introduced by zero padding in time domain.
2500 points in time domain represent 1 second duration
of data. You only have 350 ms, so the rest is zero-padded.

> Is this interpolated??

Yes.

> 3) Also the signals are sampled at 2500Hz this means there should be 2500
> samples in 1 second. However, the ERPs are only 350ms, so does this mean
> that i will only have 875 sample points in one second (350ms *2500) and
> hence have some information loss??

Not quite. You don't lose information by zero padding,
but you add known data points. The effect is -- if you
do not understand exactly what is going on -- to obfuscate
the "raw" informataion in your data.

> 4) Is it ok to use 350ms as long as the frequency resolution i want to
> look at is >= 2.85Hz??

Unless you can repeat your measurement you need to take what
you got. If I wcould design an experiment to use with Welch's
method without restrictions, I would do it as follows:

1) Decide on a required frequency resolution of the PSD
2) Determine the required data duration to reach this
   resolution (accounting for windowing effects)
3) Measure at least five times that long preferably 10x
   or 20x that long

Of course, that means that you know in advance what you
will find, and that there are no restrictions in either
the experimental setup or economy with respect to the
duration of measurements. I don't know of any experiments
which work like that...

> 5) What methods can i use to look at this? Wavelets, STFT??

Those methods are useful for examinig dynamic data.
You haven't stated whether your data are stationary
or dynamic, but for stationary data Welch is the
method of choise.

Both wavelets and the STFT (or Gabor filter) suffer
from the same time-frequency resolution issues as
Welch's method, they only use different compromises
to handle the resolution issue.

Rune

On Oct 8, 3:18 am, Rune Allnor <all...@tele.ntnu.no> wrote:
> On 8 Okt, 09:11, "cherrie" <cherriegel...@hotmail.co.uk> wrote:
> > I am working on some event-related potentials (ERP) and I wanted to find
> > the power at different frequencies. I understand that I can perform a FFT
> > to obtain the PSD, however the problem i have is that the duration of
> > these signals are very short ~350ms. The signals are sampled at 2500Hz.
>
> > What i am confuse is:
>
> > 1) When performing a FFT, the frequency resolution is defined by 1/350ms
> > ~2.85Hz. Does this mean that i will lose the information for frequency
> > intervals smaller than this resolution?
>
> Yes.

I'm not sure I would say that any information was lost,
as much as that a certain amount of unambiguous information
was never present or revealed in a signal that short,
assuming or not assuming some noise/interference floor.
There may not have been any information there to lose.

I'll also add my usual comment that it might be possible
to measure some frequency components more accurately then
the FFT resolution if one can safely make certain assumptions
about relative noise levels, or perhaps use information from
additional data samples outside the FFT window to invalidate
certain portions of an ambiguous solution set.

...

> > 3) Also the signals are sampled at 2500Hz this means there should be 2500
> > samples in 1 second. However, the ERPs are only 350ms, so does this mean
> > that i will only have 875 sample points in one second (350ms *2500) and
> > hence have some information loss??
>
> Not quite. You don't lose information by zero padding,

It's not completely clear that the OP is zero padding.
He might be analyzing 2500 valid samples containing some sort
of pulse with an envelope.  Thus it would be the natural
envelope of the signal that would revealing insufficient
information about some something within that envelope
sub-portion.  The information about the whole 2500 samples
would be complete, but perhaps contain a lot of information
about portions of the sample set which are not of interest
to the OP.



IMHO. YMMV.
--
rhn A.T nicholson d.0.t C-o-M

cherrie wrote:
> Hi all,
> 
> I am working on some event-related potentials (ERP) and I wanted to find
> the power at different frequencies. I understand that I can perform a FFT
> to obtain the PSD, however the problem i have is that the duration of
> these signals are very short ~350ms. The signals are sampled at 2500Hz.
> 
> What i am confuse is:
> 
> 1) When performing a FFT, the frequency resolution is defined by 1/350ms
> ~2.85Hz. Does this mean that i will lose the information for frequency
> intervals smaller than this resolution?

Not so much lose it as never have it in the first place.  The FFT is a 
nice general way to analyze the spectral content of a signal, but isn't 
the optimal way to analyze most signals.
> 
> 2) I used the pwelch method in matlab to compute the PSD. What i used was
> nfft = 2500, fs = 2500Hz. This gives me frequency intervals of 1Hz.
> However, according to the f = 1/T resolution i would only have valid
> information at every 2.85Hz. So my question is where does the points in
> between each 2.85Hz come from? Is this interpolated??

More or less.  Your data will have been padded out to match the FFT 
length, then (one hopes -- I'm not familiar with Matlab's functions) 
windowed to prevent end effects.  Windowing the signal will spread out 
its spectrum.
> 
> 3) Also the signals are sampled at 2500Hz this means there should be 2500
> samples in 1 second. However, the ERPs are only 350ms, so does this mean
> that i will only have 875 sample points in one second (350ms *2500) and
> hence have some information loss?? 

More like information never there (see my answer to your question 1), 
but yes.
> 
> 4) Is it ok to use 350ms as long as the frequency resolution i want to
> look at is >= 2.85Hz??

If you mean "as long as I don't need to resolve closer than 2.85Hz", 
probably yes.
> 
> 5) What methods can i use to look at this? Wavelets, STFT??

What are you really trying to find out?  What is the physical meaning of 
the power density spectrum that you're trying to extract?  Does the 
meaning you're looking for match the notion of PSD at all?

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" gives you just what it says.
See details at http://www.wescottdesign.com/actfes/actfes.html

On Oct 8, 12:11 am, "cherrie" <cherriegel...@hotmail.co.uk> wrote:
> Hi all,
>
> I am working on some event-related potentials (ERP) and I wanted to find
> the power at different frequencies. I understand that I can perform a FFT
> to obtain the PSD, however the problem i have is that the duration of
> these signals are very short ~350ms. The signals are sampled at 2500Hz.
>
> What i am confuse is:
>
> 1) When performing a FFT, the frequency resolution is defined by 1/350ms
> ~2.85Hz. Does this mean that i will lose the information for frequency
> intervals smaller than this resolution?
>
> 2) I used the pwelch method in matlab to compute the PSD. What i used was
> nfft = 2500, fs = 2500Hz. This gives me frequency intervals of 1Hz.
> However, according to the f = 1/T resolution i would only have valid
> information at every 2.85Hz. So my question is where does the points in
> between each 2.85Hz come from? Is this interpolated??
>
> 3) Also the signals are sampled at 2500Hz this means there should be 2500
> samples in 1 second. However, the ERPs are only 350ms, so does this mean
> that i will only have 875 sample points in one second (350ms *2500) and
> hence have some information loss??
>
> 4) Is it ok to use 350ms as long as the frequency resolution i want to
> look at is >= 2.85Hz??
>
> 5) What methods can i use to look at this? Wavelets, STFT??
>
> I am new to this and I appreciate all help. Thanks,
>
> cherrie

When you use an fft to analyze a signal component you know to be of
shorter duration than the data window, the signal component is
considered a transient. It should be measured as an Energy Spectral
Density (ESD) not a PSD. See:
http://www.sagetechnologies.com/library_documents/dave_qa/q&ajul0.pdf
and
http://www.bksv.com/pdf/bv0031.pdf  pages 34-38
for discussions of the differences.

Dale B. Dalrymple
http://dbdimages.com
htp://stores.lulu.com/dbd

Tim Wescott wrote:

>   ...  Your data will have been padded out to match the FFT 
> length, then (one hopes -- I'm not familiar with Matlab's functions) 
> windowed to prevent end effects.  Windowing the signal will spread out 
> its spectrum.

It should be windowed *first* and *then* zero padded.

   ...

Jerry
-- 
Engineering is the art of making what you want from things you can get.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

On Mon, 08 Oct 2007 20:53:01 -0400, Jerry Avins wrote:

> Tim Wescott wrote:
> 
>>   ...  Your data will have been padded out to match the FFT 
>> length, then (one hopes -- I'm not familiar with Matlab's functions) 
>> windowed to prevent end effects.  Windowing the signal will spread out 
>> its spectrum.
> 
> It should be windowed *first* and *then* zero padded.
> 
Well, if you'd just _read_ my _mind_ you'd see that I had visualized as
being padded, then just the non-zero part windowed.

OK, maybe that wasn't clear enough.  Here: I'll think it harder.

Better?

-- 
Tim Wescott
Control systems and communications consulting
http://www.wescottdesign.com

Need to learn how to apply control theory in your embedded system?
"Applied Control Theory for Embedded Systems" by Tim Wescott
Elsevier/Newnes, http://www.wescottdesign.com/actfes/actfes.html

>On Mon, 08 Oct 2007 20:53:01 -0400, Jerry Avins wrote:
>
>> Tim Wescott wrote:
>> 
>>>   ...  Your data will have been padded out to match the FFT 
>>> length, then (one hopes -- I'm not familiar with Matlab's functions) 
>>> windowed to prevent end effects.  Windowing the signal will spread out

>>> its spectrum.
>> 
>> It should be windowed *first* and *then* zero padded.
>> 
>Well, if you'd just _read_ my _mind_ you'd see that I had visualized as
>being padded, then just the non-zero part windowed.
>
>OK, maybe that wasn't clear enough.  Here: I'll think it harder.
>
>Better?
>
>-- 
>Tim Wescott
>Control systems and communications consulting
>http://www.wescottdesign.com
>
>Need to learn how to apply control theory in your embedded system?
>"Applied Control Theory for Embedded Systems" by Tim Wescott
>Elsevier/Newnes, http://www.wescottdesign.com/actfes/actfes.html
>

Thank you all for your valuable comments and advice. I really appreciate
your efforts, I am more clear about the idea of sampling points, frequency
resolution and FFT in general. 

The problem i have is that I am analysing some ERP data which shows the
brain's response to a particular stimuli and i do not know prior to
experiments how the results would look. These response typically take
place in a window of 100ms to 500ms and hence its not possible to design
the experiments to record longer durations. An ideal method would be one
that can tell me the frequency content in each time instant. I read that
wavelets can achieve this .. any idea??

Thanks, cherrie

Tim Wescott wrote:
> On Mon, 08 Oct 2007 20:53:01 -0400, Jerry Avins wrote:
> 
>> Tim Wescott wrote:
>>
>>>   ...  Your data will have been padded out to match the FFT 
>>> length, then (one hopes -- I'm not familiar with Matlab's functions) 
>>> windowed to prevent end effects.  Windowing the signal will spread out 
>>> its spectrum.
>> It should be windowed *first* and *then* zero padded.
>>
> Well, if you'd just _read_ my _mind_ you'd see that I had visualized as
> being padded, then just the non-zero part windowed.
> 
> OK, maybe that wasn't clear enough.  Here: I'll think it harder.
> 
> Better?

:-) I hasn't thought to enlighten *you*. I just hate to leave something 
like that dangling to bop the unwary.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

"cherrie" <cherriegeller@hotmail.co.uk> wrote in message 
news:ytCdncdvhYWC-JbanZ2dnUVZ_jCdnZ2d@giganews.com...

>>
>
> Thank you all for your valuable comments and advice. I really appreciate
> your efforts, I am more clear about the idea of sampling points, frequency
> resolution and FFT in general.
>
> The problem i have is that I am analysing some ERP data which shows the
> brain's response to a particular stimuli and i do not know prior to
> experiments how the results would look. These response typically take
> place in a window of 100ms to 500ms and hence its not possible to design
> the experiments to record longer durations. An ideal method would be one
> that can tell me the frequency content in each time instant. I read that
> wavelets can achieve this .. any idea??

To summarize much of what's been said:
If the responses are that short then there are a couple of observations one 
can make about spectral analysis:

1) The *actual* spectral resolution is generally the reciprocal of the 
temporal length.  This is true whether the data is continuous or sampled and 
whether the spectral analysis provides a continous plot or sampled data.

2) If we assume that the 2500Hz or so sample rate is OK as far as the 
sampling theory goes then this implies that the bandwidth of the signal is 
below 1250Hz and hopefully at 1000Hz or less.  This may mean that the 
"edges" of the signal will be neglected or filtered out or will contribute 
aliasing errors to a degree.  This is useful because:

Let's assume for the sake of discussion that the signal is perfectly 
bandlimited to something below 1250Hz.  If so, it can be sampled at 2500Hz 
and perfectly reconstructed by convolving the samples with a sinx/x function 
whose zeros are spaced at 1/2500 = 0.4msec.  This means that sampling loses 
no information.  But, note that the sinx/x functions have a width of around 
0.4msec.  Because of this, temporal resolution is limited by the same 
temporal value.

All of this is quite familiar to signal processing folks and is discussed 
all over the place.

But, you're interested in spectral resolution so we can "flip" this 
discussion, interchanging time and frequency:

3) If the temporal width or epoch is limited to just below 500msec then 
assuming a continuous time signal and a corresponding continuous Fourier 
Transform, the Fourier Transform can be sampled at intervals of 2Hz and the 
spectral samples can be used to completely reconstruct the spectrum using a 
sinf/f function whose zeros are spaced 2Hz apart.
This corresponds exactly with the sort of resolution you get if the temporal 
epoch is 500msec, you sample it first and then compute a DFT.
So, if all is well, you can interpolate the spectral samples with a sinf/f 
or "sinc"  function to get a pretty good interpolation.  [Actually you'd use 
a Dirichlet function (a periodic "sinc") because the spectrum of sampled 
data is periodic].
[There's a good reason why there's not the same factor of 2 in the criterion 
for the time epoch related to spectral sampling - but we needn't go into 
that].

Thus, you lose no information interpolating the spectral samples BUT the 
resolution remains around 2Hz because the sinc or Dirichlet smears the 
sample values around.  Interpolation smears....  At least with a sinc or 
Dirichlet, the sample values stay put at the sample points.

From #1 above, you are running into the limits of "physics" and no method is 
going to help make things better than physics will allow.

That said, some mathematicians will point out that you can perfectly 
reconstruct a function from a perfect set of samples if the signal bandwidth 
is perfectly limited, etc. etc.  Nice in theory but not generally a 
practical observation.

Here's a simple example:
If you have two purely sinousoidal responses that are 0.6Hz apart and are 
both 500msec in length (exactly) then observing the distance between 
zero-crossings will tell you that they are 0.6Hz apart.  But, to observe the 
zero-crossing points that precisely will take a lot more samples - it 
implies a wide bandwidth process being applied to a lower-bandwidth signal. 
That's the same as if the signal were wide bandwidth itself.

Knowing what you're trying to accomplish is often helpful.
I take it that you intend to compare frequency content of multiple 
responses.
So, is there any physical reason to expect that there will be frequencies 
that are so close together that differentiating them would be useful?
What physical mechanism in the brain is capable of differentiating between 
two 500msec purely sinusoidal responses that are .6Hz apart?  Somehow I 
doubt such a physical mechanism exists.

I hope this helps in some way...

Fred