On Aug 8, 6:29�pm, ggk <ggkm...@comcast.net> wrote:
>
> Thanks Julius, just so I'm clear... are you saying that it is always
> more mathematically correct to average FFTs as root-sum-square because
> the spectrum is defined as above?
It depends on what you are trying to do (and it sounds like you're
not sure). If you want to estimate the spectrum of a signal, then
yes, average of magnitudes is the way to go.
I think that what you are asking can be answered by reading up
on "periodogram", "spectrum estimate", "Bartlett's method",
"Welch's method".
Roughly speaking, one trades off between noise reduction and
resolution. The MATLAB docs are actually pretty decent expositions
targeted to the different methods implemented, such as pwelch,
bartlett, etc.
Reply by ggk●August 8, 20082008-08-08
On Aug 8, 4:21�pm, julius <juli...@gmail.com> wrote:
> On Aug 8, 3:44�pm, ggk <ggkm...@comcast.net> wrote:
>
> > Thanks so much Kenn,
>
> > I think my first step here is to identify where the spur frequencies
> > are located, with high accuracy. Seems like your scheme above is the
> > perfect answer.
>
> > Best regards, -GGK
>
> The spectrum of a signal is defined as:
>
> �S_{xx}(f) = E[ | X(f) |^2 ]
>
> as Dirk suggested.
Thanks Julius, just so I'm clear... are you saying that it is always
more mathematically correct to average FFTs as root-sum-square because
the spectrum is defined as above?
Reply by julius●August 8, 20082008-08-08
On Aug 8, 3:44�pm, ggk <ggkm...@comcast.net> wrote:
> Thanks so much Kenn,
>
> I think my first step here is to identify where the spur frequencies
> are located, with high accuracy. Seems like your scheme above is the
> perfect answer.
>
> Best regards, -GGK
The spectrum of a signal is defined as:
S_{xx}(f) = E[ | X(f) |^2 ]
as Dirk suggested.
Reply by ggk●August 8, 20082008-08-08
On Aug 8, 1:43�pm, dbell <bellda2...@cox.net> wrote:
> On Aug 8, 4:21�pm, kennheinr...@sympatico.ca wrote:
>
>
>
>
>
> > On Aug 8, 1:21 pm, ggk <ggkm...@comcast.net> wrote:
>
> > > Jim,
>
> > > I should clarify. The data I listed above is the raw FFT data, but
> > > when I plot the FFT to identify the spurs I only plot the magnitude
> > > (that is, I plot 2*abs(Y)). I just wanted to show the raw FFT data in
> > > case it helped. Also, the numbers are close to zero but the
> > > measurement is fairly sensitive, and the noise floor is actually an
> > > order of magnitude lower.
>
> > > Rune,
>
> > > I agree the energy is fairly small. It just so happens to be the
> > > nature of the phenomenon I'm trying to analyze. Not sure what the
> > > total energy is. Let's see, if I type sum(Y1.*conj(Y1)), it returns
> > > 4.86e-024.
>
> > > Per your definitions I am using coherent averaging to evaluate the
> > > average FFT. I think you and Ken are pointing to the same root cause
> > > of this... lack of a fixed time reference. �If coherent averaging
> > > isn't taking me where I want to go, can you talk about "cross
> > > spectra"... what is it, how to do it, how it would help determine the
> > > spur frequencies present?
>
> > > Ken,
>
> > > Thanks for the mental exercise to hightlight the point. I may not have
> > > got it without that! So if I understand what you and Rune are saying,
> > > is that because the 5 waveforms are not connected in real-time, they
> > > lack a consistent time reference between the files, which means when
> > > you throw in the phase information (the imaginary number in the FFT
> > > freq bin), things don't line up. Thus, I shouldn't use coherent
> > > averaging, but rather ... what? If I have 3 frequency bins to average,
> > > should I use...
>
> > > a1 + b1i
> > > a2 + b2i
> > > a3 + b3i
>
> > > --> ( abs(a1 + b1i) + abs(a2 + b2i) + abs(a3 + b3i)) / 3
>
> > > ... for example?
>
> > > Thanks so much! -GGK
>
> > If you have 3 captures to average, just average the co-located bins'
> > magnitude (a.k.a absolute value), not the phase. �Taking the absolute
> > value of a complex number is more or less the same thing as taking
> > x*conj(x), as suggested earlier � - that's just some alegbra to show
> > that (abs(x))^2 = x*conj(x).
>
> > In matlab, you'd do something like
> > p=(abs(fft(a))+abs(fft(b))+abs(fft(c)))/3;
> > where a,b,c are the three equal-length input signal captures from your
> > scope. Then look at the value of, for example, p(363540), to find the
> > average amplitude in bin 363540 across all three captures. If you then
> > want to average adjacent bins (say bins 363540 through 363543) you can
> > do this as a simple arithmetic average, since the values are real.
>
> > This will get you to the point where you know that the spur is there -
> > trying to extract, demodulate, or measure it accurately will be a
> > little more work.
>
> > �- Kenn- Hide quoted text -
>
> > - Show quoted text -
>
> Generally for averaging FFTs, you average the magnitude squared for
> each frequency rather than the magnitude.
>
> Dirk- Hide quoted text -
>
> - Show quoted text -
Thanks Dirk, can anyone explain the various tradeoffs in choosing to
average as a magnitude versus magnitude squared (I assume root-sum-of-
squares approach)? When is one more appropriate than another?
Reply by ggk●August 8, 20082008-08-08
Thanks so much Kenn,
I think my first step here is to identify where the spur frequencies
are located, with high accuracy. Seems like your scheme above is the
perfect answer.
Best regards, -GGK
Reply by dbell●August 8, 20082008-08-08
On Aug 8, 4:21�pm, kennheinr...@sympatico.ca wrote:
> On Aug 8, 1:21 pm, ggk <ggkm...@comcast.net> wrote:
>
>
>
>
>
> > Jim,
>
> > I should clarify. The data I listed above is the raw FFT data, but
> > when I plot the FFT to identify the spurs I only plot the magnitude
> > (that is, I plot 2*abs(Y)). I just wanted to show the raw FFT data in
> > case it helped. Also, the numbers are close to zero but the
> > measurement is fairly sensitive, and the noise floor is actually an
> > order of magnitude lower.
>
> > Rune,
>
> > I agree the energy is fairly small. It just so happens to be the
> > nature of the phenomenon I'm trying to analyze. Not sure what the
> > total energy is. Let's see, if I type sum(Y1.*conj(Y1)), it returns
> > 4.86e-024.
>
> > Per your definitions I am using coherent averaging to evaluate the
> > average FFT. I think you and Ken are pointing to the same root cause
> > of this... lack of a fixed time reference. �If coherent averaging
> > isn't taking me where I want to go, can you talk about "cross
> > spectra"... what is it, how to do it, how it would help determine the
> > spur frequencies present?
>
> > Ken,
>
> > Thanks for the mental exercise to hightlight the point. I may not have
> > got it without that! So if I understand what you and Rune are saying,
> > is that because the 5 waveforms are not connected in real-time, they
> > lack a consistent time reference between the files, which means when
> > you throw in the phase information (the imaginary number in the FFT
> > freq bin), things don't line up. Thus, I shouldn't use coherent
> > averaging, but rather ... what? If I have 3 frequency bins to average,
> > should I use...
>
> > a1 + b1i
> > a2 + b2i
> > a3 + b3i
>
> > --> ( abs(a1 + b1i) + abs(a2 + b2i) + abs(a3 + b3i)) / 3
>
> > ... for example?
>
> > Thanks so much! -GGK
>
> If you have 3 captures to average, just average the co-located bins'
> magnitude (a.k.a absolute value), not the phase. �Taking the absolute
> value of a complex number is more or less the same thing as taking
> x*conj(x), as suggested earlier � - that's just some alegbra to show
> that (abs(x))^2 = x*conj(x).
>
> In matlab, you'd do something like
> p=(abs(fft(a))+abs(fft(b))+abs(fft(c)))/3;
> where a,b,c are the three equal-length input signal captures from your
> scope. Then look at the value of, for example, p(363540), to find the
> average amplitude in bin 363540 across all three captures. If you then
> want to average adjacent bins (say bins 363540 through 363543) you can
> do this as a simple arithmetic average, since the values are real.
>
> This will get you to the point where you know that the spur is there -
> trying to extract, demodulate, or measure it accurately will be a
> little more work.
>
> �- Kenn- Hide quoted text -
>
> - Show quoted text -
Generally for averaging FFTs, you average the magnitude squared for
each frequency rather than the magnitude.
Dirk
Reply by ●August 8, 20082008-08-08
On Aug 8, 1:21 pm, ggk <ggkm...@comcast.net> wrote:
> Jim,
>
> I should clarify. The data I listed above is the raw FFT data, but
> when I plot the FFT to identify the spurs I only plot the magnitude
> (that is, I plot 2*abs(Y)). I just wanted to show the raw FFT data in
> case it helped. Also, the numbers are close to zero but the
> measurement is fairly sensitive, and the noise floor is actually an
> order of magnitude lower.
>
> Rune,
>
> I agree the energy is fairly small. It just so happens to be the
> nature of the phenomenon I'm trying to analyze. Not sure what the
> total energy is. Let's see, if I type sum(Y1.*conj(Y1)), it returns
> 4.86e-024.
>
> Per your definitions I am using coherent averaging to evaluate the
> average FFT. I think you and Ken are pointing to the same root cause
> of this... lack of a fixed time reference. If coherent averaging
> isn't taking me where I want to go, can you talk about "cross
> spectra"... what is it, how to do it, how it would help determine the
> spur frequencies present?
>
> Ken,
>
> Thanks for the mental exercise to hightlight the point. I may not have
> got it without that! So if I understand what you and Rune are saying,
> is that because the 5 waveforms are not connected in real-time, they
> lack a consistent time reference between the files, which means when
> you throw in the phase information (the imaginary number in the FFT
> freq bin), things don't line up. Thus, I shouldn't use coherent
> averaging, but rather ... what? If I have 3 frequency bins to average,
> should I use...
>
> a1 + b1i
> a2 + b2i
> a3 + b3i
>
> --> ( abs(a1 + b1i) + abs(a2 + b2i) + abs(a3 + b3i)) / 3
>
> ... for example?
>
> Thanks so much! -GGK
If you have 3 captures to average, just average the co-located bins'
magnitude (a.k.a absolute value), not the phase. Taking the absolute
value of a complex number is more or less the same thing as taking
x*conj(x), as suggested earlier - that's just some alegbra to show
that (abs(x))^2 = x*conj(x).
In matlab, you'd do something like
p=(abs(fft(a))+abs(fft(b))+abs(fft(c)))/3;
where a,b,c are the three equal-length input signal captures from your
scope. Then look at the value of, for example, p(363540), to find the
average amplitude in bin 363540 across all three captures. If you then
want to average adjacent bins (say bins 363540 through 363543) you can
do this as a simple arithmetic average, since the values are real.
This will get you to the point where you know that the spur is there -
trying to extract, demodulate, or measure it accurately will be a
little more work.
- Kenn
Reply by ggk●August 8, 20082008-08-08
Jim,
I should clarify. The data I listed above is the raw FFT data, but
when I plot the FFT to identify the spurs I only plot the magnitude
(that is, I plot 2*abs(Y)). I just wanted to show the raw FFT data in
case it helped. Also, the numbers are close to zero but the
measurement is fairly sensitive, and the noise floor is actually an
order of magnitude lower.
Rune,
I agree the energy is fairly small. It just so happens to be the
nature of the phenomenon I'm trying to analyze. Not sure what the
total energy is. Let's see, if I type sum(Y1.*conj(Y1)), it returns
4.86e-024.
Per your definitions I am using coherent averaging to evaluate the
average FFT. I think you and Ken are pointing to the same root cause
of this... lack of a fixed time reference. If coherent averaging
isn't taking me where I want to go, can you talk about "cross
spectra"... what is it, how to do it, how it would help determine the
spur frequencies present?
Ken,
Thanks for the mental exercise to hightlight the point. I may not have
got it without that! So if I understand what you and Rune are saying,
is that because the 5 waveforms are not connected in real-time, they
lack a consistent time reference between the files, which means when
you throw in the phase information (the imaginary number in the FFT
freq bin), things don't line up. Thus, I shouldn't use coherent
averaging, but rather ... what? If I have 3 frequency bins to average,
should I use...
a1 + b1i
a2 + b2i
a3 + b3i
--> ( abs(a1 + b1i) + abs(a2 + b2i) + abs(a3 + b3i)) / 3
... for example?
Thanks so much! -GGK
Reply by ●August 8, 20082008-08-08
On Aug 8, 12:42 pm, jim <"sjedgingN0sp"@m...@mwt.net> wrote:
> ggk wrote:
>
>
> > If I analyze the instantaneous signal I get one answer, and if I
> > analyze the averaged signal I get another answer. Is it that I cannot
> > use the averaged signal if I want to answer the question "where are
> > the spurs in this signal?".
>
Hint,
Imagine if you captured one sequence that went
50,100,50,100,50,100,...
then your second sequence, which didn't have synchronized start phase,
went
100,50,100,50,100,50,...
Now average them (75,75,75,...) and think about where the spur went.
If you want to "average the fact that spurs are there" in a quick and
dirty way, consider averaging the abs of the FFT points. This
effectively removes the unknown phase angle from consideration.
- Kenn
Reply by Rune Allnor●August 8, 20082008-08-08
On 8 Aug, 18:16, ggk <ggkm...@comcast.net> wrote:
> Hi Everyone,
>
> I observed an unexpected result in doing some FFT work recently, that
> I think relates to understanding the fundamentals of an FFT I'm hoping
> someone can help me understand better. I'm not a DSP expert.
>
> I captured five waveforms sequentially using an oscilloscope. Aside
> from capturing these 5 waveforms with a short time delay between each,
> these 5 waveforms are otherwise identical. I captured 5 waveforms with
> the intent of averaging them to reduce the random noise and cause any
> spurs to stand out more clearly from the background random noise.
>
> When I take an FFT in matlab and plot abs(2*Y) (that is, the
> magnitude) for each of the individual captures, there are a couple
> dozen spurs (spikes indicating sinusoidal noise) standing above a
> random noise floor. When I look at the averaged spectrum, some of
> these spurs are now gone. I would have expected this, but when I went
> back to analyze some specific spurs, I found that in several cases,
> there would be a spur standing clearly above the noise floor in the
> exact same frequency bin for EACH of the five individual captures, but
> NOT in the averaged spectrum.
>
> For example, the following spur is peaking at the same position in the
> FFT for individual captures...
>
> Y1(363540) = � � 5.754953052662731e-015 +4.485425416719249e-015i
> Y2(363540) = � � 6.933769335163839e-015 -4.932545982083068e-015i
> Y3(363540) = � �-1.212685676778186e-015 -7.810350064989210e-015i
> Y4(363540) = � �-8.210562239595824e-015 +2.188852638554278e-015i
> Y5(363540) = � � 2.012052774970021e-017 +7.208534215893042e-015i
The magnitudes are pretty small, 1e-15 isn't a very strong signal.
What is the energy in the total signal?
Try to compute sum(Y.*conj(Y1)) and post the result here.
> but is buried in the noise when averaged...
>
> avg_Y(363540) = � � 6.571189998404520e-016 +2.279832448188584e-016i
>
> I don't doubt the math here, but I'm looking for some understanding
> what this says about my data. How can a spur be present in all five
> individual captures, but absent from the average of them? I know
> there's a thing called destructive interference that means the
> amplitudes and phases can add to cancel each other, and perhaps that's
> what's going on here, but I'm simply trying to find out which
> frequencies are spurs in the original signal.
Coherent averaging of complex numbers works like this:
z1 = 1
z2 = i
z3 = -1
z4 = -i
(z1+z2+z3+z4)/4 = 0
So even if the magnitudes of each number equals 1, the
coherent average equals 0.
*In*coherent averaging is to average the magnitudes (ignoring
phases), in which case
(|z1|+|z2|+|z3|+|z4|)/4 = 1.
> If I analyze the instantaneous signal I get one answer, and if I
> analyze the averaged signal I get another answer. Is it that I cannot
> use the averaged signal if I want to answer the question "where are
> the spurs in this signal?".
If you want to use coherent averaging you need a fixed time
reference in order to match up the phases. If that's not an
option you might want to use cross spectra.
Rune