Online spectrum analyzer

Hello,

first, I want to apologize because my english is not perfect but it seems
that there are a lot of skilled people here in frequency analysis so I'll
try to be as clear as possible.

My goal is to build an online spectrum analyser given an input signal u and
an output signal y. Both signals are acquired with a sampling frequency of
200Hz and my program has to build the bode diagram of the underlying
process when new data comes (ie. amplitude y:u versus frequency and phase
versus frequency). The frequency-range of interest is [0,5] Hz.

Here is what I have done until now and which needs to be improved:
to acquire the first window signal, I wait 4seconds (window-length) and I
save my windowed signals u_w, y_w. I remove the mean of u_w and y_w and
multiply the result with a hanning window to reduce spectral leakege such
as:

u_wh=(u_w-mean(u_w))*hann(length(u_w))
y_wh=(y_w-mean(y_w))*hann(length(y_w))

I use than the FFT algorithm built in matlab:

FFT(u_wh) and FFT(y_wh)

I wrote a function to detect the peaks (global and locals), that is to say
the frequencies fi contained in the input signal u_wh. For a given detected
frequency I can calculated the amplitude (abs(fft(y_wh))/abs(fft(u_wh)))
and the Phase. 

Finally I move the window with 50% overlapping over the previous window.

This programm enables me to plot the bode diagram of a process in real time
and it works fine for perfect test-signals and for complex signals (that is
to say that I take a complex input u signal and I generate the output y
thanks to a known state space model).

Now I am testing the program for real time (ie inputs u measured AND
outputs y measured and the signals are than noisy) and the results are not
so good. The bode diagram is quite noisy (different amplitudes and phases
values for the same frequency). Do you see a way to improve my algorithm?

I thank you and wish you a good day.
Franck.

	 

_____________________________		
Posted through www.DSPRelated.com

Francky17 wrote:
> Hello,
>
> first, I want to apologize because my english is not perfect but it seems
> that there are a lot of skilled people here in frequency analysis so I'll
> try to be as clear as possible.
>
> My goal is to build an online spectrum analyser given an input signal u and
> an output signal y. Both signals are acquired with a sampling frequency of
> 200Hz and my program has to build the bode diagram of the underlying
> process when new data comes (ie. amplitude y:u versus frequency and phase
> versus frequency). The frequency-range of interest is [0,5] Hz.
>
> Here is what I have done until now and which needs to be improved:
> to acquire the first window signal, I wait 4seconds (window-length) and I
> save my windowed signals u_w, y_w. I remove the mean of u_w and y_w and
> multiply the result with a hanning window to reduce spectral leakege such
> as:
>
> u_wh=(u_w-mean(u_w))*hann(length(u_w))
> y_wh=(y_w-mean(y_w))*hann(length(y_w))
>
> I use than the FFT algorithm built in matlab:
>
> FFT(u_wh) and FFT(y_wh)
>
> I wrote a function to detect the peaks (global and locals), that is to say
> the frequencies fi contained in the input signal u_wh. For a given detected
> frequency I can calculated the amplitude (abs(fft(y_wh))/abs(fft(u_wh)))
> and the Phase.
>
> Finally I move the window with 50% overlapping over the previous window.
>
> This programm enables me to plot the bode diagram of a process in real time
> and it works fine for perfect test-signals and for complex signals (that is
> to say that I take a complex input u signal and I generate the output y
> thanks to a known state space model).
>
> Now I am testing the program for real time (ie inputs u measured AND
> outputs y measured and the signals are than noisy) and the results are not
> so good. The bode diagram is quite noisy (different amplitudes and phases
> values for the same frequency). Do you see a way to improve my algorithm?
>
> I thank you and wish you a good day.
> Franck.
>

I would suggest investigating Lissajous figures.
See:
   http://en.wikipedia.org/wiki/Lissajous_curve
   http://www-history.mcs.st-and.ac.uk/Curves/Lissajous.html

>I would suggest investigating Lissajous figures.
>See:
>   http://en.wikipedia.org/wiki/Lissajous_curve
>   http://www-history.mcs.st-and.ac.uk/Curves/Lissajous.html
>
>

Thank you for your answer. Nevertheless, I don't really see w Lissajous
figures would help me to improve my current program.

The problem is to get the right frequencies contained in each time-window
and the right corresponding amplitudes even if there is noise in the
signal. Lissajous figures would enable me to calculate the phase between to
signals or the frequency ratio of two signals, but not selecting the actual
frequencies in my signal and the corresponding amplitudes, or I am wrong?

I could use the coherence function for each time-window to check which
frequencies are indeed excited in the process and which are not (and are
due to noise or non-linearities), do you think that could be useful?	 

_____________________________		
Posted through www.DSPRelated.com

Francky17 wrote:
>> I would suggest investigating Lissajous figures.
>> See:
>>    http://en.wikipedia.org/wiki/Lissajous_curve
>>    http://www-history.mcs.st-and.ac.uk/Curves/Lissajous.html
>>
>>
>
> Thank you for your answer. Nevertheless, I don't really see w Lissajous
> figures would help me to improve my current program.
>
> The problem is to get the right frequencies contained in each time-window
> and the right corresponding amplitudes even if there is noise in the
> signal. Lissajous figures would enable me to calculate the phase between to
> signals or the frequency ratio of two signals, but not selecting the actual
> frequencies in my signal and the corresponding amplitudes, or I am wrong?
>
> I could use the coherence function for each time-window to check which
> frequencies are indeed excited in the process and which are not (and are
> due to noise or non-linearities), do you think that could be useful?	
>

I think an underlying problem was I was trying to guess what is 
the underlying problem you are working on is. The definition of 
Bode Plot I'm familiar with presumes the system described is 
Linear Time Invariant. That led me to jump to an apparently 
erroneous conclusion - that your input signal was a noisy sine wave.

What is system under test?
What is the source and shape of the input signal?
After the output is collected and analyzed, how will the results 
be used?
     ... must the results be available in near real time?

On Wed, 19 Jun 2013 06:37:06 -0500, "Francky17" <95322@dsprelated>
wrote:

>Hello,
>
>first, I want to apologize because my english is not perfect but it seems
>that there are a lot of skilled people here in frequency analysis so I'll
>try to be as clear as possible.
>
>My goal is to build an online spectrum analyser given an input signal u and
>an output signal y. Both signals are acquired with a sampling frequency of
>200Hz and my program has to build the bode diagram of the underlying
>process when new data comes (ie. amplitude y:u versus frequency and phase
>versus frequency). The frequency-range of interest is [0,5] Hz.
>
>Here is what I have done until now and which needs to be improved:
>to acquire the first window signal, I wait 4seconds (window-length) and I
>save my windowed signals u_w, y_w. I remove the mean of u_w and y_w and
>multiply the result with a hanning window to reduce spectral leakege such
>as:
>
>u_wh=(u_w-mean(u_w))*hann(length(u_w))
>y_wh=(y_w-mean(y_w))*hann(length(y_w))
>
>I use than the FFT algorithm built in matlab:
>
>FFT(u_wh) and FFT(y_wh)
>
>I wrote a function to detect the peaks (global and locals), that is to say
>the frequencies fi contained in the input signal u_wh. For a given detected
>frequency I can calculated the amplitude (abs(fft(y_wh))/abs(fft(u_wh)))
>and the Phase. 
>
>Finally I move the window with 50% overlapping over the previous window.
>
>This programm enables me to plot the bode diagram of a process in real time
>and it works fine for perfect test-signals and for complex signals (that is
>to say that I take a complex input u signal and I generate the output y
>thanks to a known state space model).
>
>Now I am testing the program for real time (ie inputs u measured AND
>outputs y measured and the signals are than noisy) and the results are not
>so good. The bode diagram is quite noisy (different amplitudes and phases
>values for the same frequency). Do you see a way to improve my algorithm?
>
>I thank you and wish you a good day.
>Franck.

It sounds to me like you're building a Network Analyzer rather than a
Spectrum Analyzer.  

Do you have control of signal u?   If you don't have control of u then
it may be difficult to assign much meaning to the output of your
process. 

Do you know the characteristics of the system through which u passes
to produce y?   

Unless you can control u to fully characterize the system under test,
or have some control of the system under test, I think it'll be
difficult to get a good calibration of your system.  For example, do
you know the source of the noise?   Is it in u or is it from the
system under test?

Eric Jacobsen
Anchor Hill Communications
http://www.anchorhill.com

>Francky17 wrote:
>>> I would suggest investigating Lissajous figures.
>>> See:
>>>    http://en.wikipedia.org/wiki/Lissajous_curve
>>>    http://www-history.mcs.st-and.ac.uk/Curves/Lissajous.html
>>>
>>>
>>
>> Thank you for your answer. Nevertheless, I don't really see w Lissajous
>> figures would help me to improve my current program.
>>
>> The problem is to get the right frequencies contained in each
time-window
>> and the right corresponding amplitudes even if there is noise in the
>> signal. Lissajous figures would enable me to calculate the phase between
to
>> signals or the frequency ratio of two signals, but not selecting the
actual
>> frequencies in my signal and the corresponding amplitudes, or I am
wrong?
>>
>> I could use the coherence function for each time-window to check which
>> frequencies are indeed excited in the process and which are not (and
are
>> due to noise or non-linearities), do you think that could be useful?	
>>
>
>I think an underlying problem was I was trying to guess what is 
>the underlying problem you are working on is. The definition of 
>Bode Plot I'm familiar with presumes the system described is 
>Linear Time Invariant. That led me to jump to an apparently 
>erroneous conclusion - that your input signal was a noisy sine wave.
>
>What is system under test?
>What is the source and shape of the input signal?
>After the output is collected and analyzed, how will the results 
>be used?
>     ... must the results be available in near real time?
>


Ok I will try to be clearer.
The system under test is a vehicle. I have a model describing the roll
dynamic of the vehicle. The input of the corresponding state-space model is
the laterel acceleration and the output is the roll angle. So in practice I
have sensors measuring the lateral acceleration and the roll angle of the
vehicle.

The input signal, the acceleration, has an unknown shape (it depends of the
action of the driver and of the shape of the road). It is than a rather
complex signal, with sinusoids of small amplitudes (lateral accelerations
between 0-1.5m/s2 + some noise due to the sensors).

The transfer function of the process has to be determined (hence the need
to perform a frequency analysis) and the parameters of my model will than
be identified in the frequency domain (but that is no the issue here). And
yes, everything has to be done online (at least my transfer function has to
be updated every 10 seconds or so).

I think my issue is that when I extract the frequencies contained (definied
by a local maximum in abs(FFT(u_w)) i do not check if the frequency is
actually relevant (i.e this frequency is also contained in the output
signal and is than the consequence of the reaction of the vehicule).
Moreover, I may have amplitude errors in abs(FFT(u_w)) that I would like to
reduce. Filtering the input and output data (with a 5Hz butterworth filter)
does not really improve my results.

Thank your for your attention.	 

_____________________________		
Posted through www.DSPRelated.com

>On Wed, 19 Jun 2013 06:37:06 -0500, "Francky17" <95322@dsprelated>
>wrote:
>
>>Hello,
>>
>>first, I want to apologize because my english is not perfect but it
seems
>>that there are a lot of skilled people here in frequency analysis so
I'll
>>try to be as clear as possible.
>>
>>My goal is to build an online spectrum analyser given an input signal u
and
>>an output signal y. Both signals are acquired with a sampling frequency
of
>>200Hz and my program has to build the bode diagram of the underlying
>>process when new data comes (ie. amplitude y:u versus frequency and
phase
>>versus frequency). The frequency-range of interest is [0,5] Hz.
>>
>>Here is what I have done until now and which needs to be improved:
>>to acquire the first window signal, I wait 4seconds (window-length) and
I
>>save my windowed signals u_w, y_w. I remove the mean of u_w and y_w and
>>multiply the result with a hanning window to reduce spectral leakege
such
>>as:
>>
>>u_wh=(u_w-mean(u_w))*hann(length(u_w))
>>y_wh=(y_w-mean(y_w))*hann(length(y_w))
>>
>>I use than the FFT algorithm built in matlab:
>>
>>FFT(u_wh) and FFT(y_wh)
>>
>>I wrote a function to detect the peaks (global and locals), that is to
say
>>the frequencies fi contained in the input signal u_wh. For a given
detected
>>frequency I can calculated the amplitude (abs(fft(y_wh))/abs(fft(u_wh)))
>>and the Phase. 
>>
>>Finally I move the window with 50% overlapping over the previous window.
>>
>>This programm enables me to plot the bode diagram of a process in real
time
>>and it works fine for perfect test-signals and for complex signals (that
is
>>to say that I take a complex input u signal and I generate the output y
>>thanks to a known state space model).
>>
>>Now I am testing the program for real time (ie inputs u measured AND
>>outputs y measured and the signals are than noisy) and the results are
not
>>so good. The bode diagram is quite noisy (different amplitudes and
phases
>>values for the same frequency). Do you see a way to improve my
algorithm?
>>
>>I thank you and wish you a good day.
>>Franck.
>
>It sounds to me like you're building a Network Analyzer rather than a
>Spectrum Analyzer.  
>
>Do you have control of signal u?   If you don't have control of u then
>it may be difficult to assign much meaning to the output of your
>process. 
>
>Do you know the characteristics of the system through which u passes
>to produce y?   
>
>Unless you can control u to fully characterize the system under test,
>or have some control of the system under test, I think it'll be
>difficult to get a good calibration of your system.  For example, do
>you know the source of the noise?   Is it in u or is it from the
>system under test?
>
>Eric Jacobsen
>Anchor Hill Communications
>http://www.anchorhill.com
>

I don't really have control of my input signal u, but I know the order of
the transfer function of my system. And that is the point, having the bode
diagram for the frequencies (and only them) which are contained in u and
for which the system responds (from 0 to 5Hz). If some points are missing
between 4 or 5  Hz or somewhere else Hz, because a few points of the bode
diagram will be enough to identify the models parameters (the parameters of
my state space model can be identified thanks to a parametrized
identification algorithm).

The issue is that I should be able to properly detect the relevant
frequencies for which I can get a reliable amplitude and phase information.
	 

_____________________________		
Posted through www.DSPRelated.com

So here are some new elements I found.

instead of computing the transfer function as the ratio H(f) =
abs(fft(y))/abs(fft(u)) I can rather calculate I can take the ratio of the
FFT of the cross-correlation and the auto-correlation function such as H(f)
= Sxy(f)/Sxx(f). Indeed my bode diagram has less noisy points when I do
that. Do you confirm that it is a better way to compute the transfer
function? Is there a mathematical explanation to this improvement?

I calculated the coherence function of my data u and y and I saw that the
coherence is > 0.5 for all frequencies in [0,5] Hz except for the range
[0.8, 1.2] Hz. My idea would to remove these points (which lead indeed to
mysterious results in the bode diagram). Nevertheless, I am not able to
explain why there is this short interval with really low coherence. Would
you have an idea?

Thanks a lot.	 

_____________________________		
Posted through www.DSPRelated.com

Hi,

you know that the system is causal. The response cannot happen before the
input. 
Probably you can also truncate the impulse response at some point. You're
doing that implicitly by windowing and block processing. 

With that in mind, one solution is to state that output should be a FIR
filter response of the input. Iterate the coefficients of the FIR filter
(example for the LMS algorithm can be found as adaptive equalizer in
communications, reverse the role of output and input). Then calculate the
frequency response of the FIR filter (zero-padding and FFT).

Whole libraries have been written about system identification, so I guess
there's more to it. Anyway, this is what I'd do to whip up a first
prototype.

 	 

_____________________________		
Posted through www.DSPRelated.com

On Thu, 20 Jun 2013 03:36:23 -0500, "Francky17" <95322@dsprelated>
wrote:

>So here are some new elements I found.
>
>instead of computing the transfer function as the ratio H(f) =
>abs(fft(y))/abs(fft(u)) I can rather calculate I can take the ratio of the
>FFT of the cross-correlation and the auto-correlation function such as H(f)
>= Sxy(f)/Sxx(f). Indeed my bode diagram has less noisy points when I do
>that. Do you confirm that it is a better way to compute the transfer
>function? Is there a mathematical explanation to this improvement?

The cross- and auto-correlation functions amplify the correlated parts
of the signals and attenuate the uncorrelated parts.   I know that
sounds obvious and simplistic, but the benefit is that the
uncorrelated noise gets attenuated.   That may explain some of the
improvement that you saw.

>I calculated the coherence function of my data u and y and I saw that the
>coherence is > 0.5 for all frequencies in [0,5] Hz except for the range
>[0.8, 1.2] Hz. My idea would to remove these points (which lead indeed to
>mysterious results in the bode diagram). Nevertheless, I am not able to
>explain why there is this short interval with really low coherence. Would
>you have an idea?

This may be related to what I was hinting at earlier, that since you
don't have control of u, or the system under test, you don't have a
means to calibrate the analysis at all frequencies, and the output may
not be meaningful at all frequencies.  Depending on what's meaningful
to you and what you're trying to get out of the process, you may be
able to do something like you've indicated to determine where the
useful information is, but that's dependent on a lot of things.


Eric Jacobsen
Anchor Hill Communications
http://www.anchorhill.com