What can we expect by taking Fourier Transform of noise or random samples?

Hi all,

I am trying to understand using Fourier Transform to do denoising.

I heard many times that people say suppose I have z(t)=y(t)+n(t), where z is 
the observations of the useful signal y and n is the noise... then we can 
look at the spectrum of z(t) and then if we know our useful signal is in 
some certain frequency range then we can do some filtering to recover the 
useful signal and get rid of the noise.

I want to understand this approach.

I feel n(t) is a random process, y(t) is a deterministic signal, then is 
z(t) wide sense stationary? My guess is that z(t) is a non-WSS random 
process... what is the meaning of taking Fourier Transform of a random 
process?

If we take FT of z(t)=y(t)+n(t), what is the meaning of taking FT of a 
random process n(t)? Commonly people say noise n(t) is white... but this 
"white" is talking about the power spectral density of the random process, 
it has nothing to do with taking the FT of n(t) and consequently z(t)... Am 
I right?

I did some experiments in Matlab:

>>plot(abs(fftshift(fft(rand(1, 20000)))))

gives a huge peak/impulse at DC frequency f=0...  what does this mean?

>>plot(abs(fftshift(fft(rand(1, 20000)))))

gives a uniformly chaotic noisy spectrum, looks like white...

>>plot(abs(fftshift(fft(random('rayleigh', 10, 1, 20000)))))

also gives a huge peak/impulse at DC frequency f=0...  what does this mean?

--------------------

Maybe I should ask what shall we expect to see if we take FT of random 
samples?

Any thoughts?

Thanks a lot!

lucy wrote:
> Hi all,
> 
> I am trying to understand using Fourier Transform to do denoising.
> 
> I heard many times that people say suppose I have z(t)=y(t)+n(t), where z is 
> the observations of the useful signal y and n is the noise... then we can 
> look at the spectrum of z(t) and then if we know our useful signal is in 
> some certain frequency range then we can do some filtering to recover the 
> useful signal and get rid of the noise.
> 
> I want to understand this approach.
> 
> I feel n(t) is a random process, y(t) is a deterministic signal, then is 
> z(t) wide sense stationary? My guess is that z(t) is a non-WSS random 
> process... what is the meaning of taking Fourier Transform of a random 
> process?
> 
> If we take FT of z(t)=y(t)+n(t), what is the meaning of taking FT of a 
> random process n(t)? Commonly people say noise n(t) is white... but this 
> "white" is talking about the power spectral density of the random process, 
> it has nothing to do with taking the FT of n(t) and consequently z(t)... Am 
> I right?
> 
> I did some experiments in Matlab:
> 
> 
>>>plot(abs(fftshift(fft(rand(1, 20000)))))
> 
> 
> gives a huge peak/impulse at DC frequency f=0...  what does this mean?
> 
> 
>>>plot(abs(fftshift(fft(rand(1, 20000)))))
> 
> 
> gives a uniformly chaotic noisy spectrum, looks like white...
> 
> 
>>>plot(abs(fftshift(fft(random('rayleigh', 10, 1, 20000)))))
> 
> 
> also gives a huge peak/impulse at DC frequency f=0...  what does this mean?
> 
> --------------------
> 
> Maybe I should ask what shall we expect to see if we take FT of random 
> samples?

Is the random function balanced around 0? If not, the transform will 
have a dc peak because it's all positive. Maybe you could do 
(rand(1,20000)-0.5)?
--
john

"lucy" <losemind@yahoo.com> wrote in message 
news:co6nq8$oaj$1@news.Stanford.EDU...
> Hi all,
>
> I am trying to understand using Fourier Transform to do denoising.
>
> I heard many times that people say suppose I have z(t)=y(t)+n(t), where z 
> is the observations of the useful signal y and n is the noise... then we 
> can look at the spectrum of z(t) and then if we know our useful signal is 
> in some certain frequency range then we can do some filtering to recover 
> the useful signal and get rid of the noise.
>
> I want to understand this approach.
>
> I feel n(t) is a random process, y(t) is a deterministic signal, then is 
> z(t) wide sense stationary? My guess is that z(t) is a non-WSS random 
> process... what is the meaning of taking Fourier Transform of a random 
> process?
>
> If we take FT of z(t)=y(t)+n(t), what is the meaning of taking FT of a 
> random process n(t)? Commonly people say noise n(t) is white... but this 
> "white" is talking about the power spectral density of the random process, 
> it has nothing to do with taking the FT of n(t) and consequently z(t)... 
> Am I right?
>
> I did some experiments in Matlab:
>
>>>plot(abs(fftshift(fft(rand(1, 20000)))))
>
> gives a huge peak/impulse at DC frequency f=0...  what does this mean?
>
>>>plot(abs(fftshift(fft(rand(1, 20000)))))
>
> gives a uniformly chaotic noisy spectrum, looks like white...
>
>>>plot(abs(fftshift(fft(random('rayleigh', 10, 1, 20000)))))
>
> also gives a huge peak/impulse at DC frequency f=0...  what does this 
> mean?
>
> --------------------
>
> Maybe I should ask what shall we expect to see if we take FT of random 
> samples?
>

You might do better to take a number of these and average their magnitudes - 
under the assumption that the signal is steady and the noise samples will 
vary from transform to transform.

It's not axiomatic that you will "see" the signal.  For every doubling of 
the Fourier Transform length, you will improve the signal to noise ratio by 
3dB (because the analysis bandwidth is reduced by a factor of 2 and the 
affect on the noise goes by sqrt(BW) - so the noise only goes down by 
sqrt(2).  So, you have to pick a transform length that will give you 
adequate noise gain without over-resolving the signal.

Example:
You have a bandpass signal with bandwidth 10Hz that is mixed with white 
noise.
You start with a Fourier Transform temporal epoch of 10msec.  The bandwidth 
is 100 Hz.
Let's say that you can't see the signal at this SNR.
Now change the temporal epoch to 100msec.
Now the bandwidth is 10Hz, the same as the signal (and we'll assume that the 
signal is centered on one of the bins - which it may not be in actual 
practice).
Now let us assume that the SNR is good enough to see the signal in an 
amplitude plot of the FFT.
Now change the temporal epoch to 1sec.  The bandwidth is now 1Hz.
The signal will now appear over 10 bins.  Maybe it's just as easy to see it 
at this SNR.
But, what if 10Hz bandwidth and the SNR didn't allow the signal to be 
"seen"?

Fred


Fred 

John O'Flaherty wrote:
> lucy wrote:
> 
>> Hi all,
>>
>> I am trying to understand using Fourier Transform to do denoising.
>>
>> I heard many times that people say suppose I have z(t)=y(t)+n(t), 
>> where z is the observations of the useful signal y and n is the 
>> noise... then we can look at the spectrum of z(t) and then if we know 
>> our useful signal is in some certain frequency range then we can do 
>> some filtering to recover the useful signal and get rid of the noise.
>>
>> I want to understand this approach.
>>
>> I feel n(t) is a random process, y(t) is a deterministic signal, then 
>> is z(t) wide sense stationary? My guess is that z(t) is a non-WSS 
>> random process... what is the meaning of taking Fourier Transform of a 
>> random process?
>>
>> If we take FT of z(t)=y(t)+n(t), what is the meaning of taking FT of a 
>> random process n(t)? Commonly people say noise n(t) is white... but 
>> this "white" is talking about the power spectral density of the random 
>> process, it has nothing to do with taking the FT of n(t) and 
>> consequently z(t)... Am I right?
>>
>> I did some experiments in Matlab:
>>
>>
>>>> plot(abs(fftshift(fft(rand(1, 20000)))))
>>
>>
>>
>> gives a huge peak/impulse at DC frequency f=0...  what does this mean?
>>
>>
>>>> plot(abs(fftshift(fft(rand(1, 20000)))))
>>
>>
>>
>> gives a uniformly chaotic noisy spectrum, looks like white...
>>
>>
>>>> plot(abs(fftshift(fft(random('rayleigh', 10, 1, 20000)))))
>>
>>
>>
>> also gives a huge peak/impulse at DC frequency f=0...  what does this 
>> mean?
>>
>> --------------------
>>
>> Maybe I should ask what shall we expect to see if we take FT of random 
>> samples?
> 
> 
> Is the random function balanced around 0? If not, the transform will 
> have a dc peak because it's all positive. Maybe you could do 
> (rand(1,20000)-0.5)?

Or use randn(1,20000), which give a set of normally distributed numbers 
with mean zero.

--
john

John O'Flaherty <quiasmox@yahoo.com> wrote in message news:<30o85oF2vib5nU1@uni-berlin.de>...

> > plot(abs(fftshift(fft(rand(1, 20000)))))
> > gives a huge peak/impulse at DC frequency f=0...  what does this mean?
>
> Is the random function balanced around 0? If not, the transform will 
> have a dc peak because it's all positive. Maybe you could do 
> (rand(1,20000)-0.5)?

Hint: use randn() not rand().

In article <co6nq8$oaj$1@news.Stanford.EDU>,
 "lucy" <losemind@yahoo.com> wrote:

> Hi all,
> 
> I am trying to understand using Fourier Transform to do denoising.
> 
> I heard many times that people say suppose I have z(t)=y(t)+n(t), where z is 
> the observations of the useful signal y and n is the noise... then we can 
> look at the spectrum of z(t) and then if we know our useful signal is in 
> some certain frequency range then we can do some filtering to recover the 
> useful signal and get rid of the noise.
> 
> I want to understand this approach.
> 
> I feel n(t) is a random process, y(t) is a deterministic signal, then is 
> z(t) wide sense stationary? My guess is that z(t) is a non-WSS random 
> process... what is the meaning of taking Fourier Transform of a random 
> process?
> 
> If we take FT of z(t)=y(t)+n(t), what is the meaning of taking FT of a 
> random process n(t)? Commonly people say noise n(t) is white... but this 
> "white" is talking about the power spectral density of the random process, 
> it has nothing to do with taking the FT of n(t) and consequently z(t)... Am 
> I right?
> 
> I did some experiments in Matlab:
> 
> >>plot(abs(fftshift(fft(rand(1, 20000)))))
> 
> gives a huge peak/impulse at DC frequency f=0...  what does this mean?

It means that the sample mean of the data is non-zero.  That's what the 
FFT is estimating at f=0.  Many people preprocess the data by 
subtracting the average from each observation to center at zero.

By the way, to have a true spectrum you should be squaring the transform 
results, not abs'ing.

> >>plot(abs(fftshift(fft(rand(1, 20000)))))
> 
> gives a uniformly chaotic noisy spectrum, looks like white...

Uniform spectrum indicates no correlation.  This follows directly from 
the Weiner-Khintchine theorem, which says that the spectrum can be found 
as the Fourier transform of the autocovariance function.  If your noise 
terms are independent, then the autocovariance is zero for all lags 
other than zero, yielding a flat spectrum.  (A common misconception is 
that the noise needs to be Gaussian for white noise, but uncorrelated is 
sufficient.)

> >>plot(abs(fftshift(fft(random('rayleigh', 10, 1, 20000)))))
> 
> also gives a huge peak/impulse at DC frequency f=0...  what does this mean?
> 
> --------------------
> 
> Maybe I should ask what shall we expect to see if we take FT of random 
> samples?
> 
> Any thoughts?
> 
> Thanks a lot!

Chatfield has written a very nice text called "Time Series Analysis".  
It's not as comprehensive or theoretical as many others, but it's quite 
accessible.

lucy wrote:
> Hi all,
> 
> I am trying to understand using Fourier Transform to do denoising.
> 
> I heard many times that people say suppose I have z(t)=y(t)+n(t), where z is 
> the observations of the useful signal y and n is the noise... then we can 
> look at the spectrum of z(t) and then if we know our useful signal is in 
> some certain frequency range then we can do some filtering to recover the 
> useful signal and get rid of the noise.
> 
> I want to understand this approach.
> 
> I feel n(t) is a random process, y(t) is a deterministic signal, then is 
> z(t) wide sense stationary? My guess is that z(t) is a non-WSS random 
> process... what is the meaning of taking Fourier Transform of a random 
> process?

This get back to the discussion of stationarity that we are discussing 
elsewhere.  If y(t) is known then z(t) is not stationary at all -- but 
you don't need z(t) to know y(t)!  If y(t) is a stationary random 
process then so is z(t).

Taking any non-trivial function of a random process results in a random 
process.  When you take the Fourier transform of a random process, say 
F{z(t)} -> Z(w), you get a random process in w.
> 
> If we take FT of z(t)=y(t)+n(t), what is the meaning of taking FT of a 
> random process n(t)? Commonly people say noise n(t) is white... but this 
> "white" is talking about the power spectral density of the random process, 
> it has nothing to do with taking the FT of n(t) and consequently z(t)... Am 
> I right?

Given that n(t) is a white random process its Fourier transform will be 
a white random process in w (I think, I'm probably oversimplifying for 
the mathematicians reading this).
> 
> I did some experiments in Matlab:
> 
> 
>>>plot(abs(fftshift(fft(rand(1, 20000)))))
> 
> 
> gives a huge peak/impulse at DC frequency f=0...  what does this mean?

It means that rand(1, 20000) returns a (hopefully) white random variable 
uniformly distributed between 1 and 20000, so it has a mean of 10000.5 
and a variance of 4 * 10^8/12.  So it's discrete Fourier transform will 
have a peak with an expected value of 10000.5, plus a bunch of random 
noise in each frequency bin from the randomness of the signal.  If you 
took the fft of rand(-10000, 10000) then you'd lose the central peak, 
assuming that rand(a,b) works with negative arguments.
> 
> 
>>>plot(abs(fftshift(fft(rand(1, 20000)))))
> 
> 
> gives a uniformly chaotic noisy spectrum, looks like white...

This is the same as your previous case -- ??
> 
> 
>>>plot(abs(fftshift(fft(random('rayleigh', 10, 1, 20000)))))
> 
> 
> also gives a huge peak/impulse at DC frequency f=0...  what does this mean?

It means the Rayliegh distribution also has a non-zero mean.
> 
> --------------------
> 
> Maybe I should ask what shall we expect to see if we take FT of random 
> samples?

If you are talking about the discrete Fourier transform of a random 
process, you can expect each bin of the transform to be a random process 
with a mean and variance determined by the frequency characteristics of 
the input process.  If the input process is zero-mean, stationary and 
white, then its discrete Fourier transform will have bins whose values 
are zero-mean and of equal variance.
> 
> Any thoughts?
> 
The parts of your post that are not answered in Freund & Walpole, 
"Mathematical Statistics", 1987 Prentice Hall are answered in Papoulis 
"Probability, Random Variables, and Stochastic Processes", 1984 
McGraw-Hill.  I was introduced to both of these books in classes, so I 
have no idea how well they will serve as self-study guides, but I find 
myself referring to "Mathematical Statistics" all the time, and 
"Probability etc." occasionally.

If you can't take a class or two, I think you could do well to get these 
or comparable books and do some studying on your own -- there's a lot of 
detail that we can't cover here.

 > Thanks a lot!
 >
You're welcome

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

"Tim Wescott" <tim@wescottnospamdesign.com> wrote in message
news:10qetg9453ha3b5@corp.supernews.com...
> lucy wrote:
> > Hi all,
> >
> > I am trying to understand using Fourier Transform to do denoising.
<snip>
> >
> > I did some experiments in Matlab:
> >
> >
> >>>plot(abs(fftshift(fft(rand(1, 20000)))))
> >
> >
> > gives a huge peak/impulse at DC frequency f=0...  what does this mean?
>
> It means that rand(1, 20000) returns a (hopefully) white random variable
> uniformly distributed between 1 and 20000,

I think this is matlabese for generate a  length 20000 vector of (hopefully)
randomly uniformly distributed numbers on the interval between 0 and 1
inclusive Tim..   May change your predictions on the DC component.

<snip>
> -- 
>
> Tim Wescott
> Wescott Design Services
> http://www.wescottdesign.com

Best of Luck - Mike

Mike Yarwood wrote:

> "Tim Wescott" <tim@wescottnospamdesign.com> wrote in message
> news:10qetg9453ha3b5@corp.supernews.com...
> 
>>lucy wrote:
>>
>>>Hi all,
>>>
>>>I am trying to understand using Fourier Transform to do denoising.
> 
> <snip>
> 
>>>I did some experiments in Matlab:
>>>
>>>
>>>
>>>>>plot(abs(fftshift(fft(rand(1, 20000)))))
>>>
>>>
>>>gives a huge peak/impulse at DC frequency f=0...  what does this mean?
>>
>>It means that rand(1, 20000) returns a (hopefully) white random variable
>>uniformly distributed between 1 and 20000,
> 
> 
> I think this is matlabese for generate a  length 20000 vector of (hopefully)
> randomly uniformly distributed numbers on the interval between 0 and 1
> inclusive Tim..   May change your predictions on the DC component.
> 
> <snip>
Could be right.  I use "help xxxx" a lot when I use MatLab.  At any 
rate, the other points (about the FFT of non-zero-mean random data) 
should still hold, with appropriate adjustments.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

lucy wrote:

> Hi all,
> 
> I am trying to understand using Fourier Transform to do denoising.
[ ... ]
> Any thoughts?

It all depends upon what your "real" problem is. But ...
You could consider denoising _without_ using a Fourier transform.

Basically, what you do then is taking the convolution of your signal
with a "filter" function, for example a bell shaped (Gauss) function:

Z(t) = norm . integral_(-oo)^(+oo) exp(-(x-t)^2/sigma^2/2) z(x) dx

Where norm = 1/(sigma*sqrt(2.pi))
The "bandwidth" sigma must be chosen properly.

Due to properties of the 'exp', the (numerical) domain of integration
is rather limited, say (t-2.pi.sigma) < x < (t+2.pi.sigma), which makes
the method feasible.

BTW. It is noted that the above convolution integral becomes a product
of functions in the Fourier domain: exp(-(omega*sigma)^2/2) * Y(omega)
Where Y(omega) = Fourier transform of z(t) .

IF your decision is in favour of the Fourier approach, nevertheless,
be well aware then that there exist subtle, but nasty, differences
between the so-called FFT (discrete version) and the continuous FT.

Han de Bruijn