Prewhitening....how to explain?

Hi All,
Is there a book or source that explains what prewhitening is, in
relatively simple terms?  Or would any of you like to explain it to me,
please?

I have a blood flow signal and I want to prewhiten it and then do some
blind deconvolution on it to see what happens.  As the prewhitener it
has been suggested to use a predictor as the signals I have are quite
short. (Some form of stochastic gradient algorithm (to prewhiten)
longer signals?) Is this correct?

I just don't quite 'get' what prewhitening is about.  Its a term I can
use but I can't explain it -- something that would be useful as I am
working with medics!


Many thanks

Vicki,

A signal is "white" if each sample is independent statistically from
every other sample.  In other words, if you knew the data up until
sample n-1, you would have no information about sample n.

A pre-whitening filter takes a signal that is not white and produces a
white signal.  This is performed by a predictor, as you mentioned.  The
way the predictor whitens the signal is that itl attempts to predict
sample n  based on the information from the previous samples.  If you
subtract this prediction from the actual sample n, you will be left
with the portion of sample n that is not related to the rest of the
samples.

To explain where the stochastic gradient algorithm comes into play,
here is a quick explanation of the system used to whiten the signal.

Say you're original signal is x[n] (discrete samples)
Delay signal by single samples to get x[n-1]
Run signal through whitening filter to get x'[n] (estimate of x[n])
Subtract x'[n] from x[n] to get e[n] (error signal)
Feed error signal to whitening filter to update filter coefficients
using stochastic gradient algorithm (also look under LMS filter)

The signal e[n] is your whitened signal.  If the system is perfect, the
predictor error will be a white signal, as that is the best performance
the predictor can hope to have.  Depending on how correlated the
samples are in your original x[n] are, you may need a very large filter
to whiten the signal.

The usefulness of whitening data is in performing data compression.
For example, digital cellular phones do not transmit your digitized
voice over the air.  Instead, they whiten the voice signal, and
transmit the whitened signal power (or impulse period)  and the filter
coefficients.

Hope this helps,
Susheem

Vicki wrote:

> I just don't quite 'get' what prewhitening is about.  Its a term I
> can use but I can't explain it -- something that would be useful as
> I am working with medics!

One valid interpretation has already been given; here's a slightly
different take.

Prewhitening aims to make the signal contain equal-strength components
at every possible frequency.  The assumption is that all frequencies
present in the signal contain _some_ information that is useful.  If
the signal is NOT white, then some information at some frequencies
might swamp other, equally valid, useful information.

Ciao,

Peter K.

Super.  Thank you for the replies.  That makes sense.

V.

Vicki wrote:
> Hi All,
> Is there a book or source that explains what prewhitening is, in
> relatively simple terms?  Or would any of you like to explain it to me,
> please?

That is what Michael Jackson does before he goes to court. The extra 
entropy generated should be enough for a different story to emerge 
everyday, to keep the media happy.

TonyF

Vicki wrote:

>Hi All,
>Is there a book or source that explains what prewhitening is, in
>relatively simple terms?  Or would any of you like to explain it to me,
>please?
>
>I have a blood flow signal and I want to prewhiten it and then do some
>blind deconvolution on it to see what happens.  As the prewhitener it
>has been suggested to use a predictor as the signals I have are quite
>short. (Some form of stochastic gradient algorithm (to prewhiten)
>longer signals?) Is this correct?
>
>I just don't quite 'get' what prewhitening is about.  Its a term I can
>use but I can't explain it -- something that would be useful as I am
>working with medics!
>  
>
The first time I encountered the term "pre-whitening filter" was in a 
block diagram. I was really puzzled. If this filter came before the 
whitening, where the heck did the whitening take place. It was a while 
before I starting probing into what the blocks in the diagram actually 
did. Then it was obvious "pre-whitening" was just a term chosen to 
confuse the beginner. I have since seen many beginners fall into the 
same trap. I hate these poorly chosen terms.

Regards,
Steve

Steve Underwood wrote:
> Vicki wrote:
> 
>> Hi All,
>> Is there a book or source that explains what prewhitening is, in
>> relatively simple terms?  Or would any of you like to explain it to me,
>> please?
>>
>> I have a blood flow signal and I want to prewhiten it and then do some
>> blind deconvolution on it to see what happens.  As the prewhitener it
>> has been suggested to use a predictor as the signals I have are quite
>> short. (Some form of stochastic gradient algorithm (to prewhiten)
>> longer signals?) Is this correct?
>>
>> I just don't quite 'get' what prewhitening is about.  Its a term I can
>> use but I can't explain it -- something that would be useful as I am
>> working with medics!
>>  
>>
> The first time I encountered the term "pre-whitening filter" was in a 
> block diagram. I was really puzzled. If this filter came before the 
> whitening, where the heck did the whitening take place. It was a while 
> before I starting probing into what the blocks in the diagram actually 
> did. Then it was obvious "pre-whitening" was just a term chosen to 
> confuse the beginner. I have since seen many beginners fall into the 
> same trap. I hate these poorly chosen terms.
> 
> Regards,
> Steve

Oh, the hyphen! A pre-whitening filter whitens the signal before it is 
analyzed. The Prefix is unnecessary, but ought not be confusing. it 
certainly isn't "pre whitening filter", equivalent to pre 
whitening-filter". But hyphens are so often neglected by otherwise 
competent authors (a reasonable neglect only in non-technical writing) 
that they can't be depended on for guidance. That's life!

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

Jerry Avins wrote:
> Steve Underwood wrote:
> 
>> Vicki wrote:
>>
>>> Hi All,
>>> Is there a book or source that explains what prewhitening is, in
>>> relatively simple terms?  Or would any of you like to explain it to me,
>>> please?
>>>
>>> I have a blood flow signal and I want to prewhiten it and then do some
>>> blind deconvolution on it to see what happens.  As the prewhitener it
>>> has been suggested to use a predictor as the signals I have are quite
>>> short. (Some form of stochastic gradient algorithm (to prewhiten)
>>> longer signals?) Is this correct?
>>>
>>> I just don't quite 'get' what prewhitening is about.  Its a term I can
>>> use but I can't explain it -- something that would be useful as I am
>>> working with medics!
>>>  
>>>
>> The first time I encountered the term "pre-whitening filter" was in a 
>> block diagram. I was really puzzled. If this filter came before the 
>> whitening, where the heck did the whitening take place. It was a while 
>> before I starting probing into what the blocks in the diagram actually 
>> did. Then it was obvious "pre-whitening" was just a term chosen to 
>> confuse the beginner. I have since seen many beginners fall into the 
>> same trap. I hate these poorly chosen terms.
>>
>> Regards,
>> Steve
> 
> 
> Oh, the hyphen! A pre-whitening filter whitens the signal before it is 
> analyzed. The Prefix is unnecessary, but ought not be confusing. it 
> certainly isn't "pre whitening filter", equivalent to pre 
> whitening-filter". But hyphens are so often neglected by otherwise 
> competent authors (a reasonable neglect only in non-technical writing) 
> that they can't be depended on for guidance. That's life!

What you say is accurate. However, a term which relies on such subtlety 
in its reading was not chosen for its clarity. Why do people add a 
completely superfluous "pre" anyway? Its a whitening filter, whether or 
not any further processing comes after it. Unnecessary extra word - must 
have been an American who coined that term. :-)

Regards,
Steve

Steve Underwood wrote:
> Jerry Avins wrote:

   ...

>> Oh, the hyphen! A pre-whitening filter whitens the signal before it is 
>> analyzed. The Prefix is unnecessary, but ought not be confusing. it 
>> certainly isn't "pre whitening filter", equivalent to pre 
>> whitening-filter". But hyphens are so often neglected by otherwise 
>> competent authors (a reasonable neglect only in non-technical writing) 
>> that they can't be depended on for guidance. That's life!
> 
> 
> What you say is accurate. However, a term which relies on such subtlety 
> in its reading was not chosen for its clarity. Why do people add a 
> completely superfluous "pre" anyway? Its a whitening filter, whether or 
> not any further processing comes after it. Unnecessary extra word - must 
> have been an American who coined that term. :-)

You're absolutely right. I pointed out -- round-aboutly enough to 
obscure the point -- that "pre-whitening filter" is formally 
unambiguous, but so cloaked with the baggage that English now carries 
that its accuracy isn't evident. The redundancy of "pre" obscures it 
further.

This morning, in Palmer's office, I saw this sign: "Doctor Palmer only 
sees acupuncture patients on Friday." Being curious, I asked the 
receptionist if Doctor Palmer sees only acupuncture patients on Friday, 
or if Doctor Palmer sees acupuncture patients on only Friday. She 
couldn't distinguish the two meanings. Sad.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

Steve Underwood wrote:

 > Jerry Avins wrote:


   ...

 >> Oh, the hyphen! A pre-whitening filter whitens the signal before it 
is analyzed. The Prefix is unnecessary, but ought not be confusing. it 
certainly isn't "pre whitening filter", equivalent to pre 
whitening-filter". But hyphens are so often neglected by otherwise 
competent authors (a reasonable neglect only in non-technical writing) 
that they can't be depended on for guidance. That's life!
 >
 >
 >
 > What you say is accurate. However, a term which relies on such 
subtlety in its reading was not chosen for its clarity. Why do people 
add a completely superfluous "pre" anyway? Its a whitening filter, 
whether or not any further processing comes after it. Unnecessary extra 
word - must have been an American who coined that term. :-)


You're absolutely right. I pointed out -- round-aboutly enough to 
obscure the point -- that "pre-whitening filter" is formally 
unambiguous, but so cloaked with the baggage that English now carries 
that its accuracy isn't evident. The redundancy of "pre" obscures it 
further.

This morning, in Palmer's office, I saw this sign: "Doctor Palmer only 
sees acupuncture patients on Friday." Being curious, I asked the 
receptionist if Doctor Palmer sees only acupuncture patients on Friday, 
or if Doctor Palmer sees acupuncture patients only on Friday. She 
couldn't distinguish the two meanings. Sad.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������