"Janwillem van Dijk" <xyz.van.dijk@hccnet.nl> wrote in message
news:4184abaf$0$161$3a628fcd@reader2.nntp.hccnet.nl...
> Fred Marshall wrote:
>> "Janwillem van Dijk" <xyz.van.dijk@hccnet.nl> wrote in message
>> news:41836bdb$0$162$3a628fcd@reader1.nntp.hccnet.nl...
>>
>>>How to remove noise from a signal.
>>>
>>>I have available a zero signal mainly consisting of noise and a signal
>>>during a measurement. The real signal consists theoretically of one or a
>>>few (overlapping) more or less gaussian shaped peaks. In general the
>>>signal is collected in 64 time samples. The peak widths are then equal to
>>>between 20 and 40 time samples. The noise has essentially a Poisson
>>>distribution (counting process). A more or less "true" signal can be
>>>obtianed from a measurement with high signal.
>>>
>>>I understand that a neat filtering method is based on the convolution of
>>>the measurement signal and the zero signal using Fourier transformations.
>>>The FT of the zero signal thereby (after some transformation I presume)
>>>acting as a filter, but how?
>>>
>>>Can anybody give me steps needed in terms understandable for somebody not
>>>ignorant of numerical math but certainly not an expert in signal
>>>processing.
>>
>>
>> I don't know exactly what you mean by a "zero signal" but I will assume
>> that you meant it is the best replica you have of the signal that you
>> want to detect.
>> But then you say that the "real signal has: peak widths are then equal to
>> between 20 and 40 time samples"????
>>
>> So, it's not clear what your starting reference is and what you're trying
>> to detect. Actually, I guess you didn't say you wanted to detect - so
>> that's an assumption. What are you wanting to do?
>> - filter an obviously present signal so that you can extract parameters?
>> - detect a not-so obviously present signal?
>> - both?
>>
>> The process with the FFT you describe sounds like a matched filter which
>> has optimum properties in the presence of white noise. Here's how that
>> goes:
>>
>> Cross-correlate the signal reference with any / all received or measured
>> signals.
>> Signals that match the reference will be "enhanced" relative to the
>> noise.
>> The ideal output that results is the cross-correlation of the reference
>> with itself. Sometimes called the "ambiguity function" - you will want
>> to ponder what this looks like and how it affects what you do with the
>> results.
>> Cross-correlation is very closely related to convolution - as you
>> probably know.
>>
>> Where does an FFT come into this?
>> You can efficiently compute a temporal convolution like this:
>> FFT a signal replica and save this array.
>> FFT measurements (in an array length the same as the replica above) in
>> succeeding array blocks.
>> Multiply each measurement by the replica FFT you have saved.
>> IFFT the product.
>> For correlation, time-reverse the replica before the FFT - I believe
>> that's all it takes (check this out).
>>
>> Now, you may have many replicas depending on what the signal of interest
>> looks like.
>> For example, in sonar and radar there would be a replica for each Doppler
>> shift and you may choose to quantize the replica parameter into a
>> discrete set of values and, thus, replicas.
>> In your case, there may be a replica for each width of the signal shape
>> you described.
>>
>> Fred
>>
>>
>>
>>
> Thanks for the answers, I will have a look at that. Sounds easy enough.
>
> What I meant with zero-signal versus measurement signal is the response of
> the instrument witha blank sample versus the response with either a
> standard sample giving relatively high signal for calibration purposus or
> a sample to be analysed with in general a rather low signal.
>
> Hope this helps in understanding my question.
> Janwillem
Janwillem,
Oh! Well, if the "zero signal" is the instrument output with no signal
present, then that would probably be of no use for any processing unless it
has structure - i.e. if it has sinusoidal components that are measureable
above the noise.
So, perhaps the "standard sample" version is a good replica....
Fred
Reply by Janwillem van Dijk●October 31, 20042004-10-31
Fred Marshall wrote:
> "Janwillem van Dijk" <xyz.van.dijk@hccnet.nl> wrote in message
> news:41836bdb$0$162$3a628fcd@reader1.nntp.hccnet.nl...
>
>>How to remove noise from a signal.
>>
>>I have available a zero signal mainly consisting of noise and a signal
>>during a measurement. The real signal consists theoretically of one or a
>>few (overlapping) more or less gaussian shaped peaks. In general the
>>signal is collected in 64 time samples. The peak widths are then equal to
>>between 20 and 40 time samples. The noise has essentially a Poisson
>>distribution (counting process). A more or less "true" signal can be
>>obtianed from a measurement with high signal.
>>
>>I understand that a neat filtering method is based on the convolution of
>>the measurement signal and the zero signal using Fourier transformations.
>>The FT of the zero signal thereby (after some transformation I presume)
>>acting as a filter, but how?
>>
>>Can anybody give me steps needed in terms understandable for somebody not
>>ignorant of numerical math but certainly not an expert in signal
>>processing.
>
>
> I don't know exactly what you mean by a "zero signal" but I will assume that
> you meant it is the best replica you have of the signal that you want to
> detect.
> But then you say that the "real signal has: peak widths are then equal to
> between 20 and 40 time samples"????
>
> So, it's not clear what your starting reference is and what you're trying to
> detect. Actually, I guess you didn't say you wanted to detect - so that's
> an assumption. What are you wanting to do?
> - filter an obviously present signal so that you can extract parameters?
> - detect a not-so obviously present signal?
> - both?
>
> The process with the FFT you describe sounds like a matched filter which has
> optimum properties in the presence of white noise. Here's how that goes:
>
> Cross-correlate the signal reference with any / all received or measured
> signals.
> Signals that match the reference will be "enhanced" relative to the noise.
> The ideal output that results is the cross-correlation of the reference with
> itself. Sometimes called the "ambiguity function" - you will want to ponder
> what this looks like and how it affects what you do with the results.
> Cross-correlation is very closely related to convolution - as you probably
> know.
>
> Where does an FFT come into this?
> You can efficiently compute a temporal convolution like this:
> FFT a signal replica and save this array.
> FFT measurements (in an array length the same as the replica above) in
> succeeding array blocks.
> Multiply each measurement by the replica FFT you have saved.
> IFFT the product.
> For correlation, time-reverse the replica before the FFT - I believe that's
> all it takes (check this out).
>
> Now, you may have many replicas depending on what the signal of interest
> looks like.
> For example, in sonar and radar there would be a replica for each Doppler
> shift and you may choose to quantize the replica parameter into a discrete
> set of values and, thus, replicas.
> In your case, there may be a replica for each width of the signal shape you
> described.
>
> Fred
>
>
>
>
Thanks for the answers, I will have a look at that. Sounds easy enough.
What I meant with zero-signal versus measurement signal is the response
of the instrument witha blank sample versus the response with either a
standard sample giving relatively high signal for calibration purposus
or a sample to be analysed with in general a rather low signal.
Hope this helps in understanding my question.
Janwillem
Reply by Fred Marshall●October 30, 20042004-10-30
"Janwillem van Dijk" <xyz.van.dijk@hccnet.nl> wrote in message
news:41836bdb$0$162$3a628fcd@reader1.nntp.hccnet.nl...
> How to remove noise from a signal.
>
> I have available a zero signal mainly consisting of noise and a signal
> during a measurement. The real signal consists theoretically of one or a
> few (overlapping) more or less gaussian shaped peaks. In general the
> signal is collected in 64 time samples. The peak widths are then equal to
> between 20 and 40 time samples. The noise has essentially a Poisson
> distribution (counting process). A more or less "true" signal can be
> obtianed from a measurement with high signal.
>
> I understand that a neat filtering method is based on the convolution of
> the measurement signal and the zero signal using Fourier transformations.
> The FT of the zero signal thereby (after some transformation I presume)
> acting as a filter, but how?
>
> Can anybody give me steps needed in terms understandable for somebody not
> ignorant of numerical math but certainly not an expert in signal
> processing.
I don't know exactly what you mean by a "zero signal" but I will assume that
you meant it is the best replica you have of the signal that you want to
detect.
But then you say that the "real signal has: peak widths are then equal to
between 20 and 40 time samples"????
So, it's not clear what your starting reference is and what you're trying to
detect. Actually, I guess you didn't say you wanted to detect - so that's
an assumption. What are you wanting to do?
- filter an obviously present signal so that you can extract parameters?
- detect a not-so obviously present signal?
- both?
The process with the FFT you describe sounds like a matched filter which has
optimum properties in the presence of white noise. Here's how that goes:
Cross-correlate the signal reference with any / all received or measured
signals.
Signals that match the reference will be "enhanced" relative to the noise.
The ideal output that results is the cross-correlation of the reference with
itself. Sometimes called the "ambiguity function" - you will want to ponder
what this looks like and how it affects what you do with the results.
Cross-correlation is very closely related to convolution - as you probably
know.
Where does an FFT come into this?
You can efficiently compute a temporal convolution like this:
FFT a signal replica and save this array.
FFT measurements (in an array length the same as the replica above) in
succeeding array blocks.
Multiply each measurement by the replica FFT you have saved.
IFFT the product.
For correlation, time-reverse the replica before the FFT - I believe that's
all it takes (check this out).
Now, you may have many replicas depending on what the signal of interest
looks like.
For example, in sonar and radar there would be a replica for each Doppler
shift and you may choose to quantize the replica parameter into a discrete
set of values and, thus, replicas.
In your case, there may be a replica for each width of the signal shape you
described.
Fred
Reply by Janwillem van Dijk●October 30, 20042004-10-30
How to remove noise from a signal.
I have available a zero signal mainly consisting of noise and a signal
during a measurement. The real signal consists theoretically of one or a
few (overlapping) more or less gaussian shaped peaks. In general the
signal is collected in 64 time samples. The peak widths are then equal
to between 20 and 40 time samples. The noise has essentially a Poisson
distribution (counting process). A more or less "true" signal can be
obtianed from a measurement with high signal.
I understand that a neat filtering method is based on the convolution of
the measurement signal and the zero signal using Fourier
transformations. The FT of the zero signal thereby (after some
transformation I presume) acting as a filter, but how?
Can anybody give me steps needed in terms understandable for somebody
not ignorant of numerical math but certainly not an expert in signal
processing.
Many thanks,
Janwillem
P.S. Any source examples in e.g. Pascal Fortran C or Mathematica most
welcome
e-mail replace xyz -> jwe