Comparing matched and Wiener filters

Rune Allnor wrote:
> On 24 Des, 23:27, Oli Charlesworth <ca...@olifilth.co.uk> wrote:
>> maury wrote:
>>> One is SIGNAL based, and the other is MODEL based.
>> I appreciate that the optimisation criteria may have stemmed from
>> different sources, but I guess what I'm having most trouble reconciling
>>  is that MSE and SNR aren't equivalent (even if we constrain S to be fixed).
> 
> The main difference is that MSE is the deviation between the
> desired and actual response.

But surely, additive noise ("v" in my notation in the OP) is the cause
of this deviation?  Of course, this has been shaped by our receive
process, but this is true in the matched-filter case as well.

> SNR includes an additive noise
> term as well, which is missing in the MSE.

>I'm not sure there's much of a distinction.  The purpose of detection is
>to determine some property of the signal, either presence or absence
>(OOK), or some more complicated modulation scheme (or something else
>entirely in e.g. radar applications).  Therefore, the requirement is to
>*estimate* some parameter behind the signal.

Although they are not the same in general, there is no distinction between
detection theory and estimation theory when the vector (to be estimated)
assumes discrete values in a finite-dimensional space [1, pg.2].

>> If you consider the outputs of the two for a binary communication
signal,
>> the output of the MF looks like triangles with peak amplitudes at the
>> sampling times; that is, the MF maximizes the SNR only at these
sampling
>> instances.  On the other hand, the output of a Wiener filter, if you
really
>> wanted to use it, would look more like the transmitted binary
sequence;
>> this could not be the last stage for a detector, though.
>
>I see what you're saying.  However, isn't this just an artifact of the
>"filter across all time and sample" viewpoint of matched-filtering, as
>opposed to the "integrate-and-dump" viewpoint, where we only calculate
>the output for the sampling points (i.e. correlation, equivalent to the
>vector formation).
>-- 
>Oli

Not really.  You are interested in comparing MF vs Wiener, and the outputs
of these two are significantly different.  MF does not maximize SNR in the
same sense as Wiener.  If you define the SNR like you define MSE, then
SNR_Wiener > SNR_MF.  If you are thinking of appending an integrator
following the Wiener filter, that's another story.

What is your application?  Perhaps the answer to this question would give
us some perspective as to why you are trying to compete these two against
each other.

Emre

[1] L.L. Scharf, Statistical Signal Processing: Detection, Estimation, and
Time Series Analysis (New York: Addison-Wesley Publishing Co., 1990).

> If you are thinking of appending an integrator following the Wiener
filter, that's another story. 

In fact, under the Gaussian noise assumption, the story is rather short: 
Matched filter is equivalent to Wiener filter followed by an integrator.

Emre

emre wrote:
>>> If you consider the outputs of the two for a binary communication
> signal,
>>> the output of the MF looks like triangles with peak amplitudes at the
>>> sampling times; that is, the MF maximizes the SNR only at these
> sampling
>>> instances.  On the other hand, the output of a Wiener filter, if you
> really
>>> wanted to use it, would look more like the transmitted binary
> sequence;
>>> this could not be the last stage for a detector, though.
>> I see what you're saying.  However, isn't this just an artifact of the
>> "filter across all time and sample" viewpoint of matched-filtering, as
>> opposed to the "integrate-and-dump" viewpoint, where we only calculate
>> the output for the sampling points (i.e. correlation, equivalent to the
>> vector formation).
> 
> Not really.  You are interested in comparing MF vs Wiener, and the outputs
> of these two are significantly different.  MF does not maximize SNR in the
> same sense as Wiener.  If you define the SNR like you define MSE, then
> SNR_Wiener > SNR_MF.

Perhaps this is the essence of what I need to understand!  My current
understanding is that:

  MSE = E|x_est - x|^2
  SNR = E|x'|^2 / E|v'|^2

where x' is the post-filter signal component, and v' is the post-filter
noise component of x_est (both scalar in my original example), i.e.
x_est = x' + v'.

If we normalise our matched filter (g = h*/||h||^2), then we can be sure
that x' = x (the original data).  This leaves v' = x_est - x' = x_est -
x.  Therefore:

  SNR = S / E|x_est - x|^2

and equivalent to the MSE expression above (they're just
inversely-proportional).

How else would we define these quantities?


> If you are thinking of appending an integrator
> following the Wiener filter, that's another story.

Bad terminology on my part, I think.  I was referring to the general
form of the colloquial "integrate-and-dump", i.e. weighted integration,
i.e. the inner-product with the filter kernel.  No second
integration/filter implied!


> What is your application?  Perhaps the answer to this question would give
> us some perspective as to why you are trying to compete these two against
> each other.

I have no application in mind, I'm just trying to piece some more of the
puzzle together!  Up until now, I've naively assumed that the matched
filter is just a special case of the Wiener/LMMSE formulation, but when
I thought about it and then tried to prove it, I got stuck.

Incidentally, it's possible we're talking at cross purposes here due to
the confusion between the two viewpoints of the operations; one as a
continuous process operating in the time domain, the other as a
"one-off" vector operation (as expressed in my OP).  I contend that
these two viewpoints are equivalent if we only look at the time-domain
viewpoint at the sampling instants (i.e. the points of interest), and
treat all other points merely as an implementation artifact.  If you
disagree, please correct me!

>Incidentally, it's possible we're talking at cross purposes here due to
>the confusion between the two viewpoints of the operations; one as a
>continuous process operating in the time domain, the other as a
>"one-off" vector operation (as expressed in my OP).  I contend that
>these two viewpoints are equivalent if we only look at the time-domain
>viewpoint at the sampling instants (i.e. the points of interest), and
>treat all other points merely as an implementation artifact.  If you
>disagree, please correct me!

Right, when talking about integration etc., I was referring to the use of
the Matched filter as in binary communication:
http://en.wikipedia.org/wiki/Matched_filter#Example_of_matched_filter_in_digital_communications


>Perhaps this is the essence of what I need to understand!  My current
>understanding is that:
>
>  MSE = E|x_est - x|^2
>  SNR = E|x'|^2 / E|v'|^2
>
>where x' is the post-filter signal component, and v' is the post-filter
>noise component of x_est (both scalar in my original example), i.e.
>x_est = x' + v'.
>
>If we normalise our matched filter (g = h*/||h||^2), then we can be sure
>that x' = x (the original data).  This leaves v' = x_est - x' = x_est -
>x.  Therefore:
>
>  SNR = S / E|x_est - x|^2
>
>and equivalent to the MSE expression above (they're just
>inversely-proportional).
>
>How else would we define these quantities?

At last I understand your question (in fact, looking back, your OP was
very well-posed.)  In fact, the answer to your question is simpler than it
looks.  SNR is defined as the ratio between the energy of the signal over
energy of the noise allowed in the estimator, not the estimation error.  So
the right definition of SNR would be:
   SNR = E[ (g h x)^2 ] / E[ (g v)^2 ]
which equals (S N/sigma^2) for the matched filter, where N is the
dimension of g and v, and sigma^2 is the variance (like you defined
before.)

Emre

On 24 Des, 23:51, Oli Charlesworth <ca...@olifilth.co.uk> wrote:
> Rune Allnor wrote:
> > On 24 Des, 23:27, Oli Charlesworth <ca...@olifilth.co.uk> wrote:
> >> maury wrote:
> >>> One is SIGNAL based, and the other is MODEL based.
> >> I appreciate that the optimisation criteria may have stemmed from
> >> different sources, but I guess what I'm having most trouble reconciling
> >> &#4294967295;is that MSE and SNR aren't equivalent (even if we constrain S to be fixed).
>
> > The main difference is that MSE is the deviation between the
> > desired and actual response.
>
> But surely, additive noise ("v" in my notation in the OP) is the cause
> of this deviation? &#4294967295;Of course, this has been shaped by our receive
> process, but this is true in the matched-filter case as well.

This is where things become philosophical. The term 'Mean Square
Error'
points to some deviation of non-descript origin, whereas 'noise'
points
to some other signal - usually of stochastic nature - than the signal
of interest. So the MSE can be due to deterministic factors, whereas
the SNR explictly contains a stochastic component.

The derivations of the matched filter usually start out without
noise included in the model (if you know of a derivation which
contains noise at the outset, please let me know). In that case,
the constraints are deterministic, and the objective of the
determinsitic derivation is to find the filter that best fits some
criterion, given the reference signal one searches for.

The Wiener filter is derived from a *similar* point of view,
but explicitly includes stochastic additive processes, noise,
in the model. You still want to find a filter that best fits a
purpose given a reference signal, but in the Wiener filter the
additive coise causes explicit effects on the result.

There are derivations of DSP algorithms that start out with
a deterministic signal model (no noise included) that aim to
reduce the impact of numerical inaccurcies etc. In my experience,
these methods perform very well in the presence of additive noise.
The Kumaresan & Tufts version of Prony's method for frequeny
estimation is one such method. It totally outperformed MUSIC
which explicitly include noise in the signal model, both what
accuracy of results and computational efficiency were concerned.
Well, the T&K method produced results at least as good as
MUSIC at a far less computational cost.

Rune

On Wed, 24 Dec 2008 17:10:56 -0500, Randy Yates <yates@ieee.org>
wrote:

>Oli Charlesworth <catch@olifilth.co.uk> writes:
>
>> emre wrote:
>>> Hi Oli,
>>> 
>>> The two are designed for different applications, even if their
>>> formulations are similar.  Matched filter applies to the *detection* of the
>>> presence of a known signal, whereas Wiener filter applies to the
>>> *estimation* of an unknown random signal with known power spectrum.  In
>>> either case, the spectrum of the signal is assumed to be known.
>>
>> I'm not sure there's much of a distinction.  The purpose of detection is
>> to determine some property of the signal, either presence or absence
>> (OOK), or some more complicated modulation scheme (or something else
>> entirely in e.g. radar applications).  Therefore, the requirement is to
>> *estimate* some parameter behind the signal.
>>
>>
>>> If you consider the outputs of the two for a binary communication signal,
>>> the output of the MF looks like triangles with peak amplitudes at the
>>> sampling times; that is, the MF maximizes the SNR only at these sampling
>>> instances.  On the other hand, the output of a Wiener filter, if you really
>>> wanted to use it, would look more like the transmitted binary sequence;
>>> this could not be the last stage for a detector, though.
>>
>> I see what you're saying.  However, isn't this just an artifact of the
>> "filter across all time and sample" viewpoint of matched-filtering, as
>> opposed to the "integrate-and-dump" viewpoint, where we only calculate
>> the output for the sampling points (i.e. correlation, equivalent to the
>> vector formation).
>
>Yeah. A matched filter in digital comms assumes you have timing.

I've just been working on a blog article about what "Matched Filter"
means in comm.   The term gets abused, but you guys have already hit
the main points.  A matched filter may be used in a more continuous
fashion (like in remote sensing, radar or sonar), or it may only be
really useful at specific locations in the signal, like in comm.

It all boils down to the same thing, but somehow the terminology
sometimes gets twisted a bit along the way.

I think it's kinda cool that we work in an area that's esoteric enough
that definitions are often not clear.  It makes for interesting
dialogue.

Eric Jacobsen
Minister of Algorithms
Abineau Communications
http://www.ericjacobsen.org

Blog: http://www.dsprelated.com/blogs-1/hf/Eric_Jacobsen.php

On Dec 24, 6:12&#4294967295;am, Oliver Charlesworth <ca...@olifilth.co.uk> wrote:
> Hi,
>
> This is something that's been bugging me for the last couple of days.
>
> With the understanding that both are linear, we define the matched
> filter as the one that maximises the output SNR, and the Wiener filter
> as the one that minimises the mean square error (MSE). &#4294967295;Superficially,
> these definitions sound almost identical. However, even in the simplest
> model (estimating a scalar value with independent AWGN), we clearly end
> up with different filters:
>
> MF: &#4294967295; &#4294967295; &#4294967295; &#4294967295; &#4294967295;g = h*
> Wiener: &#4294967295; &#4294967295; &#4294967295;g = Sh*(Shh* + sigma^2.I)^-1
>
> such that x_est = gy, where y = hx + v (where x is scalar data with S =
> E|x|^2, v is AWGN with sigma^2.I = E|v|^2, {y,h,v} are Nx1 vectors, and
> * denotes conjugate transpose).

i wonder if these results are all that different (even though they
look different).

>
> My question is, what is the intuitive reason (i.e. not just "because the
> maths says so") why these don't end up with the same result, or more
> specifically, how the Wiener filter can minimise the MSE without
> maximising the SNR?

i've always thought (and i hadn't done anything in communications
since grad school) that the essential difference between the MF and
WF, given the same signal description and noise spectra (i think
white), was that of gain.  to nail down MSE, you have to worry about
gain.  to nail down S/N, you need not worry about gain.

there is a related question regarding LPC that i remember relating
this question to.

r b-j

Rune Allnor wrote:
> The derivations of the matched filter usually start out without
> noise included in the model (if you know of a derivation which
> contains noise at the outset, please let me know). In that case,
> the constraints are deterministic, and the objective of the
> determinsitic derivation is to find the filter that best fits some
> criterion, given the reference signal one searches for.

Maybe I've understood you, but the derivation in Ch.5 of Digital Comms
(Proakis) uses a continuous-time model with AWGN, and uses the
Cauchy-Schwarz inequality to show that the MF maximises the SNR.


-- 
Oli

On 26 Des, 15:11, Oli Charlesworth <ca...@olifilth.co.uk> wrote:
> Rune Allnor wrote:
> > The derivations of the matched filter usually start out without
> > noise included in the model (if you know of a derivation which
> > contains noise at the outset, please let me know). In that case,
> > the constraints are deterministic, and the objective of the
> > determinsitic derivation is to find the filter that best fits some
> > criterion, given the reference signal one searches for.
>
> Maybe I've understood you, but the derivation in Ch.5 of Digital Comms
> (Proakis) uses a continuous-time model with AWGN, and uses the
> Cauchy-Schwarz inequality to show that the MF maximises the SNR.

If so, there seems to be a case of terminology mix-up.
I never specialized in comms, so I don't know what the
convention is there, but matched filters are derived
without noise terms in texts on general DSP. As Eric
suggested, this might be a case of imprecise terminology.

Rune