Rune Allnor wrote:> On 24 Des, 23:27, Oli Charlesworth <ca...@olifilth.co.uk> wrote: >> maury wrote: >>> One is SIGNAL based, and the other is MODEL based. >> I appreciate that the optimisation criteria may have stemmed from >> different sources, but I guess what I'm having most trouble reconciling >> is that MSE and SNR aren't equivalent (even if we constrain S to be fixed). > > The main difference is that MSE is the deviation between the > desired and actual response.But surely, additive noise ("v" in my notation in the OP) is the cause of this deviation? Of course, this has been shaped by our receive process, but this is true in the matched-filter case as well.> SNR includes an additive noise > term as well, which is missing in the MSE.

# Comparing matched and Wiener filters

Started by ●December 24, 2008

Reply by ●December 24, 20082008-12-24

Reply by ●December 24, 20082008-12-24

>I'm not sure there's much of a distinction. The purpose of detection is >to determine some property of the signal, either presence or absence >(OOK), or some more complicated modulation scheme (or something else >entirely in e.g. radar applications). Therefore, the requirement is to >*estimate* some parameter behind the signal.Although they are not the same in general, there is no distinction between detection theory and estimation theory when the vector (to be estimated) assumes discrete values in a finite-dimensional space [1, pg.2].>> If you consider the outputs of the two for a binary communicationsignal,>> the output of the MF looks like triangles with peak amplitudes at the >> sampling times; that is, the MF maximizes the SNR only at thesesampling>> instances. On the other hand, the output of a Wiener filter, if youreally>> wanted to use it, would look more like the transmitted binarysequence;>> this could not be the last stage for a detector, though. > >I see what you're saying. However, isn't this just an artifact of the >"filter across all time and sample" viewpoint of matched-filtering, as >opposed to the "integrate-and-dump" viewpoint, where we only calculate >the output for the sampling points (i.e. correlation, equivalent to the >vector formation). >-- >OliNot really. You are interested in comparing MF vs Wiener, and the outputs of these two are significantly different. MF does not maximize SNR in the same sense as Wiener. If you define the SNR like you define MSE, then SNR_Wiener > SNR_MF. If you are thinking of appending an integrator following the Wiener filter, that's another story. What is your application? Perhaps the answer to this question would give us some perspective as to why you are trying to compete these two against each other. Emre [1] L.L. Scharf, Statistical Signal Processing: Detection, Estimation, and Time Series Analysis (New York: Addison-Wesley Publishing Co., 1990).

Reply by ●December 24, 20082008-12-24

> If you are thinking of appending an integrator following the Wienerfilter, that's another story. In fact, under the Gaussian noise assumption, the story is rather short: Matched filter is equivalent to Wiener filter followed by an integrator. Emre

Reply by ●December 24, 20082008-12-24

emre wrote:>>> If you consider the outputs of the two for a binary communication > signal, >>> the output of the MF looks like triangles with peak amplitudes at the >>> sampling times; that is, the MF maximizes the SNR only at these > sampling >>> instances. On the other hand, the output of a Wiener filter, if you > really >>> wanted to use it, would look more like the transmitted binary > sequence; >>> this could not be the last stage for a detector, though. >> I see what you're saying. However, isn't this just an artifact of the >> "filter across all time and sample" viewpoint of matched-filtering, as >> opposed to the "integrate-and-dump" viewpoint, where we only calculate >> the output for the sampling points (i.e. correlation, equivalent to the >> vector formation). > > Not really. You are interested in comparing MF vs Wiener, and the outputs > of these two are significantly different. MF does not maximize SNR in the > same sense as Wiener. If you define the SNR like you define MSE, then > SNR_Wiener > SNR_MF.Perhaps this is the essence of what I need to understand! My current understanding is that: MSE = E|x_est - x|^2 SNR = E|x'|^2 / E|v'|^2 where x' is the post-filter signal component, and v' is the post-filter noise component of x_est (both scalar in my original example), i.e. x_est = x' + v'. If we normalise our matched filter (g = h*/||h||^2), then we can be sure that x' = x (the original data). This leaves v' = x_est - x' = x_est - x. Therefore: SNR = S / E|x_est - x|^2 and equivalent to the MSE expression above (they're just inversely-proportional). How else would we define these quantities?> If you are thinking of appending an integrator > following the Wiener filter, that's another story.Bad terminology on my part, I think. I was referring to the general form of the colloquial "integrate-and-dump", i.e. weighted integration, i.e. the inner-product with the filter kernel. No second integration/filter implied!> What is your application? Perhaps the answer to this question would give > us some perspective as to why you are trying to compete these two against > each other.I have no application in mind, I'm just trying to piece some more of the puzzle together! Up until now, I've naively assumed that the matched filter is just a special case of the Wiener/LMMSE formulation, but when I thought about it and then tried to prove it, I got stuck. Incidentally, it's possible we're talking at cross purposes here due to the confusion between the two viewpoints of the operations; one as a continuous process operating in the time domain, the other as a "one-off" vector operation (as expressed in my OP). I contend that these two viewpoints are equivalent if we only look at the time-domain viewpoint at the sampling instants (i.e. the points of interest), and treat all other points merely as an implementation artifact. If you disagree, please correct me!

Reply by ●December 24, 20082008-12-24

>Incidentally, it's possible we're talking at cross purposes here due to >the confusion between the two viewpoints of the operations; one as a >continuous process operating in the time domain, the other as a >"one-off" vector operation (as expressed in my OP). I contend that >these two viewpoints are equivalent if we only look at the time-domain >viewpoint at the sampling instants (i.e. the points of interest), and >treat all other points merely as an implementation artifact. If you >disagree, please correct me!Right, when talking about integration etc., I was referring to the use of the Matched filter as in binary communication: http://en.wikipedia.org/wiki/Matched_filter#Example_of_matched_filter_in_digital_communications>Perhaps this is the essence of what I need to understand! My current >understanding is that: > > MSE = E|x_est - x|^2 > SNR = E|x'|^2 / E|v'|^2 > >where x' is the post-filter signal component, and v' is the post-filter >noise component of x_est (both scalar in my original example), i.e. >x_est = x' + v'. > >If we normalise our matched filter (g = h*/||h||^2), then we can be sure >that x' = x (the original data). This leaves v' = x_est - x' = x_est - >x. Therefore: > > SNR = S / E|x_est - x|^2 > >and equivalent to the MSE expression above (they're just >inversely-proportional). > >How else would we define these quantities?At last I understand your question (in fact, looking back, your OP was very well-posed.) In fact, the answer to your question is simpler than it looks. SNR is defined as the ratio between the energy of the signal over energy of the noise allowed in the estimator, not the estimation error. So the right definition of SNR would be: SNR = E[ (g h x)^2 ] / E[ (g v)^2 ] which equals (S N/sigma^2) for the matched filter, where N is the dimension of g and v, and sigma^2 is the variance (like you defined before.) Emre

Reply by ●December 25, 20082008-12-25

On 24 Des, 23:51, Oli Charlesworth <ca...@olifilth.co.uk> wrote:> Rune Allnor wrote: > > On 24 Des, 23:27, Oli Charlesworth <ca...@olifilth.co.uk> wrote: > >> maury wrote: > >>> One is SIGNAL based, and the other is MODEL based. > >> I appreciate that the optimisation criteria may have stemmed from > >> different sources, but I guess what I'm having most trouble reconciling > >> �is that MSE and SNR aren't equivalent (even if we constrain S to be fixed). > > > The main difference is that MSE is the deviation between the > > desired and actual response. > > But surely, additive noise ("v" in my notation in the OP) is the cause > of this deviation? �Of course, this has been shaped by our receive > process, but this is true in the matched-filter case as well.This is where things become philosophical. The term 'Mean Square Error' points to some deviation of non-descript origin, whereas 'noise' points to some other signal - usually of stochastic nature - than the signal of interest. So the MSE can be due to deterministic factors, whereas the SNR explictly contains a stochastic component. The derivations of the matched filter usually start out without noise included in the model (if you know of a derivation which contains noise at the outset, please let me know). In that case, the constraints are deterministic, and the objective of the determinsitic derivation is to find the filter that best fits some criterion, given the reference signal one searches for. The Wiener filter is derived from a *similar* point of view, but explicitly includes stochastic additive processes, noise, in the model. You still want to find a filter that best fits a purpose given a reference signal, but in the Wiener filter the additive coise causes explicit effects on the result. There are derivations of DSP algorithms that start out with a deterministic signal model (no noise included) that aim to reduce the impact of numerical inaccurcies etc. In my experience, these methods perform very well in the presence of additive noise. The Kumaresan & Tufts version of Prony's method for frequeny estimation is one such method. It totally outperformed MUSIC which explicitly include noise in the signal model, both what accuracy of results and computational efficiency were concerned. Well, the T&K method produced results at least as good as MUSIC at a far less computational cost. Rune

Reply by ●December 25, 20082008-12-25

On Wed, 24 Dec 2008 17:10:56 -0500, Randy Yates <yates@ieee.org> wrote:>Oli Charlesworth <catch@olifilth.co.uk> writes: > >> emre wrote: >>> Hi Oli, >>> >>> The two are designed for different applications, even if their >>> formulations are similar. Matched filter applies to the *detection* of the >>> presence of a known signal, whereas Wiener filter applies to the >>> *estimation* of an unknown random signal with known power spectrum. In >>> either case, the spectrum of the signal is assumed to be known. >> >> I'm not sure there's much of a distinction. The purpose of detection is >> to determine some property of the signal, either presence or absence >> (OOK), or some more complicated modulation scheme (or something else >> entirely in e.g. radar applications). Therefore, the requirement is to >> *estimate* some parameter behind the signal. >> >> >>> If you consider the outputs of the two for a binary communication signal, >>> the output of the MF looks like triangles with peak amplitudes at the >>> sampling times; that is, the MF maximizes the SNR only at these sampling >>> instances. On the other hand, the output of a Wiener filter, if you really >>> wanted to use it, would look more like the transmitted binary sequence; >>> this could not be the last stage for a detector, though. >> >> I see what you're saying. However, isn't this just an artifact of the >> "filter across all time and sample" viewpoint of matched-filtering, as >> opposed to the "integrate-and-dump" viewpoint, where we only calculate >> the output for the sampling points (i.e. correlation, equivalent to the >> vector formation). > >Yeah. A matched filter in digital comms assumes you have timing.I've just been working on a blog article about what "Matched Filter" means in comm. The term gets abused, but you guys have already hit the main points. A matched filter may be used in a more continuous fashion (like in remote sensing, radar or sonar), or it may only be really useful at specific locations in the signal, like in comm. It all boils down to the same thing, but somehow the terminology sometimes gets twisted a bit along the way. I think it's kinda cool that we work in an area that's esoteric enough that definitions are often not clear. It makes for interesting dialogue. Eric Jacobsen Minister of Algorithms Abineau Communications http://www.ericjacobsen.org Blog: http://www.dsprelated.com/blogs-1/hf/Eric_Jacobsen.php

Reply by ●December 25, 20082008-12-25

On Dec 24, 6:12�am, Oliver Charlesworth <ca...@olifilth.co.uk> wrote:> Hi, > > This is something that's been bugging me for the last couple of days. > > With the understanding that both are linear, we define the matched > filter as the one that maximises the output SNR, and the Wiener filter > as the one that minimises the mean square error (MSE). �Superficially, > these definitions sound almost identical. However, even in the simplest > model (estimating a scalar value with independent AWGN), we clearly end > up with different filters: > > MF: � � � � �g = h* > Wiener: � � �g = Sh*(Shh* + sigma^2.I)^-1 > > such that x_est = gy, where y = hx + v (where x is scalar data with S = > E|x|^2, v is AWGN with sigma^2.I = E|v|^2, {y,h,v} are Nx1 vectors, and > * denotes conjugate transpose).i wonder if these results are all that different (even though they look different).> > My question is, what is the intuitive reason (i.e. not just "because the > maths says so") why these don't end up with the same result, or more > specifically, how the Wiener filter can minimise the MSE without > maximising the SNR?i've always thought (and i hadn't done anything in communications since grad school) that the essential difference between the MF and WF, given the same signal description and noise spectra (i think white), was that of gain. to nail down MSE, you have to worry about gain. to nail down S/N, you need not worry about gain. there is a related question regarding LPC that i remember relating this question to. r b-j

Reply by ●December 26, 20082008-12-26

Rune Allnor wrote:> The derivations of the matched filter usually start out without > noise included in the model (if you know of a derivation which > contains noise at the outset, please let me know). In that case, > the constraints are deterministic, and the objective of the > determinsitic derivation is to find the filter that best fits some > criterion, given the reference signal one searches for.Maybe I've understood you, but the derivation in Ch.5 of Digital Comms (Proakis) uses a continuous-time model with AWGN, and uses the Cauchy-Schwarz inequality to show that the MF maximises the SNR. -- Oli

Reply by ●December 26, 20082008-12-26

On 26 Des, 15:11, Oli Charlesworth <ca...@olifilth.co.uk> wrote:> Rune Allnor wrote: > > The derivations of the matched filter usually start out without > > noise included in the model (if you know of a derivation which > > contains noise at the outset, please let me know). In that case, > > the constraints are deterministic, and the objective of the > > determinsitic derivation is to find the filter that best fits some > > criterion, given the reference signal one searches for. > > Maybe I've understood you, but the derivation in Ch.5 of Digital Comms > (Proakis) uses a continuous-time model with AWGN, and uses the > Cauchy-Schwarz inequality to show that the MF maximises the SNR.If so, there seems to be a case of terminology mix-up. I never specialized in comms, so I don't know what the convention is there, but matched filters are derived without noise terms in texts on general DSP. As Eric suggested, this might be a case of imprecise terminology. Rune