Optimal detection of 4-level baseband after pulse shaping

Hi all,

I have been surprised while researching this topic, at the lack of
clear information concerning the implementation of optimal
detection/matched filtering in the general case - that is, when the
baseband signal is not a staircase-like function. Let me explain...

We are transmitting 4 level symbols over a wireless channel, (FM
modulation is used in the air, but the DSP sees demodulated 4-level
baseband so this is not important). To reduce adjacent channel power we
apply pulse shaping to the baseband signal at the transmitter, via
either a
gaussian or raised cosine filter (yet to be decided).

What I have been trying to find is the optimal way to
detect each 4-level symbol, taking into account the fact that pulse
shaping has been applied.

If pulse shaping was not applied then the 4-level baseband function
would be staircase-like, and optimal detection is done via a matched
filter which is essentially an integrate and dump. This is basic
receiver theory and can be found all over the net.

However after pulse shaping the 4-level baseband function becomes
'rounded', so I would expect the matched filter is no longer a simple
integrate and dump. What is the optimal detection method now?

Many days of research has turned up nothing clear on this topic, but
surely a few people in this newsgroup must have solved this problem
before.

Regards,
Nicholas Pelly

Integrate-and-dump is the matched filter corresponding only to a rectangular
transmit pulse shape. Other pulse shapes each have a different matched 
filter.
Matched filter means that the receive filter is matched to the transmit 
pulse
shape (or that shape modified by some known channel distortion).

In general, a matched filter is realized as the complex-conjugate of the
time-reversed transmit impulse response. This is similar to saying that
the matched filter correlates the received response with the transmit pulse
shape response.

    -- Tom



<npelly@gmail.com> wrote in message 
news:1103676249.859044.110270@c13g2000cwb.googlegroups.com...
> Hi all,
>
> I have been surprised while researching this topic, at the lack of
> clear information concerning the implementation of optimal
> detection/matched filtering in the general case - that is, when the
> baseband signal is not a staircase-like function. Let me explain...
>
> We are transmitting 4 level symbols over a wireless channel, (FM
> modulation is used in the air, but the DSP sees demodulated 4-level
> baseband so this is not important). To reduce adjacent channel power we
> apply pulse shaping to the baseband signal at the transmitter, via
> either a
> gaussian or raised cosine filter (yet to be decided).
>
> What I have been trying to find is the optimal way to
> detect each 4-level symbol, taking into account the fact that pulse
> shaping has been applied.
>
> If pulse shaping was not applied then the 4-level baseband function
> would be staircase-like, and optimal detection is done via a matched
> filter which is essentially an integrate and dump. This is basic
> receiver theory and can be found all over the net.
>
> However after pulse shaping the 4-level baseband function becomes
> 'rounded', so I would expect the matched filter is no longer a simple
> integrate and dump. What is the optimal detection method now?
>
> Many days of research has turned up nothing clear on this topic, but
> surely a few people in this newsgroup must have solved this problem
> before.
>
> Regards,
> Nicholas Pelly
> 

TOM wrote:
> Integrate-and-dump is the matched filter corresponding only to a
rectangular
> transmit pulse shape. Other pulse shapes each have a different
matched
> filter.
> Matched filter means that the receive filter is matched to the
transmit
> pulse
> shape (or that shape modified by some known channel distortion).
>
> In general, a matched filter is realized as the complex-conjugate of
the
> time-reversed transmit impulse response. This is similar to saying
that
> the matched filter correlates the received response with the transmit
pulse
> shape response.
>
>     -- Tom
>
>
>
> <npelly@gmail.com> wrote in message
> news:1103676249.859044.110270@c13g2000cwb.googlegroups.com...
> > Hi all,
> >
> > I have been surprised while researching this topic, at the lack of
> > clear information concerning the implementation of optimal
> > detection/matched filtering in the general case - that is, when the
> > baseband signal is not a staircase-like function. Let me explain...
> >
> > We are transmitting 4 level symbols over a wireless channel, (FM
> > modulation is used in the air, but the DSP sees demodulated 4-level
> > baseband so this is not important). To reduce adjacent channel
power we
> > apply pulse shaping to the baseband signal at the transmitter, via
> > either a
> > gaussian or raised cosine filter (yet to be decided).
> >
> > What I have been trying to find is the optimal way to
> > detect each 4-level symbol, taking into account the fact that pulse
> > shaping has been applied.
> >
> > If pulse shaping was not applied then the 4-level baseband function
> > would be staircase-like, and optimal detection is done via a
matched
> > filter which is essentially an integrate and dump. This is basic
> > receiver theory and can be found all over the net.
> >
> > However after pulse shaping the 4-level baseband function becomes
> > 'rounded', so I would expect the matched filter is no longer a
simple
> > integrate and dump. What is the optimal detection method now?
> >
> > Many days of research has turned up nothing clear on this topic,
but
> > surely a few people in this newsgroup must have solved this problem
> > before.
> >
> > Regards,
> > Nicholas Pelly
> >

Remember, the time-reversed, conjugated pulse shape is a matched filter
only in the case of AWGN. At the output of an FM discriminator the
noise
has a parabolic spectrum. 

John

Thanks Tom and John,

It sounds like producing such an optimal detector is non-trivial....

1) Assume we use a matched filter as Tom suggests.... the question is
matched to what exactly?? Each pulse that arrives is slightly different
due to ISI. Not to mention that with 4 level there are TWO distinct
pulse shapes, one for +1/-1 and one for +3/-3.

2) Is a matched filter the optimal detection method anyway - as Tom
points out, the noise is not AWGN due to FM modulation.

These seem like quite major issues, but there must be an optimal
solution. Currently we are running a hack which samples at the
mid-point of each  symbol and makes a best guess from that single
sample, but to get this product to operate at the minimum SNR we want
the best detection scheme possible. The ADC is sampling 6 times per
symbol so surely we can use all 6 samples to make a better guess than
the 1 sample we are using at the moment. I guess the question is: what
weighting do we apply to each sample?

This question must have been solved countless times in the past! Maybe
someone can recommend a good book, implementation example (simulink
would be excellent), or other resource that could help me out?
Regards,
Nick

On 21 Dec 2004 23:47:27 -0800, npelly@gmail.com wrote:

>Thanks Tom and John,
>
>It sounds like producing such an optimal detector is non-trivial....
>
>1) Assume we use a matched filter as Tom suggests.... the question is
>matched to what exactly?? Each pulse that arrives is slightly different
>due to ISI. Not to mention that with 4 level there are TWO distinct
>pulse shapes, one for +1/-1 and one for +3/-3.
>
>2) Is a matched filter the optimal detection method anyway - as Tom
>points out, the noise is not AWGN due to FM modulation.
>
>These seem like quite major issues, but there must be an optimal
>solution. Currently we are running a hack which samples at the
>mid-point of each  symbol and makes a best guess from that single
>sample, but to get this product to operate at the minimum SNR we want
>the best detection scheme possible. The ADC is sampling 6 times per
>symbol so surely we can use all 6 samples to make a better guess than
>the 1 sample we are using at the moment. I guess the question is: what
>weighting do we apply to each sample?
>
>This question must have been solved countless times in the past! Maybe
>someone can recommend a good book, implementation example (simulink
>would be excellent), or other resource that could help me out?
>Regards,
>Nick

Maybe I'm not following the question completely, but this still sounds
quite fundamental.   If you're worried about reducing ISI then this is
a well-known issue that has been discussed here many times.  Many
digital communications texts deal with matched filtering and arguably
the most common example is the Root Raised Cosine filter (since it is
probably the most commonly used).

An RRC should still match for ISI if the FM mod-demod are properly
engineered.  The non-white noise shouldn't contribute to ISI unless it
were somehow correlated with the signal.   It's still a random
process.

Am I missing something here?   This sounds like it should be
reasonably straightforward.


Eric Jacobsen
Minister of Algorithms, Intel Corp.
My opinions may not be Intel's opinions.
http://www.ericjacobsen.org

Eric Jacobsen wrote:
> On 21 Dec 2004 23:47:27 -0800, npelly@gmail.com wrote:
>
> >Thanks Tom and John,
> >
> >It sounds like producing such an optimal detector is non-trivial....
> >
> >1) Assume we use a matched filter as Tom suggests.... the question
is
> >matched to what exactly?? Each pulse that arrives is slightly
different
> >due to ISI. Not to mention that with 4 level there are TWO distinct
> >pulse shapes, one for +1/-1 and one for +3/-3.
> >
> >2) Is a matched filter the optimal detection method anyway - as Tom
> >points out, the noise is not AWGN due to FM modulation.
> >
> >These seem like quite major issues, but there must be an optimal
> >solution. Currently we are running a hack which samples at the
> >mid-point of each  symbol and makes a best guess from that single
> >sample, but to get this product to operate at the minimum SNR we
want
> >the best detection scheme possible. The ADC is sampling 6 times per
> >symbol so surely we can use all 6 samples to make a better guess
than
> >the 1 sample we are using at the moment. I guess the question is:
what
> >weighting do we apply to each sample?
> >
> >This question must have been solved countless times in the past!
Maybe
> >someone can recommend a good book, implementation example (simulink
> >would be excellent), or other resource that could help me out?
> >Regards,
> >Nick
>
> Maybe I'm not following the question completely, but this still
sounds
> quite fundamental.   If you're worried about reducing ISI then this
is
> a well-known issue that has been discussed here many times.  Many
> digital communications texts deal with matched filtering and arguably
> the most common example is the Root Raised Cosine filter (since it is
> probably the most commonly used).
>
> An RRC should still match for ISI if the FM mod-demod are properly
> engineered.  The non-white noise shouldn't contribute to ISI unless
it
> were somehow correlated with the signal.   It's still a random
> process.
>
> Am I missing something here?   This sounds like it should be
> reasonably straightforward.
>
>
> Eric Jacobsen
> Minister of Algorithms, Intel Corp.
> My opinions may not be Intel's opinions.
> http://www.ericjacobsen.org

Eric,

The typical matched filter derivation (the one that invokes the
Cauchy-Schwarz
inequality) uses a white noise assumption to show how to maximize the
SNR at
the sampling instant. ISI is not considered.

I agree that zero ISI would be achieved through the FM modulator and
demodulator
with the RRC filters, but isn't the white noise assumption invalid
after a frequency
discriminator?

If a whitening filter (deemphasis) is used, wouldn't that introduce
ISI?

John

Thanks Tom, John and Eric,

And the other issue I have is what pulse shape should be used to create
the matched filter.

Firstly, a +3/-3 symbol and a +1/-1 symbol will have distinctly
different shapes.

Not to mention that in the presence of ISI the pulse shapes are
dependant on adjacent symbols.

Nick

On 23 Dec 2004 16:27:15 -0800, npelly@gmail.com wrote:

>Thanks Tom, John and Eric,
>
>And the other issue I have is what pulse shape should be used to create
>the matched filter.
>
>Firstly, a +3/-3 symbol and a +1/-1 symbol will have distinctly
>different shapes.
>
>Not to mention that in the presence of ISI the pulse shapes are
>dependant on adjacent symbols.
>
>Nick

Nick,

Matching filters take care of the this problem.   4-PAM can be treated
the same as a single channel of a 16-QAM signal, and RRC filters work
great there to eliminate ISI.   The idea is that the matched response
is zero at the centers of the adjacent symbols, so it doesn't matter
what their levels were at those points.

There are essentially an infinite number of pulse shapes that can be
used to match for zero-ISI, the only issue to make sure that you use a
pair that match with each other.  ;)    If you know how much rolloff
(i.e., excess bandwidth) you can tolerate that might help you decide
exactly what to use.   Is there a reason not to consider RRC filters?


Eric Jacobsen
Minister of Algorithms, Intel Corp.
My opinions may not be Intel's opinions.
http://www.ericjacobsen.org

On 23 Dec 2004 05:42:22 -0800, "john" <johns@xetron.com> wrote:

>
>Eric Jacobsen wrote:
>> On 21 Dec 2004 23:47:27 -0800, npelly@gmail.com wrote:
>>
>> >Thanks Tom and John,
>> >
>> >It sounds like producing such an optimal detector is non-trivial....
>> >
>> >1) Assume we use a matched filter as Tom suggests.... the question
>is
>> >matched to what exactly?? Each pulse that arrives is slightly
>different
>> >due to ISI. Not to mention that with 4 level there are TWO distinct
>> >pulse shapes, one for +1/-1 and one for +3/-3.
>> >
>> >2) Is a matched filter the optimal detection method anyway - as Tom
>> >points out, the noise is not AWGN due to FM modulation.
>> >
>> >These seem like quite major issues, but there must be an optimal
>> >solution. Currently we are running a hack which samples at the
>> >mid-point of each  symbol and makes a best guess from that single
>> >sample, but to get this product to operate at the minimum SNR we
>want
>> >the best detection scheme possible. The ADC is sampling 6 times per
>> >symbol so surely we can use all 6 samples to make a better guess
>than
>> >the 1 sample we are using at the moment. I guess the question is:
>what
>> >weighting do we apply to each sample?
>> >
>> >This question must have been solved countless times in the past!
>Maybe
>> >someone can recommend a good book, implementation example (simulink
>> >would be excellent), or other resource that could help me out?
>> >Regards,
>> >Nick
>>
>> Maybe I'm not following the question completely, but this still
>sounds
>> quite fundamental.   If you're worried about reducing ISI then this
>is
>> a well-known issue that has been discussed here many times.  Many
>> digital communications texts deal with matched filtering and arguably
>> the most common example is the Root Raised Cosine filter (since it is
>> probably the most commonly used).
>>
>> An RRC should still match for ISI if the FM mod-demod are properly
>> engineered.  The non-white noise shouldn't contribute to ISI unless
>it
>> were somehow correlated with the signal.   It's still a random
>> process.
>>
>> Am I missing something here?   This sounds like it should be
>> reasonably straightforward.
>>
>>
>> Eric Jacobsen
>> Minister of Algorithms, Intel Corp.
>> My opinions may not be Intel's opinions.
>> http://www.ericjacobsen.org
>
>Eric,
>
>The typical matched filter derivation (the one that invokes the
>Cauchy-Schwarz
>inequality) uses a white noise assumption to show how to maximize the
>SNR at
>the sampling instant. ISI is not considered.
>
>I agree that zero ISI would be achieved through the FM modulator and
>demodulator
>with the RRC filters, but isn't the white noise assumption invalid
>after a frequency
>discriminator?
>
>If a whitening filter (deemphasis) is used, wouldn't that introduce
>ISI?
>
>John

Yes, the noise spectrum makes a difference, but I'm still not sure
what the OP is trying to match.   If it's just ISI, then that should
be pretty straightforward, I'd think, as long as the FM mod-demod are
matched as well to keep the distortion under control.

I'm pretty sure I still don't fully understand the problem.  :(


Eric Jacobsen
Minister of Algorithms, Intel Corp.
My opinions may not be Intel's opinions.
http://www.ericjacobsen.org

In article <1103701647.686315.120750@c13g2000cwb.googlegroups.com>, npelly@gmail.com wrote:

>2) Is a matched filter the optimal detection method anyway - as Tom
>points out, the noise is not AWGN due to FM modulation.
>
There are equivalent filters for non-white noise.  See Helstrom's book on 
detection theory for example.  It's just that the matched filter in AWGN is 
sooo simple compared to solving the integral equations generated by having 
non-white noise..