Signal synchronization in the presence of noise

Hello,

I am looking for ideas to synchronize an incoming signal with a template
signal. 

I also want to detect if the incoming signal is valid or not. For this, I
cross-correlate a small length of the input signal with the template. This
seems to work fine. 

How can I then find out a synchronization point between the noisy input and
the template? I have been doing this in MATLAB. My initial idea was to find
the point of maximum correlation and use this as the sync point. But this
does not seem to work for noisy signals. 

If it is important, my template size is around 6000 sample points. 

I appreciate any comments and suggestions you may have about this. 

Thanks much. 

On 19 apr, 13:58, "electrical_storm"
<gauripatil24@n_o_s_p_a_m.gmail.com> wrote:
> Hello,
>
> I am looking for ideas to synchronize an incoming signal with a template
> signal.
>
> I also want to detect if the incoming signal is valid or not. For this, I
> cross-correlate a small length of the input signal with the template. This
> seems to work fine.
>
> How can I then find out a synchronization point between the noisy input and
> the template? I have been doing this in MATLAB. My initial idea was to find
> the point of maximum correlation and use this as the sync point. But this
> does not seem to work for noisy signals.

That's the way to do it. But the reliability of the peak depends
on signal design. If you have a well-designed sync sequence you
will get a nice, narrow peak even in low SNRs. Correlate the whole
sync template with your signal, and see if the match is better.
You need the *whole* template, and the max peak only occurs if the
signal contains the the *whole* sync sequence.

Rune

Rune Allnor wrote:

> You need the *whole* template, and the max peak only occurs if the
> signal contains the the *whole* sync sequence.

When you geophysicists design the sonde signals, what properties of the 
signal do you have in mind? I.e. what is the purpose of the waveforms 
commonly referred as "T-power", "dB/Octave", "dB/Hz" ? Are those 
intended for highly dispersive media, and if so, what kind of receive 
processing do you need ?

Vladimir Vassilevsky
DSP and Mixed Signal Design Consultant
http://www.abvolt.com

Rune Allnor wrote:
> On 19 apr, 13:58, "electrical_storm"
> <gauripatil24@n_o_s_p_a_m.gmail.com> wrote:
>> Hello,
>>
>> I am looking for ideas to synchronize an incoming signal with a template
>> signal.
>>
>> I also want to detect if the incoming signal is valid or not. For this, I
>> cross-correlate a small length of the input signal with the template. This
>> seems to work fine.
>>
>> How can I then find out a synchronization point between the noisy input and
>> the template? I have been doing this in MATLAB. My initial idea was to find
>> the point of maximum correlation and use this as the sync point. But this
>> does not seem to work for noisy signals.
> 
> That's the way to do it. But the reliability of the peak depends
> on signal design. If you have a well-designed sync sequence you
> will get a nice, narrow peak even in low SNRs. Correlate the whole
> sync template with your signal, and see if the match is better.
> You need the *whole* template, and the max peak only occurs if the
> signal contains the the *whole* sync sequence.

Communications-type signals these days will often have a sync sequence 
as Rune describes, but will also have some sort of error correction 
coding -- either forward or backward, or both -- built in.  Generally 
the last word in whether you're really synchronized comes from getting 
all the right CRC checks on your data.

-- 
Tim Wescott
Control system and signal processing consulting
www.wescottdesign.com

On 19 apr, 15:42, Vladimir Vassilevsky <nos...@nowhere.com> wrote:
> Rune Allnor wrote:
> > You need the *whole* template, and the max peak only occurs if the
> > signal contains the the *whole* sync sequence.
>
> When you geophysicists design the sonde signals, what properties of the
> signal do you have in mind? I.e. what is the purpose of the waveforms
> commonly referred as "T-power", "dB/Octave", "dB/Hz" ? Are those
> intended for highly dispersive media, and if so, what kind of receive
> processing do you need ?

*Geophycisists* don't design signals, as marine geophysical signal
sources are 'tamed' explosives - air guns: A metal container that
is filled with compressed air. The air is quickly released into the
water, producing an air bubble. This oscillating air bubble is the
source of the geopysical signal.

It's very hard to control that kind of source - one can certainly
not design any sophisticated signal. Consistency between shots
(the rule-of-thumb is one shot every 4-10 seconds for weeks on end)
and some directivity, using source arrays, are hard enough to
achieve.

In *sonar*, on the other hand, the limiting factors are

1) Available pressure amplitude. Too much pressure, and the water
   cavitates.
2) Bandiwdth. Water is a messy medium, so too large bandwidth, and
   the properties of the water change too much over the bandwidth.

One common solution is to use moderate amplitude (i.e. the water's
acoustic properties stay linear), moderate bandwidth (dispersive
effects are negligeable) and instead use e.g. FM sweep pulses that
last a long time.

Designing signals is an exercise in utilizing the Time-Bandwidth
product, knowing and understanding the constraints and limit factors,
and coming up with useful trade-offs.

Rune

Rune Allnor wrote:
> On 19 apr, 15:42, Vladimir Vassilevsky <nos...@nowhere.com> wrote:
> 
>>Rune Allnor wrote:
>>
>>>You need the *whole* template, and the max peak only occurs if the
>>>signal contains the the *whole* sync sequence.
>>
>>When you geophysicists design the sonde signals, what properties of the
>>signal do you have in mind? I.e. what is the purpose of the waveforms
>>commonly referred as "T-power", "dB/Octave", "dB/Hz" ? Are those
>>intended for highly dispersive media, and if so, what kind of receive
>>processing do you need ?
> 
> 
> *Geophycisists* don't design signals, as marine geophysical signal
> sources are 'tamed' explosives - air guns: A metal container that
> is filled with compressed air. The air is quickly released into the
> water, producing an air bubble. This oscillating air bubble is the
> source of the geopysical signal.
> 
> It's very hard to control that kind of source - one can certainly
> not design any sophisticated signal. Consistency between shots
> (the rule-of-thumb is one shot every 4-10 seconds for weeks on end)
> and some directivity, using source arrays, are hard enough to
> achieve.
> 
> In *sonar*, on the other hand, the limiting factors are
> 
> 1) Available pressure amplitude. Too much pressure, and the water
>    cavitates.
> 2) Bandiwdth. Water is a messy medium, so too large bandwidth, and
>    the properties of the water change too much over the bandwidth.
> 
> One common solution is to use moderate amplitude (i.e. the water's
> acoustic properties stay linear), moderate bandwidth (dispersive
> effects are negligeable) and instead use e.g. FM sweep pulses that
> last a long time.
> 
> Designing signals is an exercise in utilizing the Time-Bandwidth
> product, knowing and understanding the constraints and limit factors,
> and coming up with useful trade-offs.

I was wondering what kind of black magic is involved in the above 
mentioned nonlinear sweep waveforms; and if plain simple convolution is 
  how they are processed. I guess geophysicists try different waveforms 
to resolve sidelobes, dispersion and valid reflections. Fortunately, 
Doppler is irrelevant in geophysics (?). However I don't know what is 
the exact reasoning for each of those sweeps  ("T-power", "dB/Oct", 
"dB/Hz").


Vladimir Vassilevsky
DSP and Mixed Signal Design Consultant
http://www.abvolt.com

On 19 apr, 22:02, Vladimir Vassilevsky <nos...@nowhere.com> wrote:
> Rune Allnor wrote:
> > On 19 apr, 15:42, Vladimir Vassilevsky <nos...@nowhere.com> wrote:
>
> >>Rune Allnor wrote:
>
> >>>You need the *whole* template, and the max peak only occurs if the
> >>>signal contains the the *whole* sync sequence.
>
> >>When you geophysicists design the sonde signals, what properties of the
> >>signal do you have in mind? I.e. what is the purpose of the waveforms
> >>commonly referred as "T-power", "dB/Octave", "dB/Hz" ? Are those
> >>intended for highly dispersive media, and if so, what kind of receive
> >>processing do you need ?
>
> > *Geophycisists* don't design signals, as marine geophysical signal
> > sources are 'tamed' explosives - air guns: A metal container that
> > is filled with compressed air. The air is quickly released into the
> > water, producing an air bubble. This oscillating air bubble is the
> > source of the geopysical signal.
>
> > It's very hard to control that kind of source - one can certainly
> > not design any sophisticated signal. Consistency between shots
> > (the rule-of-thumb is one shot every 4-10 seconds for weeks on end)
> > and some directivity, using source arrays, are hard enough to
> > achieve.
>
> > In *sonar*, on the other hand, the limiting factors are
>
> > 1) Available pressure amplitude. Too much pressure, and the water
> > &#4294967295; &#4294967295;cavitates.
> > 2) Bandiwdth. Water is a messy medium, so too large bandwidth, and
> > &#4294967295; &#4294967295;the properties of the water change too much over the bandwidth.
>
> > One common solution is to use moderate amplitude (i.e. the water's
> > acoustic properties stay linear), moderate bandwidth (dispersive
> > effects are negligeable) and instead use e.g. FM sweep pulses that
> > last a long time.
>
> > Designing signals is an exercise in utilizing the Time-Bandwidth
> > product, knowing and understanding the constraints and limit factors,
> > and coming up with useful trade-offs.
>
> I was wondering what kind of black magic is involved in the above
> mentioned nonlinear sweep waveforms; and if plain simple convolution is
> &#4294967295; how they are processed.

Matched filters is the usual method. An FM frequency sweep signal
is fairly unique as well as simple to generate.

> I guess geophysicists try different waveforms
> to resolve sidelobes, dispersion and valid reflections.

No. The geophys processing is labour intensive, where human
operators extensively interact with, and manipulate, the
processing chain.

When people talk about 'seismic interpretation' they mean
exactly that - interpretation.

> Fortunately,
> Doppler is irrelevant in geophysics (?).

In seismics, yes.

> However I don't know what is
> the exact reasoning for each of those sweeps &#4294967295;("T-power", "dB/Oct",
> "dB/Hz").

I don't recognize those terms off the top of my head, except
the latter two might indicate absorption factors as function
of bandwidth.

Rune

Rune Allnor wrote:

> On 19 apr, 22:02, Vladimir Vassilevsky <nos...@nowhere.com> wrote:
> 

>>However I don't know what is
>>the exact reasoning for each of those sweeps  ("T-power", "dB/Oct",
>>"dB/Hz").
> 
> 
> I don't recognize those terms off the top of my head, except
> the latter two might indicate absorption factors as function
> of bandwidth.

Aha! Thank you for the clue. Looks like they are trying to keep SNR vs 
frequency while having sharp autocorrelation.

Vladimir Vassilevsky
DSP and Mixed Signal Design Consultant
http://www.abvolt.com

On 19 apr, 23:36, Vladimir Vassilevsky <nos...@nowhere.com> wrote:
> Rune Allnor wrote:
> > On 19 apr, 22:02, Vladimir Vassilevsky <nos...@nowhere.com> wrote:
>
> >>However I don't know what is
> >>the exact reasoning for each of those sweeps &#4294967295;("T-power", "dB/Oct",
> >>"dB/Hz").
>
> > I don't recognize those terms off the top of my head, except
> > the latter two might indicate absorption factors as function
> > of bandwidth.
>
> Aha! Thank you for the clue. Looks like they are trying to keep SNR vs
> frequency while having sharp autocorrelation.

Coming to think of it, there might be the question of Doppler
processing. In water, the sound speed c is about 1500 m/s, so
Doppler shifts can become quite large with small velocities
(compared to EM propagation). A source speed of 15 m/s (~30 kts)
is about 0.01*c.

In such circumstances estimating the Doppler shift might be
important, so it is common to use a parbolic FM sweep in the
sonar pulse. That way the extreme frequency, at the apex of the
parabola, is easily estimated from the recieved signal, so
one can easily compute the Doppler shift. If one used a naive
linear sweep one would not necessarily detect the fact if the
upper or lower frequencies were attenuated, and thus end up
with poor Doppler shift estimates.

Rune

Thanks for your comments. 

My problem is slightly more complicated because the input signal has a
different sampling rate than the what the template was recorded at. So
there is a (strong) likelihood that major narrow peaks or valleys that are
present in the template might be lost in the input signal. Besides the
amplitude, the width of peaks/valleys may also change in the input signal.
In such a scenario, correlation does not give the right sync point. 

I applied an amateurish method of synchronization by checking for the
'relatively' widest peak/valley in the first few thousand samples of the
input signal and synchronizing that point with the widest peak/valley in
the template. 

I am not sure if this will work at very low sampling rates of the input
signal. Does anyone have better and more robust ways to go about this
problem?

Thanks. 

>On 19 apr, 23:36, Vladimir Vassilevsky <nos...@nowhere.com> wrote:
>> Rune Allnor wrote:
>> > On 19 apr, 22:02, Vladimir Vassilevsky <nos...@nowhere.com> wrote:
>>
>> >>However I don't know what is
>> >>the exact reasoning for each of those sweeps =A0("T-power", "dB/Oct",
>> >>"dB/Hz").
>>
>> > I don't recognize those terms off the top of my head, except
>> > the latter two might indicate absorption factors as function
>> > of bandwidth.
>>
>> Aha! Thank you for the clue. Looks like they are trying to keep SNR vs
>> frequency while having sharp autocorrelation.
>
>Coming to think of it, there might be the question of Doppler
>processing. In water, the sound speed c is about 1500 m/s, so
>Doppler shifts can become quite large with small velocities
>(compared to EM propagation). A source speed of 15 m/s (~30 kts)
>is about 0.01*c.
>
>In such circumstances estimating the Doppler shift might be
>important, so it is common to use a parbolic FM sweep in the
>sonar pulse. That way the extreme frequency, at the apex of the
>parabola, is easily estimated from the recieved signal, so
>one can easily compute the Doppler shift. If one used a naive
>linear sweep one would not necessarily detect the fact if the
>upper or lower frequencies were attenuated, and thus end up
>with poor Doppler shift estimates.
>
>Rune
>