bit stuffing in serial protocols

On Wed, 6 Apr 2016 15:18:36 -0700 (PDT), robert bristow-johnson
<rbj@audioimagination.com> wrote:

>On Saturday, April 2, 2016 at 6:53:39 PM UTC-4, Eric Jacobsen wrote:
>> On Fri, 1 Apr 2016 20:30:26 -0700 (PDT), robert bristow-johnson
>> <rbj@audioimagination.com> wrote:
>> 
>> >On Friday, April 1, 2016 at 1:28:43 PM UTC-4, Eric Jacobsen wrote:
>> >> As mentioned, this is very system specific.   Robert, are you thinking
>> >> of a continuous stream system or a bursty system?
>> >
>> >both.  either.  whichever is worst case.  but even if the receiver *thin>ks*=
>> > it bursty, i think it should work with a forever continuous stream.  an>d 
>> >vice-versa.  even with a forever continuous stream, there should be occa>sional
>> >frame syncs, just to keep everyone on the same page.
>> 
>> Not necessarily, because it takes up bandwidth and may do nothing but
>> complicate the system.  A system with a convolutional code that just
>> runs continuously would have to make an exception and suspend decoding
>> every time this annoying sync word showed up. It'd also have to know
>> how to distinguish the sync word from the rest of the bit stream.
>> There's really no benefit to a synch word in that sort of system.
>> 
>> For this reason the frame markers are sometimes done after decoding,
>> where they can be much shorter because they're protected from errors
>> by the FEC, and a binary instead of sampled symbols and therefore
>> simpler to detect.
>> 
>> >> For a continuous system the framing is often done after demodulation,
>> >> i.e., in the decoded bits, and is then independent of the modulation
>> >> and coding (FEC).
>> >
>> >but you don't necessarily have valid decoded bits after demodulation if >the=
>> > synchronization wasn't right to start with.
>> 
>> If your demodulator is worth its salt you'll know whether it is
>> synchronized properly or not before you ever get the data bits, or,
>> better still, you won't be handed any demodulated bits unless they're
>> known to be from a synchronized condition.
>
>but this is begging the question.  or at least *i* am begging the question.
>
>what if synchronization is lost, you have a good idea what the carrier freq>uency is (f_c) and a good idea what the bit rate is (f_d), but you have *no>* idea in what direction the IQ vector is pointing and you want to *both* g>et into synchronization *and* mark the beginning of a data frame with a sin>gle preamble word.  what do you do?  how do you do that?

Again, a good UW pattern provides an autocorrrelation peak that points
in the direction (on the complex plane) of a known phase, and the peak
will occur in time at an epoch that provides a reference for timing
synchronization.    So you get both phase synch and timing synch from
cross-correlating the UW sequence with the received sequence.

If you don't have a UW, you let the phase and timing loops lock on
their own.   This doesn't exclude putting a frame marker in the data.


>> >doesn't there have to be some unambiguous transmission, that if received> wi=
>> >thout error, tells the receiver in no uncertain terms, that we're starti>ng =
>> >a new frame?
>> 
>> Only if framing is required, which it often isn't.   And even if the
>> physical layer has framing, it may be independent from any framing
>> that it also in the data stream.
>
>seems to me that if you want to kill both birds with a single stone, if you> can.  seems to me that if you can transmit a short segment of signal that >will establish exactly when "things" are beginning, that's what you wanna d>o.

For what protocol?   Ethernet?  MPEG? IP?  Or do you want to be
generic and be able to transport whatever comes along?   Why
complicate the PHY layer with something that it doesn't need if you
already have to have more layers of protocol downstream to take
packets apart?

Sometimes protocols provide constant-length packets (like certain MPEG
encapsulations), so that you CAN simultaneously do PHY and packet
synchronization in a streaming link.  Often, though, it's a waste of
time and resources.

Often PHY framing is done to accomodate block FEC, like Turbo Codes or
LDPCs or whatevers, and the FEC block sizes may not have anything to
do with the transported data frame sizes.   Forcing the FEC framing to
be the same size as the data frames often serves no purpose,
complicates the system, and may degrade the FEC performance.

>> 
>> >>   Sometimes there are advantages to synchronizing
>> >> the coding, and in those cases Unique Word is usually used or some
>> >> like a modulated preamble to define a super frame and aid in initial
>> >> acquisition.
>> >>=20
>> >> For bursty systems the demodulator often has to fully synchronize
>> >> frequency, phase, and symbol timing from burst-to-burst, so the
>> >> preamble must contain features that facilitate all of those as well as
>> >> unambiguously mark the start of modulated data.
>> >
>> >that's what i am thinking.  i am thinking of a stream of data already ha>ppe=
>> >ning, and you turn on your receiver that has and *idea* of what f_c and >f_d=
>> > are and will be totally useless until the first frame-sync ("preamble",> th=
>> >at's the word for it, much better than "unique word") that it sees.  but> wi=
>> >th OQPSK, when the receiver is turned on and listening to a solid f_c, i>t d=
>> >oesn't have any idea where the IF vector is pointing.  could be 00, 01, >11,=
>> > or 10.  the only way it can do that is have an unambiguous preamble tha>t c=
>> >an never appear in the data.  and the only way i know of doing that is t>o d=
>> >o bit stuffing to prevent a combination of bits that happen to look like> th=
>> >e preamble from being transmitted inside the data block of the frame.
>> 
>> That's rarely done at the physical layer.   Sometimes bit stuffing is
>> done just above the physical layer, but it is certainly possible to
>> build an OQPSK modem without it.
>> 
>> >> Sadly, the patterns you mention aren't very good for those tasks,
>> >
>> >why?  those two patterns are the only patterns that will detune the carr>ier=
>> > away from its center value to the maximum deviation.  it would seem to >me =
>> >to be the easiest and cleanest to adapt a matched filter to.
>> 
>> Why do you want to move the carrier off center?   I don't see the
>> motivation.
>
>the motivation is that it establishes exactly where the IQ (or "IF") vector> should be pointing at the very end of the preamble.  and if the preamble i>s a "unique word", then it never happens in the data stream.  but the only >way to guarantee it never happens in the data stream is to stuff in a bit (>or maybe two, since this is a form of 4-QAM) when it appears that unique wo>rd is about to be sent out.

If the UW happens in a fixed-period frame sequence (which is usually
the case to accomodate fixe-length FEC blocks), you don't declare
frame lock until you've observed several of them spaced at the proper
distance in the stream.  This provides the appropriate confidence
level that you haven't found a UW in the data sequence.

Bit stuffing at the PHY layer is a Very Bad Idea since it is subject
to bit errors and can therefore seriously screw up bit count integrity
and many other things.   Missing or improperly deleting a stuffed bit
will completely hose the FEC and anything else down stream.   This is
why protocols (like HDLC) that do bit stuffing to isolate frame
markers have to be done outside of the FEC (i.e., after the FEC in the
demodulator), or the first bit error that comes along destroys
civilization.

>> Matched filters aren't normally "adapted", but are pre-set to be
>> compatible with the other end of the link.
>
>yes, i meant a matched filter to be "adapted" or "designed" to light up whe>n the "Unique Word" happens to show it's happy face in the signal.
>
>>   If there is an adaptive
>> filter for equalization (and there often isn't), that is often done
>> separately from the matched filter, so that it only has to deal with
>> channel distortion.   This can be done "blind" from modulated data, or
>> with a unique word during a preamble or framing marker.   Using the
>> patterns you mention can make it difficult to maintain
>> synchronization.
>
>"adapted" was the wrong word.  "designed" is a better word.
>
>> 
>> >> but they are often used for testing.   They're a good way to detect
>> >> distortion like channel balance, phase balance, etc.
>> >
>> >for OQPSK, what is a good preamble?  like what would be a good bit patte>rn?=
>> >  and, without knowledge of phase sync, how does a receiver detect it (a>nd =
>> >get all sync'd up)?  and if you don't do bit stuffing, like SDLC does to> pr=
>> >event a faux preamble from transmission, how do you robustly keep the re>cei=
>> >ver from getting confused?
>> 
>> OQPSK is a bit of an odd duck because there is an interdependence
>> between phase and symbol timing that doesn't exist with QPSK.
>
>but with QPSK (or 4-QAM, i think they're the same) you still need synchroni>zation.  otherwise you don't know what direction the IQ vector is pointing.

The point is just that it's a little harder with OQPSK.  Frankly OQPSK
is a pain to use (IMHO), and should only be used when necessary.
Otherwise, use QPSK as it is far less of a pain (again, IMHO).

>>   That
>> being said, the most efficient preambles tend to be carefully selected
>> patterns (usually a Unique Word of some kind with good autocorrelation
>> properties) that provide good detection (i.e., good autocorrelation),
>> as well as good phase and timing synchronization properties.
>
>i can see that if it wasn't quadrature. if it was BPSK, i could see just w>aiting for the Unique Word to appear and sync up. (and i would think we wou>ld bit-stuff to prevent that Unique Word from ever appearing happenstance i>n the data and causing spurious syncing.)  but BPSK is half the bit rate th>at QPSK is.  with QPSK, it's like you have an FM detector and you have to t>ake the output of the FM detector and integrate it to get PM.

No, just phase lock.   It's very straightforward.   The exact same
demodulator structure can be used for both BPSK and QPSK.   The
carrier phase detector is really the only change needed to recover the
constellation for both, and that's pretty trivial.

In a basic sense, one small additional change makes the same
demodulator capable of recovering OQPSK (but there are a number of
corner-cases that have to be considered).

>  but there is> the initial state of the integrator to set up correctly and that's where t>he frame syncing comes in.  and extreme frequencies that happen in OQPSK wi>th the 00110011 or with 01100110 is unambiguous to an FM detector.
>
>> Designing a good UW for a preamble and the demodulator algorithms to
>> reliably detect and synchronize it is sometimes difficult depending on
>> the expected channel impairments.   Sometimes the UW has to also train
>> the channel equalizer, if there is one.
>
>thanks Eric.  i hope you can continue to tolerate pedantacy from me.  unlik>e the "circular DFT" arguments we have had in the past, i am fully aware th>at i don't really know squat about this.  but, with OQPSK (at the "physical> layer") i think i can construct what it looks like in the frequency domain> when the IQ vector does not move (which would be 00000000, 11111111, 01010>101, or 10101010) and when it moves consistently clockwise (00110011) or co>unter-clockwise (01100110) which is the maximum detuning of the carrier.  (>referring to the constellation at https://en.wikipedia.org/wiki/Phase-shift>_keying#/media/File:Pi-by-O-QPSK_Gray_Coded.svg )  if you don't have a phas>e lock, all you know is frequency.  then, the only preamble that i understa>nd that will be detected unambiguously is one of those two extremes.
>
>do you understand what my point is (and where it's messed up)?

A carefully chosen length-N vector of complex samples with a good
autocorrelation sequence (i.e., a UW) solves carrier phase and timing.
If frequency is only roughly known, transmit two UWs in sequence and
get fine frequency estimation from the difference in phase between
them.   Then you know frequency, phase, and timing at the end of the
second UW.   What else do you need?



---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

On Thursday, April 7, 2016 at 3:07:21 PM UTC-4, Eric Jacobsen wrote:
> On Wed, 6 Apr 2016 15:18:36 -0700 (PDT), robert bristow-johnson
> <rbj@audioimagination.com> wrote:
> 
> >
> >what if synchronization is lost, you have a good idea what the carrier freq=
> >uency is (f_c) and a good idea what the bit rate is (f_d), but you have *no=
> >* idea in what direction the IQ vector is pointing and you want to *both* g=
> >et into synchronization *and* mark the beginning of a data frame with a sin=
> >gle preamble word.  what do you do?  how do you do that?
> 
> Again, a good UW pattern provides an autocorrrelation peak that points
> in the direction (on the complex plane) of a known phase, and the peak
> will occur in time at an epoch that provides a reference for timing
> synchronization.    So you get both phase synch and timing synch from
> cross-correlating the UW sequence with the received sequence.
> 
> If you don't have a UW, you let the phase and timing loops lock on
> their own.   This doesn't exclude putting a frame marker in the data.
> 
> 
...
> 
> Sometimes protocols provide constant-length packets (like certain MPEG
> encapsulations), so that you CAN simultaneously do PHY and packet
> synchronization in a streaming link.  Often, though, it's a waste of
> time and resources.
> 
> Often PHY framing is done to accomodate block FEC, like Turbo Codes or
> LDPCs or whatevers, and the FEC block sizes may not have anything to
> do with the transported data frame sizes.   Forcing the FEC framing to
> be the same size as the data frames often serves no purpose,
> complicates the system, and may degrade the FEC performance.
> 
...
> 
> If the UW happens in a fixed-period frame sequence (which is usually
> the case to accomodate fixe-length FEC blocks), you don't declare
> frame lock until you've observed several of them spaced at the proper
> distance in the stream.  This provides the appropriate confidence
> level that you haven't found a UW in the data sequence.
> 
> Bit stuffing at the PHY layer is a Very Bad Idea since it is subject
> to bit errors and can therefore seriously screw up bit count integrity
> and many other things.   Missing or improperly deleting a stuffed bit
> will completely hose the FEC and anything else down stream.   This is
> why protocols (like HDLC) that do bit stuffing to isolate frame
> markers have to be done outside of the FEC (i.e., after the FEC in the
> demodulator), or the first bit error that comes along destroys
> civilization.
> 
...
> 
> The point is just that it's a little harder with OQPSK.  Frankly OQPSK
> is a pain to use (IMHO), and should only be used when necessary.
> Otherwise, use QPSK as it is far less of a pain (again, IMHO).
> 
> >>   That
> >> being said, the most efficient preambles tend to be carefully selected
> >> patterns (usually a Unique Word of some kind with good autocorrelation
> >> properties) that provide good detection (i.e., good autocorrelation),
> >> as well as good phase and timing synchronization properties.
> >
> >i can see that if it wasn't quadrature. if it was BPSK, i could see just w=
> >aiting for the Unique Word to appear and sync up. (and i would think we wou=
> >ld bit-stuff to prevent that Unique Word from ever appearing happenstance i=
> >n the data and causing spurious syncing.)  but BPSK is half the bit rate th=
> >at QPSK is.  with QPSK, it's like you have an FM detector and you have to t=
> >ake the output of the FM detector and integrate it to get PM.
> 
> No, just phase lock.   It's very straightforward.   The exact same
> demodulator structure can be used for both BPSK and QPSK.   The
> carrier phase detector is really the only change needed to recover the
> constellation for both, and that's pretty trivial.
> 
> In a basic sense, one small additional change makes the same
> demodulator capable of recovering OQPSK (but there are a number of
> corner-cases that have to be considered).
> 
> >  but there is=
> > the initial state of the integrator to set up correctly and that's where t=
> >he frame syncing comes in.  and extreme frequencies that happen in OQPSK wi=
> >th the 00110011 or with 01100110 is unambiguous to an FM detector.
> >
> >> Designing a good UW for a preamble and the demodulator algorithms to
> >> reliably detect and synchronize it is sometimes difficult depending on
> >> the expected channel impairments.   Sometimes the UW has to also train
> >> the channel equalizer, if there is one.
> >
> >thanks Eric.  i hope you can continue to tolerate pedantacy from me.  unlik=
> >e the "circular DFT" arguments we have had in the past, i am fully aware th=
> >at i don't really know squat about this.  but, with OQPSK (at the "physical=
> > layer") i think i can construct what it looks like in the frequency domain=
> > when the IQ vector does not move (which would be 00000000, 11111111, 01010=
> >101, or 10101010) and when it moves consistently clockwise (00110011) or co=
> >unter-clockwise (01100110) which is the maximum detuning of the carrier.  (=
> >referring to the constellation at https://en.wikipedia.org/wiki/Phase-shift=
> >_keying#/media/File:Pi-by-O-QPSK_Gray_Coded.svg )  if you don't have a phas=
> >e lock, all you know is frequency.  then, the only preamble that i understa=
> >nd that will be detected unambiguously is one of those two extremes.
> >
> >do you understand what my point is (and where it's messed up)?
> 
> A carefully chosen length-N vector of complex samples with a good
> autocorrelation sequence (i.e., a UW) solves carrier phase and timing.
> If frequency is only roughly known, transmit two UWs in sequence and
> get fine frequency estimation from the difference in phase between
> them.   Then you know frequency, phase, and timing at the end of the
> second UW.   What else do you need?
> 

of course i don't need anything.  this is only for my own edification.

so, fixed-length packet or variable-length (within reason, packets should be too long) packet.  continuous-stream or bursty.  whatever.

the two issues still are:

1.  if you have a good (but not exact) idea of the carrier frequency, f_c, and a very good (nearly exact) idea of the bit rate, f_d, getting a lock on both the carrier phase and the IQ vector position with one single UW.

2.  and preventing a spurious frame sync if that UW *happens* to appear in your data (and the receiver may or may not be locked).  so i always thought that bit-stuffing was a way to arrange it so that the UW *never* appears unless you're trying to sync up a new frame or packet.  the UW accomplishes *both* syncing phase of the IF *and* signalling the beginning of a frame or packet.

i understand data scrambling and whitening.  so consider this *after* the data has been scrambled.

now, if you're the receiver and you don't have a lock, you have no friggin' idea where the IQ vector points.

now i understand in Offset QPSK that the IQ vector moves only when a bit in one of the two "substreams", the even or odd substream.  but suppose you're at the 00 position (lower left corner), moving clockwise (CW) means one substream (like the even bit substream) had a change and moving counter-clockwise (CCW) means it's the other substream had a change.  but if you're not synced, you don't know which one.  and you don't even know you're at the 00 position.  so you don't know whether is a 1-to-0 or a 0-to-1 change and you don't know if that happened to the even-bit substream or the odd-bit substream.

so how possibly are you going to correlate to a given UW?  you don't know if they're 0's or 1's, or even if it's the even bit or the odd bit.  

it seems to me that to sync under the worst case when you only know f_c and f_d but you have no friggin idea of the phase, you can only count on frequency.  with 00110011, you will detect 7 unambiguous CW movements of the IQ vector.  with 01100110, you will detect 7 unambiguous CCW movements of the IQ vector.  when you don't know shit about the phase, how can any other bit pattern tell you exactly what's goin' down?

hmmm, i guess you can detect some other prescribed sequence of IQ vector movements and light up a matched filter that says "sync right now".  is that how it's done, Eric?  if the initial position of the IQ vector is not specified, would a prescribed sequence of: {CW, CW, CCW, NM, CW, NM, CCW, ...} ("NM" means no movement) be one-to-one mapped to a specific UW?  i can say "yes" for the UWs of 00110011 or 01100110, but i don't know enough to say "yes" for any other UW.

i still think that OQPSK is sorta special, once we get that sync down (which seems to be a pain in arse).  it seems to me to require the least bandwidth (many bit patterns don't move the IQ vector at all and the worst case movement for any bit time is +\- 90 degrees) and the least amplitude variation for any 4-QAM.  and, at the transmitter, it can lined up quite elegantly to the even and odd bit substreams as i outlined in the initial post.

i'm mr. pedantic.

r b-j

robert bristow-johnson  <rbj@audioimagination.com> wrote:

>now i understand in Offset QPSK that the IQ vector moves only when a bit
>in one of the two "substreams", the even or odd substream.  but suppose
>you're at the 00 position (lower left corner), moving clockwise (CW)
>means one substream (like the even bit substream) had a change and
>moving counter-clockwise (CCW) means it's the other substream had a
>change.  but if you're not synced, you don't know which one.  and you
>don't even know you're at the 00 position.  so you don't know whether is
>a 1-to-0 or a 0-to-1 change and you don't know if that happened to the
>even-bit substream or the odd-bit substream.

>so how possibly are you going to correlate to a given UW?  you don't
>know if they're 0's or 1's, or even if it's the even bit or the odd bit.

It's good to remember that OQPSK is identical to 2-MSK.  So if
we suppose at the beginning of a UW (unique word), the transmitted phase 
is zero (that is, I = +1, Q = 0) the first bit in the unique word will 
cause the phase to transition to either I = 0, Q = +1, or I = 0, Q = -1.
That is to say, the effect of the bit is to rotate the phase
by either +pi/2 or -pi/2.  This is true for subsequent bits, so
when the UW is detected by means of these phase rotations occuring
in the expected sequence, the absolute phase is obtained as well.

You could say that the UW is detected differentially, and this
allows the subsequent segment of the transmission to be detected
non-differentially (or you could also say, it become coherent at this
point).

Steve

On Sat, 9 Apr 2016 22:19:49 -0700 (PDT), robert bristow-johnson
<rbj@audioimagination.com> wrote:

>On Thursday, April 7, 2016 at 3:07:21 PM UTC-4, Eric Jacobsen wrote:
>> On Wed, 6 Apr 2016 15:18:36 -0700 (PDT), robert bristow-johnson
>> <rbj@audioimagination.com> wrote:
>>=20
>> >
>> >what if synchronization is lost, you have a good idea what the carrier f=
>req=3D
>> >uency is (f_c) and a good idea what the bit rate is (f_d), but you have =
>*no=3D
>> >* idea in what direction the IQ vector is pointing and you want to *both=
>* g=3D
>> >et into synchronization *and* mark the beginning of a data frame with a =
>sin=3D
>> >gle preamble word.  what do you do?  how do you do that?
>>=20
>> Again, a good UW pattern provides an autocorrrelation peak that points
>> in the direction (on the complex plane) of a known phase, and the peak
>> will occur in time at an epoch that provides a reference for timing
>> synchronization.    So you get both phase synch and timing synch from
>> cross-correlating the UW sequence with the received sequence.
>>=20
>> If you don't have a UW, you let the phase and timing loops lock on
>> their own.   This doesn't exclude putting a frame marker in the data.
>>=20
>>=20
>...
>>=20
>> Sometimes protocols provide constant-length packets (like certain MPEG
>> encapsulations), so that you CAN simultaneously do PHY and packet
>> synchronization in a streaming link.  Often, though, it's a waste of
>> time and resources.
>>=20
>> Often PHY framing is done to accomodate block FEC, like Turbo Codes or
>> LDPCs or whatevers, and the FEC block sizes may not have anything to
>> do with the transported data frame sizes.   Forcing the FEC framing to
>> be the same size as the data frames often serves no purpose,
>> complicates the system, and may degrade the FEC performance.
>>=20
>...
>>=20
>> If the UW happens in a fixed-period frame sequence (which is usually
>> the case to accomodate fixe-length FEC blocks), you don't declare
>> frame lock until you've observed several of them spaced at the proper
>> distance in the stream.  This provides the appropriate confidence
>> level that you haven't found a UW in the data sequence.
>>=20
>> Bit stuffing at the PHY layer is a Very Bad Idea since it is subject
>> to bit errors and can therefore seriously screw up bit count integrity
>> and many other things.   Missing or improperly deleting a stuffed bit
>> will completely hose the FEC and anything else down stream.   This is
>> why protocols (like HDLC) that do bit stuffing to isolate frame
>> markers have to be done outside of the FEC (i.e., after the FEC in the
>> demodulator), or the first bit error that comes along destroys
>> civilization.
>>=20
>...
>>=20
>> The point is just that it's a little harder with OQPSK.  Frankly OQPSK
>> is a pain to use (IMHO), and should only be used when necessary.
>> Otherwise, use QPSK as it is far less of a pain (again, IMHO).
>>=20
>> >>   That
>> >> being said, the most efficient preambles tend to be carefully selected
>> >> patterns (usually a Unique Word of some kind with good autocorrelation
>> >> properties) that provide good detection (i.e., good autocorrelation),
>> >> as well as good phase and timing synchronization properties.
>> >
>> >i can see that if it wasn't quadrature. if it was BPSK, i could see just=
> w=3D
>> >aiting for the Unique Word to appear and sync up. (and i would think we =
>wou=3D
>> >ld bit-stuff to prevent that Unique Word from ever appearing happenstanc=
>e i=3D
>> >n the data and causing spurious syncing.)  but BPSK is half the bit rate=
> th=3D
>> >at QPSK is.  with QPSK, it's like you have an FM detector and you have t=
>o t=3D
>> >ake the output of the FM detector and integrate it to get PM.
>>=20
>> No, just phase lock.   It's very straightforward.   The exact same
>> demodulator structure can be used for both BPSK and QPSK.   The
>> carrier phase detector is really the only change needed to recover the
>> constellation for both, and that's pretty trivial.
>>=20
>> In a basic sense, one small additional change makes the same
>> demodulator capable of recovering OQPSK (but there are a number of
>> corner-cases that have to be considered).
>>=20
>> >  but there is=3D
>> > the initial state of the integrator to set up correctly and that's wher=
>e t=3D
>> >he frame syncing comes in.  and extreme frequencies that happen in OQPSK=
> wi=3D
>> >th the 00110011 or with 01100110 is unambiguous to an FM detector.
>> >
>> >> Designing a good UW for a preamble and the demodulator algorithms to
>> >> reliably detect and synchronize it is sometimes difficult depending on
>> >> the expected channel impairments.   Sometimes the UW has to also train
>> >> the channel equalizer, if there is one.
>> >
>> >thanks Eric.  i hope you can continue to tolerate pedantacy from me.  un=
>lik=3D
>> >e the "circular DFT" arguments we have had in the past, i am fully aware=
> th=3D
>> >at i don't really know squat about this.  but, with OQPSK (at the "physi=
>cal=3D
>> > layer") i think i can construct what it looks like in the frequency dom=
>ain=3D
>> > when the IQ vector does not move (which would be 00000000, 11111111, 01=
>010=3D
>> >101, or 10101010) and when it moves consistently clockwise (00110011) or=
> co=3D
>> >unter-clockwise (01100110) which is the maximum detuning of the carrier.=
>  (=3D
>> >referring to the constellation at https://en.wikipedia.org/wiki/Phase-sh=
>ift=3D
>> >_keying#/media/File:Pi-by-O-QPSK_Gray_Coded.svg )  if you don't have a p=
>has=3D
>> >e lock, all you know is frequency.  then, the only preamble that i under=
>sta=3D
>> >nd that will be detected unambiguously is one of those two extremes.
>> >
>> >do you understand what my point is (and where it's messed up)?
>>=20
>> A carefully chosen length-N vector of complex samples with a good
>> autocorrelation sequence (i.e., a UW) solves carrier phase and timing.
>> If frequency is only roughly known, transmit two UWs in sequence and
>> get fine frequency estimation from the difference in phase between
>> them.   Then you know frequency, phase, and timing at the end of the
>> second UW.   What else do you need?
>>=20
>
>of course i don't need anything.  this is only for my own edification.
>
>so, fixed-length packet or variable-length (within reason, packets should b=
>e too long) packet.  continuous-stream or bursty.  whatever.
>
>the two issues still are:
>
>1.  if you have a good (but not exact) idea of the carrier frequency, f_c, =
>and a very good (nearly exact) idea of the bit rate, f_d, getting a lock on=
> both the carrier phase and the IQ vector position with one single UW.
>
>2.  and preventing a spurious frame sync if that UW *happens* to appear in =
>your data (and the receiver may or may not be locked).  so i always thought=
> that bit-stuffing was a way to arrange it so that the UW *never* appears u=
>nless you're trying to sync up a new frame or packet.  the UW accomplishes =
>*both* syncing phase of the IF *and* signalling the beginning of a frame or=
> packet.
>
>i understand data scrambling and whitening.  so consider this *after* the d=
>ata has been scrambled.
>
>now, if you're the receiver and you don't have a lock, you have no friggin'=
> idea where the IQ vector points.

But even with random data the phase can be obtained just by letting
the synchronization loops lock on their own.   In a continuous system
this is often what is done.   UWs or any kind of specifc sequences
aren't required.

>now i understand in Offset QPSK that the IQ vector moves only when a bit in=
> one of the two "substreams", the even or odd substream.  but suppose you'r=
>e at the 00 position (lower left corner), moving clockwise (CW) means one s=
>ubstream (like the even bit substream) had a change and moving counter-cloc=
>kwise (CCW) means it's the other substream had a change.  but if you're not=
> synced, you don't know which one.  and you don't even know you're at the 0=
>0 position.  so you don't know whether is a 1-to-0 or a 0-to-1 change and y=
>ou don't know if that happened to the even-bit substream or the odd-bit sub=
>stream.
>
>so how possibly are you going to correlate to a given UW?  you don't know i=
>f they're 0's or 1's, or even if it's the even bit or the odd bit. =20

During initial acquisition of a burst of a continuous signal with UW
framing, symbol synchronization isn't known, either.   In a bursty
system acquisition of phase, symbol timing, and start of message must
be done very quickly, and often a carefully selected UW with good
autocorrelation properties is all that is needed.

In the demodulator this typically requires oversampling the system,
and often four samples per symbol or so is enough.   In that case the
UW is just an N-point complex-valued vector that can be corrrelated
against the incoming stream. 

This then becomes the usual, straightforward cross-correlation process
and if the UW is any good the output will be a spike indicating a
reference phase at a symbol epoch and will also indicate the start of
message time.

So, you have, unambiguously, carrier phase, symbol timing, and
start-of-message timing.   As I said before, if you also need a
frequency estimate, put two UWs back-to-back and measure the phase
rotation between the two correlation peaks.

The part of this that perhaps wasn't mentioned before was that it
needs to be oversampled, so then it is a pretty simple signal
processing task unencumbered by other communications or modulation
concepts.

The caveat with OQPSK is that it is harder to come up with a good UW
that behaves by OQPSK modulation and still has good autocorrelation
properties.   It is possible, and sometimes the UW winds up being
longer than it might be otherwise.


>it seems to me that to sync under the worst case when you only know f_c and=
> f_d but you have no friggin idea of the phase, you can only count on frequ=
>ency.  with 00110011, you will detect 7 unambiguous CW movements of the IQ =
>vector.  with 01100110, you will detect 7 unambiguous CCW movements of the =
>IQ vector.  when you don't know shit about the phase, how can any other bit=
> pattern tell you exactly what's goin' down?

See above.   Or just let the loops lock on random data.

>hmmm, i guess you can detect some other prescribed sequence of IQ vector mo=
>vements and light up a matched filter that says "sync right now".  is that =
>how it's done, Eric?  if the initial position of the IQ vector is not speci=
>fied, would a prescribed sequence of: {CW, CW, CCW, NM, CW, NM, CCW, ...} (=
>"NM" means no movement) be one-to-one mapped to a specific UW?  i can say "=
>yes" for the UWs of 00110011 or 01100110, but i don't know enough to say "y=
>es" for any other UW.

You want the UW to have a peaky autocorrelation sequence so that you
get carrier phase, symbol timing, and start of message timing all from
one cross-correlation operation.   This requires that it have a big
peak with low sidelobes, like the usual known sequences like PN
sequences, Barker codes, or many of the codes used for CDMA spreading
(which need the same or similar properties).  The same sorts of
principles can be applied to design a complex-valued UW preamble for
an OQPSK system.

>i still think that OQPSK is sorta special, once we get that sync down (whic=
>h seems to be a pain in arse).  it seems to me to require the least bandwid=
>th (many bit patterns don't move the IQ vector at all and the worst case mo=
>vement for any bit time is +\- 90 degrees) and the least amplitude variatio=
>n for any 4-QAM.  and, at the transmitter, it can lined up quite elegantly =
>to the even and odd bit substreams as i outlined in the initial post.

If you filter OQPSK (e.g., the usual Nyquist pulse shaping filtering),
then the spectral efficiency of OQPSK is identical to QPSK.   There
aren't very many cases where you wouldn't want to filter it, and if
you don't filter it the spectral efficiency is much worse.   The only
real advantage to OQSPK is that the PAPR is lower, and that's much
less important these days as PA technology has gotten crazy good.
And OQPSK is trickier to synchronize, but it's not that big of a deal
and certainly doable with only a bit more effort and complexity on top
of a QPSK demod.

So the cases where it is useful are getting to be fewer and fewer.


>i'm mr. pedantic.
>
>r b-j


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

On 07.04.2016 5:53, clay@claysturner.com wrote:

(snip)

> A ternary modulation is tempting in that there is a ternary Golay code (perfect code)! The ternary Golay code was invented before Golay's discovery to hedge bets in a football pool.
>
> Clay
>

On a sideways note, bits mean "binary digits". Makes me wondering what 
should be the proper term for "ternary digits".

Evgeny

Evgeny Filatov  <filatov.ev@mipt.ru> wrote:

>On a sideways note, bits mean "binary digits". Makes me wondering what 
>should be the proper term for "ternary digits".

Nice, an uplifting comment here on the group for a change.


Steve

Evgeny Filatov <filatov.ev@mipt.ru> writes:

> On 07.04.2016 5:53, clay@claysturner.com wrote:
>
> (snip)
>
>> A ternary modulation is tempting in that there is a ternary Golay
>> code (perfect code)! The ternary Golay code was invented before
>> Golay's discovery to hedge bets in a football pool.
>>
>> Clay
>>
>
> On a sideways note, bits mean "binary digits". Makes me wondering what
> should be the proper term for "ternary digits".
>
> Evgeny

Classy!
-- 
Randy Yates, DSP/Embedded Firmware Developer
Digital Signal Labs
http://www.digitalsignallabs.com

On Sun, 10 Apr 2016 23:15:25 +0300, Evgeny Filatov
<filatov.ev@mipt.ru> wrote:

>On 07.04.2016 5:53, clay@claysturner.com wrote:
>
>(snip)
>
>> A ternary modulation is tempting in that there is a ternary Golay code (perfect code)! The ternary Golay code was invented before Golay's discovery to hedge bets in a football pool.
>>
>> Clay
>>
>
>On a sideways note, bits mean "binary digits". Makes me wondering what 
>should be the proper term for "ternary digits".
>
>Evgeny
>

Since you brought it up I think you should be allowed to name it.
Make us proud.   ;)

On 11.04.2016 1:09, Eric Jacobsen wrote:
> On Sun, 10 Apr 2016 23:15:25 +0300, Evgeny Filatov
> <filatov.ev@mipt.ru> wrote:
>
>> On 07.04.2016 5:53, clay@claysturner.com wrote:
>>
>> (snip)
>>
>>> A ternary modulation is tempting in that there is a ternary Golay code (perfect code)! The ternary Golay code was invented before Golay's discovery to hedge bets in a football pool.
>>>
>>> Clay
>>>
>>
>> On a sideways note, bits mean "binary digits". Makes me wondering what
>> should be the proper term for "ternary digits".
>>
>> Evgeny
>>
>
> Since you brought it up I think you should be allowed to name it.
> Make us proud.   ;)
>
>

No idea. Perhaps, breasts? As in "Bigger than REquired, AStounding digiTS".

Evgeny

On 11.4.16 01:09, Eric Jacobsen wrote:
> On Sun, 10 Apr 2016 23:15:25 +0300, Evgeny Filatov
> <filatov.ev@mipt.ru> wrote:
>
>> On 07.04.2016 5:53, clay@claysturner.com wrote:
>>
>> (snip)
>>
>>> A ternary modulation is tempting in that there is a ternary Golay code (perfect code)! The ternary Golay code was invented before Golay's discovery to hedge bets in a football pool.
>>>
>>> Clay
>>>
>>
>> On a sideways note, bits mean "binary digits". Makes me wondering what
>> should be the proper term for "ternary digits".
>>
>> Evgeny
>>
>
> Since you brought it up I think you should be allowed to name it.
> Make us proud.   ;)


My friend Leo Ojala made a fast ternary multiplier with tunnel
diodes for his Dr of Thechnology thesis nearly 50 years ago.

He called the digits trits, but I'm not sure if they were printed
in the dissertation.

-- 

-Tauno Voipio