Really good book on coding

Suggestions?  I'm looking for something more specific and new than "Error-
Correction Coding for Digital Communications" by Clark and Cain.

I was going to ask for a really good book specifically on _Convolutional_ 
coding, and that's kind of the direction that I'm leaning at the moment, 
but one book to bring them all and in the darkness bind them would be 
acceptable.

I'm especially interested in non-binary convolutional coding.  At least 
today.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

I'm looking for work -- see my website!

Tim Wescott  <seemywebsite@myfooter.really> wrote:

>I'm especially interested in non-binary convolutional coding.  At least 
>today.

Non-binary, but linear?

Or non-linear as well?

Steve

On Mon, 12 Dec 2016 05:16:06 +0000, Steve Pope wrote:

> Tim Wescott  <seemywebsite@myfooter.really> wrote:
> 
>>I'm especially interested in non-binary convolutional coding.  At least
>>today.
> 
> Non-binary, but linear?
> 
> Or non-linear as well?
> 
> Steve

If there's a comprehensive theory of nonlinear coding theory I'm open.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

I'm looking for work -- see my website!

Tim Wescott  <seemywebsite@myfooter.really> wrote:

>On Mon, 12 Dec 2016 05:16:06 +0000, Steve Pope wrote:

>> Tim Wescott  <seemywebsite@myfooter.really> wrote:

>>>I'm especially interested in non-binary convolutional coding.  At least
>>>today.

>> Non-binary, but linear?

>> Or non-linear as well?

>> Steve

>If there's a comprehensive theory of nonlinear coding theory I'm open.

I don't know of any comprehensive theories in this area.

A large useful class of non-binary, non-linear convolutional codes
are the Ungerboek codes.

A large useful class of non-binary, linear convolutional codes are
the binary convolutional codes interleaved across nonbinary symbols.

I have created native, non-binary convolutiona codes as toy examples,
but that I could tell they were never any better than more conventional
convolutional codes.

So ... not much help here.

Steve

On 12.12.2016 7:16, Tim Wescott wrote:
> Suggestions?  I'm looking for something more specific and new than "Error-
> Correction Coding for Digital Communications" by Clark and Cain.
>
> I was going to ask for a really good book specifically on _Convolutional_
> coding, and that's kind of the direction that I'm leaning at the moment,
> but one book to bring them all and in the darkness bind them would be
> acceptable.
>
> I'm especially interested in non-binary convolutional coding.  At least
> today.
>

A couple of good books (not sure if they qualify as "really good"):

(1) Moon "Error Correcting Codes" (2005)
(2) Johannesson and Zigangirov "Fundamentals of Convolutional Encoding" 
(2015)

As far as I understand, the first book in the list is currently a 
standard college textbook. It reads really well. The second book is a 
tad more hard to read, but covers some stuff that doesn't appear in the 
first one.

The list is definitely not comprehensive, and like you I am open to any 
suggestions about new and good books.

Gene

On 12.12.2016 13:10, Evgeny Filatov wrote:
> On 12.12.2016 7:16, Tim Wescott wrote:
>> Suggestions?  I'm looking for something more specific and new than
>> "Error-
>> Correction Coding for Digital Communications" by Clark and Cain.
>>
>> I was going to ask for a really good book specifically on _Convolutional_
>> coding, and that's kind of the direction that I'm leaning at the moment,
>> but one book to bring them all and in the darkness bind them would be
>> acceptable.
>>
>> I'm especially interested in non-binary convolutional coding.  At least
>> today.
>>
>
> A couple of good books (not sure if they qualify as "really good"):
>
> (1) Moon "Error Correcting Codes" (2005)
> (2) Johannesson and Zigangirov "Fundamentals of Convolutional Encoding"
> (2015)
>
> As far as I understand, the first book in the list is currently a
> standard college textbook. It reads really well. The second book is a
> tad more hard to read, but covers some stuff that doesn't appear in the
> first one.
>
> The list is definitely not comprehensive, and like you I am open to any
> suggestions about new and good books.
>
> Gene
>

Oops! The second book I've listed is definitely "Fundamentals of 
Convolutional Coding" (and not what I've written).

Gene

On Mon, 12 Dec 2016 06:19:16 +0000 (UTC), spope33@speedymail.org
(Steve Pope) wrote:

>Tim Wescott  <seemywebsite@myfooter.really> wrote:
>
>>On Mon, 12 Dec 2016 05:16:06 +0000, Steve Pope wrote:
>
>>> Tim Wescott  <seemywebsite@myfooter.really> wrote:
>
>>>>I'm especially interested in non-binary convolutional coding.  At least
>>>>today.

>>> Non-binary, but linear?
>
>>> Or non-linear as well?
>
>>> Steve
>
>>If there's a comprehensive theory of nonlinear coding theory I'm open.
>
>I don't know of any comprehensive theories in this area.
>
>A large useful class of non-binary, non-linear convolutional codes
>are the Ungerboek codes.
>
>A large useful class of non-binary, linear convolutional codes are
>the binary convolutional codes interleaved across nonbinary symbols.

Aka Bit-Interleaved Coded Modulation (BICM), which turns out to be, in
many cases, much better than Trellis-Coded Modulation and also simpler
to implement.

A good reference paper is Bit-Interleaved Coded Modulation, Caire,
Taricco, Biglieri, "IEEE Trans. on Information Theory", Vol 44, No 3,
May 1998, p 927.

This is more or less my default go-to technique for basic systems.

>I have created native, non-binary convolutiona codes as toy examples,
>but that I could tell they were never any better than more conventional
>convolutional codes.

That's been my experience as well.  Binary codes are prevalent because
they're difficult to beat in many applications, even if the overlying
symbols are not binary.

There are some non-binary radix decoders which mostly fold hardware to
accelerate computation, if that's what you're interested in.
Probably searching on Radix-4 or Radix-16 ACS sections for Viterbi
decoders may get you some good references there.

Regarding reference texts, I find that few texts are good at covering
a broad scope of codes.  In other words, a book that is good at
covering convolutional codes may not be good at iterative or algebraic
codes or vice versa.

Modulation and Coding Techniques in Wireless Communication, Krouk and
Semenov.    I visited Prof. Krouk and his lab in St. Petersburg a few
times for a previous employer.   This book is a good translation of
the original Russian text.   It is one of the few books that covers
convolutional coding that also talks about list decoders (briefly) and
sequential decoders.

Sklar's text, Digital Communications, also has a good treatment of
convolutional codes, including sequential decoders.

I don't really know of any texts off the top of my head that treat
non-binary convolutional codes, as most deal with non-binary systems
by adding TCM or BICM or some other technique on top of binary codes.

I hope that helps a bit or two.

On Tue, 13 Dec 2016 01:14:41 +0000, eric.jacobsen wrote:

> On Mon, 12 Dec 2016 06:19:16 +0000 (UTC), spope33@speedymail.org (Steve
> Pope) wrote:
> 
>>Tim Wescott  <seemywebsite@myfooter.really> wrote:
>>
>>>On Mon, 12 Dec 2016 05:16:06 +0000, Steve Pope wrote:
>>
>>>> Tim Wescott  <seemywebsite@myfooter.really> wrote:
>>
>>>>>I'm especially interested in non-binary convolutional coding.  At
>>>>>least today.
> 
>>>> Non-binary, but linear?
>>
>>>> Or non-linear as well?
>>
>>>> Steve
>>
>>>If there's a comprehensive theory of nonlinear coding theory I'm open.
>>
>>I don't know of any comprehensive theories in this area.
>>
>>A large useful class of non-binary, non-linear convolutional codes are
>>the Ungerboek codes.
>>
>>A large useful class of non-binary, linear convolutional codes are the
>>binary convolutional codes interleaved across nonbinary symbols.
> 
> Aka Bit-Interleaved Coded Modulation (BICM), which turns out to be, in
> many cases, much better than Trellis-Coded Modulation and also simpler
> to implement.
> 
> A good reference paper is Bit-Interleaved Coded Modulation, Caire,
> Taricco, Biglieri, "IEEE Trans. on Information Theory", Vol 44, No 3,
> May 1998, p 927.
> 
> This is more or less my default go-to technique for basic systems.
> 
>>I have created native, non-binary convolutiona codes as toy examples,
>>but that I could tell they were never any better than more conventional
>>convolutional codes.
> 
> That's been my experience as well.  Binary codes are prevalent because
> they're difficult to beat in many applications, even if the overlying
> symbols are not binary.
> 

OK.  Here's the deal.  I'd wondered if this was the case, but because of 
hardware constraints I have an underlying layer that naturally breaks 
things down into multiple symbols.  The symbols aren't all equidistant 
from each other, but they're not ordered in a way that would map to a 
binary count.

I can make soft decisions, and because the symbols aren't equidistant, a 
small error results in a pair, or three or four, symbols being roughly 
equally probable.

As a for-instance, if the decision is equally divided between symbols #3 
and #4 out of 16 possible values, then if I keep things at this level, 
what would go into the decoder would be a soft decision that basically 
says "maybe number three, maybe number four, but definitely not the rest".

However, if I were to turn each of my 16 symbols into a four-bit 
sequence, then in the above case what would go into the decoder would be 
"definitely zero, erase, erase, erase" -- in other words, by going to 
binary my two possible values out of sixteen have turned into eight out 
of sixteen.

So intuitively, it seems that a decoder operating on GF(16) would be 
better in this case.

So -- you think that even in this case I should just use a binary code?

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

I'm looking for work -- see my website!

Tim Wescott  <seemywebsite@myfooter.really> wrote:

>OK.  Here's the deal.  I'd wondered if this was the case, but because of 
>hardware constraints I have an underlying layer that naturally breaks 
>things down into multiple symbols.  The symbols aren't all equidistant 
>from each other, but they're not ordered in a way that would map to a 
>binary count.
>
>I can make soft decisions, and because the symbols aren't equidistant, a 
>small error results in a pair, or three or four, symbols being roughly 
>equally probable.
>
>As a for-instance, if the decision is equally divided between symbols #3 
>and #4 out of 16 possible values, then if I keep things at this level, 
>what would go into the decoder would be a soft decision that basically 
>says "maybe number three, maybe number four, but definitely not the rest".
>
>However, if I were to turn each of my 16 symbols into a four-bit 
>sequence, then in the above case what would go into the decoder would be 
>"definitely zero, erase, erase, erase" -- in other words, by going to 
>binary my two possible values out of sixteen have turned into eight out 
>of sixteen.
>
>So intuitively, it seems that a decoder operating on GF(16) would be 
>better in this case.
>
>So -- you think that even in this case I should just use a binary code?

Well, you're certainly welcome to try using a code over GF(16)
and theoretically, such a code might ultimately be the best performer.
But, it might have other problems such as extremely long codewords
being necessary, humungous complexity, etc.

Or you could use a binary code with interleaving and compute the MAP 
probability for each bit feeding the decoder (as opposed to some 
suboptimal soft-decision and/or erasure algorithm which is my sense of
what you are above describing).

Once you have implemented or perhaps simulated the MAP algorithm
you can see how much performance is lost by cheaper, more ad-hoc
soft-decision algorithms.

You might also consider going to a binary code which is stronger than
a binary convolutional code in the first place (one of the near-channel
capacity codes such as LDPC) -- I think this is lower-hanging fruit
then searching for a nonbinary code.

Steve

On 13.12.2016 4:14, eric.jacobsen@ieee.org wrote:
>
> Modulation and Coding Techniques in Wireless Communication, Krouk and
> Semenov.    I visited Prof. Krouk and his lab in St. Petersburg a few
> times for a previous employer.   This book is a good translation of
> the original Russian text.   It is one of the few books that covers
> convolutional coding that also talks about list decoders (briefly) and
> sequential decoders.

I'm afraid there was no "original Russian text". It's common for Russian 
authors to write a book in English, and then, when the stars align, 
maybe make a Russian translation. Frankly, it's not much of an 
impediment, because everybody seems to know English. It's true that I 
read English fiction several times slower than the same sort of 
literature in my native tongue. But with technical texts my reading 
speed is limited by something else, i.e. my ability to understand the math.

Gene