"Randy Yates" <randy.yates@sonyericsson.com> wrote in message news:xxp652u13um.fsf@usrts005.corpusers.net...> > Intriguing stuff! I'm not sure I follow the sorting description, but > that's OK. I need to go back and read fully and with undrstanding > the seminal Shannon papers. > --Hello Randy, You will probably want to get "An Introduction to Information Theory, Symbols, Signals, and Noise" by John Pierce. This is a wonderful book, and it makes the material easy to grasp. Plus he throws in a lot of the historical background, so this also aides the understanding process. ISBN 0-486-24061-4 I got my copy a while back for 10 bucks. I have the 2nd edition from 1980. Clay
how to find the best ADC step size?
Started by ●December 21, 2004
Reply by ●December 22, 20042004-12-22
Reply by ●December 22, 20042004-12-22
glen herrmannsfeldt wrote:> > > Jerry Avins wrote: > >> glen herrmannsfeldt wrote: > > > (snip) > >>> The gaussian has infinite tails, so it isn't possible to cover >>> the range with a linear scale ADC. One could then ask for what >>> part of the distribution, when covered with the steps of a six >>> bit ADC, are the symbol probabilities most equal. > > >> For non-linear encoding, it actually is possible. One way is to divide >> the cumulative distribution into six parts equal (divide the density >> function into six equal areas), then have the ADC report the sextile* >> that the sample falls into. Infinite tails don't bother that scheme. > > > To me "step size" implies linear, but I could be convinced otherwise. > I would have thought "step sizes", though, for the non-linear case. > > -- glenI'll buy that, but it could go either way. You could compare warehouses by floor *areas* or floor *area*. jerry -- Engineering is the art of making what you want from things you can get. �����������������������������������������������������������������������
Reply by ●December 22, 20042004-12-22
Clay S. Turner wrote:> "Randy Yates" <randy.yates@sonyericsson.com> wrote in message > news:xxp652u13um.fsf@usrts005.corpusers.net... > >>Intriguing stuff! I'm not sure I follow the sorting description, but >>that's OK. I need to go back and read fully and with undrstanding >>the seminal Shannon papers. >>-- > > > Hello Randy, > You will probably want to get "An Introduction to Information Theory, > Symbols, Signals, and Noise" by John Pierce. This is a wonderful book, and > it makes the material easy to grasp. Plus he throws in a lot of the > historical background, so this also aides the understanding process. > > ISBN 0-486-24061-4 > > I got my copy a while back for 10 bucks. I have the 2nd edition from 1980. > > Clayhttp://powellbooks.com/cgi-bin/biblio?isbn=0-486-24061-4 Easy for you; less so for me. Jerry -- Engineering is the art of making what you want from things you can get. �����������������������������������������������������������������������
Reply by ●December 22, 20042004-12-22
"Clay S. Turner" <Physics@Bellsouth.net> writes:> "Randy Yates" <randy.yates@sonyericsson.com> wrote in message > news:xxp652u13um.fsf@usrts005.corpusers.net... >> >> Intriguing stuff! I'm not sure I follow the sorting description, but >> that's OK. I need to go back and read fully and with undrstanding >> the seminal Shannon papers. >> -- > > Hello Randy, > You will probably want to get "An Introduction to Information Theory, > Symbols, Signals, and Noise" by John Pierce. This is a wonderful book, and > it makes the material easy to grasp. Plus he throws in a lot of the > historical background, so this also aides the understanding process. > > ISBN 0-486-24061-4 > > I got my copy a while back for 10 bucks. I have the 2nd edition from 1980.For $10 how can you go wrong? Thanks Clay! Merry Christmas. --Randy -- % Randy Yates % "Watching all the days go by... %% Fuquay-Varina, NC % Who are you and who am I?" %%% 919-577-9882 % 'Mission (A World Record)', %%%% <yates@ieee.org> % *A New World Record*, ELO http://home.earthlink.net/~yatescr
Reply by ●December 22, 20042004-12-22
Randy Yates wrote:> "Clay S. Turner" <Physics@Bellsouth.net> writes: > > >>"Randy Yates" <randy.yates@sonyericsson.com> wrote in message >>news:xxpllbqjpjf.fsf@usrts005.corpusers.net... >> >>>"Clay S. Turner" <Physics@Bellsouth.net> writes: >>> >>>>[...] >>>>The idea is to try to make each symbol contribute equally to the overall >>>>process. Imagine looking at your data after a huge number of symbols was >>>>received. The idea is to make the info provided by each type of symbol >>>>contribute equally. >>> >>>But this Huffman coding won't do that. Choosing a representation for a >>>symbol doesn't change the probability of the symbol occurring. It >>>does, however, minize the average symbol rate - I certainly see >>>that. Perhaps I'm being blind? >>>-- >> >>Hello Randy, >> >>The Huffman problem and the quantization problem are related in they both >>implement methods to effectively flatten out variations. True they are >>different in that the Huffman problem is trying to minimize the average >>number of bits per sample and the quantization problem is trying to maximize >>the information per sample. The differences arise from the constraints and >>what you are starting with. In one case we have a fixed amount of >>information, so how few total bits can we fit all of the info in. The >>theoretical answer is the entropy, but the practical answer (comes from >>requiring whole numbers of bits in a symbol is the Huffman entropy.) In the >>other case we have a fixed symbol size, so how can we get the most info per >>symbol. The connection is the entropy and its maximization requires a flat >>distribution. >> >>In Huffman coding we sort the symbol probabilities into descending order and >>then we make multiple passes throught the list each time combining the two >>lowest probilites together. So we are essentially trying to make each path's >>probability be the same or at leat similar. In the quantization we are just >>diving up the total probability into N equal regions. >> >>You are not being blind, you are just asking how two seemingly different >>things are related. And the relation is through a flattening out of the >>probabilty function. >> >>I hope this helps to clear some things up. >> >>Clay > > > Intriguing stuff! I'm not sure I follow the sorting description, but > that's OK. I need to go back and read fully and with undrstanding > the seminal Shannon papers.Just a thought... don't know if it's at all relevant to the thread, but here we go anyway: I you do a Huffman coding, common symbols are given a longer code word. This will minimize the average data rate. If you adjust quantization similarly, I get the feeling this will also minimize the average quantization error. /Johan
Reply by ●December 22, 20042004-12-22
>>>>> "lucy" == lucy <losemind@yahoo.com> writes:lucy> Hi all, lucy> Suppose my input data follows Gaussian distribution with mean u and variance lucy> sigma^2. lucy> If I want to design 6-bit ADC... what should be the optimal step size for lucy> this ADC? Jayant and Noll has a section on quantizers and how to select optimal step sizes. Ray
Reply by ●December 23, 20042004-12-23
Johan Carlson <Johan.NOSPAM.Carlson@csee.ltu.se> writes:> Randy Yates wrote: > > > "Clay S. Turner" <Physics@Bellsouth.net> writes: > > > > >> "Randy Yates" <randy.yates@sonyericsson.com> wrote in message > >> news:xxpllbqjpjf.fsf@usrts005.corpusers.net... > > >> > >>>"Clay S. Turner" <Physics@Bellsouth.net> writes: > >>> > >>>>[...] > >>>>The idea is to try to make each symbol contribute equally to the overall > >>>>process. Imagine looking at your data after a huge number of symbols was > >>>>received. The idea is to make the info provided by each type of symbol > >>>>contribute equally. > >>> > >>>But this Huffman coding won't do that. Choosing a representation for a > >>>symbol doesn't change the probability of the symbol occurring. It > >>>does, however, minize the average symbol rate - I certainly see > >>>that. Perhaps I'm being blind? > >>> -- > >> > >>Hello Randy, > >> > >> The Huffman problem and the quantization problem are related in > >> they both implement methods to effectively flatten out > >> variations. True they are different in that the Huffman problem is > >> trying to minimize the average number of bits per sample and the > >> quantization problem is trying to maximize the information per > >> sample. The differences arise from the constraints and what you are > >> starting with. In one case we have a fixed amount of information, > >> so how few total bits can we fit all of the info in. The > >> theoretical answer is the entropy, but the practical answer (comes > >> from requiring whole numbers of bits in a symbol is the Huffman > >> entropy.) In the other case we have a fixed symbol size, so how can > >> we get the most info per symbol. The connection is the entropy and > >> its maximization requires a flat distribution. > > >> > >> In Huffman coding we sort the symbol probabilities into descending > >> order and then we make multiple passes throught the list each time > >> combining the two lowest probilites together. So we are essentially > >> trying to make each path's probability be the same or at leat > >> similar. In the quantization we are just diving up the total > >> probability into N equal regions. > > >> > >> You are not being blind, you are just asking how two seemingly > >> different things are related. And the relation is through a > >> flattening out of the probabilty function. > > >> > >>I hope this helps to clear some things up. > >> > >>Clay > > Intriguing stuff! I'm not sure I follow the sorting description, but > > > that's OK. I need to go back and read fully and with undrstanding > > the seminal Shannon papers. > > > Just a thought... don't know if it's at all relevant to the thread, > but here we go anyway: > > > I you do a Huffman coding, common symbols are given a longer code > word.No, they're given a shorter code word.> This will minimize the average data rate.Right.> If you adjust quantization similarly, I get the feeling this will > also minimize the average quantization error.Depends on what you mean by "similarly." Can you expound? -- Randy Yates Sony Ericsson Mobile Communications Research Triangle Park, NC, USA randy.yates@sonyericsson.com, 919-472-1124
Reply by ●December 23, 20042004-12-23
Randy Yates wrote:> Johan Carlson <Johan.NOSPAM.Carlson@csee.ltu.se> writes: > > >>Randy Yates wrote: >> >> >>>"Clay S. Turner" <Physics@Bellsouth.net> writes: >>> >> >>>>"Randy Yates" <randy.yates@sonyericsson.com> wrote in message >>>>news:xxpllbqjpjf.fsf@usrts005.corpusers.net... >> >>>>>"Clay S. Turner" <Physics@Bellsouth.net> writes: >>>>> >>>>> >>>>>>[...] >>>>>>The idea is to try to make each symbol contribute equally to the overall >>>>>>process. Imagine looking at your data after a huge number of symbols was >>>>>>received. The idea is to make the info provided by each type of symbol >>>>>>contribute equally. >>>>> >>>>>But this Huffman coding won't do that. Choosing a representation for a >>>>>symbol doesn't change the probability of the symbol occurring. It >>>>>does, however, minize the average symbol rate - I certainly see >>>>>that. Perhaps I'm being blind? >>>>>-- >>>> >>>>Hello Randy, >>>> >>>>The Huffman problem and the quantization problem are related in >>>>they both implement methods to effectively flatten out >>>>variations. True they are different in that the Huffman problem is >>>>trying to minimize the average number of bits per sample and the >>>>quantization problem is trying to maximize the information per >>>>sample. The differences arise from the constraints and what you are >>>>starting with. In one case we have a fixed amount of information, >>>>so how few total bits can we fit all of the info in. The >>>>theoretical answer is the entropy, but the practical answer (comes >>>>from requiring whole numbers of bits in a symbol is the Huffman >>>>entropy.) In the other case we have a fixed symbol size, so how can >>>>we get the most info per symbol. The connection is the entropy and >>>>its maximization requires a flat distribution. >> >>>>In Huffman coding we sort the symbol probabilities into descending >>>>order and then we make multiple passes throught the list each time >>>>combining the two lowest probilites together. So we are essentially >>>>trying to make each path's probability be the same or at leat >>>>similar. In the quantization we are just diving up the total >>>>probability into N equal regions. >> >>>>You are not being blind, you are just asking how two seemingly >>>>different things are related. And the relation is through a >>>>flattening out of the probabilty function. >> >>>>I hope this helps to clear some things up. >>>> >>>>Clay >>> >>>Intriguing stuff! I'm not sure I follow the sorting description, but >> >>>that's OK. I need to go back and read fully and with undrstanding >>>the seminal Shannon papers. >> >> >>Just a thought... don't know if it's at all relevant to the thread, >>but here we go anyway: >> >> >>I you do a Huffman coding, common symbols are given a longer code >>word. > > > No, they're given a shorter code word. >Of course... my mistake.> >>This will minimize the average data rate. > > > Right. > > >>If you adjust quantization similarly, I get the feeling this will >>also minimize the average quantization error. > > > Depends on what you mean by "similarly." Can you expound?If you a finer quantization for highly probable data, the error should be smaller. Exactly how this has anything to do with huffman coding isn't obvious. I was just rambling... Thought the discussion was interesting. /JC
Reply by ●January 1, 20052005-01-01
There is a lot of useful information and pointers in this thread. There is a paper in the IEEE Transactions on Acoustics, Speech, and Signal Processing from the mid eighties that address this subject. I'm sorry I don't have a better pointer. I may have a copy of the paper at work. There are two related issues, how do the transition points between levels get chosen and how do they get represented (what value is assigned to the data value) in further processing. Both are important in minimizing the error in the process and are more or less independent of whether equally spaced or variable spaced quantization is chosen though for a finite number of steps, variable spaced thresholds will minimize the error. In article <cqacbp$bbc$1@news.Stanford.EDU>, "lucy" <losemind@yahoo.com> wrote:>Hi all, > >Suppose my input data follows Gaussian distribution with mean u and variance >sigma^2. > >If I want to design 6-bit ADC... what should be the optimal step size for >this ADC? > >Thanks a lot > >-L > >
