information theory?

On 11/4/2011 5:31 AM, Martin Brown wrote:

> Have you never seen sheet music? That is what Shannon was estimating -
> the bitrate for describing music in the abstract. There are a finite
> number of notes, durations and amplitudes in a classical composition.
>
> I suspect 40 bits/sec is still far too tight, but a midi stream using a
> high end reconstruction codec represents a pretty good example of what
> is possible by way of compression for *music* as opposed to voice or a
> random noise stream.

Which is why we are talking about midi. Recording SOUND is a different 
thing. Note that a cymbal crash is very close to white noise. This means 
that exact reproduction allows for little compression. Of course all 
music is not cymbal crashes (or usually so) which is where MP3 comes from.

> Be interesting if the reference to this paper can be found to see how he
> allocated those 40 bits/s....
>
> I reckon at a bare minimum about 7 to the note, 8 to amplitude, 6
> duration, 5 to the instrument - and it is already obvious that you
> cannot encode more than a single note per second at this bitrate.

Sorry NOT "obvious". You need to learn more about information theory!
Note that you have encoded ONE note but looking at sheet music one 
observed that MANY notes are identical!  Hence there is redundancy that 
can be eliminated for compression.

On Nov 4, 12:50&#4294967295;pm, VWWall <vw...@large.invalid> wrote:
> k...@kymhorsell.com wrote:
> > In sci.math Robert Wessel <robertwess...@yahoo.com> wrote:
> > ...
> >> Consider that high keystroke rates pretty much require chords, and a
> >> chord is most certainly not three or four random keys, pressed at
> >> different times and forces - rather they're a small subset of the
> >> possible combinations of keys within a hand-span (or two hand-spans in
> >> some cases), hit at the same time, and with the same force (in fact
> >> there are less than about 10,000 chords). &#4294967295;And just like English, you
> >> can't really have random sequences, or you'd just have noise.
>
> >> Still, 40bps intuitively feels a bit light, but not by that much. I
> >> expect that if Shannon said something like that, he was considering
> >> music to be more along the lines of a MIDI stream than a digitized
> >> sound file.
> > ...
>
> > Perhaps we could use spoken language as a guide to the auditory bidrate
> > we've evolved to process. As Shannon estimated, each letter in English
> > has only a couple bits of entropy, with normal reading speed (that
> > uses the auditory processing -- the reason you can't count the number
> > of words in a limerick without using your fingers) around 150 wpm.
> > Let's call that 25 bps.
>
> > 40 bps is then about the "square" (in log terms :) of our evolved language
> > processing speeds.
>
> > Maybe not such a bad estimate.
>
> > The same disparity is then seen between normal good voice compression
> > and the actually bitrate we operate on. I.e. 25 bps "back-end" processing
> > speed, yet voice compression being still 1000s of bps.
>
> > In some applications &#4294967295;over the years I've used speach-to-text and text-to-speach
> > over very low speed lines as a slap-dash compression method. With a bit
> > of a phoneme "nomenclator" it wasn't too bad. No doubt the idea is still in
> > use, but I don't think I've heard of it in preference to straight dsp methods.
>
> At the 1939 World Fair in New York City, Bell Telephone had an exhibit
> featuring the "vocoder". &#4294967295;An operator could synchronize speech by
> pressing "keys" on a console. &#4294967295;Bell thought the method could be used for
> "compressing" its telephone bandwidth requirements.
>
> As it turned out, bandwidth was not the limiting factor in future
> telephone growth. &#4294967295;Simple system changes like separating the low-speed
> "housekeeping" functions from the in-band "information" were implemented.

   The cranks that work in signal processing are bandwidth obessed
with all
   technology, so that's where digital signal processing, Turing
Machines, Lasers,
   Holograms, miniature atomic clocks, Integrated Circuits, XML,
Optical Networks,
   USB,  LEDS, Flash Memory,  Orbital Solar Energy, and post New York
Times
   ergonomics all came from.


> I was involved in the testing of microwave frequencies in small circular
> wave-guide to allow for much wider band channels. &#4294967295;The need never
> evolved, and fiber optics replaced microwaves where it was needed.
>
> As far as I know, any "compression" system requires some means of
> "storing" information so that it can be processed. &#4294967295;This was just being
> developed during Shannon's time. &#4294967295;It's obvious that "digital" storage is
> easier and more flexible than analog.
>
> --
> Virg Wall

On 11/4/2011 6:31 AM, Martin Brown wrote:
> On 04/11/2011 09:33, Peter Webb wrote:

   ...

> I suspect 40 bits/sec is still far too tight, but a midi stream using a
> high end reconstruction codec represents a pretty good example of what
> is possible by way of compression for *music* as opposed to voice or a
> random noise stream.
>
>> cannot be compressed on average at all. To get a smaller figure for
>> music, you have to define what subsets of sounds are music. Lots of luck.
>
> I think that may have been his intention although I don't actually
> recall seeing the 40 bit/s number originally attributed to him above.
>
> As I said if he did anything I think he was estimating the information
> content of music in the already concise form of an orchestral score.
>>
>> The real number probably lies somewhere between 178 kbps and 40 bps. The
>> actual number is the base 2 logarithm of the number of different 1
>> second sound bites that the ear can distuingish and would consider as
>> music. I doubt even Shannon would have known the answer to that equation.
>
> Maybe he didn't consider all the alternative atonal and continuous
> frequency synthesisers of arbitrary waveforms that are possible now - or
> perhaps it was a subtle dig at Stockhausen and Schoenberg as not music.

There's a case to be made that it really isn't music. Oompah-pah bands, 
Mozart, Bach, Ellington, that characteristic sound of an orchestra 
tuning up, African jungle drums, all have a characteristic fractal 
pattern also found in mountain profiles. Schoenberg, Stockhausen, and 
Webern have patterns that clearly differ. I don't have the reference any 
more. I hope someone can supply it.

   ...

Jerry
-- 
Engineering is the art of making what you want from things you can get.

On Nov 4, 12:50&#4294967295;pm, VWWall <vw...@large.invalid> wrote:
> k...@kymhorsell.com wrote:
> > In sci.math Robert Wessel <robertwess...@yahoo.com> wrote:
> > ...
> >> Consider that high keystroke rates pretty much require chords, and a
> >> chord is most certainly not three or four random keys, pressed at
> >> different times and forces - rather they're a small subset of the
> >> possible combinations of keys within a hand-span (or two hand-spans in
> >> some cases), hit at the same time, and with the same force (in fact
> >> there are less than about 10,000 chords). &#4294967295;And just like English, you
> >> can't really have random sequences, or you'd just have noise.
>
> >> Still, 40bps intuitively feels a bit light, but not by that much. I
> >> expect that if Shannon said something like that, he was considering
> >> music to be more along the lines of a MIDI stream than a digitized
> >> sound file.
> > ...
>
> > Perhaps we could use spoken language as a guide to the auditory bidrate
> > we've evolved to process. As Shannon estimated, each letter in English
> > has only a couple bits of entropy, with normal reading speed (that
> > uses the auditory processing -- the reason you can't count the number
> > of words in a limerick without using your fingers) around 150 wpm.
> > Let's call that 25 bps.
>
> > 40 bps is then about the "square" (in log terms :) of our evolved language
> > processing speeds.
>
> > Maybe not such a bad estimate.
>
> > The same disparity is then seen between normal good voice compression
> > and the actually bitrate we operate on. I.e. 25 bps "back-end" processing
> > speed, yet voice compression being still 1000s of bps.
>
> > In some applications &#4294967295;over the years I've used speach-to-text and text-to-speach
> > over very low speed lines as a slap-dash compression method. With a bit
> > of a phoneme "nomenclator" it wasn't too bad. No doubt the idea is still in
> > use, but I don't think I've heard of it in preference to straight dsp methods.
>
> At the 1939 World Fair in New York City, Bell Telephone had an exhibit
> featuring the "vocoder". &#4294967295;An operator could synchronize speech by
> pressing "keys" on a console. &#4294967295;Bell thought the method could be used for
> "compressing" its telephone bandwidth requirements.
>
> As it turned out, bandwidth was not the limiting factor in future
> telephone growth. &#4294967295;Simple system changes like separating the low-speed
> "housekeeping" functions from the in-band "information" were implemented.
>
> I was involved in the testing of microwave frequencies in small circular
> wave-guide to allow for much wider band channels. &#4294967295;The need never
> evolved, and fiber optics replaced microwaves where it was needed.
>
> As far as I know, any "compression" system requires some means of
> "storing" information so that it can be processed. &#4294967295;This was just being
> developed during Shannon's time. &#4294967295;It's obvious that "digital" storage is
> easier and more flexible than analog.

  But that's only really the first stage or filtering set of signal
processing itself.
  So, that's also why the people that actually know how the rest of
the stuff works
  started working on satellites, lasers, digital controllers, optical
networks,
  holographics, USB, digital TV, and nanotechnology.


> --
> Virg Wall

Richard Outerbridge wrote:
 
> The seminal introductory work for me was the Undergraduate text
> "Coding and Information Theory", by the late Richard W. Hamming,
> 1980, ISBN 0-13-139139-9 (perhaps one of the 1st books to have
> an ISBN number

Hardly, they date from 1970.  Also, it's "ISBN" not "ISBN number".

> ).  

-- 
When a true genius appears in the world, you may know him by 
this sign, that the dunces are all in confederacy against him.
Jonathan Swift: Thoughts on Various Subjects, Moral and Diverting

On Fri, 04 Nov 2011 20:41:09 +0000, Frederick Williams
<freddywilliams@btinternet.com> wrote:

>Richard Outerbridge wrote:
> 
>> The seminal introductory work for me was the Undergraduate text
>> "Coding and Information Theory", by the late Richard W. Hamming,
>> 1980, ISBN 0-13-139139-9 (perhaps one of the 1st books to have
>> an ISBN number
>
>Hardly, they date from 1970.  Also, it's "ISBN" not "ISBN number".

Tight-assed pedants hate "ATM machines", too, right? 

On Nov 4, 5:59&#4294967295;pm, "k...@att.bizzzzzzzzzzzz" <k...@att.bizzzzzzzzzzzz>
wrote:
> On Fri, 04 Nov 2011 20:41:09 +0000, Frederick Williams
>
> <freddywilli...@btinternet.com> wrote:
> >Richard Outerbridge wrote:
>
> >> The seminal introductory work for me was the Undergraduate text
> >> "Coding and Information Theory", by the late Richard W. Hamming,
> >> 1980, ISBN 0-13-139139-9 (perhaps one of the 1st books to have
> >> an ISBN number
>
> >Hardly, they date from 1970. &#4294967295;Also, it's "ISBN" not "ISBN number".
>
> Tight-assed pedants hate "ATM machines", too, right?

   They even hate serial numbers themselves.
   So, that's were laser beams. holograms, and post Qwerty computation
came from.

On 11/4/2011 5:59 PM, krw@att.bizzzzzzzzzzzz wrote:
> On Fri, 04 Nov 2011 20:41:09 +0000, Frederick Williams
> <freddywilliams@btinternet.com>  wrote:
>
>> Richard Outerbridge wrote:
>>
>>> The seminal introductory work for me was the Undergraduate text
>>> "Coding and Information Theory", by the late Richard W. Hamming,
>>> 1980, ISBN 0-13-139139-9 (perhaps one of the 1st books to have
>>> an ISBN number
>>
>> Hardly, they date from 1970.  Also, it's "ISBN" not "ISBN number".
>
> Tight-assed pedants hate "ATM machines", too, right?

And DC current.

Jerry
-- 
Engineering is the art of making what you want from things you can get.

In article <Y00tq.24192$vg7.8274@newsfe04.iad>,
Jerry Avins  <jya@ieee.org> wrote:
>On 11/4/2011 5:59 PM, krw@att.bizzzzzzzzzzzz wrote:
>> On Fri, 04 Nov 2011 20:41:09 +0000, Frederick Williams
>> <freddywilliams@btinternet.com>  wrote:
>>
>>> Richard Outerbridge wrote:
>>>
>>>> The seminal introductory work for me was the Undergraduate text
>>>> "Coding and Information Theory", by the late Richard W. Hamming,
>>>> 1980, ISBN 0-13-139139-9 (perhaps one of the 1st books to have
>>>> an ISBN number
>>>
>>> Hardly, they date from 1970.  Also, it's "ISBN" not "ISBN number".
>>
>> Tight-assed pedants hate "ATM machines", too, right?
>
>And DC current.

And SPARC computer, which when fully expanded is
Scalable Processor Architecture (Reduced
Instruction Set Computer) Computer computer.

Greg.
-- 

Jerry Avins wrote:
> 
> On 11/4/2011 5:59 PM, krw@att.bizzzzzzzzzzzz wrote:
> > On Fri, 04 Nov 2011 20:41:09 +0000, Frederick Williams
> > <freddywilliams@btinternet.com>  wrote:
> >
> >> Richard Outerbridge wrote:
> >>
> >>> The seminal introductory work for me was the Undergraduate text
> >>> "Coding and Information Theory", by the late Richard W. Hamming,
> >>> 1980, ISBN 0-13-139139-9 (perhaps one of the 1st books to have
> >>> an ISBN number
> >>
> >> Hardly, they date from 1970.  Also, it's "ISBN" not "ISBN number".
> >
> > Tight-assed pedants hate "ATM machines", too, right?
> 
> And DC current.

And, for a different reason, DC voltage.

-- 
When a true genius appears in the world, you may know him by 
this sign, that the dunces are all in confederacy against him.
Jonathan Swift: Thoughts on Various Subjects, Moral and Diverting