information theory?

On 01/11/2011 17:29, RichD wrote:
> On Nov 1, Chris<chris.sant...@gmail.com>  wrote:
>> Do you have a specific question? Or just want to lurk and learn?
>
> I'd like to lurk someplace, where there are discussions
> of current topics.  It's such a rich field, there are always
> new ideas.
>
>> Information Theory was the hardest class I took in grad school. So
>> abstract, and so much probability.... but it's a cool topic
>> nonetheless.
>
> I read Shannon's paper long go, and found it fascinating
> ever since.  In the most general sense, it's applicable
> to so many areas.  For instance, people have recast
> thermodynamics as information processing.  Or, you
> could model the entire universe as a computer, and
> information theory applies.  It even describes human
> nature - manipulating symbols is what separates us
> from the apes.
>
>>>> Are there any boards, or private blogs, dedicated
>>>> to discussion of information theory?

You don't really say which part of information theory you are interested 
in - but some of the more interesting modern developments stem from 
Bayes theorem and the principle of maximum entropy.

You could do worse than look for the collected works of Ed Jaynes - it 
used to be available online at one time but now it is published.

http://www.amazon.com/T-Jaynes-Probability-Statistics-Statistical/dp/0792302133

David MacKays book Information Theory, Inference and Learning might be a 
cheaper alternative.

Not an easy read and assumes higher mathematics as a starting point.

-- 
Regards,
Martin Brown

In sci.math Martin Brown <|||newspam|||@nezumi.demon.co.uk> wrote:
> [[snip]]
> You could do worse than look for the collected works of Ed Jaynes - it 
> used to be available online at one time but now it is published.
> 
> http://www.amazon.com/T-Jaynes-Probability-Statistics-Statistical/dp/0792302133

Just curious -- if it was freely available online at one time,
and I assume legally at that time, how does later publishing
a paper copy retroactively affect online legality? I'd have
assumed any originally-legal online info remains legal regardless
of who else prints it on paper. Is that wrong?
-- 
John Forkosh  ( mailto:  j@f.com  where j=john and f=forkosh )

On 02/11/2011 09:48, JohnF wrote:
> In sci.math Martin Brown<|||newspam|||@nezumi.demon.co.uk>  wrote:
>> [[snip]]
>> You could do worse than look for the collected works of Ed Jaynes - it
>> used to be available online at one time but now it is published.
>>
>> http://www.amazon.com/T-Jaynes-Probability-Statistics-Statistical/dp/0792302133
>
> Just curious -- if it was freely available online at one time,
> and I assume legally at that time, how does later publishing
> a paper copy retroactively affect online legality? I'd have
> assumed any originally-legal online info remains legal regardless
> of who else prints it on paper. Is that wrong?

I don't know. ISTR it was online as a part of a global proofing effort. 
When that was finished it looks like the online copy was removed.

(at least if it is still there I can't see it now)

It was at: http://bayes.wustl.edu/etj/prob.html

Still linked to from Bayesian resources at Cornell:

http://www.astro.cornell.edu/staff/loredo/bayes/


-- 
Regards,
Martin Brown

Be careful going down this path. For me it was much more interesting
in theory than in practice.

Me in 2008: "Like, what *is* the nature of information, man?" (waves
hand to clear smoke from the room)

Then I signed up for Info Theory, and I spent a whole semester taking
integrals of logs of probability distributions and proving that one
was bounded by the other.


On Nov 1, 1:29=A0pm, RichD <r_delaney2...@yahoo.com> wrote:
> On Nov 1, Chris <chris.sant...@gmail.com> wrote:
>
> > Do you have a specific question? Or just want to lurk and learn?
>
> I'd like to lurk someplace, where there are discussions
> of current topics. =A0It's such a rich field, there are always
> new ideas.
>
> > Information Theory was the hardest class I took in grad school. So
> > abstract, and so much probability.... but it's a cool topic
> > nonetheless.
>
> I read Shannon's paper long go, and found it fascinating
> ever since. =A0In the most general sense, it's applicable
> to so many areas. =A0For instance, people have recast
> thermodynamics as information processing. =A0Or, you
> could model the entire universe as a computer, and
> information theory applies. =A0It even describes human
> nature - manipulating symbols is what separates us
> from the apes.
>
> > > >Are there any boards, or private blogs, dedicated
> > > >to discussion of information theory?
>
> --
> Rich

On Nov 2, 11:02=A0am, Chris <chris.sant...@gmail.com> wrote:
> Be careful going down this path. For me it was much more interesting
> in theory than in practice.
>
> Me in 2008: "Like, what *is* the nature of information, man?" (waves
> hand to clear smoke from the room)
>
> Then I signed up for Info Theory, and I spent a whole semester taking
> integrals of logs of probability distributions and proving that one
> was bounded by the other.
>

that (bounded integrals of logs of probability distributions) didn't
clear the room of the smoke?

r b-j

On 02/11/2011 10:02, Martin Brown wrote:
> On 02/11/2011 09:48, JohnF wrote:
>> In sci.math Martin Brown<|||newspam|||@nezumi.demon.co.uk> wrote:
>>> [[snip]]
>>> You could do worse than look for the collected works of Ed Jaynes - it
>>> used to be available online at one time but now it is published.
>>>
>>> http://www.amazon.com/T-Jaynes-Probability-Statistics-Statistical/dp/0792302133
>>>
>>
>> Just curious -- if it was freely available online at one time,
>> and I assume legally at that time, how does later publishing
>> a paper copy retroactively affect online legality? I'd have
>> assumed any originally-legal online info remains legal regardless
>> of who else prints it on paper. Is that wrong?
>
> I don't know. ISTR it was online as a part of a global proofing effort.
> When that was finished it looks like the online copy was removed.
>
> (at least if it is still there I can't see it now)
>
> It was at: http://bayes.wustl.edu/etj/prob.html
>
> Still linked to from Bayesian resources at Cornell:
>
> http://www.astro.cornell.edu/staff/loredo/bayes/

I know it is not usually done to follow up your own post, but in this 
case it is to say that the second link on Cornell's site is to a 
previous draft version of a similar Jaynes compendium works book.

http://omega.albany.edu:8008/JaynesBook.html


-- 
Regards,
Martin Brown

On 2/11/2011 7:48 PM, JohnF wrote:
> In sci.math Martin Brown<|||newspam|||@nezumi.demon.co.uk>  wrote:
>> [[snip]]
>> You could do worse than look for the collected works of Ed Jaynes - it
>> used to be available online at one time but now it is published.
>>
>> http://www.amazon.com/T-Jaynes-Probability-Statistics-Statistical/dp/0792302133
>
> Just curious -- if it was freely available online at one time,
> and I assume legally at that time, how does later publishing
> a paper copy retroactively affect online legality? I'd have
> assumed any originally-legal online info remains legal regardless
> of who else prints it on paper. Is that wrong?

It depends solely on the copyright status. How a work is distributed is 
entirely at the discretion of the lawful owner of the copyright owner He 
can for example offer a work free on-line for a a period of time and 
then rescind that offer and the work is no longer available on-line for 
free.

If the copyright owner surrenders their copyright on a piece of work, 
for example by stating the piece of work is in the public domain, they 
then cannot re-instate their copyright in anyway and the work remains 
freely distributable in the public domain.

On Nov 2, Martin Brown <|||newspam...@nezumi.demon.co.uk> wrote:
> >> Do you have a specific question? Or just want
> >> to lurk and learn?
>
> >I'd like to lurk someplace, where there are
> >discussions of current topics. &#4294967295;It's such a
> >rich field, there are always new ideas.
>
> >I read Shannon's paper long go, and found it fascinating
> >ever since. &#4294967295;In the most general sense, it's
> >applicable to so many areas.
>
> >>>> Are there any boards, or private blogs, dedicated
> >>>> to discussion of information theory?
>
> You don't really say which part of information
> theory you are interested in -

Coding, antenna arrays, compression. etc.

> You could do worse than look for the collected works
> of Ed Jaynes -

He's the one who published the original work
re-interpreting thermodynamics as information theory.

It was ingenious, but made no new predictions,
so the physics community yawned and said,
what's the point?

> it used to be available online at one time but
> now it is published.
>
> http://www.amazon.com/T-Jaynes-Probability-Statistics-Statistical/dp/...

I'm not looking for textbooks, but a place
to hang out with experts, discussing the
latest stuff.  Like the faculty lounge -

--
Rich

On Nov 2, Chris <chris.sant...@gmail.com> wrote:
> Be careful going down this path. For me it was
> much more interesting in theory than in practice.
>
> Me in 2008: "Like, what *is* the nature of information,
> man?" (waves hand to clear smoke from the room)
>
> Then I signed up for Info Theory, and I spent
> a whole semester taking integrals of logs of
> probability distributions and proving that one
> was bounded by the other.

It's too bad you got so little out of it.
It's not 'just a theory', it's a field
where theory and practice coincide.
The theorists concoct the recipes, the
engineers follow the instructions, and
it works, bang on.  Fukengrooven.
And it's the root of many a Silly Con
Valley fortune.

I strongly suggest, if you have some
free neurons and time and nothing's good on
tv, read Shannon's original paper.  It's
utterly fucking brilliant.  There's no doubt,
in Valhalla, Shannon shares a table with Einstein.

It's sort of funny, he called it communication
theory, it was his disciples who popularized
'information theory'.

--
Rich

["Followup-To:" header set to sci.crypt.]
On 2011-11-02, David Eather <eather@tpg.com.au> wrote:
> On 2/11/2011 7:48 PM, JohnF wrote:
>> In sci.math Martin Brown<|||newspam|||@nezumi.demon.co.uk>  wrote:
>>> [[snip]]
>>> You could do worse than look for the collected works of Ed Jaynes - it
>>> used to be available online at one time but now it is published.
>>>
>>> http://www.amazon.com/T-Jaynes-Probability-Statistics-Statistical/dp/0792302133
>>
>> Just curious -- if it was freely available online at one time,
>> and I assume legally at that time, how does later publishing
>> a paper copy retroactively affect online legality? I'd have
>> assumed any originally-legal online info remains legal regardless
>> of who else prints it on paper. Is that wrong?
>
> It depends solely on the copyright status. How a work is distributed is 
> entirely at the discretion of the lawful owner of the copyright owner He 
> can for example offer a work free on-line for a a period of time and 
> then rescind that offer and the work is no longer available on-line for 
> free.
>
> If the copyright owner surrenders their copyright on a piece of work, 
> for example by stating the piece of work is in the public domain, they 
> then cannot re-instate their copyright in anyway and the work remains 
> freely distributable in the public domain.

Well, no. They perhaps cannot sue someone who received that work under
the old license (depending on the terms of that license), but they can
certainly stop supplying that work themselves and issuing another
license for anything obtained from them. 

I believe "public domain" is a state, not a license. Something is in the
public domain if it satisfies certain requirements as listed under
copyright law. Eg, the work is more than x years old, where x is defined
in the law. Or was produced by the government, etc. It is not a license.
You can issue a license which renounces your claims to copyright but
unless you make that irrevokable, you can probably reclain your rights
under copyright in that same work thereafter (just not with respect
to copies people made under the old license.). Thus, I believe that I
could state today that I relinquish the copyright in this work, and
tomorrow could reinstate my claims under copyright. I just could not go
after people who in good faith made copies today. 


>
>