"David Martinez" <macarena.dupouy@neuf.fr> wrote in message
news:chbsi0$olb$1@apollon.grec.isp.9tel.net...
> The entropy describes the information contained in a source, and is
measured
> in terms of the symbol alphabet.
>
> Ex: 3.15 bits/symbol or X (alphabet elements)/symbol.
>
> So if you want to know the entropy of a sampled waveform you need to
decide
> first your coding alphabet.
>
> For example if you consider 1024 levels and 1024 symbols you'll have.
>
> H{X} = {-1/log{1024}}* Sum {log{p(Xi=i)}*p{Xi=i}
>
> If you want a Real-Time Entropy you can use this aproximation:
>
> for i=1.N
> for j=1.Lenght(Buffer)
> Buffer[j] = input;
> end
> P = Histogram(Buffer)/Length(Buffer); // You can
> // calculate the histogram and divide by the number of
> // samples to get an approximation of P(Xi=i).
> H(i) = {-1/log{Length Buffer}}* Sum {log{P(k)}*P(k)
> end
>
> regards,
>
> David.
>
>
>
>
Reply by David Martinez●September 4, 20042004-09-04
">
> for i=1.N
> for j=1.Lenght(Buffer)
> Buffer[j] = input;
> end
> P = Histogram(Buffer,Length{P})/Length(Buffer); // You can
> // calculate the histogram and divide by the number of
> // samples to get an approximation of P(Xi=i).
> H(i) = {-1/log{Length Buffer}}* Sum {log{P(k)}*P(k)
> end
Sorry i meant H(i) = {-1/log{length(P)}}* Sum {log{P(k)}*P(k)}
Where P length is constant, you only changes the values contained in P
Reply by David Martinez●September 4, 20042004-09-04
The entropy describes the information contained in a source, and is measured
in terms of the symbol alphabet.
Ex: 3.15 bits/symbol or X (alphabet elements)/symbol.
So if you want to know the entropy of a sampled waveform you need to decide
first your coding alphabet.
For example if you consider 1024 levels and 1024 symbols you'll have.
H{X} = {-1/log{1024}}* Sum {log{p(Xi=i)}*p{Xi=i}
If you want a Real-Time Entropy you can use this aproximation:
for i=1.N
for j=1.Lenght(Buffer)
Buffer[j] = input;
end
P = Histogram(Buffer)/Length(Buffer); // You can
// calculate the histogram and divide by the number of
// samples to get an approximation of P(Xi=i).
H(i) = {-1/log{Length Buffer}}* Sum {log{P(k)}*P(k)
end
regards,
David.
Reply by Tom●September 3, 20042004-09-03
"One Usenet Poster" <me@my.computer.org> wrote in message
news:10jhcr4hs9sag86@corp.supernews.com...
> Randy Yates wrote:
>
> > Hold on a minute there, cowboy. Where in the tarheeled' nation
> > did you get that?
>
> "The Minimum Entropy Criterion of Range Alignment in ISAR Motion
> Compensation", Genyuan Wang and Zheng Bao, Radar 97, 16 Oct 1997, p.
> 237, equation (4).
>
> OUP
Thanks but that s eems to be for binary data - I meant a sampled waveform.DO
you suggest converting everything to binary first?
Thanks
Tom
Reply by One Usenet Poster●September 3, 20042004-09-03
Randy Yates wrote:
> Hold on a minute there, cowboy. Where in the tarheeled' nation
> did you get that?
"The Minimum Entropy Criterion of Range Alignment in ISAR Motion
Compensation", Genyuan Wang and Zheng Bao, Radar 97, 16 Oct 1997, p.
237, equation (4).
OUP
Reply by ●September 3, 20042004-09-03
One Usenet Poster <me@my.computer.org> writes:
> Tom wrote:
> > How do I calculate entropy of a sampled signal of lenght N samples?
> > I assume I need the PDF (probability density function) - can I use
> > the
>
> > Periodogram P(f) is this is like the histogram and use
> > H=-sum{=1 to N of } P(k)log(P(k))
>
> > does it have to be log base 2?
>
> > I assume if I take the entropy of white noise what should be?
>
> > Also I notice that log(P(f) are the cepstrum coefficients.
>
> > Tom
>
>
> The entropy of a sequence X = { x_1 x_2 x_3 ... x_N } is defined as
>
> H(X) = sum_1_to_N ( p_i log( p_i ) )
>
> where p_i = | x_i | / || X ||
>
> and || X || = sum_1_to_N ( | x_i | ).
Hold on a minute there, cowboy. Where in the tarheeled' nation
did you get that?
Nope. It oughta be
H(X) = -1 * sum_1_to_N ( p_i log( p_i ) ),
where p_i is the probability of symbol x_i occurring and log is
base 2.
--
Randy Yates
Sony Ericsson Mobile Communications
Research Triangle Park, NC, USA
randy.yates@sonyericsson.com, 919-472-1124
Reply by One Usenet Poster●September 3, 20042004-09-03
Tom wrote:
> How do I calculate entropy of a sampled signal of lenght N samples?
>
> I assume I need the PDF (probability density function) - can I use the
> Periodogram P(f) is this is like the histogram and use
>
> H=-sum{=1 to N of } P(k)log(P(k))
>
> does it have to be log base 2?
>
> I assume if I take the entropy of white noise what should be?
>
> Also I notice that log(P(f) are the cepstrum coefficients.
>
>
> Tom
The entropy of a sequence X = { x_1 x_2 x_3 ... x_N } is defined as
H(X) = sum_1_to_N ( p_i log( p_i ) )
where p_i = | x_i | / || X ||
and || X || = sum_1_to_N ( | x_i | ).
Good luck,
OUP
Reply by Tom●September 3, 20042004-09-03
How do I calculate entropy of a sampled signal of lenght N samples?
I assume I need the PDF (probability density function) - can I use the
Periodogram P(f) is this is like the histogram and use
H=-sum{=1 to N of } P(k)log(P(k))
does it have to be log base 2?
I assume if I take the entropy of white noise what should be?
Also I notice that log(P(f) are the cepstrum coefficients.
Tom