Subband Spectral Entropy

On Feb 16, 10:18&#4294967295;am, "DWT" <zwfilte...@yahoo.com> wrote:
> >I did not notice that you said "calculate the pdf" in your step one. &#4294967295;
> >You need to calculate the power spectral density (PSD). &#4294967295;Let's start
> >from the top...
>
> >wi,k is kth sample of the ith subband of the wavelet transform of an
> >input signal xj.
>
> >Then define the DFT of wi,k as
>
> >Wi,f=dft(wi,k)
>
> >1) Calculate normalized power spectrum as
> >Qi,f=|Wi,f|^2/|sum(Wi,f)|^2
>
> >(#Qi's = # subbands)
>
> >2) Transform with the Shannon function:
> >Hi,f=Qi,f[log(1/Qi,f)]
>
> >3) Wavelet spectral entropy:
> >Ei=sum(Hi,f)/log(Ni) (where Ni is the number of frequency components in
> >the ith subband)
>
> >HTH,
>
> >KP
>
> Hi Ken,
>
> Sorry for the silence.
>
> Thanks a lot, but is there any way to avoid using FFT?

Not if you want the 'spectral entropy on each subband'. At least, not
with the most obvious interpretation of that phrase. Then Ken's answer
is correct, except for the question marks I noted in my reply to him.

illywhacker;

>Not if you want the 'spectral entropy on each subband'. At least, not
>with the most obvious interpretation of that phrase. Then Ken's answer
>is correct, except for the question marks I noted in my reply to him.
>
>illywhacker;
>

Hi,

Would it be correct to use energy-entropy then (not for the spectral
entropy, just for the entropy calculation)?

1) X=E/sum(E);
2) Ent=-sum(X*log(X));

Thanks

On Feb 16, 12:12&#4294967295;pm, "DWT" <zwfilte...@yahoo.com> wrote:
> >Not if you want the 'spectral entropy on each subband'. At least, not
> >with the most obvious interpretation of that phrase. Then Ken's answer
> >is correct, except for the question marks I noted in my reply to him.
>
> >illywhacker;
>
> Hi,
>
> Would it be correct to use energy-entropy then (not for the spectral
> entropy, just for the entropy calculation)?
>
> 1) X=E/sum(E);
> 2) Ent=-sum(X*log(X));

You need to be more careful in your notation, for two reasons. If you
wish to communicate with others, they need to know precisely what you
mean. It is counter-productive and not very respectful to make them
guess or ask you. More importantly, if you do not use good notation,
you yourself will not understand what you mean. You will make
conceptual and calculational errors.

So, what is E?

illywhacker;

>You need to be more careful in your notation, for two reasons. If you
>wish to communicate with others, they need to know precisely what you
>mean. It is counter-productive and not very respectful to make them
>guess or ask you. More importantly, if you do not use good notation,
>you yourself will not understand what you mean. You will make
>conceptual and calculational errors.
>
>So, what is E?
>
>illywhacker;
>

My warmest apologies.

1) Calculating subband energy:
E=sum(|Wij|.^2),
where Wij - k-th coefficients on the current decomposition level i.

2) Energy-entropy:
Entr=-sum(E*log(E))

I know that it's not the spectral entropy.

I'll use Ken's method as well, but I'm also looking for the quick methods.

On Feb 16, 12:37&#4294967295;pm, "DWT" <zwfilte...@yahoo.com> wrote:
> >You need to be more careful in your notation, for two reasons. If you
> >wish to communicate with others, they need to know precisely what you
> >mean. It is counter-productive and not very respectful to make them
> >guess or ask you. More importantly, if you do not use good notation,
> >you yourself will not understand what you mean. You will make
> >conceptual and calculational errors.
>
> >So, what is E?
>
> >illywhacker;
>
> My warmest apologies.
>
> 1) Calculating subband energy:
> E=sum(|Wij|.^2),
> where Wij - k-th coefficients on the current decomposition level i.
>
> 2) Energy-entropy:
> Entr=-sum(E*log(E))
>
> I know that it's not the spectral entropy.
>
> I'll use Ken's method as well, but I'm also looking for the quick methods.

Sum over what?

illywhacker;

>> My warmest apologies.
>>
>> 1) Calculating subband energy:
>> E=3Dsum(|Wij|.^2),
>> where Wij - k-th coefficients on the current decomposition level i.
>>
>> 2) Energy-entropy:
>> Entr=3D-sum(E*log(E))
>>
>> I know that it's not the spectral entropy.
>>
>> I'll use Ken's method as well, but I'm also looking for the quick
methods=
>.
>
>Sum over what?
>
>illywhacker;
>

Oops, I mistyped. Here's the revised algo:
W(i,j) - wavelet coefficients (WC), where i - decomposition level.

On each level 'i':
1) Getting each coefficient's energy:
E(i,j)=W(i,j).^2; 

2) Normalize this energy:
NE(i,j)=E(i,j)./sum(E(i,j)); [sum over j]

3) Normalized Shannon Entropy:
Entr(i)=-sum(NE(i,j)*log(E(i,j))) [sum over j]

Is this a viable method for entropy calculation? 

In article 
<78ce1f4b-f0f9-4891-af34-67162dc1a9e1@p20g2000yqi.googlegroups.com>,
 illywhacker <illywacker@gmail.com> wrote:

> On Feb 13, 3:30&#4294967295;pm, Ken Prager <pra...@ieee.org> wrote:
> > In article <Nu6dnQz_c8zhYw7UnZ2dnUVZ_qPin...@giganews.com>,
> >
> >
> >
> > &#4294967295;"DWT" <zwfilte...@yahoo.com> wrote:
> > > >Yes.
> >
> > > >KP
> >
> > > Hello Ken,
> >
> > > Could you please check it out:
> > >http://img10.imageshack.us/img10/4664/49476386kk8.jpg
> >
> > > Why did I get such a result?
> >
> > > This initial signal is divided into the frames (512 samples + 50%
> > > overlap). Applying DWT for each frame (3 decomposition levels), 
> > > calculating
> > > the entropy on each subband:
> >
> > > Part of Matlab code:
> > > % +1 is required to calculate the entropy for the approximation
> > > coefficients.
> > > % E.g. 3 levels leads to 4 coefficient sets: 3 details (HP filter)
> > > % and 1 approximation (LP filter)
> >
> > > for j=1:decomp_levels+1
> > > ZZZ=length(x);
> > > pdf=(abs(x).^2)./sum(abs(x).^2));
> > > h=pdf.*log2(1./pdf);
> > > ent(j)=sum(h)./log2(ZZZ);
> > > end;
> >
> > > Then I'll just sum(ent) to get the entropy of the current frame.
> >
> > > You suggested to use levels count in the equation [3]. Am I doing right
> > > using the coefficients per each sub-band instead?
> >
> > I did not notice that you said "calculate the pdf" in your step one. &#4294967295;
> > You need to calculate the power spectral density (PSD). &#4294967295;Let's start
> > from the top...
> >
> > wi,k is kth sample of the ith subband of the wavelet transform of an
> > input signal xj.
> >
> > Then define the DFT of wi,k as
> >
> > Wi,f=dft(wi,k)
> >
> > 1) Calculate normalized power spectrum as
> > Qi,f=|Wi,f|^2/|sum(Wi,f)|^2
> >
> > (#Qi's = # subbands)
> >
> > 2) Transform with the Shannon function:
> > Hi,f=Qi,f[log(1/Qi,f)]
> >
> > 3) Wavelet spectral entropy:
> > Ei=sum(Hi,f)/log(Ni) (where Ni is the number of frequency components in
> > the ith subband)
> 
> Are you sure about this? It does not look quite right.
> 
> 1) Surely you should divide by
> 
> sum |Wi,f|^{2}
> 
> not by
> 
> |sum Wi,f|^{2}
> 
> which might be zero?

Yes, you are correct.  Typo by me.

Also, each Qi,f is now normalized for so that the sum of all the 
frequency components, per subband, is one.

> 
> 3) Why do you need to divide by log(Ni) when Qi,f is already
> normalized as a function of f?

Additional normalization will make Ei range between 0 and 1.

Ken P.

In article <KPWdnRUKSb28VQTUnZ2dnUVZ_vqdnZ2d@giganews.com>,
 "DWT" <zwfilter09@yahoo.com> wrote:

> >> My warmest apologies.
> >>
> >> 1) Calculating subband energy:
> >> E=3Dsum(|Wij|.^2),
> >> where Wij - k-th coefficients on the current decomposition level i.
> >>
> >> 2) Energy-entropy:
> >> Entr=3D-sum(E*log(E))
> >>
> >> I know that it's not the spectral entropy.
> >>
> >> I'll use Ken's method as well, but I'm also looking for the quick
> methods=
> >.
> >
> >Sum over what?
> >
> >illywhacker;
> >
> 
> Oops, I mistyped. Here's the revised algo:
> W(i,j) - wavelet coefficients (WC), where i - decomposition level.
> 
> On each level 'i':
> 1) Getting each coefficient's energy:
> E(i,j)=W(i,j).^2; 
> 
> 2) Normalize this energy:
> NE(i,j)=E(i,j)./sum(E(i,j)); [sum over j]
> 
> 3) Normalized Shannon Entropy:
> Entr(i)=-sum(NE(i,j)*log(E(i,j))) [sum over j]
> 
> Is this a viable method for entropy calculation? 

If I follow your math, you've now calculated subband entropy instead of 
subband spectral entropy.  It might work, depending on what you are 
planning to use it for.  You might calculate both for a few different 
signals and compare how they track.  One method may be better than the 
other for your application.

Have fun,

Ken P.

On Feb 17, 12:17&#4294967295;am, Ken Prager <pra...@ieee.org> wrote:
> In article
> <78ce1f4b-f0f9-4891-af34-67162dc1a...@p20g2000yqi.googlegroups.com>,
>
>
>
> &#4294967295;illywhacker <illywac...@gmail.com> wrote:
> > On Feb 13, 3:30&#4294967295;pm, Ken Prager <pra...@ieee.org> wrote:
> > > In article <Nu6dnQz_c8zhYw7UnZ2dnUVZ_qPin...@giganews.com>,
>
> > > &#4294967295;"DWT" <zwfilte...@yahoo.com> wrote:
> > > > >Yes.
>
> > > > >KP
>
> > > > Hello Ken,
>
> > > > Could you please check it out:
> > > >http://img10.imageshack.us/img10/4664/49476386kk8.jpg
>
> > > > Why did I get such a result?
>
> > > > This initial signal is divided into the frames (512 samples + 50%
> > > > overlap). Applying DWT for each frame (3 decomposition levels),
> > > > calculating
> > > > the entropy on each subband:
>
> > > > Part of Matlab code:
> > > > % +1 is required to calculate the entropy for the approximation
> > > > coefficients.
> > > > % E.g. 3 levels leads to 4 coefficient sets: 3 details (HP filter)
> > > > % and 1 approximation (LP filter)
>
> > > > for j=1:decomp_levels+1
> > > > ZZZ=length(x);
> > > > pdf=(abs(x).^2)./sum(abs(x).^2));
> > > > h=pdf.*log2(1./pdf);
> > > > ent(j)=sum(h)./log2(ZZZ);
> > > > end;
>
> > > > Then I'll just sum(ent) to get the entropy of the current frame.
>
> > > > You suggested to use levels count in the equation [3]. Am I doing right
> > > > using the coefficients per each sub-band instead?
>
> > > I did not notice that you said "calculate the pdf" in your step one. &#4294967295;
> > > You need to calculate the power spectral density (PSD). &#4294967295;Let's start
> > > from the top...
>
> > > wi,k is kth sample of the ith subband of the wavelet transform of an
> > > input signal xj.
>
> > > Then define the DFT of wi,k as
>
> > > Wi,f=dft(wi,k)
>
> > > 1) Calculate normalized power spectrum as
> > > Qi,f=|Wi,f|^2/|sum(Wi,f)|^2
>
> > > (#Qi's = # subbands)
>
> > > 2) Transform with the Shannon function:
> > > Hi,f=Qi,f[log(1/Qi,f)]
>
> > > 3) Wavelet spectral entropy:
> > > Ei=sum(Hi,f)/log(Ni) (where Ni is the number of frequency components in
> > > the ith subband)
>
> > Are you sure about this? It does not look quite right.
>
> > 1) Surely you should divide by
>
> > sum |Wi,f|^{2}
>
> > not by
>
> > |sum Wi,f|^{2}
>
> > which might be zero?
>
> Yes, you are correct. &#4294967295;Typo by me.
>
> Also, each Qi,f is now normalized for so that the sum of all the
> frequency components, per subband, is one.
>
>
>
> > 3) Why do you need to divide by log(Ni) when Qi,f is already
> > normalized as a function of f?
>
> Additional normalization will make Ei range between 0 and 1.

Is that good?

illywhacker;

On Feb 16, 9:10&#4294967295;pm, "DWT" <zwfilte...@yahoo.com> wrote:
> >> My warmest apologies.
>
> >> 1) Calculating subband energy:
> >> E=3Dsum(|Wij|.^2),
> >> where Wij - k-th coefficients on the current decomposition level i.
>
> >> 2) Energy-entropy:
> >> Entr=3D-sum(E*log(E))
>
> >> I know that it's not the spectral entropy.
>
> >> I'll use Ken's method as well, but I'm also looking for the quick
> methods=
> >.
>
> >Sum over what?
>
> >illywhacker;
>
> Oops, I mistyped. Here's the revised algo:
> W(i,j) - wavelet coefficients (WC), where i - decomposition level.
>
> On each level 'i':
> 1) Getting each coefficient's energy:
> E(i,j)=W(i,j).^2;
>
> 2) Normalize this energy:
> NE(i,j)=E(i,j)./sum(E(i,j)); [sum over j]
>
> 3) Normalized Shannon Entropy:
> Entr(i)=-sum(NE(i,j)*log(E(i,j))) [sum over j]
>
> Is this a viable method for entropy calculation?

The situation is exactly as Ken expresses it. There are two different
notions: the entropy of the normalized wavelet coefficient energies,
and the entropy of the normalized wavelet Fourier coefficient
energies. These are not the same. Indeed you can choose any basis you
like and you will have a different answer.

What are you trying to do with this?

illywhacker;