sparse coding..

Hello Forum,

what is sparse coding exactly? 

Does it mean that we choose a set of basis functions to represent a certain
signal and the superposition involves very few basis functions?

For instance, given the signal f(t), we could express it as a sum of
sinusoids or wavelets.
The sinusoidal expansion would involve many more terms while the wavelet
transform very few...

thanks,
Brett

Am 27.06.2012 18:16, schrieb brettcooper:
> Hello Forum,
>
> what is sparse coding exactly?
>
> Does it mean that we choose a set of basis functions to represent a certain
> signal and the superposition involves very few basis functions?

Well, that is a requirement for sparse coding, namely that the signal to 
be coded is sparse. To get such a signal, you typically first apply a 
transformation into another domain under which a signal becomes sparse. 
A wavelet transformation might do that, but this depends on the signal.

The reason *why* you want a sparse signal is that there is now a theorem 
(a generalization of the sampling theorem) that tells you that you can 
reconstruct the sparse signal even if you only pick relatively few 
samples of the spare signal "at random".

 > For instance, given the signal f(t), we could express it as a sum of
> sinusoids or wavelets.
> The sinusoidal expansion would involve many more terms while the wavelet
> transform very few...

Well, this rather depends on the source signal.

Greetings,
	Thomas

On Wed, 27 Jun 2012 19:34:48 +0200, Thomas Richter
<thor@math.tu-berlin.de> wrote:

>Am 27.06.2012 18:16, schrieb brettcooper:
>> Hello Forum,
>>
>> what is sparse coding exactly?
>>
>> Does it mean that we choose a set of basis functions to represent a certain
>> signal and the superposition involves very few basis functions?
>
>Well, that is a requirement for sparse coding, namely that the signal to 
>be coded is sparse. To get such a signal, you typically first apply a 
>transformation into another domain under which a signal becomes sparse. 
>A wavelet transformation might do that, but this depends on the signal.
>
>The reason *why* you want a sparse signal is that there is now a theorem 
>(a generalization of the sampling theorem) that tells you that you can 
>reconstruct the sparse signal even if you only pick relatively few 
>samples of the spare signal "at random".

Is this in the context of compressive sensing?

I've been trying to get my head 'round that for a bit in order to
understand the limitations.

>
> > For instance, given the signal f(t), we could express it as a sum of
>> sinusoids or wavelets.
>> The sinusoidal expansion would involve many more terms while the wavelet
>> transform very few...
>
>Well, this rather depends on the source signal.
>
>Greetings,
>	Thomas
>

Eric Jacobsen
Anchor Hill Communications
www.anchorhill.com

Am 27.06.2012 20:25, schrieb Eric Jacobsen:

>> The reason *why* you want a sparse signal is that there is now a theorem
>> (a generalization of the sampling theorem) that tells you that you can
>> reconstruct the sparse signal even if you only pick relatively few
>> samples of the spare signal "at random".
>
> Is this in the context of compressive sensing?

At least my answer is. I'm not sure whether the OP had compressive 
sensing in mind.

> I've been trying to get my head 'round that for a bit in order to
> understand the limitations.

It had been a hot topic during the last years, but I believe it is 
already dying away again as only some of the expectations people had it 
it actually worked out. So for example, it is not exactly a prime tool 
for image compression.

Greetings,
	Thomas