Forums

sparse coding..

Started by brettcooper June 27, 2012
Hello Forum,

what is sparse coding exactly? 

Does it mean that we choose a set of basis functions to represent a certain
signal and the superposition involves very few basis functions?

For instance, given the signal f(t), we could express it as a sum of
sinusoids or wavelets.
The sinusoidal expansion would involve many more terms while the wavelet
transform very few...

thanks,
Brett


Am 27.06.2012 18:16, schrieb brettcooper:
> Hello Forum, > > what is sparse coding exactly? > > Does it mean that we choose a set of basis functions to represent a certain > signal and the superposition involves very few basis functions?
Well, that is a requirement for sparse coding, namely that the signal to be coded is sparse. To get such a signal, you typically first apply a transformation into another domain under which a signal becomes sparse. A wavelet transformation might do that, but this depends on the signal. The reason *why* you want a sparse signal is that there is now a theorem (a generalization of the sampling theorem) that tells you that you can reconstruct the sparse signal even if you only pick relatively few samples of the spare signal "at random". > For instance, given the signal f(t), we could express it as a sum of
> sinusoids or wavelets. > The sinusoidal expansion would involve many more terms while the wavelet > transform very few...
Well, this rather depends on the source signal. Greetings, Thomas
On Wed, 27 Jun 2012 19:34:48 +0200, Thomas Richter
<thor@math.tu-berlin.de> wrote:

>Am 27.06.2012 18:16, schrieb brettcooper: >> Hello Forum, >> >> what is sparse coding exactly? >> >> Does it mean that we choose a set of basis functions to represent a certain >> signal and the superposition involves very few basis functions? > >Well, that is a requirement for sparse coding, namely that the signal to >be coded is sparse. To get such a signal, you typically first apply a >transformation into another domain under which a signal becomes sparse. >A wavelet transformation might do that, but this depends on the signal. > >The reason *why* you want a sparse signal is that there is now a theorem >(a generalization of the sampling theorem) that tells you that you can >reconstruct the sparse signal even if you only pick relatively few >samples of the spare signal "at random".
Is this in the context of compressive sensing? I've been trying to get my head 'round that for a bit in order to understand the limitations.
> > > For instance, given the signal f(t), we could express it as a sum of >> sinusoids or wavelets. >> The sinusoidal expansion would involve many more terms while the wavelet >> transform very few... > >Well, this rather depends on the source signal. > >Greetings, > Thomas >
Eric Jacobsen Anchor Hill Communications www.anchorhill.com
Am 27.06.2012 20:25, schrieb Eric Jacobsen:

>> The reason *why* you want a sparse signal is that there is now a theorem >> (a generalization of the sampling theorem) that tells you that you can >> reconstruct the sparse signal even if you only pick relatively few >> samples of the spare signal "at random". > > Is this in the context of compressive sensing?
At least my answer is. I'm not sure whether the OP had compressive sensing in mind.
> I've been trying to get my head 'round that for a bit in order to > understand the limitations.
It had been a hot topic during the last years, but I believe it is already dying away again as only some of the expectations people had it it actually worked out. So for example, it is not exactly a prime tool for image compression. Greetings, Thomas