approximate to sum of gaussians

Hi all,
I have few sampled data (positive integer values), and I want to
approximate such data to sum of few gaussians.
As first step I removed low frequencies (filtering all the values
greater then the mean), and I obtained something that actually looks
like a sum of few gaussians. But now I'm looking forward to a method
to retrieve automatically mean and variance of each gaussian.

If it can help to understand the idea, here a plot of the data:
http://traversata.altervista.org/img_samples/sampledData.jpg

and here the data after the high pass filter:
http://traversata.altervista.org/img_samples/highpass.jpg

I think the curve it can be easily approximated to the sum of 4
gaussians, using some trick... but I don't have any clue!
any suggestion would be strong appreciated.

cheers,
Mauro

On Apr 17, 9:03 pm, mauro <mauro.austra...@gmail.com> wrote:

> I think the curve it can be easily approximated to the sum of 4
> gaussians, using some trick... but I don't have any clue!
> any suggestion would be strong appreciated.

got it for the \mu of guassian, first derivatives equals to zero. it
works.
I was mainly interested to the \mu, but if you have any idea of how to
do the same in the frequency domain, I would be curious to see how to
do that :-)

cheers,
Mauro

On Apr 17, 8:46&#4294967295;am, mauro <mauro.austra...@gmail.com> wrote:
> On Apr 17, 9:03 pm, mauro <mauro.austra...@gmail.com> wrote:
>
> > I think the curve it can be easily approximated to the sum of 4
> > gaussians, using some trick... but I don't have any clue!
> > any suggestion would be strong appreciated.
>
> got it for the \mu of guassian, first derivatives equals to zero. it
> works.
> I was mainly interested to the \mu, but if you have any idea of how to
> do the same in the frequency domain, I would be curious to see how to
> do that :-)
>
> cheers,
> Mauro

Hello Mauro,

The Gaussian (Normal Distribution) is special (yes there is an
exception- Cauchy) in that a sum of Normals is Normal. And the Fourier
transform of a Gaussian is also Gaussian. So the freq. domain still
has you working with Gaussians. Neat! So whether you are convolving,
adding, or multiplying - its all about the Gaussians.

Clay

On Apr 17, 10:10�am, c...@claysturner.com wrote:
> On Apr 17, 8:46&#4294967295;am, mauro <mauro.austra...@gmail.com> wrote:
>
> > On Apr 17, 9:03 pm, mauro <mauro.austra...@gmail.com> wrote:
>
> > > I think the curve it can be easily approximated to the sum of 4
> > > gaussians, using some trick... but I don't have any clue!
> > > any suggestion would be strong appreciated.
>
> > got it for the \mu of guassian, first derivatives equals to zero. it
> > works.
> > I was mainly interested to the \mu, but if you have any idea of how to
> > do the same in the frequency domain, I would be curious to see how to
> > do that :-)
>
> > cheers,
> > Mauro
>
> Hello Mauro,
>
> The Gaussian (Normal Distribution) is special (yes there is an
> exception- Cauchy) in that a sum of Normals is Normal. And the Fourier
> transform of a Gaussian is also Gaussian. So the freq. domain still
> has you working with Gaussians. Neat! So whether you are convolving,
> adding, or multiplying - its all about the Gaussians.
>
> Clay

Clay,

It seems that a Gaussian of a given mean and variance could come from
the convolution of an infinite number of pairs (or larger groups) of
other Gaussians, so the OPs desire to find out what the original
Gaussian components were that summed to a given Gaussian, without some
known relation between the component Gaussians,  would be fruitless.
No?

Dirk

On Apr 17, 11:04&#4294967295;am, dbell <bellda2...@cox.net> wrote:
> On Apr 17, 10:10&#4294967295;am, c...@claysturner.com wrote:
>
>
>
>
>
> > On Apr 17, 8:46&#4294967295;am, mauro <mauro.austra...@gmail.com> wrote:
>
> > > On Apr 17, 9:03 pm, mauro <mauro.austra...@gmail.com> wrote:
>
> > > > I think the curve it can be easily approximated to the sum of 4
> > > > gaussians, using some trick... but I don't have any clue!
> > > > any suggestion would be strong appreciated.
>
> > > got it for the \mu of guassian, first derivatives equals to zero. it
> > > works.
> > > I was mainly interested to the \mu, but if you have any idea of how to
> > > do the same in the frequency domain, I would be curious to see how to
> > > do that :-)
>
> > > cheers,
> > > Mauro
>
> > Hello Mauro,
>
> > The Gaussian (Normal Distribution) is special (yes there is an
> > exception- Cauchy) in that a sum of Normals is Normal. And the Fourier
> > transform of a Gaussian is also Gaussian. So the freq. domain still
> > has you working with Gaussians. Neat! So whether you are convolving,
> > adding, or multiplying - its all about the Gaussians.
>
> > Clay
>
> Clay,
>
> It seems that a Gaussian of a given mean and variance could come from
> the convolution of an infinite number of pairs (or larger groups) of
> other Gaussians, so the OPs desire to find out what the original
> Gaussian components were that summed to a given Gaussian, without some
> known relation between the component Gaussians, &#4294967295;would be fruitless.
> No?
>
> Dirk- Hide quoted text -
>
> - Show quoted text -

Dirk,

In general yes, but what if each of the 4 gaussians were identical? As
usual with questions here we are often lacking info that can sometimes
really simplify the problem. But if he is adding RVs from sources with
unique and unknown means and variances, he is in trouble, but since
nonnegative variances add together, the resulting variance sets an
upper bound on the individual variances.

Clay

On 2008-04-17 15:20:28 -0300, clay@claysturner.com said:

> On Apr 17, 11:04&#4294967295;am, dbell <bellda2...@cox.net> wrote:
>> On Apr 17, 10:10&#4294967295;am, c...@claysturner.com wrote:
>> 
>> 
>> 
>> 
>> 
>>> On Apr 17, 8:46&#4294967295;am, mauro <mauro.austra...@gmail.com> wrote:
>> 
>>>> On Apr 17, 9:03 pm, mauro <mauro.austra...@gmail.com> wrote:
>> 
>>>>> I think the curve it can be easily approximated to the sum of 4
>>>>> gaussians, using some trick... but I don't have any clue!
>>>>> any suggestion would be strong appreciated.
>> 
>>>> got it for the \mu of guassian, first derivatives equals to zero. it
>>>> works.
>>>> I was mainly interested to the \mu, but if you have any idea of how to
> 
>>>> do the same in the frequency domain, I would be curious to see how to
>>>> do that :-)
>> 
>>>> cheers,
>>>> Mauro
>> 
>>> Hello Mauro,
>> 
>>> The Gaussian (Normal Distribution) is special (yes there is an
>>> exception- Cauchy) in that a sum of Normals is Normal. And the Fourier
>>> transform of a Gaussian is also Gaussian. So the freq. domain still
>>> has you working with Gaussians. Neat! So whether you are convolving,
>>> adding, or multiplying - its all about the Gaussians.
>> 
>>> Clay
>> 
>> Clay,
>> 
>> It seems that a Gaussian of a given mean and variance could come from
>> the convolution of an infinite number of pairs (or larger groups) of
>> other Gaussians, so the OPs desire to find out what the original
>> Gaussian components were that summed to a given Gaussian, without some
>> known relation between the component Gaussians, &#4294967295;would be fruitless.
>> No?
>> 
>> Dirk- Hide quoted text -
>> 
>> - Show quoted text -
> 
> Dirk,
> 
> In general yes, but what if each of the 4 gaussians were identical? As
> usual with questions here we are often lacking info that can sometimes
> really simplify the problem. But if he is adding RVs from sources with
> unique and unknown means and variances, he is in trouble, but since
> nonnegative variances add together, the resulting variance sets an
> upper bound on the individual variances.
> 
> Clay

Adding two Gaussian RVs is not the same as mixing two Gaussian RVs.

Mixing RVs adds the distributions but adding the RVs convolves the
distributions.

When you have a mixture it means that there is some probability of
one and the suitable probability of the other. The mixture of two Gaussians
is not another Gaussian. Think of an equal mixture of two Gaussians that
have means +10 and -10 with both having a variance of 1. The distribution
function of the result has two widely separated humps.

The original problem was to fit a mixture of four components.

mauro wrote:

> I have few sampled data (positive integer values), and I want to
> approximate such data to sum of few gaussians.
> As first step I removed low frequencies (filtering all the values
> greater then the mean), and I obtained something that actually looks
> like a sum of few gaussians. But now I'm looking forward to a method
> to retrieve automatically mean and variance of each gaussian.

This usually works fine as a non-linear least squares problem
using Newton's method.  One must give an initial guess, the closer
the better.

I used to do this as a sum of three Gaussians and a polynomial
(for what you call the low frequencies, but we called background).

Since Gaussians aren't independent the initial guess is
important.  I used to do it with skewed Gaussians so four
parameters each:  Amplitude, Position, Width (FWHM), and skew,
analyzing optical absorption and emission spectra.

There are Newton's method routines in Numerical Recipes.  Many
require in addition to the appropriate function that one
compute the derivative of the function in each variable
(parameter).

-- glen

On Apr 17, 3:35&#4294967295;pm, Gordon Sande <g.sa...@worldnet.att.net> wrote:
> On 2008-04-17 15:20:28 -0300, c...@claysturner.com said:
>
>
>
>
>
> > On Apr 17, 11:04&#4294967295;am, dbell <bellda2...@cox.net> wrote:
> >> On Apr 17, 10:10&#4294967295;am, c...@claysturner.com wrote:
>
> >>> On Apr 17, 8:46&#4294967295;am, mauro <mauro.austra...@gmail.com> wrote:
>
> >>>> On Apr 17, 9:03 pm, mauro <mauro.austra...@gmail.com> wrote:
>
> >>>>> I think the curve it can be easily approximated to the sum of 4
> >>>>> gaussians, using some trick... but I don't have any clue!
> >>>>> any suggestion would be strong appreciated.
>
> >>>> got it for the \mu of guassian, first derivatives equals to zero. it
> >>>> works.
> >>>> I was mainly interested to the \mu, but if you have any idea of how to
>
> >>>> do the same in the frequency domain, I would be curious to see how to
> >>>> do that :-)
>
> >>>> cheers,
> >>>> Mauro
>
> >>> Hello Mauro,
>
> >>> The Gaussian (Normal Distribution) is special (yes there is an
> >>> exception- Cauchy) in that a sum of Normals is Normal. And the Fourier
> >>> transform of a Gaussian is also Gaussian. So the freq. domain still
> >>> has you working with Gaussians. Neat! So whether you are convolving,
> >>> adding, or multiplying - its all about the Gaussians.
>
> >>> Clay
>
> >> Clay,
>
> >> It seems that a Gaussian of a given mean and variance could come from
> >> the convolution of an infinite number of pairs (or larger groups) of
> >> other Gaussians, so the OPs desire to find out what the original
> >> Gaussian components were that summed to a given Gaussian, without some
> >> known relation between the component Gaussians, &#4294967295;would be fruitless.
> >> No?
>
> >> Dirk- Hide quoted text -
>
> >> - Show quoted text -
>
> > Dirk,
>
> > In general yes, but what if each of the 4 gaussians were identical? As
> > usual with questions here we are often lacking info that can sometimes
> > really simplify the problem. But if he is adding RVs from sources with
> > unique and unknown means and variances, he is in trouble, but since
> > nonnegative variances add together, the resulting variance sets an
> > upper bound on the individual variances.
>
> > Clay
>
> Adding two Gaussian RVs is not the same as mixing two Gaussian RVs.
>
> Mixing RVs adds the distributions but adding the RVs convolves the
> distributions.
>
> When you have a mixture it means that there is some probability of
> one and the suitable probability of the other. The mixture of two Gaussians
> is not another Gaussian. Think of an equal mixture of two Gaussians that
> have means +10 and -10 with both having a variance of 1. The distribution
> function of the result has two widely separated humps.
>
> The original problem was to fit a mixture of four components.- Hide quoted text -
>
> - Show quoted text -

OP said "sum" of gaussians repeatedly, not mix.

Dirk

On Apr 17, 2:20�pm, c...@claysturner.com wrote:
> On Apr 17, 11:04&#4294967295;am, dbell <bellda2...@cox.net> wrote:
>
>
>
>
>
> > On Apr 17, 10:10&#4294967295;am, c...@claysturner.com wrote:
>
> > > On Apr 17, 8:46&#4294967295;am, mauro <mauro.austra...@gmail.com> wrote:
>
> > > > On Apr 17, 9:03 pm, mauro <mauro.austra...@gmail.com> wrote:
>
> > > > > I think the curve it can be easily approximated to the sum of 4
> > > > > gaussians, using some trick... but I don't have any clue!
> > > > > any suggestion would be strong appreciated.
>
> > > > got it for the \mu of guassian, first derivatives equals to zero. it
> > > > works.
> > > > I was mainly interested to the \mu, but if you have any idea of how to
> > > > do the same in the frequency domain, I would be curious to see how to
> > > > do that :-)
>
> > > > cheers,
> > > > Mauro
>
> > > Hello Mauro,
>
> > > The Gaussian (Normal Distribution) is special (yes there is an
> > > exception- Cauchy) in that a sum of Normals is Normal. And the Fourier
> > > transform of a Gaussian is also Gaussian. So the freq. domain still
> > > has you working with Gaussians. Neat! So whether you are convolving,
> > > adding, or multiplying - its all about the Gaussians.
>
> > > Clay
>
> > Clay,
>
> > It seems that a Gaussian of a given mean and variance could come from
> > the convolution of an infinite number of pairs (or larger groups) of
> > other Gaussians, so the OPs desire to find out what the original
> > Gaussian components were that summed to a given Gaussian, without some
> > known relation between the component Gaussians, &#4294967295;would be fruitless.
> > No?
>
> > Dirk- Hide quoted text -
>
> > - Show quoted text -
>
> Dirk,
>
> In general yes, but what if each of the 4 gaussians were identical? As
> usual with questions here we are often lacking info that can sometimes
> really simplify the problem. But if he is adding RVs from sources with
> unique and unknown means and variances, he is in trouble, but since
> nonnegative variances add together, the resulting variance sets an
> upper bound on the individual variances.
>
> Clay- Hide quoted text -
>
> - Show quoted text -

Clay,

As I said, "without some known relation between the component
Gaussians".  OP hasn't provided any known relation.

Dirk

mauro <mauro.australia@gmail.com> writes:

> I have few sampled data (positive integer values), and I want to
> approximate such data to sum of few gaussians.
> As first step I removed low frequencies (filtering all the values
> greater then the mean), and I obtained something that actually looks
> like a sum of few gaussians. But now I'm looking forward to a method
> to retrieve automatically mean and variance of each gaussian.

Hi Mauro,

I know the answer to this question (he says, thinking of Donkey in Shrek
shouting "I know! I know! Pick me!").

What you need is an application of the Estimation Maximisation
algorithm.  This is an iterative algorithm which estimates a probability
density function as a weighted sum of Gaussians (or more generally log
linear function), that is what you want are the coefficients of:

p(x) = \sum c_i / (sigma_i sqrt(2 pi)) exp (- (x - mean_i)^2 / (2 sigma_i^2))

For a quick explanation, I'll start by assuming you know how to calculate
the mean and sigma for a single Gaussian and that you will use Google to
fill in details on the EM algorithm.

The EM algorithm goes like this, start with some reasonablish values of
c_i mean_i and sigma_i (if there are 4 Gaussians then c_i could be 1/4,
the sigma_i could be the global standard deviation and the means could
be distributed around the global mean +- a standard deviation).  Now
given this, the EM algorithm says that for each data points you:

Firstly assume the mixture weights, c_i, are fixed and calculate the
probability that each data point came from each of your Gaussians.  Using
this you can update the mean and variances of your Gaussians.

Then you assume that the mean and variances are fixed and use Bayes to
update the mixture weights.

You loop through each of these steps until your objective function (the
log likelihood of the data given the parameters) converges well enough
for you.

Finally don't be worried that your posted graphs don't sum to one,
simply normalise them, do EM and scale the c_i afterwards.

I've done all this before for very many different problems, let me know
if you want more info (use the domain name to reply).



Tony

--
Founder Cantab Research
http://www.cantabresearch.com