Reply by Randy Yates ●February 10, 20142014-02-10
Gordon Sande <Gordon.Sande@gmail.com> writes:


> On 2014-02-09 13:59:29 +0000, Randy Yates said:
>
>> gyansorova@gmail.com writes:
>>
>>> I was unaware that the idea was quite so old. Thought maybe Wiener
>>> thought of it but i was wrong
>>>
>>> http://en.wikipedia.org/wiki/Arthur_Schuster
>>>
>>> A German called Arthur Schuster in the 1890s!
>>>
>>> Schuster is perhaps most widely remembered for his periodogram
>>> analysis, a technique which was long the main practical tool for
>>> identifying statistically important frequencies present in a time
>>> series of observations. He first used this form of harmonic analysis
>>> in 1897 to disprove C. G. Knott's claim of periodicity in earthquake
>>> occurrences. He went on to apply the technique to analysing sunspot
>>> activity.
>>
>> Interesting bit of math/DSP history!
>>
>> Would the following statement be correct:
>>
>>   A periodogram is a method of frequency spectrum estimation that
>>   reduces the variance of the estiamtes over the simple Fourier
>>   transform technique for random (stochastic) signals.
>>
>> ?
>
> No. The periodogram is only the magmitude squared of the Fourier transform.
> For Gaussian data the periodogram will have two degrees of fredom, so is very
> unstable. It is not a consistent estimator as more data does not lead to
> lower variance.


Gordon,

Thanks for the correction. I keep on getting this confused.

I initially encountered the periodogram in a class on estimation theory.
We used the book by Schwartz:

@book{schwartz,
  title = "Signal Processing: Discrete Spectral Analysis, Detection, and Estimation",
  author = "{Mischa~Schwartz and Leonard~Shaw}",
  publisher = "McGraw-Hill",
  year = "1975"}

which I still have. Revisiting the relevent sections discloses agreement
with you, that the AVERAGED periodograms reduces the variance in the
estimates; the plain old periodogram is as you stated.


> It was only the independent work of Bartlett and Tukey that produced the
> power spectrum estimates with more stability. It was well known that the
> periodigram was not a good way to find hidden periodicities. So much so
> that the whole idea had a rather poor reputation.


Do you have a reference to their work in this area? Was it basically
the "averaged periodogram" approach?
-- 
Randy Yates
Digital Signal Labs
http://www.digitalsignallabs.com
Reply by Gordon Sande ●February 10, 20142014-02-10
On 2014-02-10 04:46:29 +0000, gyansorova@gmail.com said:


> On Monday, February 10, 2014 3:51:30 AM UTC+13, Gordon Sande wrote:
>> On 2014-02-09 13:59:29 +0000, Randy Yates said:
>> 
>> 
>> 
>>> gyansorova@gmail.com writes:
>> 
>>> 
>> 
>>>> I was unaware that the idea was quite so old. Thought maybe Wiener
>> 
>>>> thought of it but i was wrong
>> 
>>>> 
>> 
>>>> http://en.wikipedia.org/wiki/Arthur_Schuster
>> 
>>>> 
>> 
>>>> A German called Arthur Schuster in the 1890s!
>> 
>>>> 
>> 
>>>> Schuster is perhaps most widely remembered for his periodogram
>> 
>>>> analysis, a technique which was long the main practical tool for
>> 
>>>> identifying statistically important frequencies present in a time
>> 
>>>> series of observations. He first used this form of harmonic analysis
>> 
>>>> in 1897 to disprove C. G. Knott's claim of periodicity in earthquake
>> 
>>>> occurrences. He went on to apply the technique to analysing sunspot
>> 
>>>> activity.
>> 
>>> 
>> 
>>> Interesting bit of math/DSP history!
>> 
>>> 
>> 
>>> Would the following statement be correct:
>> 
>>> 
>> 
>>> A periodogram is a method of frequency spectrum estimation that
>> 
>>> reduces the variance of the estiamtes over the simple Fourier
>> 
>>> transform technique for random (stochastic) signals.
>> 
>>> 
>> 
>>> ?
>> 
>> 
>> 
>> No. The periodogram is only the magmitude squared of the Fourier transform.
>> 
>> For Gaussian data the periodogram will have two degrees of fredom, so is very
>> 
>> unstable. It is not a consistent estimator as more data does not lead to
>> 
>> lower variance.
>> 
>> 
>> 
>> It was only the independent work of Bartlett and Tukey that produced the
>> 
>> power spectrum estimates with more stability. It was well known that the
>> 
>> periodigram was not a good way to find hidden periodicities. So much so
>> 
>> that the whole idea had a rather poor reputation.
> 
> Usually the periodogram is averaged over successive frames and you can 
> get a pretty good estimate.



Averaging multiple periodograms is a possible spectrum estimator but it 
is no longer
a periodogram. The definition of a periodogram is rather well established.
Reply by ●February 10, 20142014-02-10
On Monday, February 10, 2014 3:51:30 AM UTC+13, Gordon Sande wrote:

> On 2014-02-09 13:59:29 +0000, Randy Yates said:
> 
> 
> 
> > gyansorova@gmail.com writes:
> 
> > 
> 
> >> I was unaware that the idea was quite so old. Thought maybe Wiener 
> 
> >> thought of it but i was wrong
> 
> >> 
> 
> >> http://en.wikipedia.org/wiki/Arthur_Schuster
> 
> >> 
> 
> >> A German called Arthur Schuster in the 1890s!
> 
> >> 
> 
> >> Schuster is perhaps most widely remembered for his periodogram
> 
> >> analysis, a technique which was long the main practical tool for
> 
> >> identifying statistically important frequencies present in a time
> 
> >> series of observations. He first used this form of harmonic analysis
> 
> >> in 1897 to disprove C. G. Knott's claim of periodicity in earthquake
> 
> >> occurrences. He went on to apply the technique to analysing sunspot
> 
> >> activity.
> 
> > 
> 
> > Interesting bit of math/DSP history!
> 
> > 
> 
> > Would the following statement be correct:
> 
> > 
> 
> >   A periodogram is a method of frequency spectrum estimation that
> 
> >   reduces the variance of the estiamtes over the simple Fourier
> 
> >   transform technique for random (stochastic) signals.
> 
> > 
> 
> > ?
> 
> 
> 
> No. The periodogram is only the magmitude squared of the Fourier transform.
> 
> For Gaussian data the periodogram will have two degrees of fredom, so is very
> 
> unstable. It is not a consistent estimator as more data does not lead to
> 
> lower variance.
> 
> 
> 
> It was only the independent work of Bartlett and Tukey that produced the
> 
> power spectrum estimates with more stability. It was well known that the
> 
> periodigram was not a good way to find hidden periodicities. So much so
> 
> that the whole idea had a rather poor reputation.


Usually the periodogram is averaged over successive frames and you can get a pretty good estimate.
Reply by Gordon Sande ●February 9, 20142014-02-09
On 2014-02-09 13:59:29 +0000, Randy Yates said:


> gyansorova@gmail.com writes:
> 
>> I was unaware that the idea was quite so old. Thought maybe Wiener 
>> thought of it but i was wrong
>> 
>> http://en.wikipedia.org/wiki/Arthur_Schuster
>> 
>> A German called Arthur Schuster in the 1890s!
>> 
>> Schuster is perhaps most widely remembered for his periodogram
>> analysis, a technique which was long the main practical tool for
>> identifying statistically important frequencies present in a time
>> series of observations. He first used this form of harmonic analysis
>> in 1897 to disprove C. G. Knott's claim of periodicity in earthquake
>> occurrences. He went on to apply the technique to analysing sunspot
>> activity.
> 
> Interesting bit of math/DSP history!
> 
> Would the following statement be correct:
> 
>   A periodogram is a method of frequency spectrum estimation that
>   reduces the variance of the estiamtes over the simple Fourier
>   transform technique for random (stochastic) signals.
> 
> ?


No. The periodogram is only the magmitude squared of the Fourier transform.
For Gaussian data the periodogram will have two degrees of fredom, so is very
unstable. It is not a consistent estimator as more data does not lead to
lower variance.

It was only the independent work of Bartlett and Tukey that produced the
power spectrum estimates with more stability. It was well known that the
periodigram was not a good way to find hidden periodicities. So much so
that the whole idea had a rather poor reputation.
Reply by Randy Yates ●February 9, 20142014-02-09
gyansorova@gmail.com writes:


> I was unaware that the idea was quite so old. Thought maybe Wiener thought of it but i was wrong 
>
> http://en.wikipedia.org/wiki/Arthur_Schuster 
>
> A German called Arthur Schuster in the 1890s! 
>
> Schuster is perhaps most widely remembered for his periodogram
> analysis, a technique which was long the main practical tool for
> identifying statistically important frequencies present in a time
> series of observations. He first used this form of harmonic analysis
> in 1897 to disprove C. G. Knott's claim of periodicity in earthquake
> occurrences. He went on to apply the technique to analysing sunspot
> activity.


Interesting bit of math/DSP history!

Would the following statement be correct: 

  A periodogram is a method of frequency spectrum estimation that
  reduces the variance of the estiamtes over the simple Fourier
  transform technique for random (stochastic) signals.

?
-- 
Randy Yates
Digital Signal Labs
http://www.digitalsignallabs.com
Reply by ●February 9, 20142014-02-09
On Sunday, February 9, 2014 1:30:46 PM UTC+13, robert bristow-johnson wrote:

> On 2/7/14 10:58 PM, gyansorova@gmail.com wrote:
> 
> > I was unaware that the idea was quite so old. Thought maybe Wiener thought of it but i was wrong
> 
> >
> 
> > http://en.wikipedia.org/wiki/Arthur_Schuster
> 
> >
> 
> > A German called Arthur Schuster in the 1890s!
> 
> >
> 
> > Schuster is perhaps most widely remembered for his periodogram analysis, a technique which was long the main practical tool for identifying statistically important frequencies present in a time series of observations. He first used this form of harmonic analysis in 1897 to disprove C. G. Knott's claim of periodicity in earthquake occurrences. He went on to apply the technique to analysing sunspot activity.
> 
> 
> 
> how did he create the periodograms, with what technology of that era?
> 
> 
> 
> -- 
> 
> 
> 
> r b-j                  rbj@audioimagination.com
> 
> 
> 
> "Imagination is more important than knowledge."


Maybe by hand! That;s all they had I think back then. Did they not have a whole room full of people called a computer?
Reply by robert bristow-johnson ●February 8, 20142014-02-08
On 2/7/14 10:58 PM, gyansorova@gmail.com wrote:

> I was unaware that the idea was quite so old. Thought maybe Wiener thought of it but i was wrong
>
> http://en.wikipedia.org/wiki/Arthur_Schuster
>
> A German called Arthur Schuster in the 1890s!
>
> Schuster is perhaps most widely remembered for his periodogram analysis, a technique which was long the main practical tool for identifying statistically important frequencies present in a time series of observations. He first used this form of harmonic analysis in 1897 to disprove C. G. Knott's claim of periodicity in earthquake occurrences. He went on to apply the technique to analysing sunspot activity.


how did he create the periodograms, with what technology of that era?

-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."
Reply by ●February 7, 20142014-02-07
I was unaware that the idea was quite so old. Thought maybe Wiener thought of it but i was wrong 

http://en.wikipedia.org/wiki/Arthur_Schuster 

A German called Arthur Schuster in the 1890s! 

Schuster is perhaps most widely remembered for his periodogram analysis, a technique which was long the main practical tool for identifying statistically important frequencies present in a time series of observations. He first used this form of harmonic analysis in 1897 to disprove C. G. Knott's claim of periodicity in earthquake occurrences. He went on to apply the technique to analysing sunspot activity.