Show me the numbers

On Monday, June 1, 2015 at 1:04:24 PM UTC-7, Cedron wrote:
> >On Saturday, May 30, 2015 at 4:54:59 AM UTC-7, Randy Yates wrote:
> >...
> >>  How do you classify an
> >> algorithm that gives you "exact" results when no noise is present and
> a
> >> good estimate when it is? 
> >...
> >> Randy Yates
> >> Digital Signal Labs
> >> http://www.digitalsignallabs.com
> >
> >"Robust" comes to mind.
> >
> >Dale B. Dalrymple
> 
> That's not what "Robust" means at all.  At least not how I learned it. 
> Robust means tolerant to perturbations without moving to a different
> equilibrium point.  In this context it means insensitive to noise.

From mathworld.wolfram.com:
"Robust Estimation
An estimation technique which is insensitive to small departures from the idealized assumptions used to optimize the algorithm."

Words mean what a community uses them to mean. What does the meaning of "robust" that you were taught have to do with dsp topics that make little use of "perturbations" and "equilibrium points"?

> 
> Julien gave an example of an estimator that was exact in the noiseless
> case, but not robust at all with noise.
> 
> I would think that "unbiased" is closest, but that implies (as I
> understand it) the error distribution remains centered on the solution as
> the noise level increases.  Perhaps "baseline unbiased" or "inherently
> unbiased" are accurate.

"Unbaised" means not having a error term independent of noise level. More formally, the mean of the sampling distribution of the estimator is the parameter being measured. So biased/unbiased is orthogonal to robust.

> 
> I think Randy asked a very good question, and I would like to know the
> answer too.  That's why I repeated it.  This is one of those jargon
> things.  Is there a specific term that defines this concept?  Come on,
> estimation theory experts, please give us an answer.
> 
> Ced

I think Randy got an answer. It isn't necessary to reinvent terms because you are not familiar with dsp terminology.

Dale B. Dalrymple

Hi Kevin,

I didn't know that there were interpolation forumla with only 2 bins.
Thank you for the link.

>
>My mistake - I'm not truncating in the sense that I'm setting
coefficients
>to zero at either end of the window. And the window coefficients do
indeed
>go from non-zero to non-zero, so, yes, it is actually truncated at the
>ends.
>
>But I have used Gaussian windows that were of such narrow beamwidth
that,
>for all practical purposes, they went to zero at the ends (when the
>coefficient was too small to be represented by the number system being
>used).

Yes, but if your window is already near zero at the ends, it means that
you are throwing away a signifiant part of the signal, which means that
you will have less accuracy in the end result (going farther and farther
away from the CRLB).

Yet, I am conscious that if it is not zero at the end, leakages appear.
Difficult problem!

>In the above, N is 32, sample rate is 32, the Gaussian beamwidth is
>6.0, and the input frequency is 7.5, so the output estimates of bin 7 are
the
>correct ones [...] :
> 7       7.500000000560  1.000000000016

This is an error of 5.60e-10 (in bins) = 1.75e-11 (in normalized
frequency)

This seems much better than the result I measured (although with a 3 bins
gaussian method): MSE noise floor at about 1e-13 (although not for the
same number of samples, in my tests, N = 512), that is a typical error of
1e-6.5 (in normalized frequency units).

This is an indication that I have probably misprogrammed the gaussian
method: I will try to improve on this by comparing with your code.

Julien (with an 'e' :-))

---------------------------------------
Posted through http://www.DSPRelated.com

On Mon, 1 Jun 2015 20:49:13 -0700 (PDT), kevinjmcee
<kevinjmcgee@netscape.net> wrote:

  [Snipped by Lyons]
>
>Cedron, I've mentioned some links to McEachern's Gaussian Ratio method in a previous post, so I thought I'd add some numbers.  I programmed an example - then I remembered that I have a tutorial (and derivation) regarding the method at:
>
>http://fftguru.com/fftguru.com.tutorial4.pdf
>
  [Snipped by Lyons]

Hi Kevin,
   Since you and I discussed the 'McEachern Method' 
via e-mails last Fall, I did a little modeling of it 
using Matlab (using floating point numbers).

With a zero-noise FFT input data the 'McEachern Method' 
performs quite well.  But when my input sinusoid 
had an SNR of less than 50 dB I found that the 
'McEachern Method' did not perform as well as 
the 'Jacobsen Method'.

Of course my modeling could have been faulty so it 
would be nice for someone else here on comp.dsp 
to model and compare the McEachern, Jacobsen, and 
Cedron methods.

[-Rick-]

On Mon, 1 Jun 2015 18:17:27 +0000 (UTC),
spope33@speedymail.org (Steve Pope) wrote:

>Bob Masta <NoSpam@daqarta.com> wrote:
>
>>spope33@speedymail.org (Steve Pope) wrote:
>
>>>Along these lines, an unrelated example.   I have always suspected
>>>that black holes do not exist in reality, for the simple reason
>>>that singularities tend to occur in mathematics but not in nature.
>>>For many decades running, all except fringe astrophysicists
>>>firmly believed (or at least would talk as though they believed)
>>>that "black hole candidates" were indeed black holes, and not some
>>>other sort of dense object.  There were always doubters, but they 
>>>were always shouted down ... until Stephen Hawking became one 
>>>of the doubters.
>
>>I'm no physicist, but can't you have a "black hole" (from
>>which light can't escape) without having a singularity?
>
>What people talk about when they talk about black holes
>is a solution to the gravitational field equations that has
>an infiniterly dense, zero-size point in its center,
>therefore, a singularity.
>
>There is a current school of thought that you can have
>an almost-black-hole that some call an "eternally collapsing
>object".  How exactly that might work is up for speculation,
>but the idea is that quantum effects and radiation pressure
>keep the thing from collapsing in finite time, and it never
>becomes infintesimal in size.
>
>It's a very dense exotic object for sure, but not a black hole by the
>book.  Hawking has not signed on to the "eternally collapsing
>object" terminology, but he did suggest there is no true event
>horizon, as had always been assumed.  
>
>(I think no event horizon means no singularity, and vice versa,
>but I'm no physicist either.)

Thanks, Steve... I'm gonna have to read up on this stuff. 

I've been assuming that having gravity strong enough to keep
light from escaping (which I *think* is the same as having
an event horizon) did not require any infinities.  I was
also assuming (wrongly, it appears) that a singularity was
grounded in physics, as the density at which matter could
not withstand the force of gravity and collapsed on itself.

Which, now that I think about it, would have been much too
convenient for the guys working on unified field theory.  It
would have related electromagnetism (light not escaping from
the hole) with all the nuclear forces preventing collapse
and with gravity.  Oh, well!

Best regards,


Bob Masta
 
              DAQARTA  v7.60
   Data AcQuisition And Real-Time Analysis
              www.daqarta.com
Scope, Spectrum, Spectrogram, Sound Level Meter
 Frequency Counter, Pitch Track, Pitch-to-MIDI 
   FREE Signal Generator, DaqMusiq generator    
          Science with your sound card!

>
>From mathworld.wolfram.com:
>"Robust Estimation
>An estimation technique which is insensitive to small departures from
the
>idealized assumptions used to optimize the algorithm."

Insensitive to noise fits very well this, I think.

>
>Words mean what a community uses them to mean. What does the meaning of
>"robust" that you were taught have to do with dsp topics that make
little
>use of "perturbations" and "equilibrium points"?
>

Catastrophe Theory.  How much you could push a system before it "went of
the edge" was a very important topic.  I was describing the word robust,
not just within a "estimation" context.

>
>"Unbaised" means not having a error term independent of noise level.
More
>formally, the mean of the sampling distribution of the estimator is the
>parameter being measured. So biased/unbiased is orthogonal to robust.
>

Then it is the correct term.  Which then begs the question: "Is there a
term for an estimator that is unbiased that shows a bias at higher noise
levels?"

>
>I think Randy got an answer. It isn't necessary to reinvent terms
because
>you are not familiar with dsp terminology.
>
>Dale B. Dalrymple

I did too.  Thank you for your lookups.  I have no desire to invent new
terms and have made it no secret that I am not steeped in DSP training.

Ced
---------------------------------------
Posted through http://www.DSPRelated.com

[...snip...]
>
>
>Cedron, I've mentioned some links to McEachern's Gaussian Ratio method
in
>a previous post, so I thought I'd add some numbers.  I programmed an
>example - then I remembered that I have a tutorial (and derivation)
regarding the
>method at:
>
>http://fftguru.com/fftguru.com.tutorial4.pdf
>
[...snip...]
>
> 7       7.500000000560  1.000000000016
> 
>
[...snip...]

>Kevin McGee

Thank you so much, Kevin.

This compares to 

10 10.40000000000 0.99999999996

from a combination of two of my blog articles.  I got your code working
and I'm going to add it to my bin count comparison program.  I'll let
Julien ensure his Gaussian is up to snuff to do the noise comparisons.

Ced
---------------------------------------
Posted through http://www.DSPRelated.com

On Tue, 02 Jun 2015 02:03:12 -0500, "tsd82" <105802@DSPRelated> wrote:

>Hi Kevin,
>
>I didn't know that there were interpolation forumla with only 2 bins.
>Thank you for the link.

There are quite a few of them, actually.   They generally don't
perform as well as the three-terminal estimators.   In many cases this
is because as the frequency offset from a bin center decreases one of
the samples will approach zero (at high SNR), and this often leads to
significant bias or MSE in that region since there's usually a ratio
involved.

Grandke and Jain were separate authors for two of the more well-known
authors for two-coefficient methods.   Those papers are very old.   

>>
>>My mistake - I'm not truncating in the sense that I'm setting
>coefficients
>>to zero at either end of the window. And the window coefficients do
>indeed
>>go from non-zero to non-zero, so, yes, it is actually truncated at the
>>ends.
>>
>>But I have used Gaussian windows that were of such narrow beamwidth
>that,
>>for all practical purposes, they went to zero at the ends (when the
>>coefficient was too small to be represented by the number system being
>>used).
>
>Yes, but if your window is already near zero at the ends, it means that
>you are throwing away a signifiant part of the signal, which means that
>you will have less accuracy in the end result (going farther and farther
>away from the CRLB).
>
>Yet, I am conscious that if it is not zero at the end, leakages appear.
>Difficult problem!
>
>>In the above, N is 32, sample rate is 32, the Gaussian beamwidth is
>>6.0, and the input frequency is 7.5, so the output estimates of bin 7 are
>the
>>correct ones [...] :
>> 7       7.500000000560  1.000000000016
>
>This is an error of 5.60e-10 (in bins) = 1.75e-11 (in normalized
>frequency)
>
>This seems much better than the result I measured (although with a 3 bins
>gaussian method): MSE noise floor at about 1e-13 (although not for the
>same number of samples, in my tests, N = 512), that is a typical error of
>1e-6.5 (in normalized frequency units).

I've tested Gasior's method in the past but don't recall much about
the results except that other estimators seemed to work better, and
for my purposes that means at low SNR.

>This is an indication that I have probably misprogrammed the gaussian
>method: I will try to improve on this by comparing with your code.
>
>Julien (with an 'e' :-))
>
>---------------------------------------
>Posted through http://www.DSPRelated.com

Eric Jacobsen
Anchor Hill Communications
http://www.anchorhill.com

Kevin wrote:
Here are the numbers for the zero noise case (bin number, frequency
estimate, amplitude estimate):
 7       7.500000000560  1.000000000016

Rick wrote:
>With a zero-noise FFT input data the 'McEachern Method' 
>performs quite well.  But when my input sinusoid 
>had an SNR of less than 50 dB I found that the 
>'McEachern Method' did not perform as well as 
>the 'Jacobsen Method'.

Hi Kevin and Rick, 
I was surprised to see that Kevin got much better result with the 2 bins
interpolation instead than the 3 bins interpolation that I had tested.
This leaded me to do some tests, and it appeared that the difference was
not really in the fact to use 2 or 3 bins (apparently it has no effect),
but in the gaussian window shape factor (sigma): we did not use the same
values.

This leaded to other tests, and found some things, maybe you could find
this interesting. I updated the small note, you can find my humble and
temporary conclusions page 14 and page 15:

http://www.tsdconseil.fr/log/scriptscilab/festim/freqestim-comp3.pdf

(note: I had to change the pdf file name, because otherwise, I kept
getting the old file from some cache...)

Rick, 
Indeed, with the small tests I had done, I observe the same thing,
McEacern method is generally not as good as Jacobsen method (but the
former has a free parameter, so if you know in advance your SNR...).

Regards,

Julien


---------------------------------------
Posted through http://www.DSPRelated.com

PS : Dear Kevin, I forgot an observation. 

This is about why you obtain very good accuracy when there is no noise. In
fact, as the curves show, you can obtain arbitrary accuracy  (again, in a
theoretical noiseless environment) by proper selection of the gaussian
shape factor (sigma, or alpha in your code).

For example, in your code, you can raise alpha a little (e.g. decrease
sigma), and the accuracy should be even better!
---------------------------------------
Posted through http://www.DSPRelated.com

Randy Yates  <yates@digitalsignallabs.com> wrote:

>spope33@speedymail.org (Steve Pope) writes:
>
>>
>> Easily could be.  You can try to use the "double double" type in C++.
>> Or apply normalizing within internal stages of the DFT (easiest
>> to do this in an FFT, actually).
>
>Or, if one is motivated, one could also try using the Gnu
>Multi-Precision (GMP) library, which allows you to extend to arbitrary
>precision.

Good point.  This also exists in Boost, which is standard (or
standards-track) for C++.  The "double double" type, last I checked,
is not part of the C++ language standard.

I have no idea whether GMP or Gnu-anything is a standard.  I 
suspect not; this may or may not mean it's less portable than
any of the above.

Steve