Show me the numbers

Hi all,

I took some time the previous few days to play with different estimators,
including Cedron's one, and you can find the results of my "playings" in
the following small note:

http://www.tsdconseil.fr/log/scriptscilab/festim/freqestim-comp.pdf

Many thanks to Eric that gave me lot of leads and advice for doing this
evaluation.

For those who have not much time to read all the preliminaries, you can go
directly to pages 9 and 10 to see the results I obtained.

I may have done mistakes, and I have surely not tested all existing
estimators, so if someone wants to reproduce the results, check the
algorithms or play with the simulation parameters, I give a link to the
SCILAB scripts.

Also, as written in the note, I am still asking myself a lot of question:
any feedback about these would be very welcome!

Kind regards,

Julien

>Julien, this page is not loading for me.  Is it still up?
>Thanks
>Steve

Indeed, this was only a first try. You can find the last graphs I obtained
in the PDF note.
---------------------------------------
Posted through http://www.DSPRelated.com

"tsd82" <105802@DSPRelated> writes:

> Hi all,
>
> I took some time the previous few days to play with different estimators,
> including Cedron's one, and you can find the results of my "playings" in
> the following small note:
>
> http://www.tsdconseil.fr/log/scriptscilab/festim/freqestim-comp.pdf
>
> Many thanks to Eric that gave me lot of leads and advice for doing this
> evaluation.
>
> For those who have not much time to read all the preliminaries, you can go
> directly to pages 9 and 10 to see the results I obtained.
>
> I may have done mistakes, and I have surely not tested all existing
> estimators, so if someone wants to reproduce the results, check the
> algorithms or play with the simulation parameters, I give a link to the
> SCILAB scripts.
>
> Also, as written in the note, I am still asking myself a lot of question:
> any feedback about these would be very welcome!
>
> Kind regards,
>
> Julien

Julien,

Whether there are mistakes in your work or not, I would like to thank
you for taking the time and effort to examine this subject through the
lens of the "real world." I think that is just excellent.
-- 
Randy Yates
Digital Signal Labs
http://www.digitalsignallabs.com

On Mon, 01 Jun 2015 08:47:22 -0400, Randy Yates
<yates@digitalsignallabs.com> wrote:

>"tsd82" <105802@DSPRelated> writes:
>
>> Hi all,
>>
>> I took some time the previous few days to play with different estimators,
>> including Cedron's one, and you can find the results of my "playings" in
>> the following small note:
>>
>> http://www.tsdconseil.fr/log/scriptscilab/festim/freqestim-comp.pdf
>>
>> Many thanks to Eric that gave me lot of leads and advice for doing this
>> evaluation.
>>
>> For those who have not much time to read all the preliminaries, you can go
>> directly to pages 9 and 10 to see the results I obtained.
>>
>> I may have done mistakes, and I have surely not tested all existing
>> estimators, so if someone wants to reproduce the results, check the
>> algorithms or play with the simulation parameters, I give a link to the
>> SCILAB scripts.
>>
>> Also, as written in the note, I am still asking myself a lot of question:
>> any feedback about these would be very welcome!
>>
>> Kind regards,
>>
>> Julien
>
>Julien,
>
>Whether there are mistakes in your work or not, I would like to thank
>you for taking the time and effort to examine this subject through the
>lens of the "real world." I think that is just excellent.

Yeah, that's some good quality work.   Julien is a fast learner and
has good intuition in this area.


Eric Jacobsen
Anchor Hill Communications
http://www.anchorhill.com

On Mon, 01 Jun 2015 07:27:37 -0500, "tsd82" <105802@DSPRelated> wrote:

>Hi all,
>
>I took some time the previous few days to play with different estimators,
>including Cedron's one, and you can find the results of my "playings" in
>the following small note:
>
>http://www.tsdconseil.fr/log/scriptscilab/festim/freqestim-comp.pdf
>
>Many thanks to Eric that gave me lot of leads and advice for doing this
>evaluation.
>
>For those who have not much time to read all the preliminaries, you can go
>directly to pages 9 and 10 to see the results I obtained.
>
>I may have done mistakes, and I have surely not tested all existing
>estimators, so if someone wants to reproduce the results, check the
>algorithms or play with the simulation parameters, I give a link to the
>SCILAB scripts.
>
>Also, as written in the note, I am still asking myself a lot of question:
>any feedback about these would be very welcome!

Regarding why the estimators diverge at ~5dB, this is probably due to
what is known as the "threshold effect", which is where the magnitude
maximizer of the DFT starts to be influenced by noise rather than the
peak location of the signal energy.  In other words, when the peak
magnitude of the DFT is the wrong bin, the estimators don't have much
chance of getting things right.

The location of the threshold depends on N and is consistent with the
usual noise reduction characteristics of the DFT.   So as N increases
the location of the knee for the threshold effect drops in SNR.

>Kind regards,
>
>Julien
>
>>Julien, this page is not loading for me.  Is it still up?
>>Thanks
>>Steve
>
>Indeed, this was only a first try. You can find the last graphs I obtained
>in the PDF note.

Awesome job on the paper and simulations. 


Eric Jacobsen
Anchor Hill Communications
http://www.anchorhill.com

>Hi all,
>
>I took some time the previous few days to play with different
estimators,

[...snip...]

>Also, as written in the note, I am still asking myself a lot of
question:
>any feedback about these would be very welcome!
>
>Kind regards,
>
>Julien
>
[...snip...]

Not so bad.

Just kidding, excellent job, especially in such a short time frame.

I would note that the real case and complex case are not really
interchangeable.  My formula is derived from the real case.  I am pleased
at how well it did in the complex case, but I have a 3-bin noise resistant
version (and its derivation) for the complex case I will send you via
email.

I'd also like to make you (and everyone) aware of these results I posted
earlier in this thread.

Bins  Dawg Error        Candan Error
----  ----------------  ----------------
 128  0.00000000000002  0.00001130078824
 161 -0.00000000000000  0.00000210472222
 203  0.00000000000000 -0.00000375601512
 256  0.00000000000001 -0.00000744452953
 323 -0.00000000000009 -0.00000976778380
 406 -0.00000000000008 -0.00001120848112
 512  0.00000000000004 -0.00001213058153
 645  0.00000000000026 -0.00001270833913
 813  0.00000000000057 -0.00001307308072
1024  0.00000000000032 -0.00001330207727
1290  0.00000000000161 -0.00001344651598
1625 -0.00000000000002 -0.00001353751043
2048 -0.00000000001037 -0.00001359495013

They seem to indicate that there is a precision limitation when using the
double data type for DFT calculations somewhere approaching 200 bins. 
This was unexpected to me.

Thank you so much for your effort.  Thanks also to RL and EJ for
assisting.

Ced

---------------------------------------
Posted through http://www.DSPRelated.com

Eric wrote:
>Regarding why the estimators diverge at ~5dB, this is probably due to
>what is known as the "threshold effect", which is where the magnitude
>maximizer of the DFT starts to be influenced by noise rather than the
>peak location of the signal energy.  In other words, when the peak
>magnitude of the DFT is the wrong bin, the estimators don't have much
>chance of getting things right.

Yes, this seems quite a good explanation!

By the way, this fact give me a very simple (maybe naive!) idea: why not
pre-filtering the DFT before selecting the max (e.g. some kind of
windowing in the time domain, but not the usual kind of window, since we
do not need any spectral leakage removal, only frequency domain
filtering)? It should enhance the accuracy at low SNR. Then we can always
use the same interpolation algorithms (but on the unwindowed DFT bins). 

Now, this may probably not enhance too much the result, I don't know, I
will try when I have some time, and this raise at least another
interesting (or at least I find) question: what is the ideal filter (in
the frequency domain) to apply? At first view, I would say that we must
apply a matched filter, e.g. a filter matched to the signal magnitude,
which is for us a pattern of spectral leakage.

**But**, the spectral leakage pattern is dépendent of the frequency, so,
what to do, compute (or calculate analytically if possible: another
interesting problem!) the mean leakage pattern when the frequency
distribution is uniform, and defines our filter with it. I am curius to
see the filter shape!

Ok, I am too much thinking while I am writing! I come back when I have
confronted a little this idea to reality (not before some days). Unless
someone has already the answers?

---------------------------------------
Posted through http://www.DSPRelated.com

>
>Not so bad.
>
>Just kidding, excellent job, especially in such a short time frame.
>

Thank you for this nice comment, that it is all the more appreciable that
I was wrongly harsh in my first comments about your methods.

>I would note that the real case and complex case are not really
>interchangeable.  My formula is derived from the real case.  I am
pleased
>at how well it did in the complex case, but I have a 3-bin noise
resistant
>version (and its derivation) for the complex case I will send you via
>email.

I didn't know that you had made a specific method for the complex case. I
can evaluate it too, if you want. But I must say I am a little surprised,
because I don't really see how the methods exploiting the 3 highest DFT
bins can make the difference between a real and complex tone?

>They seem to indicate that there is a precision limitation when using
the
>double data type for DFT calculations somewhere approaching 200 bins. 
>This was unexpected to me.

This is something I had tough to consider, but you're right, at high SNR
computations errors may have some impact!

>
>Thank you so much for your effort.  Thanks also to RL and EJ for
>assisting.

Thank you for posting this very interesting problem.

---------------------------------------
Posted through http://www.DSPRelated.com

Bob Masta <NoSpam@daqarta.com> wrote:

>spope33@speedymail.org (Steve Pope) wrote:

>>Along these lines, an unrelated example.   I have always suspected
>>that black holes do not exist in reality, for the simple reason
>>that singularities tend to occur in mathematics but not in nature.
>>For many decades running, all except fringe astrophysicists
>>firmly believed (or at least would talk as though they believed)
>>that "black hole candidates" were indeed black holes, and not some
>>other sort of dense object.  There were always doubters, but they 
>>were always shouted down ... until Stephen Hawking became one 
>>of the doubters.

>I'm no physicist, but can't you have a "black hole" (from
>which light can't escape) without having a singularity?

What people talk about when they talk about black holes
is a solution to the gravitational field equations that has
an infiniterly dense, zero-size point in its center,
therefore, a singularity.

There is a current school of thought that you can have
an almost-black-hole that some call an "eternally collapsing
object".  How exactly that might work is up for speculation,
but the idea is that quantum effects and radiation pressure
keep the thing from collapsing in finite time, and it never
becomes infintesimal in size.

It's a very dense exotic object for sure, but not a black hole by the
book.  Hawking has not signed on to the "eternally collapsing
object" terminology, but he did suggest there is no true event
horizon, as had always been assumed.  

(I think no event horizon means no singularity, and vice versa,
but I'm no physicist either.)


Steve

> On 5/30/15 12:16 AM, Cedron wrote:
>
> Honestly, my derivations are not that complicated.  I did use the n and k
> subscripts in reverse of what is conventional so that might be a hindrance
> to understanding them for some.  I intend to edit the blogs to correct
> that.  I truly am trying to get folks to understand what I have done.  Any
> one who can understand the typical DSP literature should be able to
> understand my blog articles.  They are meant to be entry level.

it wasn't just the "n" vs. "k" or the X_n vs. X[k] notation that 
bothered me.  i thought that the math was much thicker than it needed to be.

i also think that the theoretical usefulness is limited because you're 
considering only the rectangular window with the block of samples going 
into the DFT.


someone else wrote:

>> There really
>> is little place for "exact solutions" in DSP applications. Perhaps a
>> symbolic math group could be more interested. More of interest in DSP
>> applications are the details of implementation cost and the effects
>> of noise and interference.


On 5/30/15 10:45 AM, Cedron wrote:
>
>
> Don't you think it's good when practice rests on solid theory?

i like to lean on the theoretical myself, but think sometimes the theory 
guys get lost in the weeds.  at the dsp.stackexchange.com site i find 
myself to be the Neanderthal discussing DSP stuff with folks that worry 
about ROC and other stuff that i don't, and then start making claims 
that marginal instability in an otherwise LTI makes it violate 
linearity: 
http://dsp.stackexchange.com/questions/18557/does-instability-make-an-otherwise-lti-system-nonlinear-or-time-variant 
.



-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

Eric Jacobsen <eric.jacobsen@ieee.org> wrote:

>Regarding why the estimators diverge at ~5dB, this is probably due to
>what is known as the "threshold effect", which is where the magnitude
>maximizer of the DFT starts to be influenced by noise rather than the
>peak location of the signal energy.  In other words, when the peak
>magnitude of the DFT is the wrong bin, the estimators don't have much
>chance of getting things right.

One could, in simulation, conduct an experiment in which the 
maximizer-selection is geniated (that is, it uses side information
to be always correct even in low SNR) and isolate whether this is the cause.

Steve