Show me the numbers

>Hi Julien,
>
>Do you know how to code in C++ on Linux by any chance?  I'd be happy to
>send you my source code.

Hi Cedron,
Yes, I know a little of C++, and indeed it would be a good idea to publish
your code. Then I will be able to check that I have well implemented your
formula.

>Increasing the number of samples is definitely a noise mitigation
>technique.

Sure! I was surprised by the way that you did your first tests with only 8
(if I remember well) samples. I used for the graph 128 samples (it's a
little arbitrary of course!). But there are enough parameters to test for 
with the SNR and the frequency...

Kind regards,

Julien

---------------------------------------
Posted through http://www.DSPRelated.com

>And another thing that is not very clear, is that I obtain strange
results
>for all methods at near-DC or near-Nyquist frequency, although I don't
>make assumptions about the phase / amplitude. I need to think more about
>this.

PS : I know why now, this was because I used systematically a zero-phased
sinusoidal tone (e.g. sin(2pift+phi), with phi = 0), which is very
difficult to detect near DC (CRB going to infinity as f -> 0 if I
understand well).

---------------------------------------
Posted through http://www.DSPRelated.com

Randy Yates <yates@digitalsignallabs.com> wrote:
> Rick Lyons <R.Lyons@_BOGUS_ieee.org> writes:

>> For a trivial example, no computer on Earth can measure 
>> the "exact" frequency of a noise-free input sine wave 
>> sequence whose frequency is 100/3 Hz.  
 
(snip) 
> I don't think that example is correct in the general sense. 
> It depends on the sample rate, doesn't it? 
 
> Consider the specific example of Fs = 400/3 samples/second.
> Then a 100/3 Hz frequency would be at exactly pi/2 radians,
> or at bin 2^(N-2) for a 2^N-point FFT (N >= 2). Right?

But you don't know the exact sampling frequency such that it is
an exact integer multiple of the signal, unless someone tells
you that.  That is, as long as there is quantization, which there
always is for a computer on earth. 

With a finite number of samples, a sufficiently close sample rate
will give the same quantized sample values.

Another example of why Nyquist sampling theory requires an infinite
number of samples. 

-- glen

On Sat, 30 May 2015 02:38:06 -0500, "tsd82" <105802@DSPRelated> wrote:

>Hi Rick,

Hi Julien,
>
>Thank you very much for your feedback and suggested readings!
>
>I know that these kind of comparisons have already been done before, but I
>have personal motivations to do this: Having a big limitation, namely I
>don't understand really well a DSP technique if I don't program / test it.
>Also it is very interesting, because when putting the things into
>practive, some interesting questions, even more "theoretical", often
>arise.

Yes. Yes. Yes.

[Snipped by Lyons]

>And another thing that is not very clear, is that I obtain strange results
>for all methods at near-DC or near-Nyquist frequency, although I don't
>make assumptions about the phase / amplitude. I need to think more about
>this.

I'll bet you're seeing unexpected results when your 
real-valued input test sine wave is near zero Hz 
and Fs/2 Hz because the sidelobes of the negative-frequency 
sin(x)/x spectral envelope is crossing over the 
zero-Hz boundary and distorting the shape of the 
positive-frequency sin(x)/x spectral envelope.

You might try replacing your real-valued input test 
sine wave with a complex-valued exponential sinusoid 
and see if you still see unexpected results when that 
complex-valued input sinusoid is near zero Hz 
and Fs/2 Hz.  Just a thought.

[-Rick-]

Cedron <103185@DSPRelated> wrote:

>I think that my data tables show that my equations are considerably more
>accurate than Candan's.  What also distinguishes mine from the others
>examined is that the accuracy is not sample size dependent.  Ergo, it
>doesn't follow the shelf in Julien's/Candan's diagram, but lays right
>along the Cramer-Rao bound for any sample size.

Hey, that's great.  Go with it.

Steve

Rick Lyons <R.Lyons@_BOGUS_ieee.org> writes:

> On Fri, 29 May 2015 23:42:22 -0400, Randy Yates
> <yates@digitalsignallabs.com> wrote:
>
>>Randy Yates <yates@digitalsignallabs.com> writes:
>>
>>> Rick Lyons <R.Lyons@_BOGUS_ieee.org> writes:
>>>> [...]
>>>> For a trivial example, no computer on Earth can measure 
>>>> the "exact" frequency of a noise-free input sine wave 
>>>> sequence whose frequency is 100/3 Hz.  
>>>
>>> Rick,
>>>
>>> I don't think that example is correct in the general sense. 
>>> It depends on the sample rate, doesn't it? 
>>>
>>> Consider the specific example of Fs = 400/3 samples/second.
>>> Then a 100/3 Hz frequency would be at exactly pi/2 radians,
>>> or at bin 2^(N-2) for a 2^N-point FFT (N >= 2). Right?
>>
>>PS: I see your point, though. There are an infinite number of numbers
>>(even within the dynamic range of a float) which a float can't represent
>>exactly.
>>
>>However what if Cedron's techniques were considered in the analytic
>>domain only, i.e., using real/complex numbers and analytical operations?
>
> Hi Randy,
>   If you were born in the Middle Ages your 
> name would have been "Randy, Ever the Thoughtful, Yates."
> I say that with kindest regards because you're so 
> much more analytical than I.  :-)

Hi Rick,

You are more humble than Isaac Newton!

> I was somewhat hesitant to give my 'trivial example' 
> because I was worried smart guys like you might poke 
> holes in it.

Quit calling me smart. It makes me nervous when really smart (and
famous) people call me smart in front of other really, really smart
people (like Eric, glen, etc.). As I've been skimming some of the
references in this thread it's been really cool to see Eric's name
referenced. Hey, I know him!

> In any case, I was merely trying to make the point 
> that using the word "exact" in the title of a blog 
> is a bit risky.  I say that because representing a 
> decimal value based on a finite number of binary bits 
> can sometimes not be "exact".

Yes, of course I agree with that. 

To address your main point, I'm not sure if the term "exact" is improper
here or not (assuming an analytical context). How do you classify an
algorithm that gives you "exact" results when no noise is present and a
good estimate when it is? Are there other "estimators" that give exact
results when no noise is present?

Perhaps it depends on the algorithm's intended application: if it is
meant to be used to get a good estimate when noise is present, then it's
an estimator. Right? I don't know - not versed enough in the field of
estimation. Perhaps one of the experts here can clarify the terminology.

> Hope you're doin' well Randy.

Like sleeping bats, I'm hanging in there. Same to you and Ziggie.
-- 
Randy Yates
Digital Signal Labs
http://www.digitalsignallabs.com

[...snip...]
>
>I have received yesterday many good suggestions and ideas to improve the
>comparison by mail from Eric Jacobsen. As a result of these exchanges,
you
>can find on the link http://92.243.17.111/uploads/mse.pdf an updated
>graphic of the comparison. Now the RC bound is shown (minimum variance
of
>any estimators), and the results are more accurate (removed near-DC and
>near-Nyquist frequencies from the test), the gaussian method shows
finally
>its 'exact' property (no noise floor). My implementation of the Candan
>method is still bugged.

Julien,

I think this chart is even better than your first one.  I am so grateful
for you doing this work.  I am also very grateful to EJ for helping out to
improve upon it.

[...snip...]

>
>I hope to come back in a few days with more solid results,
>
>Many thanks for your help,
>
>Julien
>

I am very much looking forward to it.  Many thanks to you, too.

Ced
---------------------------------------
Posted through http://www.DSPRelated.com

>>Hi Julien,
>>
>>Do you know how to code in C++ on Linux by any chance?  I'd be happy to
>>send you my source code.
>
>Hi Cedron,
>Yes, I know a little of C++, and indeed it would be a good idea to
publish
>your code. Then I will be able to check that I have well implemented
your
>formula.

The results show you have done it right.

Here is the relevant snippet:

//============================================================================
double DftHelper::getDawgFrequency( int            argBaseBinNumber, 
                                    ComplexDouble* argBaseOfBinValues )
{
        ComplexDouble theNumerator;
        ComplexDouble theDenominator;
        
        ComplexDouble theQuotient;
        
        int theBase = 3 * argBaseBinNumber;
        
        theNumerator.setTo( 0.0, 0.0 );
        theNumerator.addProductOf( &myWB[theBase],  
&argBaseOfBinValues[0] );
        theNumerator.addProductOf( &myWB[theBase+1],
&argBaseOfBinValues[1] );
        theNumerator.addProductOf( &myWB[theBase+2],
&argBaseOfBinValues[2] );
        
        theDenominator.setTo( 0.0, 0.0 );
        theDenominator.addProductOf( &myW[0], &argBaseOfBinValues[0] );
        theDenominator.addProductOf( &myW[1], &argBaseOfBinValues[1] );
        theDenominator.addProductOf( &myW[2], &argBaseOfBinValues[2] );
        
        theQuotient.setToQuotientOf( &theNumerator, &theDenominator );
        
        if( theQuotient.myReal >  1.0 ) theQuotient.myReal =  1.0;
        if( theQuotient.myReal < -1.0 ) theQuotient.myReal = -1.0;
        
        double theAlpha = acos( theQuotient.myReal );
               
        return theAlpha * myAngleToBin;
}
//============================================================================

It is coded for clarity, not performance.  An optimizer will probably
inline the routine calls.  The bounds checks are for cases where the noise
distorts the DFT so badly the bins no longer look like they hold a tone.

I'm not too keen on having all the code published, but I'd like you to
have it in case you find it useful.  I found your email address and will
send it after I polish it up a little bit.

>
>>Increasing the number of samples is definitely a noise mitigation
>>technique.
>
>Sure! I was surprised by the way that you did your first tests with only
8
>(if I remember well) samples. I used for the graph 128 samples (it's a
>little arbitrary of course!). But there are enough parameters to test for

>with the SNR and the frequency...
>
>Kind regards,
>
>Julien
>
>---------------------------------------
>Posted through http://www.DSPRelated.com

I used a very small number of samples (10) to demonstrate the benefit of
having an exact formula.  Candan's formula improves with the sample count.
 I plan on doing another data run to show this numerically, but it
requires a little more coding.

Thanks again,
Ced

---------------------------------------
Posted through http://www.DSPRelated.com

[...snip...]
>>
>>Please don't misquote my claims.  
>
>Whoa.  Wait a minute!  I did NOT misquote your 
>claims.  You used the word "Exact" in the title of 
>your blogs and I merely suggested that the word "exact" 
>should be avoided when discussing signal processing 
>algorithms. 

The full title is "Exact Frequency Formula for a Pure Real Tone in a DFT".
 Its subject matter is theoretical.  You won't find a pure real tone in
practice any more than you will find mathematical exactness.  Does that
also mean you think that the term "pure tone" should be avoided when
discussing DSP algorithms?

This kind of goes back to Plato and the discussion of a perfect circle.

>
>Cedron, it's clear to us that you have some 
>DSP skills and knowledge.  OK, this is a good thing 
>so no problem there.  Your problem is that you "come on" 
>far too strong.  You write things similar to"
>
>   "I have solved a DSP problem that no one else 
>    has ever solved.  And my solution is perfect, 
>    it's absolutely God-damned exactly perfect!"
>
What I have said is close to that:

I have solved a fundamental DSP problem that no one else seems to have
solved.  And my solution is exact, it's absolutely mathematically exact.

I have presented the solution, and tried to explain it.  I have done an
extensive amount of searching and not found it.  I have asked the experts
in the field whether they are aware of it.  So far, no affirmative
replies.  Therefore "no one else has ever solved" appears to be holding
true.

>What you should write is:
>
>   "I have a solution to a DSP problem that I think 
>    is accurate.  Please have a look at my solution 
>    and I welcome any and all comments."
>

I know it is accurate in the noiseless case, I think it is good in the
noisy case.  You can not say I haven't been pleading for people to look at
it.  Please look at it and tell me you understand it.  

>I bought an English translation of Sir Issac Newton's 
>'Principia' at a garage sale last week.  (Newton's 
>Principia is considered by many historians to be one 
>of the greatest discourses ever produced by the 
>human intellect.)  In the Preface of that book Newton,
>with humility, tells his readers that he presents his 
>thoughts not to invite criticism but rather to invite 
>suggestions on how to make his material better.
>
>Cedron, if Sir Issac Newton can show a little humility 
>so can you.

Here are some statements of mine from earlier posts:

"I am not sure I will do an article on the impact of noise.  I am simply
not knowledgeable enough to get the terminology right."

"I do not consider myself a good writer, but the math should be able to
stand on its own."

"I've started every article with "This is an article to hopefully give a
better understanding of the Discrete Fourier Transform (DFT) by...." not
"Here is a superior method to current techniques....""

"I've brought the equations to the table.  I understand their performance
from a theoretical point of view and by testing them empirically.  I am
not the best person to do standard performance evaluation.  Anybody who is
so able is welcome to do so."

"As far as the realm of DSP goes, the DFT is my specialty area, although I
would not call myself an expert or a specialist in the field."

I am totally inviting criticism of the math.  Suggestions on how to
improve my presentation are also welcome.  (Including this post of
yours).

[...snip...]

>>Did you happen to see the delineation between "Mathematically Exact"
and
>>"Engineering Exact" that I made in the Matlab thread?
>
>What does a Matlab thread have to do with my 
>claim that the word "exact" should be avoided in
>the literature of DSP?
>
>[-Rick-]

What I said is:
"In mathematics, "exact" means accurate with infinite precision.  In
engineering, "exact" means matching in value to the limit of the
measurement resolution."

A more concise way of putting it is "Analytical vs Numerical".  

I think the word "exact" is perfectly appropriate in theoretical
discussions.  I also think there is room in DSP literature for theoretical
topics, not all of it should application oriented.  Your opinion may
differ, I don't really want to argue about preferences.

Ced
---------------------------------------
Posted through http://www.DSPRelated.com

I wrote:

>It is coded for clarity, not performance.  

The word wrapping kind of ruins that.  Sorry.

Ced
---------------------------------------
Posted through http://www.DSPRelated.com