Cedron <103185@DSPRelated> replies to my post,>In a sense, my solution is "applying trigonometry to any three bin >values". And in the pure tone case, can be solved from any three bin set. > It can also be derived to use non-adjacent bins, but there is no >advantage to that.Agreed.>>My gut feeling (also unsupported by exact math) is that the best >>estimate will use all of the bins, but will weight them using >>some variant of maximal ratio combining so that the bins with >>the least energy do not contribute disproportionate noise to the result.>This conjecture flies in direct contradiction to the argument I made to >rbj in the Matlab thread concerning the DFT acting like a noise filter.I'll look back for that discussion.>Let me try to disavow you of your notion using some supported approximate >math.I haven't suggested using approximate math.>In the near integer case, the bin magnitudes drop off rapidly from >the peak bin. In these cases the amount of signal information in the >further distant bins are insignificant. They will quickly be swamped my >the noise level, so the SNR level will be so low that they can only bring >your average down, no matter what weighting scheme you use.I agree that you definitely want to underweight the signals with the least energy. The idea of throwing out the the N-3 worst signals is not a bad idea. That is one way of underweighting the weaker signals. But, it is not maximal ratio weighting (wherein the weaker signals are underweighted in proportion to their SNR, and which is provably optimal signal combining under very broad conditions.) For my 2-hypothesis example, from the post you just replied to, MRC is even tractable. There may not be much advantage but you should be able to see a definite fraction of a dB improvement relative to choosing just three signals and discarding the rest. You would have to simulate over a quite large dataset to observe this. I think the only reason you'd want to do such an investigation is to determine that your preferred method is close enough (in its low SNR performance) to a (much more complex) method more firmly rooted in estimation theory. Steve
Show me the numbers
Started by ●May 28, 2015
Reply by ●May 31, 20152015-05-31
Reply by ●May 31, 20152015-05-31
Hi Julien,>Hi Cedron, > >>1) How do you classify an algorithm that gives you "exact" results when >no >>noise is present and a good estimate when it is? > >Word wars... >I don't have a dog in this fight. I am truly curious whether there is a term specific to this. I don't think "unbiased" is the correct answer.>>2) Are there other "estimators" that give exact results when no noiseis>>present? > >For real tones, I don't know yet. But for complex tones, the answer is >yes! >For example, this very simple one (Lank-Reed-Pollon) : >w = tan^{-1}(E[x_n x_{n-1}^star]) > > > >--------------------------------------- >Posted through http://www.DSPRelated.comFor real tones, I have not found any other than the ones I derived. Complex tones are simpler to handle than real ones. I derived a frequency formula that is not as slick as the one you just gave. exp( i alpha ) = exp( i beta_n ) * ( Z_n - Z_{n+1} ) / ( Z_n - R_1 Z_{n+1} ) R_1 = exp( -i 2Pi/N ) Pulling the alpha out or pulling ( alpha - beta_n ) out requires using the atan2 function. Ced --------------------------------------- Posted through http://www.DSPRelated.com
Reply by ●May 31, 20152015-05-31
Cedron <103185@DSPRelated> wrote:>For real tones, I have not found any other than the ones I derived. > >Complex tones are simpler to handle than real ones.Why would the tones not be complex? (Assuming the sinusoid being estimated can have arbitrary phase.) Steve
Reply by ●May 31, 20152015-05-31
>Cedron <103185@DSPRelated> wrote: > >>For real tones, I have not found any other than the ones I derived. >> >>Complex tones are simpler to handle than real ones. > >Why would the tones not be complex? (Assuming the sinusoid being >estimated can have arbitrary phase.) > >Stevecos( alpha n + phi ) vs exp( i ( alpha n + phi ) ) The real tone can, of course, be considered the average of two complex tones. Having one exp instead of two (and no sum) makes the math a lot easier to handle. Ced --------------------------------------- Posted through http://www.DSPRelated.com
Reply by ●May 31, 20152015-05-31
> >1) Complex tone case: simply omega = tan^{-1}(x_1 * conj(x_0)) >Difficult to do simpler. And it is an exact solution. And completly >useless and inefficient in practice too. I hope this can convince you of >the non revelence of this criterium of "exactness". >I'm a precision freak. I can't help it. I understand your "practical uselessness" point. You can derive equations for the amplitude and phase from the same equations you use to derive the frequency formula. However, these equations are very noise sensitive, to the point of being useless. On a theoretical level though, they give a description of how these values are impacted by the signal parameters. My last blog article gives an alternative approach to solving for the amplitude and phase that is both exact in the noiseless case and robust in the presence of noise. Note it says "Method 1". "Method 2" (not yet written up) is an approximation that rapidly converges to the exact answer.>2) Real tone case: this is more complex (pun intended). >But a minimal answer could be found by writing 3 équations, but inthe>time domain, instead of the frequency domain as your solution: >A*cos(phi) = x0 >A*cos(2pi*f+phi) = x1 >A*cos(4pi*f+phi) = x2 > >And solving this 3 unknown / 3 équations systems (although non linear)(I>let you do the solving, I am not brave enough), will give you an >absolutely exact and absolutely inefficient solution too...And one that won't work in the presence of other tones.> >Fortunately, your formula is processing the 3 highest bins of the DFT, >this is why it is relatively efficient... > >--------------------------------------- >Posted through http://www.DSPRelated.comActually, my formula will work on any three bin set in the noiseless case. When noise (or other tones) is present, it will work best at the peak. Ced --------------------------------------- Posted through http://www.DSPRelated.com
Reply by ●May 31, 20152015-05-31
On Sun, 31 May 2015 20:45:01 +0000 (UTC), spope33@speedymail.org (Steve Pope) wrote:>Cedron <103185@DSPRelated> wrote: > >>[...snip...] >> >>>There really >>>is little place for "exact solutions" in DSP applications. Perhaps a >>>symbolic math group could be more interested. More of interest in DSP >>applications >>>are the details of implementation cost and the effects of noise and >>>interference. >> >>Don't you think it's good when practice rests on solid theory? > >I'm going to side with Cedron on this one. > >"Exact" statements can be made in mathematics. Everybody [*] knows >that you don't get these "exact" results in the real world, therefore, >nobody is confused when DSP engineers use the term "exact". >It simply means the claimed statement is correct as a mathematical >derivation. No big deal.It's even blurrier in the context of Fourier Transforms for this case. Since sinusoids aren't absolutely integrable, the exact FT doesn't "exist" (in the context of that definition), and is instead produced using the distribution properties of the dirac delta. It's not like the dirac delta hasn't caused its share of theoretical arguments, and it certainly doesn't exist in nature, so, as usual, the door is always open for debate. It's certainly close enough and powerful enough to do many useful things, but I think there are more than one valid viewpoints on how to interpret the analysis. Since all real-world analyses have limited time windows, this also stretches the meanings of "exact" in an absolute sense.>Along these lines, an unrelated example. I have always suspected >the black holes do not exist in reality, for the simple reason >that singularities tend to occur in mathematics but not in nature. >For many decades running, all except fringe astrophysicists >firmly believed (or at least would talk as though they believed) >that "black hole candidates" were indeed black holes, and not some >other sort of dense object. There were always doubters, but they >were always shouted down ... until Stephen Hawking became one >of the doubters.We do sometimes see the effects of natural similarities to mathematical singularities in systems that can run away or become unstable. It's not hard to make a filter or a feedback or control system with a pole in a bad place and watch it go nuts when the input pushes it to that region. Mechanical systems that do this can be pretty frightening. But, yeah, it often pays to be diligent about the differences between the assumptions made in the math and reality. Exact often isn't.>Steve > >[*] Everybody except astrophysicists apparently...Eric Jacobsen Anchor Hill Communications http://www.anchorhill.com
Reply by ●May 31, 20152015-05-31
Steve Pope <spope33@speedymail.org> wrote:>Cedron <103185@DSPRelated> replies to my post,>> spope33 wrote,>>>My gut feeling (also unsupported by exact math) is that the best >>>estimate will use all of the bins, but will weight them using >>>some variant of maximal ratio combining so that the bins with >>>the least energy do not contribute disproportionate noise to the result.>>This conjecture flies in direct contradiction to the argument I made to >>rbj in the Matlab thread concerning the DFT acting like a noise filter.>I'll look back for that discussion.Okay I believe I found the passage you are referencing: Cedron writes, The advantage of doing it in the frequency space, the DFT bins, is that the noise will be spread out over all the bins and thus only the noise that impacts the three bins will have any effect. So the DFT is acting as a noise filter. rbj replies, you can make the same case with 3 time-domain samples. the "noise" is spread out over all of the samples and only the noise that impacts the three samples selected will have any effect. estimating frequency from only 3 bins or from only 3 samples is not always the best idea. Here I largely agree with rbj. It is a happy coincidence that your equation requires values from three bins as its input, and under the application conditions much of the energy ends up in the strongest three bins, but this is all it really is -- a coincidence that you have used to good advantage. I am lukewarm on the expression that the DFT is acting as a noise filter, largely because a discrete transform is not the same as a bank of bandpass filters, although the two may share a few qualities so there is some truth here. (To extend the analogy, discarding the weak bins amounts to using a "noise gate".) "estimating frequency from only 3 bins or from only 3 samples is not always the best idea". I agree 100% with that. It may be as good as any effective algorithm in a given situation, but you are still throwing away some of your information, so it cannot be the optimal approach in general. Steve
Reply by ●May 31, 20152015-05-31
[...snip...]> >>I'll look back for that discussion. > >Okay I believe I found the passage you are referencing:This was actually prior to the post I was referring to. I've copied the text at the bottom of this post so you don't have to search again. [...snip...]> > estimating frequency from only 3 bins or from only 3 samples > is not always the best idea. > >Here I largely agree with rbj. > >It is a happy coincidence that your equation requires >values from three bins as its input, and under the application >conditions much of the energy ends up in the strongest three bins, >but this is all it really is -- a coincidence that you have used to good >advantage.Not a coincidence at all. It's the minimal solution space I've found so far. It is easy to extend it to larger bin sets, but I have not figured out how to reduce it to a two-bin formula.> >I am lukewarm on the expression that the DFT is acting as a noise >filter, largely because a discrete transform is not the same as a bank >of bandpass filters, although the two may share a few qualities so >there is some truth here. (To extend the analogy, discarding the weak >bins amounts to using a "noise gate".) > >"estimating frequency from only 3 bins or from only 3 samples is >not always the best idea". I agree 100% with that. It may >be as good as any effective algorithm in a given situation, but >you are still throwing away some of your information, so it >cannot be the optimal approach in general. > >SteveThe explanation below it to explain why using three bins from the DFT is superior to using 3 samples from the signal. It was a followup to the post you quoted. The real point being made was that one of the rationales for using more sample points (and thus a larger DFT) from the same interval was a noise mitigation technique. I can relate to your sentiment about throwing away information by discarding bin values that still have some signal in them. Unless you know the nature of the noise, you can not remove any of it and therefore it thwarts your ability to recover it. The information in those bins is redundant if you are talking about a single pure tone with added noise as the signal. If you have a more complex waveform, you may be able to improve your accuracy by finding the frequency of a higher harmonic and dividing by the proper number. I never claimed that my formula will have the best results of all techniques in a noisy case. What I have claimed is that it is the mathematically correct formula for a single pure real tone. I have also claimed that I expect it's behavior in the presence of noise to be comparable to Jacobsen's estimator. Ced ==================== excerpt from Matlab thread (May 27, page 8 on dsprelated forums interface) ================= I guess I kind of explained that poorly. You can only do it in the time domain if the signal is a single pure tone. Any harmonics are technically other tones. I need to explain the noise filter better. Suppose I have two dice and roll them. The expected outcome of their sum is 7, and there is a variance measure. I can consider there sum a biased random number. If I subtract 7, it becomes unbiased, the variance stays the same. If I roll the dice 10 times and take the average sum, the expected value is still 7, but the variance is much smaller. If I roll it 1000 times, the expected average value is still 7, but the variance is much much smaller. The more times I roll, the smaller the variance becomes, meaning the more exceptional it is that I would get a result very far from 7. The probability distribution approaches a narrower and narrower bell curve. So, if your noise is unbiased, the expected value is zero. When you take three points in the time domain, it is like rolling the dice three times. When you use three bins from the DFT, it is sort of like rolling the dice N times for each bin. I say "sort of" because you aren't doing a straight average, you are doing a weighted average based on the sine and cosine functions of the bin frequency. This is why I made the point to the OP, way earlier in this thread, that one of the reasons you might want use more sample points and a larger DFT is for noise mitigation. > >estimating frequency from only 3 bins or from only 3 samples is not>always the best idea. >The frequency formula, expressed as cos( alpha ) = WBZ / WZ can be arbitrarily extended to as many bins as you want with the proper selection of W. Each bin you add gives you one more degree of freedom in your selection. For simple tones, I can see no reason to do this. --------------------------------------- Posted through http://www.DSPRelated.com
Reply by ●June 1, 20152015-06-01
tsd82 <105802@DSPRelated> wrote:>PS: The first link is: http://92.243.17.111/uploads/mse.pdf >(without the comma at the end) >--------------------------------------- >Posted through http://www.DSPRelated.comJulien, this page is not loading for me. Is it still up? Thanks Steve
Reply by ●June 1, 20152015-06-01
On Sun, 31 May 2015 20:45:01 +0000 (UTC), spope33@speedymail.org (Steve Pope) wrote: <snip>>Along these lines, an unrelated example. I have always suspected >the black holes do not exist in reality, for the simple reason >that singularities tend to occur in mathematics but not in nature. >For many decades running, all except fringe astrophysicists >firmly believed (or at least would talk as though they believed) >that "black hole candidates" were indeed black holes, and not some >other sort of dense object. There were always doubters, but they >were always shouted down ... until Stephen Hawking became one >of the doubters.I'm no physicist, but can't you have a "black hole" (from which light can't escape) without having a singularity? Best regards, Bob Masta DAQARTA v7.60 Data AcQuisition And Real-Time Analysis www.daqarta.com Scope, Spectrum, Spectrogram, Sound Level Meter Frequency Counter, Pitch Track, Pitch-to-MIDI FREE Signal Generator, DaqMusiq generator Science with your sound card!
