Sign in

username:

password:



Not a member?

Search blogs



Search tips

Articles by category

Our Bloggers

See Also

Embedded SystemsFPGAElectronics

DSP Blogs > Steve Smith > Benford’s law solved with DSP

Steve Smith
Steve Smith specializes in developing novel imaging systems for medical, security, and industrial applications. His interests include: digital signal processing, analog electronics, x-ray physics and sensory systems. Dr. Smith is the author of "The Scientist and Engineer's Guide to Digital Signal Processing," freely distributed over the internet in electronic form. For the last ten years he has been the president and technical director of Spectrum San Diego, Inc., a research and development group specializing in imaging and instrumentation systems.

RSS Feed

Would you like to be notified by email when Steve Smith publishes a new blog?

  

Benford’s law solved with DSP

Posted by Steve Smith on Feb 22 2008 under Academia / Research   

I have a longtime interest in the mystery of 1/f noise. A few years ago I came across Benford’s law, another puzzle that seemed to have many of the same characteristics.

Suppose you collect a large group of seemingly random numbers, such as might appear in a newspaper or financial report. Benford’s law relates to the leading digit of each number, such as "4" in 4.268, "3" in 0.0312, and "9" in -932.34. Since there are nine possible leading digits (1-9), you would expect that each of the leading digits would occur 1/9th of the time. The problem is, Nature doesn’t agree with your thinking-- the leading digit is a "1" over 30% of the time. This is hard to believe at first encounter, but it is absolutely true. And this is only the start-- there are other properties that seem simply bizarre.

The search to explain Benford’s law has a colorful history extending over more than 80 years. Explanations abound on the internet, from well respected mathematicians to kooks and clowns. One of the most interesting claims is that Benford’s law represents an underlying property of our existence, something akin to the paranormal. And this isn’t from the kooks, it's from the mathematicians! On balance, most mathematicians have viewed Benford’s law as a minor mystery, not a major problem to solve.

So, I started looking at Benford’s law in an attempt to understand 1/f noise. It shouldn’t be surprising that I used the tools I’m familiar with, convolution, Fourier analysis, and the like. The big surprise is that it worked unbelievably well. Signal processing provides an elegant solution to the mystery of Benford’s law.

Now you are probably wondering why I didn’t publish this result in some prominent journal. Well, I tried. No argument with the math, just not enough interest in the topic from professional mathematicians. They suggested it be submitted to a lesser known journal.

On reflection, I realized that there was something much more valuable in this work. The solution to Benford’s law is interesting in itself, but the way it is solved is far more important. Those in DSP are familiar with signals in the time domain. They are also familiar with signals in the spacial domain, such as images. The solution to Benford’s law shows how signal processing techniques can be applied to another important domain, the number line.

So I made the decision that I would forego traditional publication, and make the material available as a chapter of my on-line book. And here it is:

 

html: http://www.dspguide.com/ch34.htm

pdf (better): http://www.dspguide.com/CH34.PDF

 

Now the big question: Does the explanation of Benford’s law provide insight into 1/f noise? Not a damn bit. Back to where I started.

Steve Smith

2/22/2008



Rate this article:
5
Rating: 5 | Votes: 2
 
posted by Steve Smith
Steve Smith specializes in developing novel imaging systems for medical, security, and industrial applications. His interests include: digital signal processing, analog electronics, x-ray physics and sensory systems. Dr. Smith is the author of "The Scientist and Engineer's Guide to Digital Signal Processing," freely distributed over the internet in electronic form. For the last ten years he has been the president and technical director of Spectrum San Diego, Inc., a research and development group specializing in imaging and instrumentation systems.

Previous post by Steve Smith: Waveforms that are their own Fourier Transform
all articles by Steve Smith

Would you like to be notified by email when Steve Smith publishes a new blog?

  


Comments


 

steveu wrote:

2/25/2008
 
That's s a really nice way of analysing the Benford rule/effect/not-really-a-law, as it homes right in on the rhythmic behaviour of working through successive decades. Although the result is basically just an artifact of using base 10 numbers, I understand it does seem to have practical application. I believe it is used to scan for fraud in financial markets. If someone selectively tampers with numbers, and doesn't appreciate the need to maintain the Benford behaviour, they might be spotted. However, I guess that will only home in on the less able. The smart guys know the numeric patterns they need to achieve to fool the watchdog.
 

steph_tsf wrote:

3/16/2008
 
Fantastic constribution, as usual from Steve Smith.
Would be interesting to see if a "genuine" audio .wav file recorded from nature sounds like wild life, human voice, or music, are following benford's law, when converted into decimal of floating point. Surely they do ! Or maybe not ?
Benfor's law might be THE tool for detecting (alien) intelligence, but we are immediately losing the clue when digitizing the (alien) signal using 0/1 digital numbering scheme, isn't ?
Or not ?
Tere is maybe a thin chance for Benfor's law to become another tool for assessing audio equipment quality and fidelity.
Thanks, Steve !
Steph
 

JonTitus wrote:

2/9/2009
 
Good to find your site. Benford's 30% rule triggered a look at my old Post slide rule. About 31% of the distance covers values between 1 and 2. The other 69% covers values from >2 to 10. Some companies use Benford's rule to monitor travel and entertainment expenses that help identify employees who make up random numbers for phony expenses.

Add a Comment
You need to login before you can post a comment (best way to prevent spam). ( Not a member? )

Fatal error: Call to a member function finish() on a non-object in /home/dsprelat/public_html/new/showarticle.php on line 432