Detecting DTMF Pulses

OK, determining the two frequencies of a single DTMF pulse where the
starting location of the pulse (mark) is known, under ideal conditions,
using Goertzel , is like taking candy from a baby.

After recording a sequence of marks and spaces I can once again determine
their associated frequencies.  The way I do this is to visually look at the
recorded waveform. From there I can see the starting location of each mark.
I then just manually enter them into my program. With Goertzel to the
rescue the desired results are achieved.

However, my question is how can I automate this process by finding the
approximate location of each mark so I do not have to visually inspect the
wave sequence?

Due note, this is to be done in S/W on a PC, and that mark and space
durations are of random length.  Where a mark is the DTMF part and a space
is what is between a mark. The total amount of samples will be made
available for block processing as opposed to a real-time solution. It can
be assumed that the mark amplitudes are much larger than the space
amplitudes. Mark amplitudes are similar. For now, background noise and
harmonics need not be considered.

Suggestions would be appreciated on how to go about this task or what
methods can be used so I can google.

Thank-you


	 

_____________________________		
Posted through www.DSPRelated.com

On Thu, 06 Mar 2014 20:50:58 -0600, "Stacy" <22429@dsprelated> wrote:

>OK, determining the two frequencies of a single DTMF pulse where the
>starting location of the pulse (mark) is known, under ideal conditions,
>using Goertzel , is like taking candy from a baby.
>
>After recording a sequence of marks and spaces I can once again determine
>their associated frequencies.  The way I do this is to visually look at the
>recorded waveform. From there I can see the starting location of each mark.
>I then just manually enter them into my program. With Goertzel to the
>rescue the desired results are achieved.
>
>However, my question is how can I automate this process by finding the
>approximate location of each mark so I do not have to visually inspect the
>wave sequence?
>
>Due note, this is to be done in S/W on a PC, and that mark and space
>durations are of random length.  Where a mark is the DTMF part and a space
>is what is between a mark. The total amount of samples will be made
>available for block processing as opposed to a real-time solution. It can
>be assumed that the mark amplitudes are much larger than the space
>amplitudes. Mark amplitudes are similar. For now, background noise and
>harmonics need not be considered.

If this is a continuous process, I would suggest simulating an
envelope detection circuit to track the peak volume.  Use appropriate
filtering and slew-limiting to avoid responding to extremely short
noise spikes.  Then set a threhhold for mark detection based on a
fraction of the filtered peak amplitude.  If you want to get fancier,
run a separate filtered track of the minimum volume to discover the
noise floor.  Then use a threshhold that is about half way between
these two tracks.  If this is not a continuous process and you have
the luxury of processing an entire block of data multiple times, then
you can do an even better job by setting the threshold equal to the
median volume level.  As before, use slew-limiting and filtering to
avoid responding to spike noise.  And if you have all the time in the
world, you can do even better by performing multiple DTMF trial
extractions on all possible segments of the time series and keeping
only those trial DTMF extractions for which the quality of the pitch
extraction was high.  The quality would be defined by how much of the
energy was concentrated in the neighborhood of the expected DTMF
frequencies.  This last method can dig the information out of noise
that would otherwise foil the envelope detection scheme, but this
method is usually not practical from a computing resources point of
view.  If you are able to visually distinguish marks and spaces in the
waveform, chances are the envelope detection and volume threshhold
method will work just fine.

On Thu, 06 Mar 2014 20:50:58 -0600, "Stacy" <22429@dsprelated> wrote:

>OK, determining the two frequencies of a single DTMF pulse where the
>starting location of the pulse (mark) is known, under ideal conditions,
>using Goertzel , is like taking candy from a baby.
>
>After recording a sequence of marks and spaces I can once again determine
>their associated frequencies.  The way I do this is to visually look at the
>recorded waveform. From there I can see the starting location of each mark.
>I then just manually enter them into my program. With Goertzel to the
>rescue the desired results are achieved.
>
>However, my question is how can I automate this process by finding the
>approximate location of each mark so I do not have to visually inspect the
>wave sequence?
>
>Due note, this is to be done in S/W on a PC, and that mark and space
>durations are of random length.  Where a mark is the DTMF part and a space
>is what is between a mark. The total amount of samples will be made
>available for block processing as opposed to a real-time solution. It can
>be assumed that the mark amplitudes are much larger than the space
>amplitudes. Mark amplitudes are similar. For now, background noise and
>harmonics need not be considered.
>
>Suggestions would be appreciated on how to go about this task or what
>methods can be used so I can google.
>
>Thank-you

Search on "energy detection" or "envelope detection" to start with.  

You can combine the DTMF detection and energy detection as well if
you're careful, since the DFT turns the tone(s) into spike(s) there's
a lot of processing gain to exploit.   If you use a sliding-window DFT
instead of Goertzel you can let it run continuously and detect the
presence and frequency of the tones simultaneously using some
straightforward threshold techniques.


Eric Jacobsen
Anchor Hill Communications
http://www.anchorhill.com

Thank you Robert and Eric for the timely responses. 


____||||_____||||__||||____||||__|||||___
    ||||     ||||  ||||    ||||  |||||   


Above is what my samples may look like. The vertical hashes represent a sum
of two sinusoids. The vertical axis represents amplitude. The horizontal
axis represents each sample number. 

With my computer program I manually enter the starting point of each
sequence of vertical hashes. And the DTMF number is decoded.

However, I wish for the approximate location of each starting location to
be determined using only S/W. 
So, I do not believe an envelope detector can be used for a S/W only
solution.

I found the below link.

http://netwerkt.wordpress.com/2011/08/25/goertzel-filter/

Maybe implementing the "tandemRTgoertzelFilter" will work.



	 

_____________________________		
Posted through www.DSPRelated.com

On 3/6/14 9:50 PM, Stacy wrote:
> OK, determining the two frequencies of a single DTMF pulse where the
> starting location of the pulse (mark) is known, under ideal conditions,
> using Goertzel , is like taking candy from a baby.
>

well, i dunno if Goertzel is the best way to do it anyway.

...
> Suggestions would be appreciated on how to go about this task or what
> methods can be used so I can google.


long ago, a little after i got my very first email address, i did a DTMF 
detector for someone and i didn't use Goertzel, but instead tuned a 
bunch of these peaking EQs (a wire in parallel with a BPF and gain) 
tuned to the various DTMF frequencies (i think there were 8) and, i 
think, some ring tones and a hangup blip, too.

i think i had a peak gain of 40 dB and the half dB bandwidth was equal 
to the spacing of the DTMF frequencies.

then a simple squared-envelope-follower after each one and a set of 
thresholds (that were relative to the total power of the signal).  and a 
little logic in the software (like make sure there are not two or more 
columns lit up and same with the rows).  worked fine, but i didn't do 
any quantitative study on it about how good.  it just worked.


-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

On Fri, 07 Mar 2014 14:36:02 -0600, "Stacy" <22429@dsprelated> wrote:

>Thank you Robert and Eric for the timely responses. 
>
>
>____||||_____||||__||||____||||__|||||___
>    ||||     ||||  ||||    ||||  |||||   
>
>
>Above is what my samples may look like. The vertical hashes represent a sum
>of two sinusoids. The vertical axis represents amplitude. The horizontal
>axis represents each sample number. 
>
>With my computer program I manually enter the starting point of each
>sequence of vertical hashes. And the DTMF number is decoded.
>
>However, I wish for the approximate location of each starting location to
>be determined using only S/W. 

You can do that in either the time or frequency domain, as I described
previously.   There are some advantages to doing it in the frequency
domain since the tone locations are known.

>So, I do not believe an envelope detector can be used for a S/W only
>solution.

Why not?  An envelope or energy detector isn't any more complex than
the Goertzel algorithm or other DTMF detectors.

>I found the below link.
>
>http://netwerkt.wordpress.com/2011/08/25/goertzel-filter/
>
>Maybe implementing the "tandemRTgoertzelFilter" will work.
>
>
>
>	 
>
>_____________________________		
>Posted through www.DSPRelated.com

Eric Jacobsen
Anchor Hill Communications
http://www.anchorhill.com

On Fri, 07 Mar 2014 14:36:02 -0600, Stacy wrote:

> Thank you Robert and Eric for the timely responses.
> 
> 
> ____||||_____||||__||||____||||__|||||___
>     ||||     ||||  ||||    ||||  |||||
> 
> 
> Above is what my samples may look like. The vertical hashes represent a
> sum of two sinusoids. The vertical axis represents amplitude. The
> horizontal axis represents each sample number.
> 
> With my computer program I manually enter the starting point of each
> sequence of vertical hashes. And the DTMF number is decoded.
> 
> However, I wish for the approximate location of each starting location
> to be determined using only S/W.
> So, I do not believe an envelope detector can be used for a S/W only
> solution.

Then you don't understand what Robert and Eric are trying to tell you.

If envelope detection can work at all (and it can work well here), then 
it can work just fine in a "software only" case (by which I assume you 
mean "unsupervised", not "software aided by hardware").

Implemented properly, bandpass filtering of any sort (of which the 
sliding DFT is one sort) followed by envelope detection should give you 
eight "envelope" outputs (seven if you're only implementing a 3x4 DTMF 
keypad).  When a tone happens, two of these outputs should stand head and 
shoulders above the rest.  You should be able to detect the beginning and 
ending of each digit quite easily, unless there is noise present.

-- 
Tim Wescott
Control system and signal processing consulting
www.wescottdesign.com

On Thursday, March 6, 2014 9:50:58 PM UTC-5, Stacy wrote:
> OK, determining the two frequencies of a single DTMF pulse where the
> 
> starting location of the pulse (mark) is known, under ideal conditions,
> 
> using Goertzel , is like taking candy from a baby.
> 
> 
> 
> After recording a sequence of marks and spaces I can once again determine
> 
> their associated frequencies.  The way I do this is to visually look at the
> 
> recorded waveform. From there I can see the starting location of each mark.
> 
> I then just manually enter them into my program. With Goertzel to the
> 
> rescue the desired results are achieved.
> 
> 
> 
> However, my question is how can I automate this process by finding the
> 
> approximate location of each mark so I do not have to visually inspect the
> 
> wave sequence?
> 
> 
> 
> Due note, this is to be done in S/W on a PC, and that mark and space
> 
> durations are of random length.  Where a mark is the DTMF part and a space
> 
> is what is between a mark. The total amount of samples will be made
> 
> available for block processing as opposed to a real-time solution. It can
> 
> be assumed that the mark amplitudes are much larger than the space
> 
> amplitudes. Mark amplitudes are similar. For now, background noise and
> 
> harmonics need not be considered.
> 
> 
> 
> Suggestions would be appreciated on how to go about this task or what
> 
> methods can be used so I can google.
> 
> 
> 
> Thank-you
> 
> 
> 
> 
> 
> 	 
> 
> 
> 
> _____________________________		
> 
> Posted through www.DSPRelated.com

Besides what the others have added, an advantage of using a DFT based approach, is you can use parseval's relation to ensure that when you do detect tones, that most of the energy is in just those tones and not across the whole band. Try running voice through your DTMF detector and see how much spoofing you get.

Another approach passes the data through a pair of band pass filters. One band covers the row frequencies and the other band covers the column frequences. After each of these two filters you perform a 2nd order LPC analysis which will directly yield the frequency of the row and column tones. A 2nd order LPC is not very complicated. There are some appnotes that cover this process. While some LPC variations on DTMF decoding were patented, some of those patents are now expired. If you are just doing this for yourself, then the patent issue doesn't really matter; if not just for you - be advised.

Clay

Clay, thanks for taking the time to respond with your suggestions. 

This project is for my own curiosity so patent issues do not apply.

To reiterate,  at this stage I have either ‘spaces’ that have a value
close to zero or two sine waves (marks).  The challenge right now is
finding where the ‘marks’ are. If I expand this project I know there
will a whole plethora of other issues to resolve.

I) It has been suggested to use a sliding DFT.
   Q1) Is that an actual plain vanilla DFT or an FFT?
   Q2) If it is a DFT, can I assume it would mean making the calculation  
          using only the 8 DTMF tones?

II) It has also been suggested to use an envelope detector. My search
entering the phrase  “envelope detector” has not been too fruitful
since I wish to implement this in s/w. I seem to be getting the notion from
my search that using the Hilbert Transform for an envelope detector is the
way to go. I’m also still trying to find a s/w  algorithm to do the
Hilbert Transform.

III)  Of course there is the old fashion way. Just use brute force and
count the samples that exceed an absolute value threshold and there is the
location of a ‘mark’. Knowing that, Goertzel works fine.

But honestly, I would prefer to achieve desired results with a more elegant
solution such as (I) or (II), rather than what amounts to counting on my
fingers and toes.

Cheers

	 

_____________________________		
Posted through www.DSPRelated.com

On Monday, March 10, 2014 2:54:30 PM UTC-4, Stacy wrote:
> Clay, thanks for taking the time to respond with your suggestions. 
> 
> 
> 
> This project is for my own curiosity so patent issues do not apply.
> 
> 
> 
> To reiterate,  at this stage I have either 'spaces' that have a value
> 
> close to zero or two sine waves (marks).  The challenge right now is
> 
> finding where the 'marks' are. If I expand this project I know there
> 
> will a whole plethora of other issues to resolve.
> 
> 
> 
> I) It has been suggested to use a sliding DFT.
> 
>    Q1) Is that an actual plain vanilla DFT or an FFT?
> 
>    Q2) If it is a DFT, can I assume it would mean making the calculation  
> 
>           using only the 8 DTMF tones?
> 
> 
> 
> II) It has also been suggested to use an envelope detector. My search
> 
> entering the phrase  "envelope detector" has not been too fruitful
> 
> since I wish to implement this in s/w. I seem to be getting the notion from
> 
> my search that using the Hilbert Transform for an envelope detector is the
> 
> way to go. I'm also still trying to find a s/w  algorithm to do the
> 
> Hilbert Transform.
> 
> 
> 
> III)  Of course there is the old fashion way. Just use brute force and
> 
> count the samples that exceed an absolute value threshold and there is the
> 
> location of a 'mark'. Knowing that, Goertzel works fine.
> 
> 
> 
> But honestly, I would prefer to achieve desired results with a more elegant
> 
> solution such as (I) or (II), rather than what amounts to counting on my
> 
> fingers and toes.
> 
> 
> 
> Cheers
> 
> 
> 
> 	 
> 
> 
> 
> _____________________________		
> 
> Posted through www.DSPRelated.com

Stacy,

Yes, a sliding DFT works with a single bin frequency. You can certainy adapt it to work with 8 bins. You certainly don't need to do the FFT as that is likely more computationaly costly than just 8 DFTS. The advantage of parseval's that I mentioned earlier is it relates the total energy in the time domain to the total energy in the frequency domain. In the standard case, you calculate the total energy in the frequency domain by using the two loadest frequencies and comparing that to the total time domain energy. Others have done antispoofing by not only doing DFTS for the 8 row/column frequencies but also at their 1st harmonic frequencies (double the fundamentals). Tones will have little energy in the 1st harmonic bins whereas voice often does.

You can do envelope detection with a Hilbert method, but it may be simpler to do by simply rectifying the signal (absolute value it) and feed that to an attack/decay function (fast attack - slow release). 



Very simple example - perform as iteration

1)  M=M*gamma
2)  if (|X|), then M= |X|
3) compare M to a trigger threshold

M is a memory element, and X is the current signal value. gamma is a decay factor; try something like 0.998


Since this is a personal project, there are lots of things for you to experiment with. Search for some appnotes for DTMF detection from the 1980s. There were quite a few back then dealing with DTMF, MF-R1, MFC-R2, and supervisory tone detection.

Clay