newbee fft question, decimate or mix??

I am attempting to use a PC soundcard and Spectrum Lab software to
detect the presence of a very weak input signal at 600 Hz.

I know the frequency of the input signal and I know it does not drift
(drift is less than 1 ppm), so I am only interested in a very limited
portion of the spectrum around 600 Hz.

I would like to get .001 hz bins, (or narrower) in order to have the
best chance of seeing the weak input signal.

Should I decimate the input signal by 128 X (sampled at 5.512 Khz
originally), then do a 32768 point fft on the remaining 0 to 43 hz in
order to get my .001 Hz bins???

Or.... should I mix a 599 Hz sinewave and the 600 Hz signal and try to
detect it at 1 Hz (1024 point fft), which should yield .001 Hz bins??

I think both methods will work, but which method will give me the
quickest indication that the 600 Hz weak signal is present in the
least amount of elapsed time?

I can sample up to 96 Khz rate, but am not sure if there is any
benefit to sampling at a higher rate.

Also, I DO NOT care about the power distribution in the rest of the
spectrum, so what happens at 30 , 400 or 700 hz is not important, so I
do not need to see those results.

Thanks for  any an all guidance.

Albert

On Thu, 08 Jan 2004 23:27:47 -0500, albert <> wrote:

>I am attempting to use a PC soundcard and Spectrum Lab software to
>detect the presence of a very weak input signal at 600 Hz.
>
>I know the frequency of the input signal and I know it does not drift
>(drift is less than 1 ppm), so I am only interested in a very limited
>portion of the spectrum around 600 Hz.
>
>I would like to get .001 hz bins, (or narrower) in order to have the
>best chance of seeing the weak input signal.
>
>Should I decimate the input signal by 128 X (sampled at 5.512 Khz
>originally), then do a 32768 point fft on the remaining 0 to 43 hz in
>order to get my .001 Hz bins???
>
>Or.... should I mix a 599 Hz sinewave and the 600 Hz signal and try to
>detect it at 1 Hz (1024 point fft), which should yield .001 Hz bins??
>
>I think both methods will work, but which method will give me the
>quickest indication that the 600 Hz weak signal is present in the
>least amount of elapsed time?
>
>I can sample up to 96 Khz rate, but am not sure if there is any
>benefit to sampling at a higher rate.
>
>Also, I DO NOT care about the power distribution in the rest of the
>spectrum, so what happens at 30 , 400 or 700 hz is not important, so I
>do not need to see those results.
>
>Thanks for  any an all guidance.
>
>Albert

Hi Albert,

  Maybe you should consider the "Goertzel 
algorithm", a "2nd-order complex IIR filter", 
or the "sliding DFT" to detect 
the signal at 600 Hz.  All of these methods are 
more efficient than an FFT.

Good Luck,
[-Rick-]

On Fri, 09 Jan 2004 05:28:14 GMT, r.lyons@REMOVE.ieee.org (Rick Lyons)
wrote:

>On Thu, 08 Jan 2004 23:27:47 -0500, albert <> wrote:
>
>>I am attempting to use a PC soundcard and Spectrum Lab software to
>>detect the presence of a very weak input signal at 600 Hz.
>>
>>I know the frequency of the input signal and I know it does not drift
>>(drift is less than 1 ppm), so I am only interested in a very limited
>>portion of the spectrum around 600 Hz.
>>
>>I would like to get .001 hz bins, (or narrower) in order to have the
>>best chance of seeing the weak input signal.
>>
>>Should I decimate the input signal by 128 X (sampled at 5.512 Khz
>>originally), then do a 32768 point fft on the remaining 0 to 43 hz in
>>order to get my .001 Hz bins???
>>
>>Or.... should I mix a 599 Hz sinewave and the 600 Hz signal and try to
>>detect it at 1 Hz (1024 point fft), which should yield .001 Hz bins??
>>
>>I think both methods will work, but which method will give me the
>>quickest indication that the 600 Hz weak signal is present in the
>>least amount of elapsed time?
>>
>>I can sample up to 96 Khz rate, but am not sure if there is any
>>benefit to sampling at a higher rate.
>>
>>Also, I DO NOT care about the power distribution in the rest of the
>>spectrum, so what happens at 30 , 400 or 700 hz is not important, so I
>>do not need to see those results.
>>
>>Thanks for  any an all guidance.
>>
>>Albert
>
>Hi Albert,
>
>  Maybe you should consider the "Goertzel 
>algorithm", a "2nd-order complex IIR filter", 
>or the "sliding DFT" to detect 
>the signal at 600 Hz.  All of these methods are 
>more efficient than an FFT.
>
>Good Luck,
>[-Rick-]


Hi Rick,

At this point, I will need 'good luck'.

I've read a lot in this newsgroup and managed to get a basic (but
probably flawed) understanding of FFT. Unfortunately, I am challenged
with respect to programming ability, so I have to use software written
by others.

I have a perfectly good Analog Devices E-Z lite DSP development board
sitting in a box right now because I bought it, only to discover that
I can't write software to make it do what I'd like it to do.

I just googled for 'IIRFilter' and 'sliding DFT' and found some
promising links. But, they are all meant for people who know heavy
duty math and have abilities well beyond mine.

I thank you for your suggestion very much though, if FFT is barking up
the wrong tree, I appreciate knowing it at this early stage!

Regards

Albert

albert wrote:

> I am attempting to use a PC soundcard and Spectrum Lab software to
> detect the presence of a very weak input signal at 600 Hz.
> 
> I know the frequency of the input signal and I know it does not drift
> (drift is less than 1 ppm), so I am only interested in a very limited
> portion of the spectrum around 600 Hz.
>[MASSIVE snip]

As a "newbie to newbie" suggestion

1. Sample at highest possible rate.
2. Save as a *.wav file
3. Download Scilab ( or other Mathlab substitute/clone )
4. Scilab ( at least ) seems to nicely handle L-O-N-G FFT's.
    So do an FFT on a second's worth of data.
5. Does that meet your (many unstated) requirements?
    If so, DONE
    If not, ask more questions

albert wrote:

> I am attempting to use a PC soundcard and Spectrum Lab software to
> detect the presence of a very weak input signal at 600 Hz.
> 
> I know the frequency of the input signal and I know it does not drift
> (drift is less than 1 ppm), so I am only interested in a very limited
> portion of the spectrum around 600 Hz.
> 
> I would like to get .001 hz bins, (or narrower) in order to have the
> best chance of seeing the weak input signal.
> 
> Should I decimate the input signal by 128 X (sampled at 5.512 Khz
> originally), then do a 32768 point fft on the remaining 0 to 43 hz in
> order to get my .001 Hz bins???
> 
> Or.... should I mix a 599 Hz sinewave and the 600 Hz signal and try to
> detect it at 1 Hz (1024 point fft), which should yield .001 Hz bins??
> 
> I think both methods will work, but which method will give me the
> quickest indication that the 600 Hz weak signal is present in the
> least amount of elapsed time?
> 
> I can sample up to 96 Khz rate, but am not sure if there is any
> benefit to sampling at a higher rate.
> 
> Also, I DO NOT care about the power distribution in the rest of the
> spectrum, so what happens at 30 , 400 or 700 hz is not important, so I
> do not need to see those results.
> 
> Thanks for  any an all guidance.

Hi Albert,

Let me give you another tack to Rick's Goertzel suggestion: try FIR filtering
the signal through a very narrow bandpass 600 Hz filter, then look at
the energy at the output. It's a simple-minded approach that may work.
You can come up with a threshold experimentally.

Myself or others around here could provide you with the coefficients. An
FIR is easy to implement, and if you're doing this on a PC you probably have
gobs of cycles to waste anyway (FIR is more computationally expensive than
Goertzel).
-- 
%  Randy Yates                  % "...the answer lies within your soul
%% Fuquay-Varina, NC            %       'cause no one knows which side
%%% 919-577-9882                %                   the coin will fall."
%%%% <yates@ieee.org>           %  'Big Wheels', *Out of the Blue*, ELO
http://home.earthlink.net/~yatescr

albert wrote:
 > [...]
> I think both methods will work, but which method will give me the
> quickest indication that the 600 Hz weak signal is present in the
> least amount of elapsed time?

How fast do you need it and what is the smallest SNR you need to
detect?
-- 
%  Randy Yates                  % "...the answer lies within your soul
%% Fuquay-Varina, NC            %       'cause no one knows which side
%%% 919-577-9882                %                   the coin will fall."
%%%% <yates@ieee.org>           %  'Big Wheels', *Out of the Blue*, ELO
http://home.earthlink.net/~yatescr

On Fri, 09 Jan 2004 15:52:10 -0600, Richard Owlett
<rowlett@atlascomm.net> wrote:

>albert wrote:
>
>> I am attempting to use a PC soundcard and Spectrum Lab software to
>> detect the presence of a very weak input signal at 600 Hz.
>> 
>> I know the frequency of the input signal and I know it does not drift
>> (drift is less than 1 ppm), so I am only interested in a very limited
>> portion of the spectrum around 600 Hz.
>>[MASSIVE snip]
>
>As a "newbie to newbie" suggestion
>
>1. Sample at highest possible rate.
>2. Save as a *.wav file
>3. Download Scilab ( or other Mathlab substitute/clone )
>4. Scilab ( at least ) seems to nicely handle L-O-N-G FFT's.
>    So do an FFT on a second's worth of data.
>5. Does that meet your (many unstated) requirements?
>    If so, DONE
>    If not, ask more questions
>


I need real time, or near real time, but I could work around the
'record file, then analyze'  procedure you suggest.

With .001 Hz (or narrower) bins, expect I'd need about 1000 seconds
(over 15 minutes) to get my first set of data points, I'm not sure
what good a 1 minute wav recording would do. This isn't DSP, it's
physics.....even if I could build a mechanical filter with that high
of a Q, it would still take atleast 15 minutes to get any output
because of the extreme narrow band output.

Now, if I do use FFT, will your SCILAB software do .001 Hz bins over a
750 Hz bandwidth so I can look at the spectrum around 750 Hz with the
required narrow bins??

At a 5 Khz sample rate (lowest that PC's will go when using a sound
card), I would need to decimate the input signal by 4 (giving me 0 to
1200 Hz bandwidth). I would then have to do a  1.04 million point fft.
I have no knowledge of anyone using larger than 65k point fft's, but
it might be possible. I think it might be somewhat impossible or at
the very least, it would not be practical because of the large quan of
temporary data that has to be stored and manipulated.

My idea was to mix the input signal with a 650 Hz oscillator and
extract the difference frequency. Then, do an FFT on that difference
frequency since it is much lower after the mixing.

Say my desired signal is at 654 Hz. I mix 650 from a crystal
oscillator and extract dc to 4 Hz (after running the mixer output
through a low pass filter). At this point, my 654 Hz signal is now at
4 Hz! Feed this 4 Hz signal into an 4096 fft, and I now get .001 Hz
bins rather easily and without dedicating a super computer to do a
1.04 mb.

So, should I mix (downconvert) or try to do the fft at 700 Hz with a 1
mb point fft?

Al 

On Fri, 09 Jan 2004 22:07:32 GMT, Randy Yates <yates@ieee.org> wrote:

>albert wrote:
>
>> I am attempting to use a PC soundcard and Spectrum Lab software to
>> detect the presence of a very weak input signal at 600 Hz.
>> 
>> I know the frequency of the input signal and I know it does not drift
>> (drift is less than 1 ppm), so I am only interested in a very limited
>> portion of the spectrum around 600 Hz.
>> 
>> I would like to get .001 hz bins, (or narrower) in order to have the
>> best chance of seeing the weak input signal.
>> 
>> Should I decimate the input signal by 128 X (sampled at 5.512 Khz
>> originally), then do a 32768 point fft on the remaining 0 to 43 hz in
>> order to get my .001 Hz bins???
>> 
>> Or.... should I mix a 599 Hz sinewave and the 600 Hz signal and try to
>> detect it at 1 Hz (1024 point fft), which should yield .001 Hz bins??
>> 
>> I think both methods will work, but which method will give me the
>> quickest indication that the 600 Hz weak signal is present in the
>> least amount of elapsed time?
>> 
>> I can sample up to 96 Khz rate, but am not sure if there is any
>> benefit to sampling at a higher rate.
>> 
>> Also, I DO NOT care about the power distribution in the rest of the
>> spectrum, so what happens at 30 , 400 or 700 hz is not important, so I
>> do not need to see those results.
>> 
>> Thanks for  any an all guidance.
>
>Hi Albert,
>
>Let me give you another tack to Rick's Goertzel suggestion: try FIR filtering
>the signal through a very narrow bandpass 600 Hz filter, then look at
>the energy at the output. It's a simple-minded approach that may work.
>You can come up with a threshold experimentally.
>
>Myself or others around here could provide you with the coefficients. An
>FIR is easy to implement, and if you're doing this on a PC you probably have
>gobs of cycles to waste anyway (FIR is more computationally expensive than
>Goertzel).


OK Rick,

I know nothing of FIR or Goertzel.

My signals are all very small, buried in tons of noise. With FFT at
.05 Hz bins, they are still very undectable. I can get .08 Hz bins
easily, but anything narrower than that seems to be easier said than
done:>:

I need bins narrow enough to see a 6db margin, something I can be
pretty sure isn't random noise.

I have no desire to hear the actual signal with my ear, if I can
display it in a waterfall type display, I'm a happy camper.

When you say 'very narrow 600 Hz filter'....how narrow can it really
get??? 

I'll look into FIR and see if it's something I can use, but I know
that others always use fft in similar aps.

By the way, I'm trying to detect over the horizon (non line of sight
troposcatter) laser signals. That's why I know the frequency signature
of the transmitter and how I know it is stable enough to employ narrow
fft bins.

Should I be looking into FIR, DFT or FFT?

Thanks,

A

albert wrote:
> On Fri, 09 Jan 2004 15:52:10 -0600, Richard Owlett
> <rowlett@atlascomm.net> wrote:
> 
> 
>>albert wrote:
>>
>>
>>>I am attempting to use a PC soundcard and Spectrum Lab software to
>>>detect the presence of a very weak input signal at 600 Hz.
>>>
>>>I know the frequency of the input signal and I know it does not drift
>>>(drift is less than 1 ppm), so I am only interested in a very limited
>>>portion of the spectrum around 600 Hz.
>>>[MASSIVE snip]
>>
>>As a "newbie to newbie" suggestion
>>
>>1. Sample at highest possible rate.
>>2. Save as a *.wav file
>>3. Download Scilab ( or other Mathlab substitute/clone )
>>4. Scilab ( at least ) seems to nicely handle L-O-N-G FFT's.
>>   So do an FFT on a second's worth of data.
>>5. Does that meet your (many unstated) requirements?
>>   If so, DONE
>>   If not, ask more questions
>>
> 
> 
> 
> I need real time, or near real time, but I could work around the
> 'record file, then analyze'  procedure you suggest.
> 
> With .001 Hz (or narrower) bins, expect I'd need about 1000 seconds
> (over 15 minutes) to get my first set of data points, I'm not sure
> what good a 1 minute wav recording would do. 

Not much ;)
That "minute" obviously should have been "15 minutes".


> Now, if I do use FFT, will your SCILAB software do .001 Hz bins over a
> 750 Hz bandwidth so I can look at the spectrum around 750 Hz with the
> required narrow bins??
> 
> At a 5 Khz sample rate (lowest that PC's will go when using a sound
> card), I would need to decimate the input signal by 4 (giving me 0 to
> 1200 Hz bandwidth). I would then have to do a  1.04 million point fft.


Scilab 2.7.2 on WinXP can do at least a 4 million point fft.

>> With .001 Hz (or narrower) bins, I'd need about 1000 seconds
>> (over 15 minutes) to get my first set of data points, I'm not sure
>> what good a 1 minute wav recording would do. 
>
>Not much ;)
>That "minute" obviously should have been "15 minutes".
>
>

OK, the 15 minute time required to get the first data point would be
for .001 Hz bins. To go to .0001 Hx bins, it would require about 160
minutes! I'm not sure my hard drive is large enough to store a 5.5 Khz
sampled wav for 1.5 hours:>:

 
>> At a 5 Khz sample rate (lowest that PC's will go when using a sound
>> card), I would need to decimate the input signal by 4 (giving me 0 to
>> 1200 Hz bandwidth). I would then have to do a  1.04 million point fft.
>
>
>Scilab 2.7.2 on WinXP can do at least a 4 million point fft.

OK, this is good news indeed. I don't look forward to learning a new
software package, but atleast I know it is within the realm of
possible options. I thank you for making me aware of Scilab and it's
multi mega point fft processing ability.

I wonder a 4 million point fft is practical though...when I did a 512
point fft on my laptop, just running the fft alone used 1.5 percent of
my processor! I think the processor load increases exponentially.

But, I wonder if scilab will allow me to mix (downconvert) the signal
in software and then do a much smaller point fft on the remainder
(1000 or 10000 point fft on a 1 Hz difference frequency)? This might
be more practical in terms of processor utilization???

Again, thanks for the letting me know about scilab.

A