Elusive signal in the noise

Hello,

Bear with me while I explain my situation here.  I have a test setup. 
This test setup is a big analog contraption that works when I connect a
wire between two aluminum blocks within the test setup.  The contraption
"comes to life" when the wire is connected and "dies off" when the wire is
disconnected.  Don't worry about the "comes alive" and "dies off".  Don't
even worry about what the rest of the contraption is there isn't any more
information there on what the signal is other than what I will relate here.
 The important thing is that I have a signal in a wire and I can tell when
it is there and when it isn't by behavior of the system at large.  

I know that the signal is contained within 100 Hz centered at 5 kHz.  When
I put my test setup in an anechoic chamber and look at the signal in the
wire with one of Agilent's best spectrum analyzers, I can see no power
attributable to this signal.  All I see is noise.  Even when I integrate
the signal over long periods of time.  So apparently this signal doesn't
come out of the noise using an FFT.

I can break the wire and put one end into a laptop sound input and bring
the other end out of the output.  I can run Spectrum Lab (DL4YHFs free and
excellent software) and do an FFT on the input, filter the signal, and
output the signal and the system at large will work.  If I pass any 30 Hz
band between 4.95 and 5.05 kHz the system "comes alive" and if I move the
filter down to say 4.9 kHz, the system dies off.  So obviously with a sound
card I am capturing this signal, which is somewhat bizarre because I can't
see any power with that Agilent setup which has a noise floor way down
below the sound card setup.  

I have recorded this digital signal that I have captured and tried to
looked at the power spectral density with Matlab and can't see anything. 

I already tried listening for the signal and I can't pick it out by ear. 

What I would like to do is capture the signal in real time with Matlab and
then insert various filters, then output the signal and see what kills the
signal and what allows it to pass.  I almost have the Matlab framework for
this in place but my problem is that I don't know what filters to use and
once I do figure out some filters, how do I interpret the results?  For
example if I use filter x and change the y parameter to five different
values and the first three work, what will that tell me about the signal. 
I would greatly appreciate any suggestions on what filters to use, what
parameters to change, and some insight into what that would mean for what
the signal is.

Thank you!  

-xerxes

xerxes73 wrote:
> Hello,
> 
> Bear with me while I explain my situation here.  I have a test setup. 
> This test setup is a big analog contraption that works when I connect a
> wire between two aluminum blocks within the test setup.  The contraption
> "comes to life" when the wire is connected and "dies off" when the wire is
> disconnected.  Don't worry about the "comes alive" and "dies off".  Don't
> even worry about what the rest of the contraption is there isn't any more
> information there on what the signal is other than what I will relate here.
>  The important thing is that I have a signal in a wire and I can tell when
> it is there and when it isn't by behavior of the system at large.  
> 
> I know that the signal is contained within 100 Hz centered at 5 kHz.  When
> I put my test setup in an anechoic chamber and look at the signal in the
> wire with one of Agilent's best spectrum analyzers, I can see no power
> attributable to this signal.  All I see is noise.  Even when I integrate
> the signal over long periods of time.  So apparently this signal doesn't
> come out of the noise using an FFT.
> 
> I can break the wire and put one end into a laptop sound input and bring
> the other end out of the output.  I can run Spectrum Lab (DL4YHFs free and
> excellent software) and do an FFT on the input, filter the signal, and
> output the signal and the system at large will work.  If I pass any 30 Hz
> band between 4.95 and 5.05 kHz the system "comes alive" and if I move the
> filter down to say 4.9 kHz, the system dies off.  So obviously with a sound
> card I am capturing this signal, which is somewhat bizarre because I can't
> see any power with that Agilent setup which has a noise floor way down
> below the sound card setup.  
> 
> I have recorded this digital signal that I have captured and tried to
> looked at the power spectral density with Matlab and can't see anything. 
> 
> I already tried listening for the signal and I can't pick it out by ear. 
> 
> What I would like to do is capture the signal in real time with Matlab and
> then insert various filters, then output the signal and see what kills the
> signal and what allows it to pass.  I almost have the Matlab framework for
> this in place but my problem is that I don't know what filters to use and
> once I do figure out some filters, how do I interpret the results?  For
> example if I use filter x and change the y parameter to five different
> values and the first three work, what will that tell me about the signal. 
> I would greatly appreciate any suggestions on what filters to use, what
> parameters to change, and some insight into what that would mean for what
> the signal is.

Is it an acoustic signal? If not, what is the use of an anechoic 
chamber? You might try analyzing the system with a Hieronymus machine.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

>xerxes73 wrote:
>> Hello,
>> 
>> Bear with me while I explain my situation here.  I have a test setup. 
>> This test setup is a big analog contraption that works when I connect
a
>> wire between two aluminum blocks within the test setup.  The
contraption
>> "comes to life" when the wire is connected and "dies off" when the wire
is
>> disconnected.  Don't worry about the "comes alive" and "dies off". 
Don't
>> even worry about what the rest of the contraption is there isn't any
more
>> information there on what the signal is other than what I will relate
here.
>>  The important thing is that I have a signal in a wire and I can tell
when
>> it is there and when it isn't by behavior of the system at large.  
>> 
>> I know that the signal is contained within 100 Hz centered at 5 kHz. 
When
>> I put my test setup in an anechoic chamber and look at the signal in
the
>> wire with one of Agilent's best spectrum analyzers, I can see no power
>> attributable to this signal.  All I see is noise.  Even when I
integrate
>> the signal over long periods of time.  So apparently this signal
doesn't
>> come out of the noise using an FFT.
>> 
>> I can break the wire and put one end into a laptop sound input and
bring
>> the other end out of the output.  I can run Spectrum Lab (DL4YHFs free
and
>> excellent software) and do an FFT on the input, filter the signal, and
>> output the signal and the system at large will work.  If I pass any 30
Hz
>> band between 4.95 and 5.05 kHz the system "comes alive" and if I move
the
>> filter down to say 4.9 kHz, the system dies off.  So obviously with a
sound
>> card I am capturing this signal, which is somewhat bizarre because I
can't
>> see any power with that Agilent setup which has a noise floor way down
>> below the sound card setup.  
>> 
>> I have recorded this digital signal that I have captured and tried to
>> looked at the power spectral density with Matlab and can't see
anything. 
>> 
>> I already tried listening for the signal and I can't pick it out by
ear. 
>> 
>> What I would like to do is capture the signal in real time with Matlab
and
>> then insert various filters, then output the signal and see what kills
the
>> signal and what allows it to pass.  I almost have the Matlab framework
for
>> this in place but my problem is that I don't know what filters to use
and
>> once I do figure out some filters, how do I interpret the results? 
For
>> example if I use filter x and change the y parameter to five different
>> values and the first three work, what will that tell me about the
signal. 
>> I would greatly appreciate any suggestions on what filters to use,
what
>> parameters to change, and some insight into what that would mean for
what
>> the signal is.
>
>Is it an acoustic signal? If not, what is the use of an anechoic 
>chamber? You might try analyzing the system with a Hieronymus machine.
>
>Jerry
>-- 
>Engineering is the art of making what you want from things you can get.
>�����������������������������������������������������������������������
The anechoic chamber is just for a quiet RF space in those frequencies,
i.e. just dropping the noise floor so the power of the signal would be
visible.  It wasn't.  The Heironymus machine is an "interesting" idea, but
I would like to know what filtering methods I use to pull a hard to find
signal out of a captured digital stream of data....   -xerxes

>xerxes73 wrote:
>> Hello,
>> 
>> Bear with me while I explain my situation here.  I have a test setup. 
>> This test setup is a big analog contraption that works when I connect
a
>> wire between two aluminum blocks within the test setup.  The
contraption
>> "comes to life" when the wire is connected and "dies off" when the wire
is
>> disconnected.  Don't worry about the "comes alive" and "dies off". 
Don't
>> even worry about what the rest of the contraption is there isn't any
more
>> information there on what the signal is other than what I will relate
here.
>>  The important thing is that I have a signal in a wire and I can tell
when
>> it is there and when it isn't by behavior of the system at large.  
>> 
>> I know that the signal is contained within 100 Hz centered at 5 kHz. 
When
>> I put my test setup in an anechoic chamber and look at the signal in
the
>> wire with one of Agilent's best spectrum analyzers, I can see no power
>> attributable to this signal.  All I see is noise.  Even when I
integrate
>> the signal over long periods of time.  So apparently this signal
doesn't
>> come out of the noise using an FFT.
>> 
>> I can break the wire and put one end into a laptop sound input and
bring
>> the other end out of the output.  I can run Spectrum Lab (DL4YHFs free
and
>> excellent software) and do an FFT on the input, filter the signal, and
>> output the signal and the system at large will work.  If I pass any 30
Hz
>> band between 4.95 and 5.05 kHz the system "comes alive" and if I move
the
>> filter down to say 4.9 kHz, the system dies off.  So obviously with a
sound
>> card I am capturing this signal, which is somewhat bizarre because I
can't
>> see any power with that Agilent setup which has a noise floor way down
>> below the sound card setup.  
>> 
>> I have recorded this digital signal that I have captured and tried to
>> looked at the power spectral density with Matlab and can't see
anything. 
>> 
>> I already tried listening for the signal and I can't pick it out by
ear. 
>> 
>> What I would like to do is capture the signal in real time with Matlab
and
>> then insert various filters, then output the signal and see what kills
the
>> signal and what allows it to pass.  I almost have the Matlab framework
for
>> this in place but my problem is that I don't know what filters to use
and
>> once I do figure out some filters, how do I interpret the results? 
For
>> example if I use filter x and change the y parameter to five different
>> values and the first three work, what will that tell me about the
signal. 
>> I would greatly appreciate any suggestions on what filters to use,
what
>> parameters to change, and some insight into what that would mean for
what
>> the signal is.
>
>Is it an acoustic signal? If not, what is the use of an anechoic 
>chamber? You might try analyzing the system with a Hieronymus machine.
>
>Jerry
>-- 
>Engineering is the art of making what you want from things you can get.
>�����������������������������������������������������������������������
The anechoic chamber is just for a quiet RF space in those frequencies,
i.e. just dropping the noise floor so the power of the signal would be
visible.  It wasn't.  The Heironymus machine is an "interesting" idea, but
I would like to know what filtering methods I use to pull a hard to find
signal out of a captured digital stream of data....   -xerxes

xerxes73 wrote:
>> xerxes73 wrote:
>>> Hello,
>>>
>>> Bear with me while I explain my situation here.  I have a test setup. 
>>> This test setup is a big analog contraption that works when I connect
> a
>>> wire between two aluminum blocks within the test setup.  The
> contraption
>>> "comes to life" when the wire is connected and "dies off" when the wire
> is
>>> disconnected.  Don't worry about the "comes alive" and "dies off". 
> Don't
>>> even worry about what the rest of the contraption is there isn't any
> more
>>> information there on what the signal is other than what I will relate
> here.
>>>  The important thing is that I have a signal in a wire and I can tell
> when
>>> it is there and when it isn't by behavior of the system at large.  
>>>
>>> I know that the signal is contained within 100 Hz centered at 5 kHz. 
> When
>>> I put my test setup in an anechoic chamber and look at the signal in
> the
>>> wire with one of Agilent's best spectrum analyzers, I can see no power
>>> attributable to this signal.  All I see is noise.  Even when I
> integrate
>>> the signal over long periods of time.  So apparently this signal
> doesn't
>>> come out of the noise using an FFT.
>>>
>>> I can break the wire and put one end into a laptop sound input and
> bring
>>> the other end out of the output.  I can run Spectrum Lab (DL4YHFs free
> and
>>> excellent software) and do an FFT on the input, filter the signal, and
>>> output the signal and the system at large will work.  If I pass any 30
> Hz
>>> band between 4.95 and 5.05 kHz the system "comes alive" and if I move
> the
>>> filter down to say 4.9 kHz, the system dies off.  So obviously with a
> sound
>>> card I am capturing this signal, which is somewhat bizarre because I
> can't
>>> see any power with that Agilent setup which has a noise floor way down
>>> below the sound card setup.  
>>>
>>> I have recorded this digital signal that I have captured and tried to
>>> looked at the power spectral density with Matlab and can't see
> anything. 
>>> I already tried listening for the signal and I can't pick it out by
> ear. 
>>> What I would like to do is capture the signal in real time with Matlab
> and
>>> then insert various filters, then output the signal and see what kills
> the
>>> signal and what allows it to pass.  I almost have the Matlab framework
> for
>>> this in place but my problem is that I don't know what filters to use
> and
>>> once I do figure out some filters, how do I interpret the results? 
> For
>>> example if I use filter x and change the y parameter to five different
>>> values and the first three work, what will that tell me about the
> signal. 
>>> I would greatly appreciate any suggestions on what filters to use,
> what
>>> parameters to change, and some insight into what that would mean for
> what
>>> the signal is.
>> Is it an acoustic signal? If not, what is the use of an anechoic 
>> chamber? You might try analyzing the system with a Hieronymus machine.
>>
>> Jerry
>> -- 
>> Engineering is the art of making what you want from things you can get.
> The anechoic chamber is just for a quiet RF space in those frequencies,
> i.e. just dropping the noise floor so the power of the signal would be
> visible.

There's no reason to think that an anechoic chamber would provide any 
electromagnetic shielding. Do you perhaps mean a screen room?

>       It wasn't.  The Heironymus machine is an "interesting" idea, but
> I would like to know what filtering methods I use to pull a hard to find
> signal out of a captured digital stream of data....   -xerxes

Well, a Heironymus machine is said to be uniquely capable of responding 
to signals that aren't there.

Jerry
-- 
Engineering is the art of making what you want from things you can get.

>>> -- 
>>> Engineering is the art of making what you want from things you can
get.
>> The anechoic chamber is just for a quiet RF space in those
frequencies,
>> i.e. just dropping the noise floor so the power of the signal would be
>> visible.  It wasn't.  
>
>There's no reason to think that an anechoic chamber would provide any 
>electromagnetic shielding. Do you perhaps mean a screen room?

The wikipedia article on anechoic chambers indicates that I am using the
term correctly: http://en.wikipedia.org/wiki/Anechoic_chamber although it
would probably be clearer if I said rf anechoic chamber.

>
>>       The Heironymus machine is an "interesting" idea, but
>> I would like to know what filtering methods I use to pull a hard to
find
>> signal out of a captured digital stream of data....   -xerxes
>
>Well, a Heironymus machine is said to be uniquely capable of responding 
>to signals that aren't there.
>
This is a signal that is clearly there but invisible in the digital
analysis I have done thus far.  Perhaps a filter based on wavelets will
detect a signal that is invisible to an fft because of the nature of the
fft as it grabs a large chunk of data in time.  The wavelet grabbing
differently scaled time data may pick this up but I don't have enough
experience to know if this is worth implementing or not.  Or perhaps some
sort of adaptive filter.  That's what I am looking for advice on what
filters to use when an fft-based filter is not working.  Thanks.  -xerxes

xerxes73 wrote:
>>>> -- 
>>>> Engineering is the art of making what you want from things you can
> get.
>>> The anechoic chamber is just for a quiet RF space in those
> frequencies,
>>> i.e. just dropping the noise floor so the power of the signal would be
>>> visible.  It wasn't.  
>> There's no reason to think that an anechoic chamber would provide any 
>> electromagnetic shielding. Do you perhaps mean a screen room?
> 
> The wikipedia article on anechoic chambers indicates that I am using the
> term correctly: http://en.wikipedia.org/wiki/Anechoic_chamber although it
> would probably be clearer if I said rf anechoic chamber.

As I read that article, RF anechoic chambers are used at microwave 
frequencies to prevent reflections off the walls, floor, and ceiling of 
signals generated inside the chamber. Acoustic chambers also greatly 
attenuate sound from outside. The low frequencies you use are not 
amenable to anechoic techniques in structures of reasonable size. (The 
wavelength of EM radiation at 5 KHz is 37.5 miles.) Rour anexhoic 
chamber _may_ include electrostatic shielding (Faraday shield). If it 
does, it is also a screen room.

>>>       The Heironymus machine is an "interesting" idea, but
>>> I would like to know what filtering methods I use to pull a hard to
> find
>>> signal out of a captured digital stream of data....   -xerxes
>> Well, a Heironymus machine is said to be uniquely capable of responding 
>> to signals that aren't there.

I meant it as a joke. http://en.wikipedia.org/wiki/Hieronymus_machine

> This is a signal that is clearly there but invisible in the digital
> analysis I have done thus far.  Perhaps a filter based on wavelets will
> detect a signal that is invisible to an fft because of the nature of the
> fft as it grabs a large chunk of data in time.  The wavelet grabbing
> differently scaled time data may pick this up but I don't have enough
> experience to know if this is worth implementing or not.  Or perhaps some
> sort of adaptive filter.  That's what I am looking for advice on what
> filters to use when an fft-based filter is not working.  Thanks.  -xerxes

Knowledge of how the signal is generated and what it affects should put 
a lower limit on its strength. Recall that if the circuit impedance is 
very low, measuring voltage could be futile. Can you measure the current 
in the wire?

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

>> The wikipedia article on anechoic chambers indicates that I am using
the
>> term correctly: http://en.wikipedia.org/wiki/Anechoic_chamber although
it
>> would probably be clearer if I said rf anechoic chamber.
>
>As I read that article, RF anechoic chambers are used at microwave 
>frequencies to prevent reflections off the walls, floor, and ceiling of 
>signals generated inside the chamber. Acoustic chambers also greatly 
>attenuate sound from outside. The low frequencies you use are not 
>amenable to anechoic techniques in structures of reasonable size. (The 
>wavelength of EM radiation at 5 KHz is 37.5 miles.) Rour anexhoic 
>chamber _may_ include electrostatic shielding (Faraday shield). If it 
>does, it is also a screen room.
>
Ah, I see.  That's right that kHz will go through most things.  I went
back through my notes (it was about a year and a half ago that it was in
the chamber).  We actually built another faraday cage in the chamber to get
the noise way down there.  We didn't see it with an FFT and a really low
noise floor.


>>>>       The Heironymus machine is an "interesting" idea, but
>>>> I would like to know what filtering methods I use to pull a hard to
>> find
>>>> signal out of a captured digital stream of data....   -xerxes
>>> Well, a Heironymus machine is said to be uniquely capable of
responding 
>>> to signals that aren't there.
>
>I meant it as a joke. http://en.wikipedia.org/wiki/Hieronymus_machine
>
Okay good.  I was worried about bringing this conversation back to
hardcore dsp techniques.

>> This is a signal that is clearly there but invisible in the digital
>> analysis I have done thus far.  Perhaps a filter based on wavelets
will
>> detect a signal that is invisible to an fft because of the nature of
the
>> fft as it grabs a large chunk of data in time.  The wavelet grabbing
>> differently scaled time data may pick this up but I don't have enough
>> experience to know if this is worth implementing or not.  Or perhaps
some
>> sort of adaptive filter.  That's what I am looking for advice on what
>> filters to use when an fft-based filter is not working.  Thanks. 
-xerxes
>
>Knowledge of how the signal is generated and what it affects should put 
>a lower limit on its strength. Recall that if the circuit impedance is 
>very low, measuring voltage could be futile. Can you measure the current

>in the wire?
>
I did try a Keithley 6430 Electrometer which is a heck of an instrument
but just connecting the electrometer seemed to screw up the signal both
when trying to measure amperage and voltage.  It is an extremely low
impedance I believe.  I can't say I am an expert in this though so perhaps
I could have done it better.  I talked it over with their app engineers
quite a bit and although they were very helpful, we couldn't figure out an
approach that worked.  It seems like the signal is right in front of me now
in this captured data but I can't figure out a way to figure out what it
is.  Very strange.

christopher wrote:

...

> I have recorded this digital signal that I have captured and tried to
> looked at the power spectral density with Matlab and can't see anything.

Can you post the data for download?

xerxes73 wrote:

I wrote

>> Knowledge of how the signal is generated and what it affects should put 
>> a lower limit on its strength. Recall that if the circuit impedance is 
>> very low, measuring voltage could be futile. Can you measure the current
>> in the wire?
>>
> I did try a Keithley 6430 Electrometer which is a heck of an instrument
> but just connecting the electrometer seemed to screw up the signal both
> when trying to measure amperage and voltage.  It is an extremely low
> impedance I believe.  I can't say I am an expert in this though so perhaps
> I could have done it better.  I talked it over with their app engineers
> quite a bit and although they were very helpful, we couldn't figure out an
> approach that worked.  It seems like the signal is right in front of me now
> in this captured data but I can't figure out a way to figure out what it
> is.  Very strange.

An electrometer is a voltmeter with an extremely high impedance. The 
6430 is a sub-femtoamp instrument.

Since you can apparently break the wire to run the signal through a 
computer's sound card, it can't be low impedance. I hadn't considered 
that bit you wrote. Since the computer can deal with the signal, it must 
see it. Why don't you record what the computer sees?

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������