Deboosting signal

Hi all,

I've searched the board but I'm not too sure what I'm searching for...

My question is quite obscure, please bear with me.

I have a voltage to which a known transfer function has been applied.
Basically there is a frequency dependent amplification, ~ x5 at DC,
x50 at 1 kHz and x500 at 100 kHz. Previously, someone using the same
amplifiers applied a method using convolutions (FFTs were used to
reduce calculation time) to retrieve the input signal. I cannot get
this method to work accurately. There is a lot of tedious maths
involved too, so I can't go into it here.

However, if I apply an FFT to the signal and then reduce the magnitude
of each frequency component (for example reduce the DC component by 5,
etc)  and then perform an inverse FFT I seem to get correct results
(Tested with DC signals, 10 kHz, 50 kHz and various other
superpositions). I also compensate for the phase by altering the
argument of each frequency.

Now, this method is trivial and so I wonder why it was not implemented
before... Is there something important that I have missed?

If anyone can suggest any appropriate reading, books or journal
papers, that would be much appreciated.

Thanks for any advice!

ntward@hotmail.com writes:
> [...]
> If anyone can suggest any appropriate reading, books or journal
> papers, that would be much appreciated.

Google frequency-domain filtering, overlap-add, overlap-save.
There is something important you're missing. 
-- 
%  Randy Yates                  % "Watching all the days go by...    
%% Fuquay-Varina, NC            %  Who are you and who am I?"
%%% 919-577-9882                % 'Mission (A World Record)', 
%%%% <yates@ieee.org>           % *A New World Record*, ELO
http://www.digitalsignallabs.com

Also, the "Subject of the post was "Deboosting".On Wed, 06 Feb 2008
12:48:55 -0500, Randy Yates <yates@ieee.org> wrote:

>ntward@hotmail.com writes:
>> [...]
>> If anyone can suggest any appropriate reading, books or journal
>> papers, that would be much appreciated.
>
>Google frequency-domain filtering, overlap-add, overlap-save.
>There is something important you're missing. 

Hi Randy,
  did ya' think is was sort of strange that the 
the original poster said he had "a voltage to which 
a known transfer function has been applied"?

How do you apply a transfer function to a voltage?
Humm, ... those words "invite meditation".  Ha ha.

Also, the "Subject" of the post contained the 
word "Deboosting".  I checked my Funk & Wagnells 
Dictionary and couldn't find the word "deboosting".
Does "Deboosting" mean attenuation?

See Ya',
[-Rick-]

Rick Lyons <R.Lyons@_BOGUS_ieee.org> writes:

> Also, the "Subject of the post was "Deboosting".On Wed, 06 Feb 2008
> 12:48:55 -0500, Randy Yates <yates@ieee.org> wrote:
>
>>ntward@hotmail.com writes:
>>> [...]
>>> If anyone can suggest any appropriate reading, books or journal
>>> papers, that would be much appreciated.
>>
>>Google frequency-domain filtering, overlap-add, overlap-save.
>>There is something important you're missing. 
>
> Hi Randy,
>   did ya' think is was sort of strange that the 
> the original poster said he had "a voltage to which 
> a known transfer function has been applied"?
>
> How do you apply a transfer function to a voltage?
> Humm, ... those words "invite meditation".  Ha ha.

Hey Rick,

Damn you can be anal sometimes [1]! Yeah, he used "unconventional
terminology" but I think I understood him.

> Also, the "Subject" of the post contained the 
> word "Deboosting".  I checked my Funk & Wagnells 
> Dictionary and couldn't find the word "deboosting".
> Does "Deboosting" mean attenuation?

Last time I heard the Funk & Wagnall dictionary referenced, I was
watching an episode of "Rowan and Martin's Laugh-In." 

  "sock-it-to-me-sock-it-to-me-sock-it-to-me-sock-it-to-me-sock-it-to-me..."
  "Verrrrryy Eeeenterestink, but it STINKS!"

--RY

[1] Let no one else with less than 10 years in this group and several years
knowing Rick personally speak to him like this. 
--
%  Randy Yates                  % "Watching all the days go by...    
%% Fuquay-Varina, NC            %  Who are you and who am I?"
%%% 919-577-9882                % 'Mission (A World Record)', 
%%%% <yates@ieee.org>           % *A New World Record*, ELO
http://www.digitalsignallabs.com

Hi,

As you can tell I've no experience in this field so I'm not familiar
with the correct terminology.

Basically, I have a voltage, and it is being amplified, such that:
Vr(s) = G(s) Vg(s)
where G(s) is the transfer function, Vr(s) is the Laplace transform of
the recorded voltage, and Vg(s) is the Laplace transform of the actual
voltage at my gauge. G(s) can be written as (a(s)^2 + bs + c)/(d(s)^2
+ es + f).
What should I have written to explain that?

Deboosting just seemed like a good word since I am boosting the
original signal, and want to do the inverse to get it back. Apologies
for any confusion.


On a similar theme, if I had a signal, Fourier transformed it, removed
a component I didn't want, and transformed it back, is there a problem
with that?!

Thanks for your help, any further advice much appreciated!

On Thu, 7 Feb 2008 01:15:59 -0800 (PST), ntward@hotmail.com wrote:

>Hi,
>
>As you can tell I've no experience in this field so I'm not familiar
>with the correct terminology.
>
>Basically, I have a voltage, and it is being amplified, such that:
>Vr(s) = G(s) Vg(s)
>where G(s) is the transfer function, Vr(s) is the Laplace transform of
>the recorded voltage, and Vg(s) is the Laplace transform of the actual
>voltage at my gauge. G(s) can be written as (a(s)^2 + bs + c)/(d(s)^2
>+ es + f).
>What should I have written to explain that?
>
>Deboosting just seemed like a good word since I am boosting the
>original signal, and want to do the inverse to get it back. Apologies
>for any confusion.
>
>
>On a similar theme, if I had a signal, Fourier transformed it, removed
>a component I didn't want, and transformed it back, is there a problem
>with that?!
>
>Thanks for your help, any further advice much appreciated!

Better words than deboosting would be deemphasis or equalisation.

Does (a(s)^2 + bs + c) have roots with a positive real part?  That
could cause problems, because the equalisation filter you want to
approximate is:

H(s) = (d(s)^2 + es + f)/(a(s)^2 + bs + c)

and that will be unstable when the poles are in the right half plane.
If we assume the roots are in nice spots, then the result of

H(s)G(s)Vr(s) should be Vg(s).

You could approximate the filter H(s) a number of ways.  An FT+IFT is
probably not the best choice in the same way that an F16 is probably
not the best choice of transport when going down to the local shop to
buy milk.
Google for filter synthesis to get some ideas.

Regards,
Allan

On Wed, 06 Feb 2008 22:06:07 -0500, Randy Yates <yates@ieee.org>
wrote:

>Rick Lyons <R.Lyons@_BOGUS_ieee.org> writes:
>
   (snipped by Lyons)
>
>Hey Rick,
>
>Damn you can be anal sometimes [1]! Yeah, he used "unconventional
>terminology" but I think I understood him.
>
>> Also, the "Subject" of the post contained the 
>> word "Deboosting".  I checked my Funk & Wagnells 
>> Dictionary and couldn't find the word "deboosting".
>> Does "Deboosting" mean attenuation?
>
>Last time I heard the Funk & Wagnall dictionary referenced, I was
>watching an episode of "Rowan and Martin's Laugh-In." 
>
>  "sock-it-to-me-sock-it-to-me-sock-it-to-me-sock-it-to-me-sock-it-to-me..."
>  "Verrrrryy Eeeenterestink, but it STINKS!"
>
>--RY
>
>[1] Let no one else with less than 10 years in this group and several years
>knowing Rick personally speak to him like this.

Hi Randy,
   Ha ha.  You're being awfully diplomatic Randy!!
That scares me.

Don't worry about hurting my feelings.  I've been 
married twice, ...so I no longer have feelings.

See Ya',
[-Rick-]

On Thu, 7 Feb 2008 01:15:59 -0800 (PST), ntward@hotmail.com wrote:

>Hi,
>
>As you can tell I've no experience in this field so I'm not familiar
>with the correct terminology.

Hello ntward,
  One of the nice things about this newsgroup is that 
here people teach us correct terminology.
As far as I can tell, what you're doing 
is called "deconvolution".  (You have the "output 
spectrum" of a system, you have the system's 
transfer function, and you'd like to determine 
the system's input signal that produced that output.)   

>Basically, I have a voltage, and it is being amplified, such that:
>Vr(s) = G(s) Vg(s)
>where G(s) is the transfer function, Vr(s) is the Laplace transform of
>the recorded voltage, and Vg(s) is the Laplace transform of the actual
>voltage at my gauge. G(s) can be written as (a(s)^2 + bs + c)/(d(s)^2
>+ es + f).
>What should I have written to explain that?

Your above explanation was quite clear.
As you probably know, given that you have 
Vr(s), and you know G(s), your desired Vg(s) is 

         Vg(s) = Vr(s)/G(s).    

Steven Smith's DSP book has a nice explanation 
(with a couple of examples) of deconvolution.
You can access Smith's material at:

  http://www.dspguide.com/ch17.htm

Go that web page and click on "Deconvolution".

>Deboosting just seemed like a good word since I am boosting the
>original signal, and want to do the inverse to get it back. Apologies
>for any confusion.

No need to apologize.  I was being snide, so now 
I apologize.  Perhaps the word "deboosting" isn't 
so bad after all. 

>On a similar theme, if I had a signal, Fourier transformed it, removed
>a component I didn't want, and transformed it back, is there a problem
>with that?!

No, there is no problem with that.  But Mother Nature 
forces us to be careful.  If you multiply a signal 
spectrum by the following:

     *************   ****************

 0................***
                         Freq ->

to attenuate some very narrow band of spectral energy,
those abrupt "down & up" transitions in the freq domain 
will cause unwanted "ripples" (a kind of amplitude 
oscillations) in the time signal that you obtain after 
performing your inverse Fourier transform.  (The above "0" 
means zero gain.)

What the professionals" do is multiply a signal spectrum 
by something like the following:

     ************     ****************
                 *   *
 0............... ***                
                        Freq ->

or perhaps this:

      **********       ***************
                *     *
                 *   *
 0............... ***                
                        Freq ->


Those more gentle "down & up"freq-domain transitions 
will cause the inverse Fourier transformed time 
signal to have greatly-reduced amplitude ripples.

But here's the "kicker": there's no equation that 
we can solve to determine the "best" values for 
the above intermediate gain values between full 
gain and zero gain!  Early DSP pioneers studied 
the living heck out of this problem (in conjunction
with designing tapped-delay line FIR filters).
I can give you the name of an IEEE paper that 
details this thorny problem if you wish.  The 
results of their study are "design tables". 
(Of course, you can experiement with various 
intermediate gain values to see what are their 
effects in the final time domain signal.)

Good Luck,
[-Rick-]
P.S.  By the way, do really have an amplifier 
whose voltage gain is 500?  That's a surprisingly 
large gain!

Rick Lyons wrote:
> On Thu, 7 Feb 2008 01:15:59 -0800 (PST), ntward@hotmail.com wrote:
> 
>> Hi,
>>
>> As you can tell I've no experience in this field so I'm not familiar
>> with the correct terminology.
> 
> Hello ntward,
>   One of the nice things about this newsgroup is that 
> here people teach us correct terminology.
> As far as I can tell, what you're doing 
> is called "deconvolution".  (You have the "output 
> spectrum" of a system, you have the system's 
> transfer function, and you'd like to determine 
> the system's input signal that produced that output.)   
> 
>> Basically, I have a voltage, and it is being amplified, such that:
>> Vr(s) = G(s) Vg(s)
>> where G(s) is the transfer function, Vr(s) is the Laplace transform of
>> the recorded voltage, and Vg(s) is the Laplace transform of the actual
>> voltage at my gauge. G(s) can be written as (a(s)^2 + bs + c)/(d(s)^2
>> + es + f).
>> What should I have written to explain that?
> 
> Your above explanation was quite clear.
> As you probably know, given that you have 
> Vr(s), and you know G(s), your desired Vg(s) is 
> 
>          Vg(s) = Vr(s)/G(s).    
> 
> Steven Smith's DSP book has a nice explanation 
> (with a couple of examples) of deconvolution.
> You can access Smith's material at:
> 
>   http://www.dspguide.com/ch17.htm
> 
> Go that web page and click on "Deconvolution".
> 
>> Deboosting just seemed like a good word since I am boosting the
>> original signal, and want to do the inverse to get it back. Apologies
>> for any confusion.
> 
> No need to apologize.  I was being snide, so now 
> I apologize.  Perhaps the word "deboosting" isn't 
> so bad after all. 
> 
>> On a similar theme, if I had a signal, Fourier transformed it, removed
>> a component I didn't want, and transformed it back, is there a problem
>> with that?!
> 
> No, there is no problem with that.  But Mother Nature 
> forces us to be careful.  If you multiply a signal 
> spectrum by the following:
> 
>      *************   ****************
> 
>  0................***
>                          Freq ->
> 
> to attenuate some very narrow band of spectral energy,
> those abrupt "down & up" transitions in the freq domain 
> will cause unwanted "ripples" (a kind of amplitude 
> oscillations) in the time signal that you obtain after 
> performing your inverse Fourier transform.  (The above "0" 
> means zero gain.)
> 
> What the professionals" do is multiply a signal spectrum 
> by something like the following:
> 
>      ************     ****************
>                  *   *
>  0............... ***                
>                         Freq ->
> 
> or perhaps this:
> 
>       **********       ***************
>                 *     *
>                  *   *
>  0............... ***                
>                         Freq ->
> 
> 
> Those more gentle "down & up"freq-domain transitions 
> will cause the inverse Fourier transformed time 
> signal to have greatly-reduced amplitude ripples.
> 
> But here's the "kicker": there's no equation that 
> we can solve to determine the "best" values for 
> the above intermediate gain values between full 
> gain and zero gain!  Early DSP pioneers studied 
> the living heck out of this problem (in conjunction
> with designing tapped-delay line FIR filters).
> I can give you the name of an IEEE paper that 
> details this thorny problem if you wish.  The 
> results of their study are "design tables". 
> (Of course, you can experiement with various 
> intermediate gain values to see what are their 
> effects in the final time domain signal.)
> 
> Good Luck,
> [-Rick-]
> P.S.  By the way, do really have an amplifier 
> whose voltage gain is 500?  That's a surprisingly 
> large gain!

Rick, I think you're making a mountain out of a molehill here. That's a 
trouble that comes from knowing too much. Ntward's signal went through 
an amplifier with some frequency response he doesn't like -- he knows 
what it is -- and he wants to run it through a filter that will get the 
original signal back. Since his sloped response is gradual, tapering the 
results of an FFT and then IFFTing won't be too terrible, except for the 
influence on the phase. Still, for continuous signals, that means he'll 
need to get into the details of overlap-save or overlap-add. While 
Smith's book and yours both explain that well, it's a complication 
ntward doesn't need. Besides, the original boost network is very likely 
analog, and a linear-phase equalizer is probably not his best choice.

If the sample rate is high enough, a couple of biquads ought to be 
enough to restore things. Again, Your book and Smith's cover that very well.

Ntward, The trouble with diddling FFT components is that it ignores the 
effect on the time response. (The impulse response is easiest to deal 
with. The response to a step is it's time integral.) Abrupt transitions 
in frequency response cause ringing in time. Although it's not as 
evident, abrupt transitions in impulse response cause a very lumpy 
frequency response. What's more, these approaches extend the signal in 
time, making the FFT too short to do the job adequately. I won't try to 
write a treatise on how to do it right. For you, for now, it's better 
just not to do it. Use an IIR and get it over with.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

Hello again,

Thanks for all the info. All of that + the books I am reading is quite
a lot to take in so I can't really offer constructive replies yet!
Also I need to understand it all to my satisfaction which I think will
take some time.

Here's some more general background information for those interested:
 - Sample rate 500 kHz.
 - Duration of sample ~ 300 ms.
 - Signal is filtered by the ADC at 100 kHz, but I'm not certain what
type of filter it is. Signal is also attenuated by the amplifiers at >
100 kHz so the roll off is quite steep.
 - Gain of (analogue) amplifiers is very smooth, as is their
associated phase shift.


In response to various people:

@ Allan:
The roots are all negative and real.

@Rick:
 - Deconvolution, that sounds right!
 - OK, so I understand attenuating a very narrow band can cause
problems. In my case I want to remove a smoothly varying gain, so that
should be less of a problem?
 - The paper sounds a bit over my head, but I'll take a look if it's
no trouble to give me the reference?
 - Yes, it's just an OP37, 63MHz gain bandwidth, so it starts rolling
off at about 100 kHz.

@Jerry:
 - The frequency response of the amplifier is a necessary evil,
without it the high frequency signals would be attenuated before I can
sample them. But I can't interpret my signals without first removing
the gain.
 - I'm also changing the phases of each component, so that should not
cause too much problem?
 - Correct, I don't need extra complications, but I need as many
pointers as possible so I can understand exactly what I can and can't
do, and why.
 - Your second paragraph has lost me, so I'm off to read some more...

My issue is that the original method performs a convolution (using
FFTs, so any 'intermediate' frequencies should cause problems here too
no?) and doesn't seem to give sensible results. Whereas the straight
forward method "diddling" with FFT components apparently (I can't
address Jerrys last paragraph yet) does work.

Admittedly the phase error increases with frequency, but it's
tolerable especially considering that the previous method only allowed
calibration of values below 10 kHz as the polynomial deviates from the
measured response of the amplifiers... Also, the original method
doesn't allow me to independently calibrate the phases - fitting the
polynomial to the gain fixes how the phase will be changed and this
doesn't match the measured phase change as well as perhaps it should.

As I said, I have a *lot* of reading and learning to do. Thanks again
for all the advice...