Applying a custom filter to an FFT for IQ imbalance

 think I am missing something here and I do not quite understand why.
Background: I am collecting quadrature signals from a software defined
radio. There are two signals, the I, and the Q which is 90 degrees out of
phase to the I. These are passed into a complex FFT and the results plotted
as Power vs frequency. This all works fine. However, the nature of the
signal gives a two sided FFT. In other words you need to keep the negative
frequency output from the FFT. The problem is that the gain and phase
relationship between the two signals are not always 1:1 and 90 degrees. The
result is that you obtain images of the signal symmetrical around DC. It is
possible to correct these differences with the right equation however the
correction varies with the frequency. So, I need to apply a correction in
the frequency domain and this correction will be different at each
frequency (or bin in the FFT results). So I started by looking at this
reference: http://www.dxatlas.com/rocky/Advanced.asp (under IQ Balancing)
In this case, the author determines the gain and phase differences via a
synchronous detector. For various reasons I will be providing a manual
calibration procedure where the gain and phase correction of a signal will
be adjusted at several frequencies. From the gain vs frequency data, and
the phase vs frequency data, I will construct a filter that corrects the
FFT, or so I thought.

So here is the code that manipulates the FFT
output:Mitov.SignalLab.ComplexBuffer ComplexDataBuffer = new
Mitov.SignalLab.ComplexBuffer(16384);

        uint j;

        if (correctionAvailable)
        {
            for (uint i = 1; i < fftOrder; i++)
            {
                j = (fftOrder - i) % fftOrder;

                var Sp = Args.InBuffer[i];
                var Sm = Args.InBuffer[j];

                realBuffer[i] = Sp.Real + Sm.Real - (Sp.Imaginary +
Sm.Imaginary) * imaginaryCorrectionArray[j] + (Sp.Real - Sm.Real) *
realCorrectionArray[j];
                imaginaryBuffer[i] = Sp.Imaginary - Sm.Imaginary +
(Sp.Imaginary + Sm.Imaginary) * realCorrectionArray[j] + (Sp.Real -
Sm.Real) * imaginaryCorrectionArray[j];
            }
        }

        else //no correction.  Use the gain and phase sliders
        {
            for (uint i = 1; i < fftOrder; i++)
            {
                j = (fftOrder - i) % fftOrder;

                var Sp = Args.InBuffer[i];
                var Sm = Args.InBuffer[j];

                realBuffer[i] = Sp.Real + Sm.Real - (Sp.Imaginary +
Sm.Imaginary) * gain * Math.Sin(phaseCorrection) + (Sp.Real - Sm.Real) *
gain * Math.Cos(phaseCorrection);
                imaginaryBuffer[i] = Sp.Imaginary - Sm.Imaginary +
(Sp.Imaginary + Sm.Imaginary) * gain * Math.Cos(phaseCorrection) + (Sp.Real
- Sm.Real) * gain * Math.Sin(phaseCorrection);


            }
        } 

        ComplexDataBuffer.Set(realBuffer, imaginaryBuffer);

        Args.SendOutputData = false;

        genericComplex1.SendData(ComplexDataBuffer);

Now, the "gain" and "phaseCorrection" values are obtained interactively
with trackbars in the user interface. I can reduce the image right into the
noise. If I hard code a particular gain and phase value at one frequency
then I am applying this across the whole spectrum. It works but at only one
frequency. So I do this for several frequencies, calculate a quadratic
curve and generate an array that contains the correction values at every
bin (frequency) in the FFT. This is done like this:
public void createCorrectionArrays()
    {
        // we have two arrays.  One for the real data and one for the
imaginary data
        // H[w].Re = Gain[i] * cos(Phase [i]))
        // H[w].Im = gain[i] * sin(phase[i]))
        // 
        // Regression equation y=ax^2 + bx + c

        var x = 8192;

        for (int i = 0; i <8192; i++) //the lower half corresponds to the
upper frequencies
        {
            realCorrectionArray[i] = aTermGain * x * x + bTermGain * x +
cTermGain + Math.Cos(aTermPhase * x * x + bTermPhase * x + cTermPhase);
            imaginaryCorrectionArray[i] = aTermGain * x * x + bTermGain * x
+ cTermGain + Math.Sin(aTermPhase * x * x + bTermPhase * x + cTermPhase);


            x = x + 1;
        }

        x = 0;

        for (int i = 8192; i < 16384; i++) // upper half is the lower
frequencies
        {
            realCorrectionArray[i] = aTermGain * x * x + bTermGain * x +
cTermGain + Math.Cos(aTermPhase * x * x + bTermPhase * x + cTermPhase);
            imaginaryCorrectionArray[i] = aTermGain * x * x + bTermGain * x
+ cTermGain + Math.Sin(aTermPhase * x * x + bTermPhase * x + cTermPhase);
            x = x + 1;
        }

        correctionAvailable = true;

Now when I do this, there is no correction at all. If I replace these
correction arrays with a few calculated values, and place them into the
buffer correction equations, there is no correction. Now in the link above,
in the programming section the source code in Pascal is available however I
do not quite understand why the above works at one frequency, but not when
I try to correct at all frequencies. So it appears to me, based on his code
that I need to take my filters (y=ax^2 + bx + c) and run a FFT on them?
Then apply a blackman window and then do an IFFT? But evrey this I am not
sure about at this point. Can someone explain this please?

	 

_____________________________		
Posted through www.DSPRelated.com

On Mon, 19 Jan 2015 15:46:11 -0600, tomb18 wrote:

> think I am missing something here and I do not quite understand why.
> Background: I am collecting quadrature signals from a software defined
> radio. There are two signals, the I, and the Q which is 90 degrees out
> of phase to the I. These are passed into a complex FFT and the results
> plotted as Power vs frequency. This all works fine. However, the nature
> of the signal gives a two sided FFT. In other words you need to keep the
> negative frequency output from the FFT. The problem is that the gain and
> phase relationship between the two signals are not always 1:1 and 90
> degrees. The result is that you obtain images of the signal symmetrical
> around DC. It is possible to correct these differences with the right
> equation however the correction varies with the frequency. So, I need to
> apply a correction in the frequency domain and this correction will be
> different at each frequency (or bin in the FFT results). So I started by
> looking at this reference: http://www.dxatlas.com/rocky/Advanced.asp
> (under IQ Balancing) In this case, the author determines the gain and
> phase differences via a synchronous detector. For various reasons I will
> be providing a manual calibration procedure where the gain and phase
> correction of a signal will be adjusted at several frequencies. From the
> gain vs frequency data, and the phase vs frequency data, I will
> construct a filter that corrects the FFT, or so I thought.
> 
> So here is the code that manipulates the FFT
> output:

< code snipped >

> Now when I do this, there is no correction at all. If I replace these
> correction arrays with a few calculated values, and place them into the
> buffer correction equations, there is no correction. Now in the link
> above,
> in the programming section the source code in Pascal is available
> however I do not quite understand why the above works at one frequency,
> but not when I try to correct at all frequencies. So it appears to me,
> based on his code that I need to take my filters (y=ax^2 + bx + c) and
> run a FFT on them? Then apply a blackman window and then do an IFFT? But
> evrey this I am not sure about at this point. Can someone explain this
> please?

I don't read code unless someone is paying me.  If you were to write down 
a succinct description of your _algorithm_ in a mix of plain English and 
equations I will happily critique that.  I suspect others in the group 
feel the same, although I have seen people helping with code.

If you do want people to comment on your code, however, say what @#$% 
language it is!  I suspect Matlab, because it's usually Matlab users who 
are under the impression that knowing how to code in their tool is the 
same as knowing DSP, but I don't know -- and neither do the folks you 
want to help you.

I have a few general comments on your situation, however:

* "The hardware's broken, go fix the software" is the central pillar of 
some very common jokes among software engineers.  It's a joke because 
often (a) getting the hardware right often takes less engineering time 
than compensating for it in software, and (b) compensating for poor 
hardware in software often never works quite right.  You really want to 
take a hard look and see if you can fix your hardware.

* In general, you ought to be able to figure out the frequency-by-
frequency correction you need, take an FFT of it, and make a filter out 
of that.  You probably want to roll the frequency-domain description off 
to zero gain as you get close to the Nyquist rate, and look at the time-
domain FIR filter to make sure that it's nicely rolled off.

* Beyond that -- knowing how to code in Matlab is not knowing DSP.  I 
suggest that you get comfortable with, and understand the math.  Correct 
code will flow from that -- if not automatically, then at least easier.

-- 
www.wescottdesign.com

Hi, Thanks for the reply. The code is C# not matlab.  However, it's the
concept I do not understand so I will take your suggestion and explain it
without the code.
I am displaying a spectrum obtained from a software defined radio.  This
radio has two outputs. One is 90% out of phase with the other.
One characteristic of these radios is that the two signals are not always
equal in amplitude nor exactly at 90 degrees.  Consequently you need to
adjust the gain of one channel and the phase relationship between them.  If
you do not, you will see images of the signals in the opposite side of the
spectrum. The FFT is a two sided FFT, in other words you need to keep both
the positive and negative frequencies.
Now I have an equation that can be used directly against the FFT output. 
This equation has corrections for gain and phase.  Now what I have done is
wrote a program that will allow a user to use trackbars for gain and phase
and vary the values until the image is gone.  This works great at one
frequency only.  So I had the idea to have the user do this at six points
in the spectrum.  At each frequency  point the gain and phase was recorded.
 I then take these values and determine the coefficients of a quadratic
equation.  So I end up with gain = a^2*Freq + a*Freq + c;  I also have
another equation for the phase.
So my idea was, based on the bin in the FFT (corresponding to the
frequency) I would substitute apply the original equation, using the values
of the quadratic equations.
However, it doesn't work.  I do have some code in Pascal that does all of
this but I cannot understand what is going on after the coefficients are
calculated.  It seems that he applies an inverse FFT, does windowing and
then does another FFT.  This is the area that I do not understand.  It also
seems that he may drop half of the values?  I'm not sure, so I was looking
for a little guidance here.  I am a beginner in DSP.
Thanks

>On Mon, 19 Jan 2015 15:46:11 -0600, tomb18 wrote:
>
>> think I am missing something here and I do not quite understand why.
>> Background: I am collecting quadrature signals from a software defined
>> radio. There are two signals, the I, and the Q which is 90 degrees out
>> of phase to the I. These are passed into a complex FFT and the results
>> plotted as Power vs frequency. This all works fine. However, the nature
>> of the signal gives a two sided FFT. In other words you need to keep
the
>> negative frequency output from the FFT. The problem is that the gain
and
>> phase relationship between the two signals are not always 1:1 and 90
>> degrees. The result is that you obtain images of the signal symmetrical
>> around DC. It is possible to correct these differences with the right
>> equation however the correction varies with the frequency. So, I need
to
>> apply a correction in the frequency domain and this correction will be
>> different at each frequency (or bin in the FFT results). So I started
by
>> looking at this reference: http://www.dxatlas.com/rocky/Advanced.asp
>> (under IQ Balancing) In this case, the author determines the gain and
>> phase differences via a synchronous detector. For various reasons I
will
>> be providing a manual calibration procedure where the gain and phase
>> correction of a signal will be adjusted at several frequencies. From
the
>> gain vs frequency data, and the phase vs frequency data, I will
>> construct a filter that corrects the FFT, or so I thought.
>> 
>> So here is the code that manipulates the FFT
>> output:
>
>< code snipped >
>
>> Now when I do this, there is no correction at all. If I replace these
>> correction arrays with a few calculated values, and place them into the
>> buffer correction equations, there is no correction. Now in the link
>> above,
>> in the programming section the source code in Pascal is available
>> however I do not quite understand why the above works at one frequency,
>> but not when I try to correct at all frequencies. So it appears to me,
>> based on his code that I need to take my filters (y=ax^2 + bx + c) and
>> run a FFT on them? Then apply a blackman window and then do an IFFT?
But
>> evrey this I am not sure about at this point. Can someone explain this
>> please?
>
>I don't read code unless someone is paying me.  If you were to write down

>a succinct description of your _algorithm_ in a mix of plain English and 
>equations I will happily critique that.  I suspect others in the group 
>feel the same, although I have seen people helping with code.
>
>If you do want people to comment on your code, however, say what @#$% 
>language it is!  I suspect Matlab, because it's usually Matlab users who 
>are under the impression that knowing how to code in their tool is the 
>same as knowing DSP, but I don't know -- and neither do the folks you 
>want to help you.
>
>I have a few general comments on your situation, however:
>
>* "The hardware's broken, go fix the software" is the central pillar of 
>some very common jokes among software engineers.  It's a joke because 
>often (a) getting the hardware right often takes less engineering time 
>than compensating for it in software, and (b) compensating for poor 
>hardware in software often never works quite right.  You really want to 
>take a hard look and see if you can fix your hardware.
>
>* In general, you ought to be able to figure out the frequency-by-
>frequency correction you need, take an FFT of it, and make a filter out 
>of that.  You probably want to roll the frequency-domain description off 
>to zero gain as you get close to the Nyquist rate, and look at the time-
>domain FIR filter to make sure that it's nicely rolled off.
>
>* Beyond that -- knowing how to code in Matlab is not knowing DSP.  I 
>suggest that you get comfortable with, and understand the math.  Correct 
>code will flow from that -- if not automatically, then at least easier.
>
>-- 
>www.wescottdesign.com
>	 

_____________________________		
Posted through www.DSPRelated.com