Energy Measurement

Randy Yates wrote:

> Jerry Avins wrote:
> 
>>Randy Yates wrote:
>>
>>>What (DFT) method does one deem best, so that the test result
>>>is conclusive?
>>>
>>>--Randy
>>
>>A DFT is a DFT.
> 
> 
> Really? So none of the possible implementations make
> any difference?  :
> 
>   1. Take 32768 two-point DFTs and average the result.
>   2. Take 16374 four-point DFTs and average the result.
>   .
>   .
>   .
>   14. Take 2 32768-point DFTs and average the result
>   15. Take 1 65536-point DFT
>   16. Zero-extend the 65536 points to 131032 points
>    and take a 131032-point DFT.
>   .
>   .
>   .
> 
> Combine these choices with multiple choices in windowing
> and combining of the resulting bins and I think you have
> pretty much an infinite number of ways to perform the
> operation, and I'm pretty sure they're not all going to
> give the same result.

Fulfilling the terms of the power division you hypothesized is best done
with a DFT by minimizing the spillover between the bands, hence the
spectral leakage of the bins. I guess that zero padding probably won't
help much; that guess can be tested. Using a subset of the data in
smaller DFTs and averaging the results results in wider bins, and so
spreads the leakage. I'm pretty sure that will hurt.

If you had ten times as many samples, I'm pretty sure the filter would
be better, but the filtered samples contain start and end transients
that need to be discarded, and that 400 discarded out of 65536 may not
be negligible.

In the absence of a developed theory, the crossover number of samples is
best determined by experiment. The same applies to the question of
all-at-once or divide-and-conquer in the DFT, although I think I know
the outcome.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

Jerry Avins wrote:

   ...

> Fulfilling the terms of the power division you hypothesized is best done
> with a DFT by minimizing the spillover between the bands, hence the
> spectral leakage of the bins. ...

I didn't mean that exactly. I meant to get across that
_when_using_a_DFT_, fulfilling the terms of the power division you
hypothesized is best done with a DFT by minimizing the spillover between
the bands. The filter may be better.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

Randy Yates wrote:

> Gentle comp.dsp'ers,
> 
> I again have a question that is essentially the same as the one I
> posted back at the beginning of April, but better stated.
> 
> Assume one has a buffer of 65536 samples, and the goal is to measure
> the energy between Fs/4 and Fs/2 (just arbitrary frequencies selected
> for this example) - half the available digital bandwidth - with as
> little energy from the 0 to Fs/4 band in the measurement as possible.
> 
> Method one: use a DFT. The manner in which the DFT is applied is open
> for discussion.
> 
> Method two: construct a large FIR, say, 500 taps or more, with a
> near-brick-wall frequency response and measure the energy of the
> output of this filter with the input data applied.
> 
> Which do you think is more accurate in rejecting energy from 0 to
> Fs/4?
> 
> --Randy

I don't have opinion on which method would be better, however, I think
filtering has problems at the borders of the signal (i.e., at the start and
end). Perhaps symmetric extension would help but would it be even better to
design 2*499 extra filters for the border cases? Those filters would have
lengths ranging from 1 to 499 taps and they would only be used at the
borders.

-- 
Jani Huhtanen
Tampere University of Technology, Pori

Rick Lyons wrote:
> On 6 Jun 2006 17:27:22 -0700, "Randy Yates" <yates@ieee.org> wrote:
>
> >Gentle comp.dsp'ers,
> >
> >I again have a question that is essentially the same as the one I
> >posted back at the beginning of April, but better stated.
> >
> >Assume one has a buffer of 65536 samples, and the goal is to measure
> >the energy between Fs/4 and Fs/2 (just arbitrary frequencies selected
> >for this example) - half the available digital bandwidth - with as
> >little energy from the 0 to Fs/4 band in the measurement as possible.
> >
> >Method one: use a DFT. The manner in which the DFT is applied is open
> >for discussion.
> >
> >Method two: construct a large FIR, say, 500 taps or more, with a
> >near-brick-wall frequency response and measure the energy of the
> >output of this filter with the input data applied.
> >
> >Which do you think is more accurate in rejecting energy from 0 to
> >Fs/4?
> >
> >--Randy
>
> Hi Randy,
>   what an interesting question.  So many things to
> consider.
>
> Assuming that your time samples are not windowed
> before they are applied to an N-point DFT,
> leakage from your "unwanted" N/2 DFT bins will show
> up, to some degree, in your N/2 "bins of interest".
>
> My vote is that you use your Method Two (high-performance
> FIR filter).  500 taps ought to give you one
> butt-kickin' filter!
>
> Good Luck,
> [-Rick-]

Rick,
Isn't this exact question addressed very well in your book in Section
13.20 "A Paractical Spectrum Analyzer"?   The "weighted overlad-add or
window presum FFT can be used to imprve the shape factor of the FFT.

Randy,
I think the "manager answer" is that both methods can do the job but
that the FIR is more computationaly efficient because the FFT is also
calculating the energy in all the other bins as well and the problem
calls for the energy in only one bin.


OK.....so then what about the Goertzel which if I understand correctly
is equivalent to an IIR filter... so another question...why consider
only FIR, why not IIR as well?

Seems the most computationally efficent would be an IIR filter.....no?

thanks
Mark

Mark wrote:
> Rick Lyons wrote:
> > On 6 Jun 2006 17:27:22 -0700, "Randy Yates" <yates@ieee.org> wrote:
> >
> > >Gentle comp.dsp'ers,
> > >
> > >I again have a question that is essentially the same as the one I
> > >posted back at the beginning of April, but better stated.
> > >
> > >Assume one has a buffer of 65536 samples, and the goal is to measure
> > >the energy between Fs/4 and Fs/2 (just arbitrary frequencies selected
> > >for this example) - half the available digital bandwidth - with as
> > >little energy from the 0 to Fs/4 band in the measurement as possible.
> > >
> > >Method one: use a DFT. The manner in which the DFT is applied is open
> > >for discussion.
> > >
> > >Method two: construct a large FIR, say, 500 taps or more, with a
> > >near-brick-wall frequency response and measure the energy of the
> > >output of this filter with the input data applied.
> > >
> > >Which do you think is more accurate in rejecting energy from 0 to
> > >Fs/4?
> > >
> > >--Randy
> >
> > Hi Randy,
> >   what an interesting question.  So many things to
> > consider.
> >
> > Assuming that your time samples are not windowed
> > before they are applied to an N-point DFT,
> > leakage from your "unwanted" N/2 DFT bins will show
> > up, to some degree, in your N/2 "bins of interest".
> >
> > My vote is that you use your Method Two (high-performance
> > FIR filter).  500 taps ought to give you one
> > butt-kickin' filter!
> >
> > Good Luck,
> > [-Rick-]
>
> Rick,
> Isn't this exact question addressed very well in your book in Section
> 13.20 "A Paractical Spectrum Analyzer"?   The "weighted overlad-add or
> window presum FFT can be used to imprve the shape factor of the FFT.

Yes, and damned if Rick didn't point me to that section a few weeks
ago and I dropped the ball. Thanks for the reminder, Mark. What an
excellent piece of work it is, too. The only thing I'm wondering is
if this can be extended to an indefinite sequence.

Note also that this technique extends the dynamic range of
the underlying data. For example, if you had 16-bit data and
P was 16, I think you'd extend your resolution to the equivalent
of 20 bits.

> Randy,
> I think the "manager answer" is that both methods can do the job but
> that the FIR is more computationaly efficient because the FFT is also
> calculating the energy in all the other bins as well and the problem
> calls for the energy in only one bin.

Yes, that's a "hand-waving" answer, but probably good enough for
the managers...

> OK.....so then what about the Goertzel which if I understand correctly
> is equivalent to an IIR filter... so another question...why consider
> only FIR, why not IIR as well?

Answer: Plain laziness. Having to build and tweak the SOSs of an IIR
with an eye towards the round-off problems just seems like so much
work... But that could be a great "beta" algorithm.

> Seems the most computationally efficent would be an IIR filter.....no?

Yes, I agree. Thanks for your comments, Mark.

--Randy

Jani Huhtanen wrote:
> Randy Yates wrote:
>
> > Gentle comp.dsp'ers,
> >
> > I again have a question that is essentially the same as the one I
> > posted back at the beginning of April, but better stated.
> >
> > Assume one has a buffer of 65536 samples, and the goal is to measure
> > the energy between Fs/4 and Fs/2 (just arbitrary frequencies selected
> > for this example) - half the available digital bandwidth - with as
> > little energy from the 0 to Fs/4 band in the measurement as possible.
> >
> > Method one: use a DFT. The manner in which the DFT is applied is open
> > for discussion.
> >
> > Method two: construct a large FIR, say, 500 taps or more, with a
> > near-brick-wall frequency response and measure the energy of the
> > output of this filter with the input data applied.
> >
> > Which do you think is more accurate in rejecting energy from 0 to
> > Fs/4?
> >
> > --Randy
>
> I don't have opinion on which method would be better, however, I think
> filtering has problems at the borders of the signal (i.e., at the start and
> end). Perhaps symmetric extension would help but would it be even better to
> design 2*499 extra filters for the border cases? Those filters would have
> lengths ranging from 1 to 499 taps and they would only be used at the
> borders.

Why less taps?  Just make the FIR filters at the borders asymmetric,
but still with all 500 taps.

An interesting question is how to design a family of filters with
the same frequency response, but a whole range of offsets (1 to 250).
The standard method of going towards minimum phase only gives you
a finite number of zeros to flip inside the unit circle.  Maybe
used in conjunction with a series of forward interpolation
formulas?


IMHO. YMMV.
-- 
Ron N.
http://www.nicholson.com/rhn

Ron N. wrote:

> Jani Huhtanen wrote:
>> Randy Yates wrote:
>>
>> > Gentle comp.dsp'ers,
>> >
>> > I again have a question that is essentially the same as the one I
>> > posted back at the beginning of April, but better stated.
>> >
>> > Assume one has a buffer of 65536 samples, and the goal is to measure
>> > the energy between Fs/4 and Fs/2 (just arbitrary frequencies selected
>> > for this example) - half the available digital bandwidth - with as
>> > little energy from the 0 to Fs/4 band in the measurement as possible.
>> >
>> > Method one: use a DFT. The manner in which the DFT is applied is open
>> > for discussion.
>> >
>> > Method two: construct a large FIR, say, 500 taps or more, with a
>> > near-brick-wall frequency response and measure the energy of the
>> > output of this filter with the input data applied.
>> >
>> > Which do you think is more accurate in rejecting energy from 0 to
>> > Fs/4?
>> >
>> > --Randy
>>
>> I don't have opinion on which method would be better, however, I think
>> filtering has problems at the borders of the signal (i.e., at the start
>> and end). Perhaps symmetric extension would help but would it be even
>> better to design 2*499 extra filters for the border cases? Those filters
>> would have lengths ranging from 1 to 499 taps and they would only be used
>> at the borders.
> 
> Why less taps?  Just make the FIR filters at the borders asymmetric,
> but still with all 500 taps.

True, I wasn't thinking straight. The design would be probably easier with
variable length filters, however, the frequency response would suffer close
to borders. 

> 
> An interesting question is how to design a family of filters with
> the same frequency response, but a whole range of offsets (1 to 250).

Offsets? Do you mean group delays? First filter should have group delay
close to 0, second filter with gd of 1 and so on.


-- 
Jani Huhtanen
Tampere University of Technology, Pori

Jerry Avins wrote:

(snip of power (energy) calculation using FFT or FIR)

> Fulfilling the terms of the power division you hypothesized is best done
> with a DFT by minimizing the spillover between the bands, hence the
> spectral leakage of the bins. I guess that zero padding probably won't
> help much; that guess can be tested. Using a subset of the data in
> smaller DFTs and averaging the results results in wider bins, and so
> spreads the leakage. I'm pretty sure that will hurt.

I would probably want to do the FFT (or DFT) wider than the source
data to avoid excessive rounding in the intermediate results.

> If you had ten times as many samples, I'm pretty sure the filter would
> be better, but the filtered samples contain start and end transients
> that need to be discarded, and that 400 discarded out of 65536 may not
> be negligible.

Well, the FFT is done on the assumption that the source data is 
periodic.  If you run the source data through the FIR more than once
(as in concatenated) and then take the appropriate 65536 samples you
minimize the start transient.  I don't know about the end transient, though.

The OP didn't say what to do with the Fs/4 bin (the boundary case).
It seems to me, though, that the FFT can be considered as a set of FIR 
filters with the appropriate taps and the transients removed with 
periodic boundary conditions (or periodic input data).

> In the absence of a developed theory, the crossover number of samples is
> best determined by experiment. The same applies to the question of
> all-at-once or divide-and-conquer in the DFT, although I think I know
> the outcome.

-- glen