Hi everyone,

I've implemented a 6-order IIR in an FPGA (technically an order 2 with scattered look-ahead with
pole-zero cancellation, check http://www.ece.umn.edu/users/parhi/SLIDES/chap10.pdf).

I've tried using noise shaping (1st-order) to get away with using less bits in my feedback loop
(as I did sucessfully in a 1st-order IIR) but it only added oscillation to the output. So
basically, I was wondering are there way to use noise-shaping with high-order IIR filters? Can
anyone suggest a good reference on the subject?

Thanks

Benjamin

I remember a somewhat amusing discussion about 10 years ago where someone proposed that the
noise-shaping of the truncation error should be matched to the frequency response of the pole
section of the filter. However when you follow this through to it's logical conclusion, you end
up with using the same coefficients in your noise shaping loop as you use in the recursive iir
section, which turns out to be simply double precision.

If you want to shave some bits to save power in a custom implementation, you can probably use
some and not all of the low bits left in the accumulator to recirculate.


Bob

"Robert Adams" <r...@analog.com> wrote:
>I remember a somewhat amusing discussion about 10 years ago where someone 
>proposed that the noise-shaping of the truncation >error should be matched 
>to the frequency response of the pole section of the filter.

If the resultant noise spectrum is flat, this corresponds to minimum total 
noise at the output.

> However when you follow this through to it's logical conclusion, you end 
> up with using the same coefficients in your noise shaping
> loop as you use in the recursive iir section, which turns out to be simply 
> double precision.

This is not the same as doing filter in double precision.

Vladimir Vassilevsky
DSP and Mixed Signal Consultant
www.abvolt.com

On 8/14/12 6:41 PM, Vladimir Vassilevsky wrote:
>
> "Robert Adams"<r...@analog.com>  wrote:
>> I remember a somewhat amusing discussion about 10 years ago where someone
>> proposed that the noise-shaping of the truncation error should be matched
>> to the frequency response of the pole section of the filter.
>
> If the resultant noise spectrum is flat, this corresponds to minimum total
> noise at the output.

i don't quite get this.  can you be a little more specific about what 
was proposed?

>> However when you follow this through to it's logical conclusion, you end
>> up with using the same coefficients in your noise shaping
>> loop as you use in the recursive iir section, which turns out to be simply
>> double precision.
>
> This is not the same as doing filter in double precision.
>

yeah, Bob.  i agree with Vlad.  it's not the same.



-- 

r b-j                  r...@audioimagination.com

"Imagination is more important than knowledge."

On Tuesday, August 14, 2012 8:59:01 PM UTC-4, robert bristow-johnson wrote:
> On 8/14/12 6:41 PM, Vladimir Vassilevsky wrote:
> 
> >
> 
> > "Robert Adams"<r...@analog.com>  wrote:
> 
> >> I remember a somewhat amusing discussion about 10 years ago where someone
> 
> >> proposed that the noise-shaping of the truncation error should be matched
> 
> >> to the frequency response of the pole section of the filter.
> 
> >
> 
> > If the resultant noise spectrum is flat, this corresponds to minimum total
> 
> > noise at the output.
> 
> 
> 
> i don't quite get this.  can you be a little more specific about what 
> 
> was proposed?
> 
> 
> 
> >> However when you follow this through to it's logical conclusion, you end
> 
> >> up with using the same coefficients in your noise shaping
> 
> >> loop as you use in the recursive iir section, which turns out to be simply
> 
> >> double precision.
> 
> >
> 
> > This is not the same as doing filter in double precision.
> 
> >
> 
> 
> 
> yeah, Bob.  i agree with Vlad.  it's not the same.
> 
> 
> 
> 
> 
> 
> 
> -- 
> 
> 
> 
> r b-j                  r...@audioimagination.com
> 
> 
> 
> "Imagination is more important than knowledge."

Assume you have a 2nd order biquad using direct form 1. The only quantization error occurs at
the output of the summer that combines the b0, b1, b2, a1, and a2 terms. The output contains the
quantization shaped by 1/(1 + a1*z^-1 + a2*z^-2).
So now lets take the low bits from that summer that we would have thrown away, and apply to 2
cascaded delays with coefficients aa1 and aa2 feeding back to the same summation node. Since we
know that the quantization noise reponse has a peak at the pole frequency, we might want to have
the aa1 and aa2 coefficients selected such that the error feedback 2-tap FIR has a corresponding
dip to minimize the output quantization noise. If you work through the math you will discover
that to whiten the output noise you should set aa1 = a1 and aa2 = a2. When this is done, it can
be seen that you have really just applied double-precision math to the recursive portion of the
IIR.
I'll try to dig up the references on this, but it was well documented some years ago.

I'm starting to remember how this all started. Jon Dattorro gave a paper some years ago
about error shaping, see link below.

https://ccrma.stanford.edu/~dattorro/DigitalTimesI.pdf

Fig 11 shows the 2nd-order shaping diagram. When this paper was presented, an audience member
pointed out that the optimum settings for the error feedback coefficients were exactly the same
as a1 and a2, and that this was in fact just double-precision. I was present at the presentation
which is why I remember this, but I'm not sure it ever appeared in print; although it is
discussed in some spirited letters to the editor in the AES Journal;

http://www.aes.org/tmpFiles/elib/20120814/6042.pdf





On Tuesday, August 14, 2012 1:17:47 PM UTC-4, benjamin....@gmail.com wrote:
> Hi everyone,
> 
> 
> 
> I've implemented a 6-order IIR in an FPGA (technically an order 2 with scattered look-ahead
with pole-zero cancellation, check http://www.ece.umn.edu/users/parhi/SLIDES/chap10.pdf).
> 
> 
> 
> I've tried using noise shaping (1st-order) to get away with using less bits in my feedback
loop (as I did sucessfully in a 1st-order IIR) but it only added oscillation to the output. So
basically, I was wondering are there way to use noise-shaping with high-order IIR filters? Can
anyone suggest a good reference on the subject?
> 
> 
> 
> Thanks
> 
> 
> 
> Benjamin

On Aug 14, 10:11 pm, Robert Adams <robert.ad...@analog.com> wrote:
> I'm starting to remember how this all started. Jon Dattorro gave a paper some years ago
about error shaping, see link below.
>
> https://ccrma.stanford.edu/~dattorro/DigitalTimesI.pdf
>
> Fig 11 shows the 2nd-order shaping diagram. When this paper was presented, an audience
member pointed out that the optimum settings for the error feedback coefficients were exactly
the same as a1 and a2, and that this was in fact just double-precision. I was present at the
presentation which is why I remember this, but I'm not sure it ever appeared in print; although
it is discussed in some spirited letters to the editor in the AES Journal;
>
> http://www.aes.org/tmpFiles/elib/20120814/6042.pdf
>
>
>
>

When I try the AES link it doesn't appear to work.

Dave

On Tuesday, August 14, 2012 1:17:47 PM UTC-4, benjamin....@gmail.com wrote:
> Hi everyone,
> 
> 
> 
> I've implemented a 6-order IIR in an FPGA (technically an order 2 with scattered look-ahead
with pole-zero cancellation, check http://www.ece.umn.edu/users/parhi/SLIDES/chap10.pdf).
> 
> 
> 
> I've tried using noise shaping (1st-order) to get away with using less bits in my feedback
loop (as I did sucessfully in a 1st-order IIR) but it only added oscillation to the output. So
basically, I was wondering are there way to use noise-shaping with high-order IIR filters? Can
anyone suggest a good reference on the subject?
> 
> 
> 
> Thanks
> 
> 
> 
> Benjamin

Sorry I think you have to be a member of the AES and subscribe to the E library to get that.

I realize that my previous postings left out a bit of information, so let me try to explain my
reasoning here again, bearing in mind that I might be all wrong about this.

1) Assume you have a direct form 1 biquad with a single adder that sums together all the
feed-forward and feedback terms. This adder will have extended resolution due to the
multiplication of the data and the coefficients. Typically the output of this adder is truncated
before feeding back into recursive delay memory. In a commercial fixed-point DSP this extended
resolution is typically carried in the LOW portion of the accumulator. Most DSP's allow you to
access these low bits separately in order to do double-precision operations. If we assume the
injected truncation noise is uncorrelated, then the output spectrum will look like white noise
filtered by the recursive section of the filter, so if your filter has high-Q poles then there
will be a large peak in the output noise spectrum. If the injected quantization noise is
correlated then you can get limit cycles but let's not worry about that here.

2) Now assume you want to do 2nd-order shaping of the quantization noise that is injected into
the system when those LOW accumulator bits are truncated. The way this is done is to take those
LOW bits and apply them to a 2-tap FIR filter which is then fed back to the summer (with
appropriate scaling to account for the fact that you shifted the LOW bits up to process them, so
they need to be shifted back down again before the addition). This is the "textbook"
signal flow diagram of error shaping that you will find in the paper I referenced.


3) The traditional view of error shaping is that you view the feedback filter as a noise-shaping
filter, and design it to minimize the output noise. In systems where you have high-Q poles in
the recursive section of the filter, you would normally design the noise transfer function to
have notches at that frequency.

4) If you take it one step further and place the zeros of the noise-shaping function directly on
top of the poles of the filter, then the feedback coefficients of the noise-shaping filter and
the feedback coefficients of the recursive section of the biquad are exactly the same. In this
case the quantization noise will be whitened. Remember that just because the error is white
doesn't mean it hasn't been shaped; you need to compare the spectrum with and without the
shaping filter applied.

5) The last step is to realize that this is the same as double precision just applied to the
recursive section of the filter, assuming that the coefficients in the recursive part of the
biquad filter are the same as the coefficients used in the noise-shaping filter;

a1*HIGH_BITS + a1*LOW_BITS  =   a1*(HIGH_BITS + LOW_BITS) = double-precision

and the same of course for a2. Note that technically this is "single X double", since
the coefficients are still single-precision.

Note that we are assuming that the final output of the filter is the high bits only, so maybe
that is one difference between true double precision and this scheme, in that for true double
precision you would pass the full accumulator (HIGH + LOW) to the outside world, whereas we are
only passing on the HIGH bits.


I have read papers that indicate that if you want to minimize the rms noise, the best thing you
can do is to whiten the noise spectrum, so I think this scheme is optimal, and that other
alignments of the noise-shaping filter give worse performance. In other words, putting an
infinite noise-shaping notch at the pole frequency may minimize the noise in that particular
part of the spectrum, but you will pay the price in that the energy in other parts of the
spectrum will become larger. I think there was a famous result by Gerzon and Craven in AESJ
about this, but the math was over my head.

Again I might have this all wrong, and I know rbj has enormous experience here so that fact that
he is skeptical worries me greatly. If I end up embarrassing myself I reserve the right to use
the Men-in-Black memory-erase option, adapted for newsgroups.

Bob

Back to my case

My transfer function looks something like this

H(z) = Num(z)/(1 - a3*z^-3 - a6 * z^-6).

If I understand correctly, the noise shaping block should have a transfer function like (1 + d3
* z^-3 + d6 * z^-6)? 

Yes I think you have the right idea. I would start with 


D3 = a3
D6= a6

Bob