When is noise shaping useful?

Hi everyone,

I've implemented a 6-order IIR in an FPGA (technically an order 2 with scattered look-ahead with pole-zero cancellation, check http://www.ece.umn.edu/users/parhi/SLIDES/chap10.pdf).

I've tried using noise shaping (1st-order) to get away with using less bits in my feedback loop (as I did sucessfully in a 1st-order IIR) but it only added oscillation to the output. So basically, I was wondering are there way to use noise-shaping with high-order IIR filters? Can anyone suggest a good reference on the subject?

Thanks

Benjamin

I remember a somewhat amusing discussion about 10 years ago where someone proposed that the noise-shaping of the truncation error should be matched to the frequency response of the pole section of the filter. However when you follow this through to it's logical conclusion, you end up with using the same coefficients in your noise shaping loop as you use in the recursive iir section, which turns out to be simply double precision.

If you want to shave some bits to save power in a custom implementation, you can probably use some and not all of the low bits left in the accumulator to recirculate.


Bob

"Robert Adams" <robert.adams@analog.com> wrote:
>I remember a somewhat amusing discussion about 10 years ago where someone 
>proposed that the noise-shaping of the truncation >error should be matched 
>to the frequency response of the pole section of the filter.

If the resultant noise spectrum is flat, this corresponds to minimum total 
noise at the output.

> However when you follow this through to it's logical conclusion, you end 
> up with using the same coefficients in your noise shaping
> loop as you use in the recursive iir section, which turns out to be simply 
> double precision.

This is not the same as doing filter in double precision.

Vladimir Vassilevsky
DSP and Mixed Signal Consultant
www.abvolt.com

On 8/14/12 6:41 PM, Vladimir Vassilevsky wrote:
>
> "Robert Adams"<robert.adams@analog.com>  wrote:
>> I remember a somewhat amusing discussion about 10 years ago where someone
>> proposed that the noise-shaping of the truncation error should be matched
>> to the frequency response of the pole section of the filter.
>
> If the resultant noise spectrum is flat, this corresponds to minimum total
> noise at the output.

i don't quite get this.  can you be a little more specific about what 
was proposed?

>> However when you follow this through to it's logical conclusion, you end
>> up with using the same coefficients in your noise shaping
>> loop as you use in the recursive iir section, which turns out to be simply
>> double precision.
>
> This is not the same as doing filter in double precision.
>

yeah, Bob.  i agree with Vlad.  it's not the same.



-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

On Tuesday, August 14, 2012 8:59:01 PM UTC-4, robert bristow-johnson wrote:
> On 8/14/12 6:41 PM, Vladimir Vassilevsky wrote:
>=20
> >
>=20
> > "Robert Adams"<robert.adams@analog.com>  wrote:
>=20
> >> I remember a somewhat amusing discussion about 10 years ago where some=
one
>=20
> >> proposed that the noise-shaping of the truncation error should be matc=
hed
>=20
> >> to the frequency response of the pole section of the filter.
>=20
> >
>=20
> > If the resultant noise spectrum is flat, this corresponds to minimum to=
tal
>=20
> > noise at the output.
>=20
>=20
>=20
> i don't quite get this.  can you be a little more specific about what=20
>=20
> was proposed?
>=20
>=20
>=20
> >> However when you follow this through to it's logical conclusion, you e=
nd
>=20
> >> up with using the same coefficients in your noise shaping
>=20
> >> loop as you use in the recursive iir section, which turns out to be si=
mply
>=20
> >> double precision.
>=20
> >
>=20
> > This is not the same as doing filter in double precision.
>=20
> >
>=20
>=20
>=20
> yeah, Bob.  i agree with Vlad.  it's not the same.
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> --=20
>=20
>=20
>=20
> r b-j                  rbj@audioimagination.com
>=20
>=20
>=20
> "Imagination is more important than knowledge."

Assume you have a 2nd order biquad using direct form 1. The only quantizati=
on error occurs at the output of the summer that combines the b0, b1, b2, a=
1, and a2 terms. The output contains the quantization shaped by 1/(1 + a1*z=
^-1 + a2*z^-2).
So now lets take the low bits from that summer that we would have thrown aw=
ay, and apply to 2 cascaded delays with coefficients aa1 and aa2 feeding ba=
ck to the same summation node. Since we know that the quantization noise re=
ponse has a peak at the pole frequency, we might want to have the aa1 and a=
a2 coefficients selected such that the error feedback 2-tap FIR has a corre=
sponding dip to minimize the output quantization noise. If you work through=
 the math you will discover that to whiten the output noise you should set =
aa1 =3D a1 and aa2 =3D a2. When this is done, it can be seen that you have =
really just applied double-precision math to the recursive portion of the I=
IR.
I'll try to dig up the references on this, but it was well documented some =
years ago.

I'm starting to remember how this all started. Jon Dattorro gave a paper so=
me years ago about error shaping, see link below.

https://ccrma.stanford.edu/~dattorro/DigitalTimesI.pdf

Fig 11 shows the 2nd-order shaping diagram. When this paper was presented, =
an audience member pointed out that the optimum settings for the error feed=
back coefficients were exactly the same as a1 and a2, and that this was in =
fact just double-precision. I was present at the presentation which is why =
I remember this, but I'm not sure it ever appeared in print; although it is=
 discussed in some spirited letters to the editor in the AES Journal;

http://www.aes.org/tmpFiles/elib/20120814/6042.pdf





On Tuesday, August 14, 2012 1:17:47 PM UTC-4, benjamin....@gmail.com wrote:
> Hi everyone,
>=20
>=20
>=20
> I've implemented a 6-order IIR in an FPGA (technically an order 2 with sc=
attered look-ahead with pole-zero cancellation, check http://www.ece.umn.ed=
u/users/parhi/SLIDES/chap10.pdf).
>=20
>=20
>=20
> I've tried using noise shaping (1st-order) to get away with using less bi=
ts in my feedback loop (as I did sucessfully in a 1st-order IIR) but it onl=
y added oscillation to the output. So basically, I was wondering are there =
way to use noise-shaping with high-order IIR filters? Can anyone suggest a =
good reference on the subject?
>=20
>=20
>=20
> Thanks
>=20
>=20
>=20
> Benjamin

On Aug 14, 10:11=A0pm, Robert Adams <robert.ad...@analog.com> wrote:
> I'm starting to remember how this all started. Jon Dattorro gave a paper =
some years ago about error shaping, see link below.
>
> https://ccrma.stanford.edu/~dattorro/DigitalTimesI.pdf
>
> Fig 11 shows the 2nd-order shaping diagram. When this paper was presented=
, an audience member pointed out that the optimum settings for the error fe=
edback coefficients were exactly the same as a1 and a2, and that this was i=
n fact just double-precision. I was present at the presentation which is wh=
y I remember this, but I'm not sure it ever appeared in print; although it =
is discussed in some spirited letters to the editor in the AES Journal;
>
> http://www.aes.org/tmpFiles/elib/20120814/6042.pdf
>
>
>
>

When I try the AES link it doesn't appear to work.

Dave

On Tuesday, August 14, 2012 1:17:47 PM UTC-4, benjamin....@gmail.com wrote:
> Hi everyone,
>=20
>=20
>=20
> I've implemented a 6-order IIR in an FPGA (technically an order 2 with sc=
attered look-ahead with pole-zero cancellation, check http://www.ece.umn.ed=
u/users/parhi/SLIDES/chap10.pdf).
>=20
>=20
>=20
> I've tried using noise shaping (1st-order) to get away with using less bi=
ts in my feedback loop (as I did sucessfully in a 1st-order IIR) but it onl=
y added oscillation to the output. So basically, I was wondering are there =
way to use noise-shaping with high-order IIR filters? Can anyone suggest a =
good reference on the subject?
>=20
>=20
>=20
> Thanks
>=20
>=20
>=20
> Benjamin

Sorry I think you have to be a member of the AES and subscribe to the E lib=
rary to get that.

I realize that my previous postings left out a bit of information, so let m=
e try to explain my reasoning here again, bearing in mind that I might be a=
ll wrong about this.

1) Assume you have a direct form 1 biquad with a single adder that sums tog=
ether all the feed-forward and feedback terms. This adder will have extende=
d resolution due to the multiplication of the data and the coefficients. Ty=
pically the output of this adder is truncated before feeding back into recu=
rsive delay memory. In a commercial fixed-point DSP this extended resolutio=
n is typically carried in the LOW portion of the accumulator. Most DSP's al=
low you to access these low bits separately in order to do double-precision=
 operations. If we assume the injected truncation noise is uncorrelated, th=
en the output spectrum will look like white noise filtered by the recursive=
 section of the filter, so if your filter has high-Q poles then there will =
be a large peak in the output noise spectrum. If the injected quantization =
noise is correlated then you can get limit cycles but let's not worry about=
 that here.

2) Now assume you want to do 2nd-order shaping of the quantization noise th=
at is injected into the system when those LOW accumulator bits are truncate=
d. The way this is done is to take those LOW bits and apply them to a 2-tap=
 FIR filter which is then fed back to the summer (with appropriate scaling =
to account for the fact that you shifted the LOW bits up to process them, s=
o they need to be shifted back down again before the addition). This is the=
 "textbook" signal flow diagram of error shaping that you will find in the =
paper I referenced.


3) The traditional view of error shaping is that you view the feedback filt=
er as a noise-shaping filter, and design it to minimize the output noise. I=
n systems where you have high-Q poles in the recursive section of the filte=
r, you would normally design the noise transfer function to have notches at=
 that frequency.

4) If you take it one step further and place the zeros of the noise-shaping=
 function directly on top of the poles of the filter, then the feedback coe=
fficients of the noise-shaping filter and the feedback coefficients of the =
recursive section of the biquad are exactly the same. In this case the quan=
tization noise will be whitened. Remember that just because the error is wh=
ite doesn't mean it hasn't been shaped; you need to compare the spectrum wi=
th and without the shaping filter applied.

5) The last step is to realize that this is the same as double precision ju=
st applied to the recursive section of the filter, assuming that the coeffi=
cients in the recursive part of the biquad filter are the same as the coeff=
icients used in the noise-shaping filter;

a1*HIGH_BITS + a1*LOW_BITS  =3D   a1*(HIGH_BITS + LOW_BITS) =3D double-prec=
ision

and the same of course for a2. Note that technically this is "single X doub=
le", since the coefficients are still single-precision.

Note that we are assuming that the final output of the filter is the high b=
its only, so maybe that is one difference between true double precision and=
 this scheme, in that for true double precision you would pass the full acc=
umulator (HIGH + LOW) to the outside world, whereas we are only passing on =
the HIGH bits.


I have read papers that indicate that if you want to minimize the rms noise=
, the best thing you can do is to whiten the noise spectrum, so I think thi=
s scheme is optimal, and that other alignments of the noise-shaping filter =
give worse performance. In other words, putting an infinite noise-shaping n=
otch at the pole frequency may minimize the noise in that particular part o=
f the spectrum, but you will pay the price in that the energy in other part=
s of the spectrum will become larger. I think there was a famous result by =
Gerzon and Craven in AESJ about this, but the math was over my head.

Again I might have this all wrong, and I know rbj has enormous experience h=
ere so that fact that he is skeptical worries me greatly. If I end up embar=
rassing myself I reserve the right to use the Men-in-Black memory-erase opt=
ion, adapted for newsgroups.

Bob

Back to my case

My transfer function looks something like this

H(z) = Num(z)/(1 - a3*z^-3 - a6 * z^-6).

If I understand correctly, the noise shaping block should have a transfer function like (1 + d3 * z^-3 + d6 * z^-6)? 

Yes I think you have the right idea. I would start with 


D3 = a3
D6= a6

Bob