# Quantizing the filter coefficients, what are the criterias?

Started by June 8, 2006
```I am quantizing an FIR filter's coefficients into 8-bit signed integers,
which will be coded into hardware. How do I describe the errors in
the quantization in my report? I only know of frequency response
before and after quantization.

TIA

```
```For FIRs, that's all there is.

--Randy

Mr. Ken wrote:
> I am quantizing an FIR filter's coefficients into 8-bit signed integers,
> which will be coded into hardware. How do I describe the errors in
> the quantization in my report? I only know of frequency response
> before and after quantization.
>
> TIA

```
```On Thu, 8 Jun 2006 11:24:53 +0800, "Mr. Ken" <Mr. Ken@asdf> wrote:

>I am quantizing an FIR filter's coefficients into 8-bit signed integers,
>which will be coded into hardware. How do I describe the errors in
>the quantization in my report?

Since the coefficients of a FIR filter are also its impulse response,
the set of quantization errors can be thought of as the impulse response
of an "error filter" that is added to the "true filter" to obtain the
"quantized filter".  Similarly, the Discrete Fourier Transform of the
set of quantization errors can be thought of as an "error frequency
response" that is added to the "true frequency response" to obtain the
"quantized frequency response".  You can even manipulate the
quantization errors to obtain more desirable characteristics in this
"error filter".

Greg
```
```Greg Berchin wrote:
>
> Since the coefficients of a FIR filter are also its impulse response,
> the set of quantization errors can be thought of as the impulse response
> of an "error filter" that is added to the "true filter" to obtain the
> "quantized filter".  Similarly, the Discrete Fourier Transform of the
> set of quantization errors can be thought of as an "error frequency
> response" that is added to the "true frequency response" to obtain the
> "quantized frequency response".  You can even manipulate the
> quantization errors to obtain more desirable characteristics in this
> "error filter".

one way to do that is with noise shaping the impulse response as it is
input to the 8-bit quantizer.  but we steer the noise from the stopband
(where the absolute error in the FIR gain is more of a problem) into
the passband where it is less of a problem.

r b-j

```
```I am also working on a cpld project where I need to see the frequency
responses of different coefficient wordlengths in order to obtain the
optimum result as space requirements on the device are very tight. Are
there any free utilities on the web to achieve this?

Rick

Mr. Ken wrote:
> I am quantizing an FIR filter's coefficients into 8-bit signed integers,
> which will be coded into hardware. How do I describe the errors in
> the quantization in my report? I only know of frequency response
> before and after quantization.
>
> TIA

```
```"realimag" <realimag@hotmail.com> wrote in message
> I am also working on a cpld project where I need to see the frequency
> responses of different coefficient wordlengths in order to obtain the
> optimum result as space requirements on the device are very tight. Are
> there any free utilities on the web to achieve this?
>
> Rick
>

If you use Xilinx FPGA in ISE, you may as well write some scripts to
automate the
entire procedure, then you can run fitting for all the wordlength.

For Altera, I am not sure.

```
```

>
> one way to do that is with noise shaping the impulse response as it is
> input to the 8-bit quantizer.  but we steer the noise from the stopband
> (where the absolute error in the FIR gain is more of a problem) into
> the passband where it is less of a problem.
>

Sounds interesting...do you have any more details on how to perform this
noise shaping, or a pointer to a paper/book?

```
```Rick,

ONEoverT digital filter designer will let you see the responses of
different coefficient wordlengths. you won't get any code output with
the demo version however, but you probably don't need the full version
anyway.

Bob

realimag wrote:
> I am also working on a cpld project where I need to see the frequency
> responses of different coefficient wordlengths in order to obtain the
> optimum result as space requirements on the device are very tight. Are
> there any free utilities on the web to achieve this?
>
> Rick
>
> Mr. Ken wrote:
> > I am quantizing an FIR filter's coefficients into 8-bit signed integers,
> > which will be coded into hardware. How do I describe the errors in
> > the quantization in my report? I only know of frequency response
> > before and after quantization.
> >
> > TIA

```
```in article 4489e9fe\$0\$1216\$ba620e4c@news.skynet.be, NewLine at
umts_remove_all_this@skynet.be wrote on 06/09/2006 17:37:

>>
>> one way to do that is with noise shaping the impulse response as it is
>> input to the 8-bit quantizer.  but we steer the noise from the stopband
>> (where the absolute error in the FIR gain is more of a problem) into
>> the passband where it is less of a problem.
>>
>
> Sounds interesting...do you have any more details on how to perform this
> noise shaping, or a pointer to a paper/book?

quantizing:

.------.
x[n] ------>| Q{.} |---------> xq[n]
'------'

xq[n] = Q{ x[n] } = x[n] + e[n]

Q{.} is the quantization operator (rounding or truncation) and

e[n] =def  xq[n] - x[n]  =  Q{x[n]} - x[n]

is the quantization error (or "quantization noise").  it can be computed by
taking the output of the quantizer and subtracting the input.  what this
does is model the quantizer as some kind of additive error source:

.----- e[n] = xq[n] - x[n]
|
v
x[n] ---------->(+)----------> xq[n]

here, the transfer function from e[n] to xq[n] is Xq(z)/E(z) = 1 but e[n] is
usually considered to be much smaller than x[n].  now if that is true (x[n]
much larger than the round-off or truncation error of the quantizer), then
the round-off error could be anything within -1/2 LSB and +1/2 LSB (and
pretty random) and the q[n] error will be pretty white (up to Nyquist).  so
the spectrum of xq[n] is pretty much the same as x[n] except for some sorta
flat white noise spectrum added in.  if the spectrum of x[n] had some gaps
in it where the energy got very low, then the whitish spectrum of e[n] fills
in those gaps in the spectrum.

noise shaping:

.-------------.
e1[n] .---| (z^-1)*G(z) |<----.----- e[n] = xq[n] - x1[n]
|   '-------------'     |
v -                     v
x[n] ---->(+)-------------------->(+)-------> xq[n]
+               x1[n]

notice that the output of the filter G(z) is subtracted from x[n] to result
in x1[n] which is what is applied to the quantizer Q{.}.  so the output now
is

xq[n] = Q{ x1[n] } = x1[n] + e[n]

where

x1[n] = x[n] - e1[n]

and

e1[n] = g[n] (*) e[n-1]         (*) means discrete convolution

also, the filter (z^-1)*G(z) must have at least one sample delay built in
because it does not know the present roundoff error (the roundoff of the
present sample hasn't happened yet) only the past roundoff errors, e[n-1],
e[n-2], etc.

G(z) can be any realizable (causal) filter and is actually the filter that
is implemented (not (z^-1)*G(z)), but it only sees the past sample
quantization errors, not the current quantization error.

now the transfer function of quantization noise to output is

Xq(z)/E(z) = 1 - (z^-1)*G(z)

now G(z) has to be designed in such a way so that |Xq(z)/E(z)| will steer or
"shape" the spectrum of the noise away from where you don't want it and put
it where you don't care as much.  the simplest noise shaper is one where
G(z) = 1 so the noise shaping filter would be

Xq(z)/E(z) = 1 - (z^-1)

with frequency response:

|Xq(e^jw)/E(e^jw)| = |1 - e^(-jw)| = 2*sin(w/2)

and that is a high pass filter with no gain at DC at all.  that means there
is no noise at DC, little noise at low frequencies and increased
quantization noise around Nyquist.

now, for FIR filter design, what we are talking about is h[n] (with spectrum
H(z) where z = e^jw) instead of x[n] and the quantized impulse response is
hq[n] with spectrum Hq(z).

if we look at the gain of the FIR filter with a "dB lens", we will care more
about errors in the spectrum Hq(e^jw) at frequencies where |H(e^jw)| is
small (resulting in bigger dB error) than where |H(e^jw)| is large.  so we
want

|Hq(e^jw)/E(e^jw)| = |1 - (e^-jw)*G(e^jw)|  proportional to  |H(e^jw)|

or

|1 - (e^-jw)*G(e^jw)|^2  =  A*|H(e^jw)|^2

for some real constant A>0 .

so given the "ideal" FIR, h[n]. with spectrum H(e^jw), you want to design
G(z) so that the above equation is as accurate as possible, then run your
h[n] through the noise-shaped quantizer:

.-------------.
e1[n] .---| (z^-1)*G(z) |<----.----- e[n] = hq[n] - h1[n]
|   '-------------'     |
v -                     v
h[n] ---->(+)-------------------->(+)-------> hq[n]
+               h1[n]

or

.--------.
e1[n] .---|  G(z)  |<----- e[n-1] = hq[n-1] - h1[n-1]
|   '--------'
v -                 .------.
h[n] ---->(+)----------------->| Q{.} |------> hq[n] = Q{h1[n]}
+            h1[n]    '------'

and see what you get out: hq[n].

does that make sense.

otherwise, all i can recommend is Google.

--

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

```
```in article C0AF954C.153EA%rbj@audioimagination.com, robert bristow-johnson
at rbj@audioimagination.com wrote on 06/09/2006 21:14:

> otherwise, all i can recommend is Google.

Google:  quantized FIR coefficients     (no quotes)

and you get a lot of nice hits including

http://www.ece.vt.edu/fac_support/dspcl/docs/SPMag03.pdf

and

http://hjem.get2net.dk/jjn/quantfir.htm