Digital Atten&Ampl. ?

Hello all,

I have a really simple question to ask. So simple that I almost feel
embarassed asking it. I have looked on the web for the answer, but
sometimes trying to find an answer to the really simple questions can
prove more difficult than information about cryptic phenomena.
Everybody WANTS to talk about the mysterious and complicated stuff.
12^12=144 is common knowledge but I doubt someone will dedicated a web
page to it. But I digress.

Anyway here goes:
Imagine a 10 bit wide databus.
If that databus were to be filtered via a digital filter whose stopband
attenuation is -100dB,  what would that mean extacly? would that be
overkill because a 10bit system can only up to 1023 -->
20log(1024)=60dB.  Similarly, if a digital filter were to have DCgain
of 20dB, then wouldn't that limit the range of the input databus or
risk causing overflow at the output?
Please advise.

(It was so much easy to put attenuation and amplification in their
respective places back in the analog world ;) )

-Roger

Roger Bourne wrote:
> Hello all,
> 
> I have a really simple question to ask. So simple that I almost feel
> embarassed asking it. I have looked on the web for the answer, but
> sometimes trying to find an answer to the really simple questions can
> prove more difficult than information about cryptic phenomena.
> Everybody WANTS to talk about the mysterious and complicated stuff.
> 12^12=144 is common knowledge but I doubt someone will dedicated a web
> page to it. But I digress.
> 
> Anyway here goes:
> Imagine a 10 bit wide databus.
> If that databus were to be filtered via a digital filter whose stopband
> attenuation is -100dB,  what would that mean extacly? would that be
> overkill because a 10bit system can only up to 1023 -->
> 20log(1024)=60dB. 

If you accepted a wider output bus than the input it's perfectly 
reasonable to ask for 100dB of attenuation in the stop band -- but you 
_do_ have to make sure that your output bus is wide enough for it to 
make sense.

> Similarly, if a digital filter were to have DCgain
> of 20dB, then wouldn't that limit the range of the input databus or
> risk causing overflow at the output?

If you could guarantee that the input signal had little content at DC 
then a high DC gain would be safe.  If not, then once again you'd have 
to provide a wider bus on the output than the input.

> Please advise.
> 
> (It was so much easy to put attenuation and amplification in their
> respective places back in the analog world ;) )
> 
> -Roger
> 
Choosing the right width for a data path is an important part of DSP 
design.  It sets the dynamic range of your system, as well as affecting 
the execution speed and/or price.  This is usually not expressed in 
terms of 'amplification' because signal amplification is almost a null 
concept in DSP -- just think of the number differently, and voila! it's 
got a different amplitude.

What you're talking about is more how the frequency response of a filter 
affects the data paths.  It is not at all uncommon for data to have more 
information buried in it than is apparent by it's width -- an extreme 
case of this is the sigma-delta converter, which dithers a 1-bit ADC to 
get theoretically infinite precision, with practical, inexpensive 24-bit 
converters on the market.  In such cases you just have to design your 
filter and pay attention to the data widths if you want to succeed.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google?  See http://cfaj.freeshell.org/google/

"Roger Bourne" <rover8898@hotmail.com> writes:
> [...]

Hi Roger,

First off, let me say that you're questions are not trivial. They
are good questions to ask. Let me respond below.

> Anyway here goes: Imagine a 10 bit wide databus.  If that databus
> were to be filtered via a digital filter whose stopband attenuation
> is -100dB, what would that mean extacly? would that be overkill
> because a 10bit system can only up to 1023 --> 20log(1024)=60dB.

If you do things "right," you can think of a digital signal as an
analog signal that contains a certain amount of wideband noise. The
amount of wideband noise is indirectly proportional to the number of
bits used in the amplitude quantization of the signal. Few bits mean
more noise.

So, the answer to this question is "no," it wouldn't be overkill,
because when you attenuate (filter) the signal there would still be
some of the original signal present, but just deeper down in the
noise.  Nota Bena: This is *if* you do things "right." 

I noticed Tim Wescott said you'd have to extend the bus width
to keep the extra attenuation. I would disagree with this and
clarify that if you maintain the same bus width (10 bits in the
example stated), you would have the situation I described above
(the signal is buried deeper in the noise). If you do extend the
bus width, then you would have a situation where the wideband
noise from the original 10 bits is attenuated, along with the
signal, in the stopband.

> Similarly, if a digital filter were to have DCgain of 20dB, then
> wouldn't that limit the range of the input databus or risk causing
> overflow at the output?  Please advise.

Also a good question. Yes, you are precisely correct - you'd have to
either risk overflows (or saturations), or limit the original signal.

This is why digital volume controls are tough to implement - you need
extra bits to allow for the dynamic range of the volume control if you
don't want to lose resolution in the input signal.
-- 
%  Randy Yates                  % "So now it's getting late,
%% Fuquay-Varina, NC            %    and those who hesitate
%%% 919-577-9882                %    got no one..."
%%%% <yates@ieee.org>           % 'Waterfall', *Face The Music*, ELO
http://home.earthlink.net/~yatescr

Randy Yates <yates@ieee.org> writes:
> [...]
> "Roger Bourne" <rover8898@hotmail.com> writes:
>> Similarly, if a digital filter were to have DCgain of 20dB, then
>> wouldn't that limit the range of the input databus or risk causing
>> overflow at the output?  

Sorry - I didn't notice the term "DCgain" at first. If you meant a
filter with a frequency-dependent gain rather than a constant gain, 
then I'd respond with "it depends on what you want."

If you want to maintain the same scaling on the input and output data,
then again you are correct - you must either limit the range of the
input or risk overflow/saturation.

However, if you are only interested in applying a filter of a certain
shape and don't care that input and output scalings match (or
equivalently, if you don't really care about the absolute gain of the
filter), then there is usually a way to ensure there are no overflows
in the output, but at the risk of decreasing dynamic range in other
parts of the spectrum. The absolute answer depends on the details of
the architecture you are using such as the number of bits in the
accumulator and the multiplier output word length.
-- 
%  Randy Yates                  % "Bird, on the wing,
%% Fuquay-Varina, NC            %   goes floating by
%%% 919-577-9882                %   but there's a teardrop in his eye..."
%%%% <yates@ieee.org>           % 'One Summer Dream', *Face The Music*, ELO
http://home.earthlink.net/~yatescr

Roger Bourne wrote:
> Hello all,
> 
> I have a really simple question to ask. So simple that I almost feel
> embarassed asking it. I have looked on the web for the answer, but
> sometimes trying to find an answer to the really simple questions can
> prove more difficult than information about cryptic phenomena.
> Everybody WANTS to talk about the mysterious and complicated stuff.
> 12^12=144 is common knowledge but I doubt someone will dedicated a web
> page to it. But I digress.

My calculator tells me that 12^12 is 8,916,100,448,256

> Anyway here goes:
> Imagine a 10 bit wide databus.
> If that databus were to be filtered via a digital filter whose stopband
> attenuation is -100dB,  what would that mean extacly?

Whoa! You don't filter a data bus except maybe to reduce glitches. Do 
you mean that you have a 10-bit digital data stream that represents a 
signal in time?

> would that be overkill because a 10bit system can only up to 1023 -->
> 20log(1024)=60dB.

Probably. It is possible with dither to represent a signal whose peak 
amplitude is less than one bit, but not much less.

> Similarly, if a digital filter were to have DCgain
> of 20dB, then wouldn't that limit the range of the input databus or
> risk causing overflow at the output?
> Please advise.

A filter with a gain of 20 dB produces output (at at least some 
frequency) with 10 times the amplitude of the input. Although it is 
unusual, there is no reason that the data out can't be represented by 
more bits than the data going in.

> (It was so much easy to put attenuation and amplification in their
> respective places back in the analog world ;) )

It is easy to overload analog circuits too. High-dynamic-range wide-band 
low-noise low-distortion circuits were never easy.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

> If you accepted a wider output bus than the input it's perfectly
> reasonable to ask for 100dB of attenuation in the stop band -- but you
> _do_ have to make sure that your output bus is wide enough for it to
> make sense.

This is a topic of interest to me:
Increasing the output bus (w.r.t input bus) of an IIR filter is giving
me a severe headache. Not in the implementaion sense of it, but rather
in the "what does it exactly mean?" kind of sense.

If the IIR filter input and output bus were the same width (number of
bits), then there would be no dilemma. You get a 16 bit data stream,
you dsp-filter it, and then you get the new filtered 16 bit data
stream. Voila, no problem.

But if the IIR filter output bus is bigger than the input bus, lets say
32 bits(O) and 16 bits(I), then the following question arise in my
head:
1. How is it possible to improve the accuracy of a data stream ? (Just
from a black box point of view, filtered data should have the number of
bits as the input data. Otherwise it is apples and oranges)
2. Are we really INCREASING the precision or are we rather attempting
to PRESERVE an existing infinite precison of ????? [something that I
can not put my thumb on].
3. If we really COULD increase the output precision with respect to the
input precision, then if we were to cascade several stages we could
obtain an extremely high precison from only several bits. That seems
mad. However, I get the impression that it is feasible and quite
common.
4. From where is this increased precision coming from? If my input of
the IIR filter is stemming from an ADC, how can the IIR filter
attribute a higher res than the ADC is already providing?

Please advise,

(and a big thanks for the preceding answers to my earlier posts)
-Roger

Roger Bourne wrote:

>>If you accepted a wider output bus than the input it's perfectly
>>reasonable to ask for 100dB of attenuation in the stop band -- but you
>>_do_ have to make sure that your output bus is wide enough for it to
>>make sense.
> 
> 
> This is a topic of interest to me:
> Increasing the output bus (w.r.t input bus) of an IIR filter is giving
> me a severe headache. Not in the implementaion sense of it, but rather
> in the "what does it exactly mean?" kind of sense.
> 
> If the IIR filter input and output bus were the same width (number of
> bits), then there would be no dilemma. You get a 16 bit data stream,
> you dsp-filter it, and then you get the new filtered 16 bit data
> stream. Voila, no problem.
> 
> But if the IIR filter output bus is bigger than the input bus, lets say
> 32 bits(O) and 16 bits(I), then the following question arise in my
> head:
> 1. How is it possible to improve the accuracy of a data stream ? (Just
> from a black box point of view, filtered data should have the number of
> bits as the input data. Otherwise it is apples and oranges)
> 2. Are we really INCREASING the precision or are we rather attempting
> to PRESERVE an existing infinite precison of ????? [something that I
> can not put my thumb on].
> 3. If we really COULD increase the output precision with respect to the
> input precision, then if we were to cascade several stages we could
> obtain an extremely high precison from only several bits. That seems
> mad. However, I get the impression that it is feasible and quite
> common.
> 4. From where is this increased precision coming from? If my input of
> the IIR filter is stemming from an ADC, how can the IIR filter
> attribute a higher res than the ADC is already providing?
> 
> Please advise,
> 
> (and a big thanks for the preceding answers to my earlier posts)
> -Roger
> 
Let's do a thought experiment, then, in the grand tradition of the old 
Greek philosophers.

Let's say that you have a signal that can only take on the values 0 and 
1 -- so it can be represented by just one bit in the grand old tradition 
of bits.

Now let's say that this signal is tied somehow to an analog signal, and 
done so in such a way that the _average_ value of the digital, 1-bit 
signal is exactly equal to the average value of the analog signal*.

If we were to take 16384 samples of that digital signal and averaged 
them, we'd have a much better idea of the value of the original analog 
signal than just '1' or '0'.  Depending on how the digital signal is 
tied to the analog signal you could expect that the result of this 2^14 
point sample would be good to anywhere between 7 and 14 bits**.

So a 'sensible' output from the averaging operation would have between 7 
and 14 bits -- _not_ 1.

We have, in fact, answered questions 1 through 3 above -- the accuracy 
has been improved because there was high-frequency noise on the data 
that we've filtered out, the precision has indeed been improved, and we 
did it starting with just one bit to get 14.

A subject that is somewhat tangential to your question, but one which 
you may find useful, is sigma-delta converters.  In a S-D ADC you take 
the output of a (usually) 1-bit DAC, subtract it from your signal of 
interest, filter it with an integrating filter, and run it to a 
(usually) 1-bit ADC.  The output of this process is just one bit wide -- 
but there are fairly inexpensive 24 bit S-D ADCs out there.  That's as 
much as you can expect from any ADC, and it presents a severe challenge 
to any front-end electronics you may use in front of the ADC.  I have 
written a simplified article on sigma-delta DACs, which you can get to 
from http://www.wescottdesign.com/articles/sigmadelta.html; it may help.

As for question 4, well, think of averaging as just another low-pass 
filtering process.  You can substitute the IIR low-pass filter of your 
choice and still see significant averaging going on.  \

You used exactly the right word when you asked about extending ADC 
_resolution_, by the way.  Your average 16-bit, 100ksps ADC will have 
internal noise in its front end, with an RMS value that's going to be 
several LSBs.  This means that any one reading of the ADC could be off 
by many counts and still be within spec.  It also means that several 
successive readings of the ADC will each be different from the other, 
but they will center around some value that's specific to the input 
value of the ADC but may not be the _right_ value (hitting the right 
value would be accuracy).  If you take such an ADC and average its 
output you can get way more than 16 bits of resolution, even if you're 
not really improving the accuracy.  This can be useful in motion control 
systems where short-term repeatability is essential but absolute 
accuracy isn't quite that critical.

* I'd say equal to the value of the analog signal, but it might be changing.

** Actually it could be worse than 7 bits, but never better than 14.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Posting from Google?  See http://cfaj.freeshell.org/google/

Roger Bourne wrote:
> > If you accepted a wider output bus than the input it's perfectly
> > reasonable to ask for 100dB of attenuation in the stop band -- but you
> > _do_ have to make sure that your output bus is wide enough for it to
> > make sense.
...
> 2. Are we really INCREASING the precision or are we rather attempting
> to PRESERVE an existing infinite precison of ????? [something that I
> can not put my thumb on].

Tim already answered this in great detail.

I'll just throw in an example in a more "everyday" context in order to
illustrate the concept of fixed-width integer arithmetics.

Suppose you want to multiply the numbers

A = 10
B = 20

C = A*B = 200.

As you can see, multiplying two two-digit numbers produces a
trhee-digit
number. Next, assume that you for some reason are only allowed to write

down two digits in the answer, and you see the dillema: How can one
accurately represent a three-digit number when one only is allowed to
write down two of the digits.

This is the dillema in the IIR filter. The IIR filter consists of
several
multiplications of N-bit numbers. If you start out with an N-bit
input x0 and performs M+1 multiplications with N-bit numbers am, like

x1 = a0*x0
x2 = a1*x1
:
xM+1= aM*xM

then the result xM+1 will, in the extreme case, contain something
like M*(N-1)+N bits.

So you are left with a choise as how to deal with this: Either get a
numerically accurate result, obtained at the cost of increasing the
number of bits in the bus after each multiplication, or keep a fixed
number of bits all through and accept the numerical error creeping in.
 
Rune

Rune Allnor wrote:
> Roger Bourne wrote:
>>> If you accepted a wider output bus than the input it's perfectly
>>> reasonable to ask for 100dB of attenuation in the stop band -- but you
>>> _do_ have to make sure that your output bus is wide enough for it to
>>> make sense.
> ...
>> 2. Are we really INCREASING the precision or are we rather attempting
>> to PRESERVE an existing infinite precison of ????? [something that I
>> can not put my thumb on].
> 
> Tim already answered this in great detail.
> 
> I'll just throw in an example in a more "everyday" context in order to
> illustrate the concept of fixed-width integer arithmetics.
> 
> Suppose you want to multiply the numbers
> 
> A = 10
> B = 20
> 
> C = A*B = 200.
> 
> As you can see, multiplying two two-digit numbers produces a
> trhee-digit
> number. Next, assume that you for some reason are only allowed to write
> 
> down two digits in the answer, and you see the dillema: How can one
> accurately represent a three-digit number when one only is allowed to
> write down two of the digits.
> 
> This is the dillema in the IIR filter. The IIR filter consists of
> several
> multiplications of N-bit numbers. If you start out with an N-bit
> input x0 and performs M+1 multiplications with N-bit numbers am, like
> 
> x1 = a0*x0
> x2 = a1*x1
> :
> xM+1= aM*xM
> 
> then the result xM+1 will, in the extreme case, contain something
> like M*(N-1)+N bits.
> 
> So you are left with a choise as how to deal with this: Either get a
> numerically accurate result, obtained at the cost of increasing the
> number of bits in the bus after each multiplication, or keep a fixed
> number of bits all through and accept the numerical error creeping in.
>  
> Rune
> 
Of course, for many applications those are not the only 2 options. You 
could noise shape as you go, getting a lot of the benefit of retaining 
the extra bits while actually throwing them away.

Steve

>
> I'll just throw in an example in a more "everyday" context in order to
> illustrate the concept of fixed-width integer arithmetics.
>
> Suppose you want to multiply the numbers
>
> A = 10
> B = 20
>
> C = A*B = 200.
>
> As you can see, multiplying two two-digit numbers produces a
> trhee-digit
> number. Next, assume that you for some reason are only allowed to write
>
> down two digits in the answer, and you see the dillema: How can one
> accurately represent a three-digit number when one only is allowed to
> write down two of the digits.

Actually, if you were to describe the above situation, as one input one
output situation, (rather than a 2 input system) the dillemma would
vanish:
A is the input (10)
B is a transformation process (*20 or *potato)
C is the output

Thus C is 10 chunks of 20. Or 10 potatos. The output resolution is the
same as the input resolution.


> This is the dillema in the IIR filter. The IIR filter consists of
> several
> multiplications of N-bit numbers. If you start out with an N-bit
> input x0 and performs M+1 multiplications with N-bit numbers am, like
>
> x1 = a0*x0
> x2 = a1*x1
> :
> xM+1= aM*xM
>
> then the result xM+1 will, in the extreme case, contain something
> like M*(N-1)+N bits.
>
> So you are left with a choise as how to deal with this: Either get a
> numerically accurate result, obtained at the cost of increasing the
> number of bits in the bus after each multiplication, or keep a fixed
> number of bits all through and accept the numerical error creeping in.

This thougth ocurred to me too. If I do not want to relenquish any of
the precison of the products obtained from the [coeffcients] and the
[delayed-input-and-output-samples], the databus width at the output of
the IIR filter would have to be INFINITY. According, to the direct-form
I diagram of the IIR filter, if the precision of the output is not
trimmed down when the output is delayed and [multiplied and summed
back], then the width of the output bus will perputually keep on
increasing. Are there guidelines concerning the databus width at
different points in the IIR filter?

-Roger



> Rune