## Round-Off Error Variance

This appendix shows how to derive that the noise power of amplitude quantization error is , where is the quantization step size. This is an example of a topic in*statistical signal processing*, which is beyond the scope of this book. (Some good textbooks in this area include [27,51,34,33,65,32].) However, since the main result is so useful in practice, it is derived below anyway, with needed definitions given along the way. The interested reader is encouraged to explore one or more of the above-cited references in statistical signal processing.

^{G.10}

Each round-off error in quantization noise is modeled as a uniform

*random variable*between and . It therefore has the following

*probability density function*(pdf) [51]:

^{G.11}

*probability*that a given roundoff error lies in the interval is given by

*probability distribution*, but technically it is a probability

*density*function, and to obtain probabilities, we have to integrate over one or more intervals, as above. We use probability

*distributions*for variables which take on

*discrete*values (such as dice), and we use probability

*densities*for variables which take on

*continuous*values (such as round-off errors). The

*mean*of a random variable is defined as

*rounding*(as opposed to truncation, etc.). The

*mean of a signal*is the same thing as the

*expected value*of , which we write as . In general, the expected value of

*any*function of a random variable is given by

*estimate*the mean by

*averaging*the signal over time. Such an estimate is called a

*sample mean*. Probability distributions are often characterized by their

*moments*. The th moment of the pdf is defined as

*first*moment of the pdf. The second moment is simply the expected value of the random variable squared,

*i.e.*, . The

*variance*of a random variable is defined as the

*second central moment*of the pdf:

*mean*, that is, looking at instead of . In the case of round-off errors, the mean is zero, so subtracting out the mean has no effect. Plugging in the constant pdf for our random variable which we assume is uniformly distributed on , we obtain the variance

*mean square*. Such an estimate is called the

*sample variance*. For sampled physical processes, the sample variance is proportional to the

*average power*in the signal. Finally, the square root of the sample variance (the

*rms level*) is sometimes called the

*standard deviation*of the signal, but this term is only precise when the random variable has a Gaussian pdf.

**Next Section:**

Matrix Multiplication

**Previous Section:**

Logarithmic Number Systems for Audio