Dither audio signal to 24 bits

"Jerry Avins" <jya@ieee.org> wrote in message
news:40eb168d$0$3088$61fed72c@news.rcn.com...
> Jon Harris wrote:
>
> > The short answer is that triangular "sounds better".  I've read technical
> > explanations about this but don't understand them enough to explain it.  I
think
> > that triangular allows creating a "perfect" analog-like constant noise floor
> > with no noise modulation with the minimum added noise.  Rectangular PDF
noise
> > can be used in a pinch since it is cheap to create.  My understanding is
that
> > you need less rectangular noise than you do triangular, but the resulting
noise
> > floor has some input-dependent modulations, i.e. is not constant.
> >
> > Since dithering trades off distortion for noise, various intermediate
tradeoffs
> > are possible, such as reducing but not eliminating the distortion and adding
a
> > bit less noise.  But most of the research revolves around optimal solutions,
> > i.e. completely eliminating the distortion.
> >
> > <scorey_plonk@waterford.org> wrote in message
> > news:MPG.1b546e061fe02380989687@news.xmission.com...
> >
> Triangular is only twice as hard as rectangular, plus an addition.

Agreed, but sometimes every cycle counts.  As mentioned above, it also allows
less additive noise, at the expense of more distortion.  It may be a useful
option in certain circumstances, though by far I think triangular is and should
be used in most cases.

"Jon Harris" <goldentully@hotmail.com> wrote in message
news:2l0fnpF766stU1@uni-berlin.de...
> The short answer is that triangular "sounds better".  I've read technical
> explanations about this but don't understand them enough to explain it.

The technical reason is so simple that people feel a need to make it sound
complicated when they write about it. Here's the two-line version:

Dithering with a TPDF before quantization makes the quantization error have
0 mean, and variance independent of the input signal.   Whether the signal
is very close to a quantization level, or halfway between two levels, the
mean and variance of the quantization error are the same, and the
quantization error is white, like good ol' analog hiss.

With most other dithering techniques, the quantization noise is modulated by
the signal and can become concentrated at particular frequencies, making it
more audible and annoying.

--
Matt

Jon Harris wrote:

   ...

> Agreed, but sometimes every cycle counts.  As mentioned above, it also allows
> less additive noise, at the expense of more distortion.  It may be a useful
> option in certain circumstances, though by far I think triangular is and should
> be used in most cases.

Agreed, all around.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

"Matt Timmermans" <mt0000@sympatico.nospam-remove.ca> wrote in message
news:IRIGc.23185$WM5.1056616@news20.bellglobal.com...
>
> "Jon Harris" <goldentully@hotmail.com> wrote in message
> news:2l0fnpF766stU1@uni-berlin.de...
> > The short answer is that triangular "sounds better".  I've read technical
> > explanations about this but don't understand them enough to explain it.
>
> The technical reason is so simple that people feel a need to make it sound
> complicated when they write about it. Here's the two-line version:
>
> Dithering with a TPDF before quantization makes the quantization error have
> 0 mean, and variance independent of the input signal.   Whether the signal
> is very close to a quantization level, or halfway between two levels, the
> mean and variance of the quantization error are the same, and the
> quantization error is white, like good ol' analog hiss.

Thanks for the succint answer, Matt.

> With most other dithering techniques, the quantization noise is modulated by
> the signal and can become concentrated at particular frequencies, making it
> more audible and annoying.

So, what you are saying is that triangular sounds better!  ;-)

Surely "usable" is the arguable issue here? You can't really get all 
that quantization at the same time. My comparison was  between 24bit int 
and normalized floats (+-1.0 range); to which you can add 24bit 
fractional (fixed point).

The mantissa is 24bits, so that it can as you say represent a maximum of 
2^^24 numbers.  How can you then add more numbers to that, if it is 
already the maximum?

With 24bit mantissa, you can either use:

xxxxxxxxxxxxxxxxxxxxxxxx.0  (quasi 24bit int)

or
0.xxxxxxxxxxxxxxxxxxxxxxxx  (quasi fractional/fixed)

or

xxxxxxxxxxxx.yyyyyyyyyyyy  (Cool Edit Pro!)

But you can't use:

xxxxxxxxxxxxxxxxxxxxxxxx.yyyyyyyyyyyyyyyyyyyyyyyy

Adding a large numer to a small number (i.e. involving a shift of the 
mantissa) is a good way to get bad errors; not what I would call "fine 
quantization".

Richard Dobson




Andor wrote:
> Richard Dobson wrote:
> 
>>The standard range for 32bit float audio is +- 1.0, which makes full use 
>>of the 24bit mantissa (which then gives an effective 25bits becasue of 
>>the hidden bit); hence the very close relationship between 24bit int and 
>>32bit float. SO I would say the quantization is about the same
> 
> 
> I heartily disagree. A 24bit two's complement can represent 2^24
> numbers. A 32bit floating-point can represent the same range of
> integers, ie. [-2^23, 2^23-1] plus more than 24 * 2^22 usable
> fractional numbers in between. This is definitely finer quantisation!
> 
...

Richard Dobson wrote:

> Surely "usable" is the arguable issue here? You can't really get all 
> that quantization at the same time. My comparison was  between 24bit int 
> and normalized floats (+-1.0 range); to which you can add 24bit 
> fractional (fixed point).

You can when you're multiplying or dividing.

> The mantissa is 24bits, so that it can as you say represent a maximum of 
> 2^^24 numbers.  How can you then add more numbers to that, if it is 
> already the maximum?

25 bits. The floating-point format includes an implied leading 1.

Of course, I may be missing the point entirely.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

Unlikely for you, very likely for me. Nice pun,  BTW!

I had mentioned the 25bit aspect earlier, maybe I need to be more 
rigorous in using that number.

My proposal was:

"
 > The standard range for 32bit float audio is +- 1.0, which makes full 
use of the 24bit mantissa (which then gives an effective 25bits becasue 
of the hidden bit); hence the very close relationship between 24bit int 
and 32bit float. So I would say the quantization is about the same
"

So I had the 25 bits in there.

The post from Andor was:

"
I heartily disagree. A 24bit two's complement can represent 2^24
numbers. A 32bit floating-point can represent the same range of
integers, ie. [-2^23, 2^23-1] plus more than 24 * 2^22 usable
fractional numbers in between. This is definitely finer quantisation!
"

Now, taking the 25 bit matter entirely on board, by my calculations 
2^^25 = 33554432 possible values that a 32bit float can have.

Andor's calculation is:

   2^24 + "more than" 24 * 2^22

Leaving out the "more than", my calculator gives this as 117440512, 
which seem to be 3.5 times as big. So the question is, is this actually 
the number of numbers a 32bit float can represent? I suggested the 
quantization between 24bit int and 32bit float (understood as effective 
25bit mantissa) was "about the same". OK, lazy language. I can't claim 
that a factor of 3.5 is "about the same". It's undeniably "finer 
quantization", but is it right? Can we replace "more than" with 
something exact?  I can't believe this number hasn't been written down 
somehere; after all, a 32bit float is deterministic (excl denormals), 
and must have a findable maximum number of numbers. So what is it?




Richard Dobson

Jerry Avins wrote:

> Richard Dobson wrote:
> 
>> Surely "usable" is the arguable issue here? You can't really get all 
>> that quantization at the same time. My comparison was  between 24bit 
>> int and normalized floats (+-1.0 range); to which you can add 24bit 
>> fractional (fixed point).
> 
> 
> You can when you're multiplying or dividing.
> 
>> The mantissa is 24bits, so that it can as you say represent a maximum 
>> of 2^^24 numbers.  How can you then add more numbers to that, if it is 
>> already the maximum?
> 
> 
> 25 bits. The floating-point format includes an implied leading 1.
> 
> Of course, I may be missing the point entirely.
> 
> Jerry

Richard Dobson wrote:

> Unlikely for you, very likely for me. Nice pun,  BTW!

Don't give me too much credit. Thanks, though.

> I had mentioned the 25bit aspect earlier, maybe I need to be more 
> rigorous in using that number.

I missed that. Anyhow, it's not a big deal.

> My proposal was:
> 
> "
>  > The standard range for 32bit float audio is +- 1.0, which makes full 
> use of the 24bit mantissa (which then gives an effective 25bits becasue 
> of the hidden bit); hence the very close relationship between 24bit int 
> and 32bit float. So I would say the quantization is about the same
> "
> 
> So I had the 25 bits in there.
> 
> The post from Andor was:
> 
> "
> I heartily disagree. A 24bit two's complement can represent 2^24
> numbers. A 32bit floating-point can represent the same range of
> integers, ie. [-2^23, 2^23-1] plus more than 24 * 2^22 usable
> fractional numbers in between. This is definitely finer quantisation!
> "

Give Andor some credit. Floating point and fixed point have about the
same resolution when it comes to counting bits, but floats allow the
significant bits to be positioned after the fact where you need them
most.

To work with fixed point, I have to be careful not to add small numbers
to large ones repeatedly without accumulating the truncated remainders
and adding their sum back. Floating point does that for me
automatically. The danger of fixed-point overflow forces me to use some
of those 24 bits for guard bits. With floating point, they're available
for added precision.

Paying close attention to the techniques that make fixed-point
calculations succeed make floating-point calculations more precise in
the end. Still with floating point, I can apply a final multiplier to
make my results fill the fixed-point output word size with considerably
less internal scaling gymnastics. Using floating point as a replacement
for careful algorithm design is a bad idea. There are ways to use it
wisely that I'm just beginning to appreciate.

   ...

> Jerry Avins wrote:

   ...

>> Of course, I may be missing the point entirely.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

"Jon Harris" <goldentully@hotmail.com> wrote in message
news:2l2pqdF88nvsU1@uni-berlin.de...
>
> So, what you are saying is that triangular sounds better!  ;-)
>

Precisely, but it doesn't just sound better in audio -- it also sounds
better in print!  That's where the money is, you know ;-)

Richard Dobson wrote:

> My proposal was:
> 
> "
>  > The standard range for 32bit float audio is +- 1.0, which makes full 
> use of the 24bit mantissa (which then gives an effective 25bits becasue 
> of the hidden bit); hence the very close relationship between 24bit int 
> and 32bit float. So I would say the quantization is about the same
> "
> 
> So I had the 25 bits in there.
> 
> The post from Andor was:
> 
> "
> I heartily disagree. A 24bit two's complement can represent 2^24
> numbers. A 32bit floating-point can represent the same range of
> integers, ie. [-2^23, 2^23-1] plus more than 24 * 2^22 usable
> fractional numbers in between. This is definitely finer quantisation!
> "
> 
> Now, taking the 25 bit matter entirely on board, by my calculations 
> 2^^25 = 33554432 possible values that a 32bit float can have.
> 
> Andor's calculation is:
> 
>    2^24 + "more than" 24 * 2^22
> 
> Leaving out the "more than", my calculator gives this as 117440512, 
> which seem to be 3.5 times as big. So the question is, is this actually 
> the number of numbers a 32bit float can represent? 

The "more than" came from me being too lazy to look up the IEEE
754/854 specs, so I just took a trivial lower bound.

But we can work it more accurately:
Let's assume our 24bit audio sample is a signed integer in the range
[-2^-23, 2^23 -1]. I convert this to floating-point (without
normalizing to +- 1.0, but it doesn't really matter, as you'll see
below), and get the range [-8388608.0 , 8388607.0]. The floating-point
numbers representing the two bounds have to have an exponent of 22
because

8388607.0 = Sum[2^k, {k,0,22}] = 2^22 Sum[2^-k, {k,0,22}]

That last term is the normlized floating-point representation, with
all mantissa bits = 1 except the 25th, and the exponent = 22. Looking
at only the positive values for the moment, all numbers in between
2^22 = 4194304.0 and 2^23 - 1 = 8388607.0 have this exponent. There
are 2^23 of them. Similarly, all numbers between 2^21 and 2^22 - 1
have an exponent of 21. There are also 2^23 of them. etc.

You can reduce the exponent all the way to -126, so you have 149
exponent steps for a positive number range of [2^-126, 2^23-1]. Each
exponent step has 2^23 possible mantissa values, so we get

149 * 2^23 = 1 241 513 984

valid positive 32bit floating-point numbers (149 more than 24bit
signed integer can represent). Multiply by two for negative numbers. I
was using only 24 steps in my estimate to reflect the range [1.0,
2^23-1].

When you normalize the floating-point number to [-1.0, 1.0[, you just
loose 22 exponent steps, but you still have way more (127 to be exact)
numbers than in 24bit signed integer format.

However, this is just the quantisation issue. Arithmetic is a
different story.

Regards,
Andor