Am 04.11.2012 14:29, schrieb robert bristow-johnson:
> On 11/4/12 5:16 AM, SG wrote:
> ...
>> But we were talking about noise shaping.  My point was that if you want
>> to shape the noise (if there is any!) you better use _at_least_ a 1LSB
>> rectangular dither so that you know the "original" shape that the noise
>> shaper will alter.
> 
> but you don't.

Sure I do.  We've been over this.  I tried to explain myself a couple of
times already and I'm starting to lose interest in furthering this
discussion...

> you don't have a consistent power spectrum 

I know.  The power is however less than the power of the overall error
you'd get with a TPDF dither.  And this even extends to the spectral
power density across all frequencies as far as I can tell -- which is
kind of my point (!).  We have a low upper bound for the power spectral
density in this case.  This does not conflict with anything you said and
I fail to see why you have to bring up the "the power depends on the
signal and may drop to zero" bit.  I fail to see the relevance.  Yes,
the power depends on the signal.  In case it's zero it's zero. In case
it's greater than zero, it as a flat shape.

> these birdies or limit cycles are a concern in audio primarily when the
> signal level fades down to very close to zero.  in the order of 1 LSB.
> and the magnitude of the birdie is in the same ballpark.  (sorry to use
> an American colloquialism.)

Of course, you can rely on luck or say "well, it works most of the
time".  I guess that's what the DSD proponents still do... ;-)

Cheers!
SG

On 11/4/2012 8:29 AM, robert bristow-johnson wrote:
> On 11/4/12 5:16 AM, SG wrote:
> ...
>> But we were talking about noise shaping. My point was that if you want
>> to shape the noise (if there is any!) you better use _at_least_ a 1LSB
>> rectangular dither so that you know the "original" shape that the noise
>> shaper will alter.
>
> but you don't. you don't have a consistent power spectrum until you at
> least have consistent power. and you don't have consistent power until
> you decouple both the first and second moments. and to decouple both the
> first and second moments, you need triangular p.d.f. dither. and it need
> not be white.

I have copied this paragraph to my notes, although I think it is 
permanently burned into my brain at this point.  Repetition is a primary 
learning tool.  Thanks.

>> Doing so will give you a very low upper limit of the
>> spectral power density instead of the possibility of huge peaks
>> (birdies) due to limit cycles, for example.
>
> these birdies or limit cycles are a concern in audio primarily when the
> signal level fades down to very close to zero. in the order of 1 LSB.
> and the magnitude of the birdie is in the same ballpark. (sorry to use
> an American colloquialism.)

I think this is something I need to learn more about.  I expect my 
signal will be down in the noise, possibly in the quantization noise, 
don't know for sure yet.  It is very narrow band, in the ball park of 1 
Hz bandwidth.  So I will take advantage of that to pull it out of the 
noise.  I'm concerned that these "birdies" may cause problems with this 
idea.

Rick

On 11/4/12 5:16 AM, SG wrote:
...
> But we were talking about noise shaping.  My point was that if you want
> to shape the noise (if there is any!) you better use _at_least_ a 1LSB
> rectangular dither so that you know the "original" shape that the noise
> shaper will alter.

but you don't.  you don't have a consistent power spectrum until you at 
least have consistent power.  and you don't have consistent power until 
you decouple both the first and second moments.  and to decouple both 
the first and second moments, you need triangular p.d.f. dither.  and it 
need not be white.

>  Doing so will give you a very low upper limit of the
> spectral power density instead of the possibility of huge peaks
> (birdies) due to limit cycles, for example.

these birdies or limit cycles are a concern in audio primarily when the 
signal level fades down to very close to zero.  in the order of 1 LSB. 
and the magnitude of the birdie is in the same ballpark.  (sorry to use 
an American colloquialism.)

>
> What I initially responded to was:
>
>> some kind of noise shaping is feasible in pretty much any case.  it
>> doesn't require dither to do what it does.
>
> True, it doesn't require dither.  But it makes only little sense without
> dither since you don't really know what kind of "noise" it is the noise
> shaper will filter.

it makes more than little sense in an algorithm with many internal 
nodes, each of which have quantization.  e.g. a 24-bit processor (like 
the good ol' Mot or Freescale DSP56xxx) that eventually outputs to a 16 
or 18-bit DAC.  then the signal "swing" is 48 dB larger than the 
quantization level.  in that case, the textbook treatment is close to 
correct and the simple rounding quantization is going to be quite white.

simple undithered noise shaping with a zero at z=1 (i really like the 
label "fraction saving" for this kind of dither) is quite useful (even 
if only to kill the DC limit cycle) at the 24-bit level and much cheaper 
computationally than full dithered quantization.  with the 56K, i have 
done this fraction saving with two additional instructions.  generating 
a good uniform p.d.f. dither requires a lot more than that, and i 
wouldn't spend the effort to do that for *every* node where there is 
quantization.

what makes little sense to me is going through the bother of dithering 
with RPDF and not bothering to do it right with TPDF.  especially if the 
node is *not* noise shaped, then high-pass TPDF dither requires only one 
new uniform p.d.f. random number per sample because you pass it through 
a simple differentiator to get the TPDF dither.

>  With a RPDF dither I know enough about this noise
> that I can rule out birdies.

i can too if the birdies are 48 dB below a later quantization level that 
is fully dithered (and possibly noise shaped).  those birdies will be 
swamped by the big quantization which really counts.

-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

Am 01.11.2012 18:02, schrieb robert bristow-johnson:
> On 10/31/12 3:13 PM, SG wrote:
>> Am 31.10.2012 18:44, schrieb robert bristow-johnson:
>>> [...]
>>> so when S is midway between quantization levels and you add zero-mean
>>> rectangular dither, what is the expected PSD?  then when S, the sample
>>> about to be quantized, is dead on a quantization level, what is the PSD?
>>> are they the same?
>>
>> They have the same shape: flat.
> 
> no.  it's wrong.

Well, it's constant zero or "uniformly zero". But I agree that the word
"shape" is not appropriate in this corner case.

> [...]
> here is what's annoying:  as you slowly sweep a virtual DC level (a
> *very* slow ramp) for S through adjacent post-quantization values,
> assuming the DC component of the 1 LSB rectangular dither is zero, then
> when S lands directly on a quantized level, even after adding the
> dither, the quantizer always returns the output (S+X) back to that
> level.  the 1 LSB rectangular dither is not big enough to get the output
> to quantize to another level.  the output, S+X, will sound pretty quiet.
> 
> now as you slowly sweep the level of S from dead on to halfway between
> two adjacent quantization levels, the *mean* of S+U will follow the mean
> of S which is just S.  1 LSB rectangular dither is sufficient to do
> that, to get rid of the mean error from the quantized output.  but the
> quantized output will start toggling between the lower level and the
> higher level and that is variance.  the apparent *power* of the
> quantization error increases as S goes from floor(S) to floor(S)+1/2 and
> will peak when it's midway between floor(S) and floor(S+1).
> 
> that's the point i've been trying to make.

I know.  This is not news to me.

But we were talking about noise shaping.  My point was that if you want
to shape the noise (if there is any!) you better use _at_least_ a 1LSB
rectangular dither so that you know the "original" shape that the noise
shaper will alter.  Doing so will give you a very low upper limit of the
spectral power density instead of the possibility of huge peaks
(birdies) due to limit cycles, for example.

What I initially responded to was:

> some kind of noise shaping is feasible in pretty much any case.  it
> doesn't require dither to do what it does.

True, it doesn't require dither.  But it makes only little sense without
dither since you don't really know what kind of "noise" it is the noise
shaper will filter.  With a RPDF dither I know enough about this noise
that I can rule out birdies.

Cheers!
SG

On 10/31/2012 4:37 PM, Eric Jacobsen wrote:
> On Wed, 31 Oct 2012 16:09:11 -0400, rickman<gnuarm@gmail.com>  wrote:
>
>> On 10/30/2012 3:52 AM, SG wrote:
>>> Am 29.10.2012 21:43, schrieb rickman:
>>>> On 10/29/2012 4:06 PM, SG wrote:
>>>>> Am 29.10.2012 19:29, schrieb rickman:
>>>>>> I had just come to this conclusion as I read the Wikipedia article on
>>>>>> delta sigma converters.  The circuit they showed didn't look right,
>>>>>> because it isn't clocked!  Once I understood it was really a voltage to
>>>>>> frequency converter, it made perfect sense.  They then use a counter as
>>>>>> the digital filter to accumulate pulses into samples.
>>>>>
>>>>> That's not exactly how it works. Lookup "noise shaping".
>>>>
>>>> Yes, I know.  But I like to have an understanding of the mechanics, then
>>>> it is easier to understand the "magic".
>>>
>>> Hmmm, that sounds like you want to understand noise shaping without
>>> having to understand noise shaping.
>>
>> No, I'd like to understand noise shaping, but I don't think I'll ever
>> understand "magic".
>>
>> I'm starting to get it.  The noise shaping comes as a result of the
>> quantization noise being "whitened" by the dithering, then filtered by
>> an "inverse" filter before being fed back.  The problem is I can't use
>> this model in the circuit I can build.  I created a separate thread
>> about this and maybe I can get a little insight to the circuit that
>> Lattice provides which has the same limitations I am stuck with.
>>
>> I put whitened in quotes because I understand that optimally it should
>> have a triangular distribution.  Some others in this thread seem to have
>> some issues about this and I just want to make clear that I understand,
>> at least I think I do...  It's still sinking in.
>
> "White" has to do with the spectral density.   "Triangular" has to do
> with the statistical pdf of the amplitude.   They are different.
> "White" noise can have arbitrary pdf.  Noise with a triangular pdf may
> not be white.
>
>
> Eric Jacobsen
> Anchor Hill Communications
> http://www.anchorhill.com

Thanks.

Rick

On 10/31/12 3:13 PM, SG wrote:
> Am 31.10.2012 18:44, schrieb robert bristow-johnson:
>> [...]
>> so when S is midway between quantization levels and you add zero-mean
>> rectangular dither, what is the expected PSD?  then when S, the sample
>> about to be quantized, is dead on a quantization level, what is the PSD?
>> are they the same?
>
> They have the same shape: flat.

no.  it's wrong.

> Like I said a couple of times. I know
> what point you are trying to make here.

it has to do with them first and second moments.

> It's just not relevant to the
> discussion because I claimed the PSD to be flat.

and zero doesn't have a shape.  not really.

or put another way, zero could have any shape (before that shape gets 
multiplied by zero).

> I did not say anything
> about its offset being independent from S. I still maintain that the PSD
> is predictable.

"constant and predictable" is better than non-constant and predictable.

> It has a low upper bound and it it has a flat shape (no
> annoying spectral peaks).

here is what's annoying:  as you slowly sweep a virtual DC level (a 
*very* slow ramp) for S through adjacent post-quantization values, 
assuming the DC component of the 1 LSB rectangular dither is zero, then 
when S lands directly on a quantized level, even after adding the 
dither, the quantizer always returns the output (S+X) back to that 
level.  the 1 LSB rectangular dither is not big enough to get the output 
to quantize to another level.  the output, S+X, will sound pretty quiet.

now as you slowly sweep the level of S from dead on to halfway between 
two adjacent quantization levels, the *mean* of S+U will follow the mean 
of S which is just S.  1 LSB rectangular dither is sufficient to do 
that, to get rid of the mean error from the quantized output.  but the 
quantized output will start toggling between the lower level and the 
higher level and that is variance.  the apparent *power* of the 
quantization error increases as S goes from floor(S) to floor(S)+1/2 and 
will peak when it's midway between floor(S) and floor(S+1).

that's the point i've been trying to make.

-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

Am 31.10.2012 18:44, schrieb robert bristow-johnson:
> [...]
> so when S is midway between quantization levels and you add zero-mean
> rectangular dither, what is the expected PSD?  then when S, the sample
> about to be quantized, is dead on a quantization level, what is the PSD?
> are they the same?

They have the same shape: flat. Like I said a couple of times. I know
what point you are trying to make here. It's just not relevant to the
discussion because I claimed the PSD to be flat. I did not say anything
about its offset being independent from S. I still maintain that the PSD
is predictable. It has a low upper bound and it it has a flat shape (no
annoying spectral peaks).

On Wed, 31 Oct 2012 16:09:11 -0400, rickman <gnuarm@gmail.com> wrote:

>On 10/30/2012 3:52 AM, SG wrote:
>> Am 29.10.2012 21:43, schrieb rickman:
>>> On 10/29/2012 4:06 PM, SG wrote:
>>>> Am 29.10.2012 19:29, schrieb rickman:
>>>>> I had just come to this conclusion as I read the Wikipedia article on
>>>>> delta sigma converters.  The circuit they showed didn't look right,
>>>>> because it isn't clocked!  Once I understood it was really a voltage to
>>>>> frequency converter, it made perfect sense.  They then use a counter as
>>>>> the digital filter to accumulate pulses into samples.
>>>>
>>>> That's not exactly how it works. Lookup "noise shaping".
>>>
>>> Yes, I know.  But I like to have an understanding of the mechanics, then
>>> it is easier to understand the "magic".
>>
>> Hmmm, that sounds like you want to understand noise shaping without
>> having to understand noise shaping.
>
>No, I'd like to understand noise shaping, but I don't think I'll ever 
>understand "magic".
>
>I'm starting to get it.  The noise shaping comes as a result of the 
>quantization noise being "whitened" by the dithering, then filtered by 
>an "inverse" filter before being fed back.  The problem is I can't use 
>this model in the circuit I can build.  I created a separate thread 
>about this and maybe I can get a little insight to the circuit that 
>Lattice provides which has the same limitations I am stuck with.
>
>I put whitened in quotes because I understand that optimally it should 
>have a triangular distribution.  Some others in this thread seem to have 
>some issues about this and I just want to make clear that I understand, 
>at least I think I do...  It's still sinking in.

"White" has to do with the spectral density.   "Triangular" has to do
with the statistical pdf of the amplitude.   They are different.
"White" noise can have arbitrary pdf.  Noise with a triangular pdf may
not be white.


Eric Jacobsen
Anchor Hill Communications
http://www.anchorhill.com

On 10/30/2012 3:52 AM, SG wrote:
> Am 29.10.2012 21:43, schrieb rickman:
>> On 10/29/2012 4:06 PM, SG wrote:
>>> Am 29.10.2012 19:29, schrieb rickman:
>>>> I had just come to this conclusion as I read the Wikipedia article on
>>>> delta sigma converters.  The circuit they showed didn't look right,
>>>> because it isn't clocked!  Once I understood it was really a voltage to
>>>> frequency converter, it made perfect sense.  They then use a counter as
>>>> the digital filter to accumulate pulses into samples.
>>>
>>> That's not exactly how it works. Lookup "noise shaping".
>>
>> Yes, I know.  But I like to have an understanding of the mechanics, then
>> it is easier to understand the "magic".
>
> Hmmm, that sounds like you want to understand noise shaping without
> having to understand noise shaping.

No, I'd like to understand noise shaping, but I don't think I'll ever 
understand "magic".

I'm starting to get it.  The noise shaping comes as a result of the 
quantization noise being "whitened" by the dithering, then filtered by 
an "inverse" filter before being fed back.  The problem is I can't use 
this model in the circuit I can build.  I created a separate thread 
about this and maybe I can get a little insight to the circuit that 
Lattice provides which has the same limitations I am stuck with.

I put whitened in quotes because I understand that optimally it should 
have a triangular distribution.  Some others in this thread seem to have 
some issues about this and I just want to make clear that I understand, 
at least I think I do...  It's still sinking in.

>>> See http://wiki.hydrogenaudio.org/index.php?title=Noise_shaping
>>> See http://sjeng.org/ftp/SACD.pdf
>>> See https://code.google.com/p/audio-pwm/wiki/NoiseShaping
>>
>> Thanks for the links.  The wiki link seems to be useless to me at the
>> moment.  The SACD link might actually help me a bit.
>
> The wiki link includes some real pictures that show the effect of noise
> shaping.  That's why I brought it up.

Now that it is starting to gel the explanation in the hydrogenaudio wiki 
is what helped it make sense last night.  I had to really work at it to 
follow their logic.  I don't see any images in it other than one circuit 
diagram.  A few images would have helped me a lot!  I like to anchor the 
math to reality when I can.

I understand what noise shaping *is*.  The spectrograms were pretty much 
what I expected.  I had trouble understanding how the noise shaping 
happens.  But thanks for the help.  I think I've got it now.

>>>> Wikipedia then shows a clocked conversion also with a counter.  The part
>>>> I don't get is how a train of pulses can represent the full signal.  For
>>>
>>> Through the magic of noise shaping....snip...

>> Yes, but that doesn't answer the question really.
>
> I think it does.

Anytime you answer a question with "magic" I think it didn't really 
answer anything.  :-(

>> I guess I'd like to
>> see the bit stream that results from a full scale sine wave at 1/2 the
>> output sample rate, or as close to that as the filter will allow.
>
> There is not much to see, I think.  The sine wave is practically just
> superimposed with lots and lots of noise.  The trick is to keep this
> noise out of the audible band.  And that's where noise shaping comes in.

But you have to get the output to full scale with a minimal number of 
equally weighted bits.  That is the part I don't get.

>> I understand that noise shaping would help, but what is shaping the
>> noise?  In the SACD paper they show a filter in the feedback to the
>> input quantizer.  I don't see that in real world designs.  Or is that
>> what the RC filter in the LVDS input design is doing?  Does this filter
>> need to be a low pass in order to shape the noise to the high end?
>
> I don't know exactly how this is done in hardware.  But there has to be
> some kind of error feedback loop.  Isn't that where sigma-delta gets is
> name from?  The error is computed (delta) and feed back to an integrator
> (sigma).  This would be equivalent to a noise shaper with the following
> impulse response: [1 -1].  One can improve upon this with a 2nd order
> sigma-delta (two integrators) which should be equivalent to a noise
> shaper with the following impulse response: [1 -2 1].  But I think the
> noise would still be perceivable if you run this at 48000*128 Hz.  More
> sophisticated noise shapers are needed to keep the audible band "clean".

Ok, the hardware is the part I'm working with and so want to understand. 
  I'm building the hardware part in an FPGA.

Thanks again for your help.

Rick