Reply by robert bristow-johnson●May 27, 20142014-05-27
On 5/27/14 2:54 PM, Tim Wescott wrote:
> On Tue, 27 May 2014 13:55:09 -0400, robert bristow-johnson wrote:
>
>> On 5/27/14 1:20 PM, Tim Wescott wrote:
>>> On Mon, 26 May 2014 23:03:53 -0700, gyansorova wrote:
>>>
>>>
>>>> I seem to remember them saying that if you sample too fast the poles
>>>> can drift outside the unit circle. That sound like rubbish to me
>>>> unless the original system is open-loop unstable in the first place.
>>>
>>> Even today, you still have to be mindful of the cost of high-rate
>>> sampling, and some sensors (or drivers, if you're using PWM) choose the
>>> sampling interval for you. So choosing a sampling interval, while not
>>> as critical a tradeoff between cost (or straight feasibility) and
>>> performance as it used to be, is still a task that must be undertaken.
>>>
>>> As RBJ mentioned, the higher your sampling rate, the closer all the
>>> poles get to 1,
>>
>> i think gyansorova mentioned that...
>>
>>> ... and the more precision you need to distinguish them from 1.
>>> That means that as your sampling rate goes up so does your required
>>> word width. You can play tricks to alleviate the problem, but in the
>>> end you still often need to go to wider data paths.
>>
>> maybe. if you replace all the cos(w0) with 1 - 2*(sin(w0/2))^2 and
>> you're in single-precision floating point, and if you're careful to do
>> the arithmetic with the smaller numbers first (and later start adding in
>> the bigger numbers), you may very well not need to go to wider words.
>>
>> this cos(w0) issue shows up in audio when we make a decent DC-blocking
>> filter or other filters tuned to around 20 Hz or lower and it shows up
>> when you are trying to plot frequency response from the coefficients and
>> the frequencies are very low. just remember that all of the frequency
>> information lies in the *difference* cos(w0) is from 1.
>
> You can play tricks like that and stretch the usability of single-
> precision math to some extent. But ultimately, even if you solve the
> problem of distinguishing your coefficients from 1, you still need to
> have wide enough words in your data so that the distinction is not lost.
yes, i said "maybe". but to caveat your caveat, if the "trick" avoids
the problem of subtracting two numbers that are nearly equal, i.e. forms
an addition series that looks like
c0 + c1*x + c2*x^2 + ...
and that tiny difference between the two numbers you would otherwise be
subtracting is x, then the "trick" rearranged your numerical problem
into something that works with floating-point. and, since |x| is much
smaller than 1, you start adding with the higher powers first. you can
do that without making *all* of the words wider, unless it turns out
that c1 and c2 are closely negatives of each other. then you do the
factoring thing (i think it's called "Horner's rule"):
c0 + x*(c1 + x*(c2 + ... ) )
all's i am saying with this "trick" is to identify what gets tiny when
frequency is much much less than Nyquist. eventually, tiny numbers get
added to bigger numbers. but if you can identify where all the tiny
arithmetic is happening, then do that first and *then* add it to the
bigger numbers. follow these principles and it might be surprizing how
much you can get away with single-precision floats.
a similar numerical "trick" (i just think it's numerical prudence) for
FIR filters done in floating-point is to do the outer taps first since
they will have the smallest coefficients. if it's linear-phase, the
last tap you do is the center tap and if it's minimum-phase, the last
tap you add in is the zeroth tap.
Duane Wise did a paper a couple decades ago where he did a similar thing
for fixed-point FIRs, but there was right-shifting of the sum of
products after each group of taps. it helped him nail the frequency
response with 16-bit "fixed"-point coefs in a way that you could not
with a simple fixed-point FIR.
--
r b-j rbj@audioimagination.com
"Imagination is more important than knowledge."
Reply by Tim Wescott●May 27, 20142014-05-27
On Tue, 27 May 2014 12:13:33 -0700, gyansorova wrote:
> On Wednesday, May 28, 2014 6:54:43 AM UTC+12, Tim Wescott wrote:
>> On Tue, 27 May 2014 13:55:09 -0400, robert bristow-johnson wrote:
>>
>>
>>
>> > On 5/27/14 1:20 PM, Tim Wescott wrote:
>>
>> >> On Mon, 26 May 2014 23:03:53 -0700, gyansorova wrote:
>>
>>
>> >>
>>
>> >>
>> >>> I seem to remember them saying that if you sample too fast the
>> >>> poles
>>
>> >>> can drift outside the unit circle. That sound like rubbish to me
>>
>> >>> unless the original system is open-loop unstable in the first
>> >>> place.
>>
>>
>> >>
>> >> Even today, you still have to be mindful of the cost of high-rate
>>
>> >> sampling, and some sensors (or drivers, if you're using PWM) choose
>> >> the
>>
>> >> sampling interval for you. So choosing a sampling interval, while
>> >> not
>>
>> >> as critical a tradeoff between cost (or straight feasibility) and
>>
>> >> performance as it used to be, is still a task that must be
>> >> undertaken.
>>
>>
>> >>
>> >> As RBJ mentioned, the higher your sampling rate, the closer all the
>>
>> >> poles get to 1,
>>
>>
>> >
>> > i think gyansorova mentioned that...
>>
>>
>> >
>> >> ... and the more precision you need to distinguish them from 1.
>>
>> >> That means that as your sampling rate goes up so does your required
>>
>> >> word width. You can play tricks to alleviate the problem, but in
>> >> the
>>
>> >> end you still often need to go to wider data paths.
>>
>>
>> >
>> > maybe. if you replace all the cos(w0) with 1 - 2*(sin(w0/2))^2 and
>>
>> > you're in single-precision floating point, and if you're careful to
>> > do
>>
>> > the arithmetic with the smaller numbers first (and later start adding
>> > in
>>
>> > the bigger numbers), you may very well not need to go to wider words.
>>
>>
>> >
>> > this cos(w0) issue shows up in audio when we make a decent
>> > DC-blocking
>>
>> > filter or other filters tuned to around 20 Hz or lower and it shows
>> > up
>>
>> > when you are trying to plot frequency response from the coefficients
>> > and
>>
>> > the frequencies are very low. just remember that all of the
>> > frequency
>>
>> > information lies in the *difference* cos(w0) is from 1.
>>
>>
>>
>> You can play tricks like that and stretch the usability of single-
>>
>> precision math to some extent. But ultimately, even if you solve the
>>
>> problem of distinguishing your coefficients from 1, you still need to
>>
>> have wide enough words in your data so that the distinction is not
>> lost.
>>
>>
>>
>> --
>>
>>
>>
>> Tim Wescott
>>
>> Wescott Design Services
>>
>> http://www.wescottdesign.com
>
> Here is some info from the net
>
> 10.6 CHOICE OF SAMPLING INTERVAL
>
> Whenever a digital control system is designed, a suitable sampling
> interval must be chosen. Choosing a large sampling time has
> destabilizing effects on the system. In addition, information loss
> occurs when large sampling times are selected. Also, the errors that
> occur when a continuous system is discretized increase as the sampling
> interval increases.
Ah. Read carefully. They're talking about increasing the sampling
_interval_, meaning the time between samples. So they're talking about
_decreasing_ the sampling rate.
I wouldn't say that choosing a too-large sampling time has destabilizing
effects -- a more accurate way to put it would be that it makes it more
difficult (or impossible) for a controller to achieve all of its goals,
including stability.
Aside from that quibble, what is said there is accurate: you sample more
slowly, it's harder to stabilize, you're throwing away information you
may need, and it's harder to get an accurate plant model at your (slow)
sampling rate.
--
Tim Wescott
Wescott Design Services
http://www.wescottdesign.com
Reply by ●May 27, 20142014-05-27
On Wednesday, May 28, 2014 6:54:43 AM UTC+12, Tim Wescott wrote:
> On Tue, 27 May 2014 13:55:09 -0400, robert bristow-johnson wrote:
>
>
>
> > On 5/27/14 1:20 PM, Tim Wescott wrote:
>
> >> On Mon, 26 May 2014 23:03:53 -0700, gyansorova wrote:
>
> >>
>
> >>
>
> >>> I seem to remember them saying that if you sample too fast the poles
>
> >>> can drift outside the unit circle. That sound like rubbish to me
>
> >>> unless the original system is open-loop unstable in the first place.
>
> >>
>
> >> Even today, you still have to be mindful of the cost of high-rate
>
> >> sampling, and some sensors (or drivers, if you're using PWM) choose the
>
> >> sampling interval for you. So choosing a sampling interval, while not
>
> >> as critical a tradeoff between cost (or straight feasibility) and
>
> >> performance as it used to be, is still a task that must be undertaken.
>
> >>
>
> >> As RBJ mentioned, the higher your sampling rate, the closer all the
>
> >> poles get to 1,
>
> >
>
> > i think gyansorova mentioned that...
>
> >
>
> >> ... and the more precision you need to distinguish them from 1.
>
> >> That means that as your sampling rate goes up so does your required
>
> >> word width. You can play tricks to alleviate the problem, but in the
>
> >> end you still often need to go to wider data paths.
>
> >
>
> > maybe. if you replace all the cos(w0) with 1 - 2*(sin(w0/2))^2 and
>
> > you're in single-precision floating point, and if you're careful to do
>
> > the arithmetic with the smaller numbers first (and later start adding in
>
> > the bigger numbers), you may very well not need to go to wider words.
>
> >
>
> > this cos(w0) issue shows up in audio when we make a decent DC-blocking
>
> > filter or other filters tuned to around 20 Hz or lower and it shows up
>
> > when you are trying to plot frequency response from the coefficients and
>
> > the frequencies are very low. just remember that all of the frequency
>
> > information lies in the *difference* cos(w0) is from 1.
>
>
>
> You can play tricks like that and stretch the usability of single-
>
> precision math to some extent. But ultimately, even if you solve the
>
> problem of distinguishing your coefficients from 1, you still need to
>
> have wide enough words in your data so that the distinction is not lost.
>
>
>
> --
>
>
>
> Tim Wescott
>
> Wescott Design Services
>
> http://www.wescottdesign.com
Here is some info from the net
10.6 CHOICE OF SAMPLING INTERVAL
Whenever a digital control system is designed, a suitable sampling interval must be chosen. Choosing a large sampling time has destabilizing effects on the system. In addition, information loss occurs when large sampling times are selected. Also, the errors that occur when a continuous system is discretized increase as the sampling interval increases.
Reply by Tim Wescott●May 27, 20142014-05-27
On Tue, 27 May 2014 13:55:09 -0400, robert bristow-johnson wrote:
> On 5/27/14 1:20 PM, Tim Wescott wrote:
>> On Mon, 26 May 2014 23:03:53 -0700, gyansorova wrote:
>>
>>
>>> I seem to remember them saying that if you sample too fast the poles
>>> can drift outside the unit circle. That sound like rubbish to me
>>> unless the original system is open-loop unstable in the first place.
>>
>> Even today, you still have to be mindful of the cost of high-rate
>> sampling, and some sensors (or drivers, if you're using PWM) choose the
>> sampling interval for you. So choosing a sampling interval, while not
>> as critical a tradeoff between cost (or straight feasibility) and
>> performance as it used to be, is still a task that must be undertaken.
>>
>> As RBJ mentioned, the higher your sampling rate, the closer all the
>> poles get to 1,
>
> i think gyansorova mentioned that...
>
>> ... and the more precision you need to distinguish them from 1.
>> That means that as your sampling rate goes up so does your required
>> word width. You can play tricks to alleviate the problem, but in the
>> end you still often need to go to wider data paths.
>
> maybe. if you replace all the cos(w0) with 1 - 2*(sin(w0/2))^2 and
> you're in single-precision floating point, and if you're careful to do
> the arithmetic with the smaller numbers first (and later start adding in
> the bigger numbers), you may very well not need to go to wider words.
>
> this cos(w0) issue shows up in audio when we make a decent DC-blocking
> filter or other filters tuned to around 20 Hz or lower and it shows up
> when you are trying to plot frequency response from the coefficients and
> the frequencies are very low. just remember that all of the frequency
> information lies in the *difference* cos(w0) is from 1.
You can play tricks like that and stretch the usability of single-
precision math to some extent. But ultimately, even if you solve the
problem of distinguishing your coefficients from 1, you still need to
have wide enough words in your data so that the distinction is not lost.
--
Tim Wescott
Wescott Design Services
http://www.wescottdesign.com
Reply by ●May 27, 20142014-05-27
> >
>
> > Reminds me of complaints about early CD playes which used one DAC,
>
> > multiplexed between two channels. The result is a half sample delay
>
> > between the two.
>
>
>
>
> > Unless you are very careful where you put your head, you won't notice
>
> > a half sample delay.
>
>
>
probably not a problem when listening with speakers..
but it might matter with headphones
and it will certainly matter if the the two channels are combined back to mono because it will create comb filtering
Mark
Reply by Rob Gaddi●May 27, 20142014-05-27
On Tue, 27 May 2014 14:01:18 -0400
robert bristow-johnson <rbj@audioimagination.com> wrote:
> On 5/27/14 1:29 PM, Rob Gaddi wrote:
> >
> > It's not that the poles of the transformed plant model sneak outside
> > the unit circle, it's that the poles of the controller do because your
> > numerical precision is insufficient to keep them in.
>
> but you should have enough numerical precision to express the inside
> distance your pole is *from* the unit circle, even with low precision
> words. of course, if the precision gets *way* too low (like 16-bits)
> you have all sorts of problems, but you should *still* be able to
> represent poles dancing close to the unit circle. and you do that by
> re-arranging your difference equation, which is a lot like changing the
> "form" or topology of the filter or controller from a DF1 or DF2 to
> something else.
>
> --
>
> r b-j rbj@audioimagination.com
>
> "Imagination is more important than knowledge."
>
>
Can and did are very different things. Doubly so in educational environments.
--
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order. See above to fix.
Reply by robert bristow-johnson●May 27, 20142014-05-27
On 5/27/14 1:29 PM, Rob Gaddi wrote:
>
> It's not that the poles of the transformed plant model sneak outside
> the unit circle, it's that the poles of the controller do because your
> numerical precision is insufficient to keep them in.
but you should have enough numerical precision to express the inside
distance your pole is *from* the unit circle, even with low precision
words. of course, if the precision gets *way* too low (like 16-bits)
you have all sorts of problems, but you should *still* be able to
represent poles dancing close to the unit circle. and you do that by
re-arranging your difference equation, which is a lot like changing the
"form" or topology of the filter or controller from a DF1 or DF2 to
something else.
--
r b-j rbj@audioimagination.com
"Imagination is more important than knowledge."
Reply by robert bristow-johnson●May 27, 20142014-05-27
On 5/27/14 1:20 PM, Tim Wescott wrote:
> On Mon, 26 May 2014 23:03:53 -0700, gyansorova wrote:
>
>> I seem to remember them saying that if you sample too fast the
>> poles can drift outside the unit circle. That sound like rubbish to me
>> unless the original system is open-loop unstable in the first place.
>
> Even today, you still have to be mindful of the cost of high-rate
> sampling, and some sensors (or drivers, if you're using PWM) choose the
> sampling interval for you. So choosing a sampling interval, while not as
> critical a tradeoff between cost (or straight feasibility) and
> performance as it used to be, is still a task that must be undertaken.
>
> As RBJ mentioned, the higher your sampling rate, the closer all the poles
> get to 1,
i think gyansorova mentioned that...
> ... and the more precision you need to distinguish them from 1.
> That means that as your sampling rate goes up so does your required word
> width. You can play tricks to alleviate the problem, but in the end you
> still often need to go to wider data paths.
maybe. if you replace all the cos(w0) with 1 - 2*(sin(w0/2))^2 and
you're in single-precision floating point, and if you're careful to do
the arithmetic with the smaller numbers first (and later start adding in
the bigger numbers), you may very well not need to go to wider words.
this cos(w0) issue shows up in audio when we make a decent DC-blocking
filter or other filters tuned to around 20 Hz or lower and it shows up
when you are trying to plot frequency response from the coefficients and
the frequencies are very low. just remember that all of the frequency
information lies in the *difference* cos(w0) is from 1.
--
r b-j rbj@audioimagination.com
"Imagination is more important than knowledge."
Reply by Rob Gaddi●May 27, 20142014-05-27
On Tue, 27 May 2014 12:20:44 -0500
Tim Wescott <tim@seemywebsite.really> wrote:
> As RBJ mentioned, the higher your sampling rate, the closer all the poles
> get to 1, and the more precision you need to distinguish them from 1.
> That means that as your sampling rate goes up so does your required word
> width. You can play tricks to alleviate the problem, but in the end you
> still often need to go to wider data paths.
>
> The poles drifting outside the unit circle stuff doesn't sound right -- a
> stable plant is a stable plant, and the z-domain poles of a linear time-
> invariant plant as seen by the controller always has poles that map
> straight from the plant's s-domain pole positions. I suspect the author
> was referring to some specific circumstance, but he may have been just
> plain incorrect.
>
> --
>
> Tim Wescott
> Wescott Design Services
> http://www.wescottdesign.com
>
I think that might be the thing you're saying. It's not that the poles
of the transformed plant model sneak outside the unit circle, it's that
the poles of the controller do because your numerical precision is
insufficient to keep them in.
--
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order. See above to fix.
Reply by Tim Wescott●May 27, 20142014-05-27
On Mon, 26 May 2014 23:03:53 -0700, gyansorova wrote:
> On Tuesday, May 27, 2014 5:12:30 PM UTC+12, robert bristow-johnson
> wrote:
>> On 5/27/14 12:12 AM, gyansorova@gmail.com wrote:
>>
>> > Is there really such a thing as a First Order Hold as you see in some
>> > textbooks?
>>
>>
>>
>> i've never seen one.
>>
>>
>>
>>
>> >
>> > I have looked many times at DAC outputs and all I see are staircase
>> > type waveforms ie Zero-Order Holds.
>>
>>
>>
>> but you could make a circuit that would put an analog integrator (or a
>>
>> simple RC LPF with very low pole frequency) on the output of the DAC
>>
>> and, inside the DSP, put in a digital differentiator on the data going
>>
>> out to the DAC. that would get you a first-order hold.
>>
>>
>>
>> > Also, with a high sampling rate the effect of the ZOH more or less
>> > vanishes - do we need to still analyze it?
>>
>>
>>
>> not for sigma delta. but if you were doing a control system with
>>
>> digital control and an analog plant, and with conventional DACs (i
>> dunno
>>
>> why you would, i remember an audio board with sigma-deltas and it was
>>
>> good all the way down to DC), then you might need to model it, if the
>>
>> sample rate was low.
>>
>>
>>
>> > There is a phase shift of course which is important (-90 degrees at
>> > half sampling but this gets smaller too the faster you sample).
>>
>>
>>
>> oversampling covers many, many digital sins. perhaps all of them.
>> with
>>
>> enough oversampling, you get virtual analog.
>>
>>
> Yes that was my impression. I agree. If you look in some control books
> they talk of the "choice of sampling interval"! Surely you sample as
> fast as you can get away with since as you point out we get back to
> analogue. I seem to remember them saying that if you sample too fast the
> poles can drift outside the unit circle. That sound like rubbish to me
> unless the original system is open-loop unstable in the first place.
Even today, you still have to be mindful of the cost of high-rate
sampling, and some sensors (or drivers, if you're using PWM) choose the
sampling interval for you. So choosing a sampling interval, while not as
critical a tradeoff between cost (or straight feasibility) and
performance as it used to be, is still a task that must be undertaken.
As RBJ mentioned, the higher your sampling rate, the closer all the poles
get to 1, and the more precision you need to distinguish them from 1.
That means that as your sampling rate goes up so does your required word
width. You can play tricks to alleviate the problem, but in the end you
still often need to go to wider data paths.
The poles drifting outside the unit circle stuff doesn't sound right -- a
stable plant is a stable plant, and the z-domain poles of a linear time-
invariant plant as seen by the controller always has poles that map
straight from the plant's s-domain pole positions. I suspect the author
was referring to some specific circumstance, but he may have been just
plain incorrect.
--
Tim Wescott
Wescott Design Services
http://www.wescottdesign.com