Continuous-time DSP with no sampling

Yannis wrote:
> Yes, in principle no levels are skipped, since there is no "time
> between samples" - there are no samples, and everything is done in
> continuous time. This, indeed, if implemented brute-force, makes the
> speed-resolution product low. Now, one could skip *some* steps with
> small penalty, but I will refrain from proposing this until there is
> more work to back it. And, indeed, the reason that Gray codes will not
> save us directly, is that the signals from the taps could be arriving
> at instances differing by arbitrarily small amounts, and this can
> produce glitches - it is a problem we are working on.
> 
> The above having being said, I would not be as quick as to pronounce
> the technique "truly and fundamentally impractical", as you have done.
> In my reply to your first message, I offered an example of earlier
> skeptical attitudes with another idea, and what eventually happened. I
> will give you another example: In the early 80s, we had no resistors to
> speak of on CMOS chips, so we proposed a method to make filters using
> the (heavily nonlinear) MOSFETs in lieu of resistors, and still attain
> input-output linearity. People thought it was truly and fundamentally
> impractical back then, too. But it worked (see the IEEE Journal of
> Solid-State Circuits, Dec. 85), and such filters became a mainstream
> technology and were produced in the millions. As it turned out, they
> found a niche we had not even dreamt of when we propose them:
> equalization in the read electronics of computer disk drives.

OK, lets start again skipping the gradiose comparisons with past glory. :-)
> 
> I am not saying all this to brag, but rather to make people refrain
> from dispensing with the idea, so that they will take the time to keep
> offering us their valuable input.
> 
> Yannis Tsividis

Maybe the problem here is you are calling this continuous time when is 
clearly can't be. The settling time of an A/D has very different 
behviour from the settling time of an op-amp, because its outputs switch.

It seems what you are decribing is not a continuous time system, but one 
which follows its own natural timing. Something similar to the 
difference betweem the behaviour of synchronous logic, and self-clocked 
asynchronous logic. Now, if that is the case I can see how there could 
well be structures that behave in meaningful ways without clocking. 
However, that would appear to lead to massive problems of its own. The 
majority of the things we do in DSP are predicated on the samples we use 
being extremely even in time. The transitions of an A/D's output are far 
from that. Different transitions will have very different delays. This 
seems to mean that you would need a whole new mathematical basis for any 
processing. More than that, I can't see how you could know each of your 
delays with precision, so the maths could not allow for it.

Regards,
Steve

in article dkhp9p$6rt$1@nnews.pacific.net.hk, Steve Underwood at
steveu@dis.org wrote on 11/05/2005 03:08:

> Maybe the problem here is you are calling this continuous time when is
> clearly can't be. The settling time of an A/D has very different
> behviour from the settling time of an op-amp, because its outputs switch.

i don't quite get the difference, Steve.  the traditional flash A/D with
output word of N bits has 2^N - 1 comparators with the + terminal connected
together to the input voltage and the - terminal connected to a tap on a
resistive voltage divider ladder.  the 2^N - 1 comparator outputs (which
looks like a "thermometer code") are connected to a whole shitload of
combinatorial logic and N bits of 2's complement binary (or Gray code, if
one designs it as such) come out.

no clocks.  no latching.  no samples.

there is some settling time after there is a transition from one
quantization level to another, and in the time transpiring between that
level change and when the bits settle, there are glitches.

-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

robert bristow-johnson wrote:
> in article dkhp9p$6rt$1@nnews.pacific.net.hk, Steve Underwood at
> steveu@dis.org wrote on 11/05/2005 03:08:
> 
> 
>>Maybe the problem here is you are calling this continuous time when is
>>clearly can't be. The settling time of an A/D has very different
>>behviour from the settling time of an op-amp, because its outputs switch.
> 
> 
> i don't quite get the difference, Steve.  the traditional flash A/D with
> output word of N bits has 2^N - 1 comparators with the + terminal connected
> together to the input voltage and the - terminal connected to a tap on a
> resistive voltage divider ladder.  the 2^N - 1 comparator outputs (which
> looks like a "thermometer code") are connected to a whole shitload of
> combinatorial logic and N bits of 2's complement binary (or Gray code, if
> one designs it as such) come out.
> 
> no clocks.  no latching.  no samples.
> 
> there is some settling time after there is a transition from one
> quantization level to another, and in the time transpiring between that
> level change and when the bits settle, there are glitches.

Which you fix with a clock and latches.


> 

Yannis Tsividis wrote:
> In principle, sampling is not necessary in order to do filtering
> digitally. This is discussed in the following paper:
> 
> Y. Tsividis, "Digital signal processing in continuous time: a
> possibility for avoiding aliasing and reducing quantization error",
> Proc. 2004 IEEE Int. Conf. on Acoustics, Speech, and Signal Processing,
> vol. II, pp. 589-592, Montreal, May 2004.
> (If you are interested but cannot obtain this paper, please let me
> know.)
> 
> In the above paper, I discuss a method to do DSP in continuous time,
> without sampling, resulting in a system with no aliasing. The system
> has no quantization error at non-harmonic frequencies, and exhibits
> 10-15 dB lower total quantization error than classical DSP, for a given
> number of bits. Power dissipation decreases when the input frequency is
> low, or in general when there is little activity. However, although
> breadboard measurements and simulations show that the idea works, there
> is a lot of work to be done before one can know whether all this is
> practically feasible. This work is at the early research stage, and no
> commercial feasibility is claimed at this point.
> 
> I would be very interested in the opinion of DSP experts on this idea.
> We are currently looking for an appropriate application in order to
> demonstrate the concept. I welcome any comments!

Yannis,

I read through the paper quickly. I will need to go over it again to 
understand how the quantization noise disappears, but although it hasn't 
been focused on here, it is one of the more interesting results. (In 
comparison, the absence of aliasing is trivial since the process can be 
modeled as having an infinite sample rate.)

The most obvious problem is glitching, as has been noted. I don't agree 
that Gray codes can't help there, even if they aren't the entire 
solution. For the moment, let's optimistically assume that glitching is 
solved and touch on another issue.

Consider one of Zeno's paradoxes and its resolution.

             *Paradox: Motion is Impossible*
      Before one can move to some new location, one must go
      halfway. To reach the half-way point, one must pass the
      quarter point. Before that, 1/8th, then 1/16, then ....
      There is no end to the enumeration, so the goal can't
      be reached in finite time.

                        *Resolution*
      The enumeration of way points /in that way/ can't be
      accomplished, but there is no link between the motion
      itself and the enumeration. The supposed paradox
      is simply bad logic.

Digitizing amounts to an enumeration of (typically) voltages. The usual 
way using uniform time steps is conceptually simple and mathematically 
tractable. Another way, made possible by quantizing, is an enumeration 
of the time required for the voltage to change from one step to the 
next, and indicating of the direction of the change. We don't do that 
because it's too hard. One difficulty is that before the voltage take a 
new value, it must go half way -- shades of Zeno. We are exempted from 
making the enumeration infinite only by accepting its truncation. Still, 
small step size (read "allowed truncation error") implies very high data 
rates and very fast converters. I don't know what the future may bring, 
but one can reduce quantization noise and aliasing greatly right now by 
using those same resources on more conventional ways.

Many will know me as a habitual wet blanket. My reaction to most novel 
ideas and authoritative pronouncements is a search for counterexamples. 
The aim is not to attack the originator of the idea -- my own ideas get 
the most scrutiny -- but to probe the idea itself for weakness. I think 
that what you've shown is very interesting. I'm not sure how useful it 
will be, but time will tell.

Jerry
-- 
Never ascribe to malice what might be be ignorance, stupidity, or sloth.
������������������������������������������������������������������������

Stan Pawlukiewicz wrote:
> robert bristow-johnson wrote:
> 
>> in article dkhp9p$6rt$1@nnews.pacific.net.hk, Steve Underwood at
>> steveu@dis.org wrote on 11/05/2005 03:08:
>>
>>
>>> Maybe the problem here is you are calling this continuous time when is
>>> clearly can't be. The settling time of an A/D has very different
>>> behviour from the settling time of an op-amp, because its outputs 
>>> switch.
>>
>>
>>
>> i don't quite get the difference, Steve.  the traditional flash A/D with
>> output word of N bits has 2^N - 1 comparators with the + terminal 
>> connected
>> together to the input voltage and the - terminal connected to a tap on a
>> resistive voltage divider ladder.  the 2^N - 1 comparator outputs (which
>> looks like a "thermometer code") are connected to a whole shitload of
>> combinatorial logic and N bits of 2's complement binary (or Gray code, if
>> one designs it as such) come out.
>>
>> no clocks.  no latching.  no samples.
>>
>> there is some settling time after there is a transition from one
>> quantization level to another, and in the time transpiring between that
>> level change and when the bits settle, there are glitches.
> 
> 
> Which you fix with a clock and latches.

How do you time the clock? Latches, sure, but what switches them?

I imagine suitable delays, a transition detector (XOR with one input 
slightly delayed) from each output bit feeding a grand OR gate, which 
provides the latch signal. Baroque!

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

robert bristow-johnson wrote:
> in article dkhp9p$6rt$1@nnews.pacific.net.hk, Steve Underwood at
> steveu@dis.org wrote on 11/05/2005 03:08:
> 
> 
>>Maybe the problem here is you are calling this continuous time when is
>>clearly can't be. The settling time of an A/D has very different
>>behviour from the settling time of an op-amp, because its outputs switch.
> 
> 
> i don't quite get the difference, Steve.  the traditional flash A/D with
> output word of N bits has 2^N - 1 comparators with the + terminal connected
> together to the input voltage and the - terminal connected to a tap on a
> resistive voltage divider ladder.  the 2^N - 1 comparator outputs (which
> looks like a "thermometer code") are connected to a whole shitload of
> combinatorial logic and N bits of 2's complement binary (or Gray code, if
> one designs it as such) come out.
> 
> no clocks.  no latching.  no samples.
> 
> there is some settling time after there is a transition from one
> quantization level to another, and in the time transpiring between that
> level change and when the bits settle, there are glitches.
> 
An analogue amp shows a continuous progressive output change as it 
settles. The A/D converter shows no change for a while, then a burst of 
glitching, then a new value. There is nothing continuous about that at 
all. Its analagous to the self-clocked asychronous logic that people 
have been trying to drive into the mainstream for years, except there 
may not strictly be a clock in this case. It shares that "do it as fast 
as you can, but don't do it in a continuous progressive manner" quality.

Regards,
Steve

in article dkigov$j12$1@newslocal.mitre.org, Stan Pawlukiewicz at
spam@spam.mitre.org wrote on 11/05/2005 09:49:

> robert bristow-johnson wrote:
>> in article dkhp9p$6rt$1@nnews.pacific.net.hk, Steve Underwood at
>> steveu@dis.org wrote on 11/05/2005 03:08:
>> 
>> 
>>> Maybe the problem here is you are calling this continuous time when is
>>> clearly can't be. The settling time of an A/D has very different
>>> behviour from the settling time of an op-amp, because its outputs switch.
>> 
>> i don't quite get the difference, Steve.  the traditional flash A/D with
>> output word of N bits has 2^N - 1 comparators with the + terminals connected
>> together to the input voltage and the - terminals connected to taps on a
>> resistive voltage divider ladder.  the 2^N - 1 comparator outputs (which
>> looks like a "thermometer code") are connected to a whole shitload of
>> combinatorial logic and N bits of 2's complement binary (or Gray code, if
>> one designs it as such) come out.
>> 
>> no clocks.  no latching.  no samples.
>> 
>> there is some settling time after there is a transition from one
>> quantization level to another, and in the time transpiring between that
>> level change and when the bits settle, there are glitches.
> 
> Which you fix with a clock and latches.

hey, fine!  but then it isn't "Continuous-time DSP with **no** sampling".

guys, i am not trying to defend the practicality of Yannis's idea.  i'm only
trying to keep his idea of "continuous-time" with "no sampling" from being
adulterated by concepts of discrete-time and sampled data systems.  then we
can talk (or type) about Yannis's idea as it is, not as what it might be if
it were different.

-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

in article 1131042015.321469.234090@f14g2000cwb.googlegroups.com, Yannis at
ytctdsp@yahoo.com wrote on 11/03/2005 13:20:

> ...  I hope I was careful not to claim industrial
> feasibility - in fact, my message made that very clear. On the other
> hand, this is a university, and we are supposed to be doing research.

yeah, but if the discipline is engineering, i would hope there is some
possibility of application.  otherwize what Real_McCoy says in article
b5jaf.480$xD6.26416@news.xtra.co.nz at 11/03/2005 02:35:

>> Well I always say 'Don't let trifling problems like the practicalities of
>> whether it works or not get in the way of a good paper'!!

... is applicable.

publishing for the sake of publishing (or for the sake of getting a Ph.D. or
a grant or tenure, etc.) is part of the problem, not the solution.  (check
out Martin Anderson "Imposters in the Temple: American Intellectuals Are
Destroying Our Universities and Cheating Our Students of Their Future" some
quotes: http://pegasus.cc.ucf.edu/~fabianic/anderson.html

people who know me know that the Hoover Institute resonates very little with
my politics and worldview but, with the exception of Anderson's complaint
about a politically correct witch hunt on campus (which i don't really think
exists in the vast majority of departments, particularly in science or
engineering), i really think this ex Reagan administrator is correct about
what has happened to the academe.

Yannis, i dunno what school you're at and i am no enemy to the concept and
original purpose of higher education or academic research, but i would hope
that you would choose to be part of the solution rather than part of the
problem.

...

> And, again, my purpose in
> mentioning this was to make an analogy for the possibility that
> continuous-time DSP may turn out to be feasible;

okay, Yannis, i don't believe that you have dissuaded any of my doubts, but
i wouldn't mind if did actually address what ostensibly appears to be this
possible contradiction in your words.

in what sense can this *ever* be feasible?  (if not "industrially", then
how?)  if you cannot precisely synchronize your bit delays and the delays
through the combinatorial logic, how can you *ever* beat the glitches?  i
see no hope in accomplishing that.

with sampling rates as high as they are (and only to get higher), and with
finite-bandwidth data channels and storage, how can this *ever* compete with
sampled-date systems even if you *were* able to beat the glitches.

-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

Robert,

You are beating the wrong horse. I would have assumed that my messages
of Nov. 3 and Nov. 4 in this thread would have convinved you, beyond
doubt, that our group has a track record of research that leads to
important industrial applications. As to your question, "how can you
*ever* beat the glitches?", let me say this: If, in 1974, someone asked
me "how can you *ever* make an op amp out of MOS trasistors", my answer
would have been "I don't know *yet*"; if I had let myself be convinced
by several people back then that what I was trying to do was
impossible, I would not have tried. Fortunately, I did not take their
advice, and in the end I was proven right. I hope history will repeat
itself with this work.

Being relevant in university research does not mean you do not jump
into difficult territory. 

Yannis

in article 1131399617.730331.219850@f14g2000cwb.googlegroups.com, Yannis at
ytctdsp@yahoo.com wrote on 11/07/2005 16:40:

> Robert,
> 
> You are beating the wrong horse.

i'm sure he deserves it for some other reason i'm not aware of yet.

> I would have assumed that my messages of Nov. 3 and Nov. 4 in this thread
> would have convinved you, beyond doubt, that our group has a track record of
> research that leads to important industrial applications.

i'm only commenting on the content of the article that you brought up.
whether or not your "group has a track record of research that leads to
important industrial applications" is immaterial to that.

> As to your question, "how can you *ever* beat the glitches?", let me say this:
> If, in 1974, someone asked me "how can you *ever* make an op amp out of MOS
> trasistors", my answer would have been "I don't know *yet*";

i understand that this is the argument you've been making, but it's not
persuasive.  you're talking about an issue with the natural variance of
delay times through *different* delay elements hooked up to different bits.
considering the MSB, you have to get that down to the pico- or femto-second
to prevent those glitches from being a problem.

> if I had let myself be convinced by several people back then that what I was
> trying to do was impossible, I would not have tried.

then build us a faster than light transportation device, because this is
sorta similar.

> Fortunately, I did not take their advice, and in the end I was proven right.
> I hope history will repeat itself with this work.
> 
> Being relevant in university research does not mean you do not jump
> into difficult territory.

it should also have promise.  and the promise of something better than what
is existing technology.  that's where i also remain skeptical.

-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."