Reply by Ilmari Karonen●January 19, 20112011-01-19
On 2011-01-17, VV <vanamali@netzero.net> wrote:
>
> Ilmari Karonen: Based on what you have explained, it seems that the
> example has more to it than what meets the eye of a less
> mathematically sophisticated engineer. Thanks. I still need to
> digest what you've said.
It's not really that complicated. The input space as defined does not
contain the delta function, but we could still approximate it using
nascent delta functions which do belong in the input space.
For example, let x(t) = g(t/a)/a, where g is a bounded piecewise
smooth function with integral 1. Then x(t) belongs in the input
space, and x(t) -> delta(t) as a -> 0 (in the usual weak sense).
Now, let's say g(t) is the "hat function" g(t) = min(0, 1-abs(t)).
This function is continuous, and so is therefore x(t), which makes
y(t) = 0 regardless of a.
What if we choose a discontinuous g(t), like the "top hat function"
g(t) = 1 for -1/2 < t < 1/2, g(t) = 0 otherwise? Since x(t) is now
piecewise constant (and 0 for t < -1/2a), y(t) = x(t), and therefore
y(t) -> delta(t) as a -> 0.
Even worse, what if we combine these two into an asymmetric function
like g(t) = 1 if 1/2 < t <= 0, g(t) = 1-t if 0 < t <= 1 and g(t) = 0
otherwise? Now x(t) has a single discontinuity at -1/2a, where it
jumps up by 1/a, and thus y(t) = 0 for t <= -1/2a and y(t) = 1/a
otherwise. Therefore, as a -> 0, lim y(t) = 0 for t < 0, but lim y(t)
diverges to infinity for all t >= 0.
The fact that all these limits are different should make it obvious
that this system in fact does not have a well definable impulse
response, and that it's not just a matter of the input space being
arbitrarily constrained.
--
Ilmari Karonen
To reply by e-mail, please replace ".invalid" with ".net" in address.
Reply by VV●January 17, 20112011-01-17
On Jan 14, 1:30�am, HardySpicer <gyansor...@gmail.com> wrote:
> Never even heard of a Banach space and been using sig processing for a
> very long time!
"Signal Theory" by Lewis Frank is referenced not infrequently in
signal processing literature.
Tim Wescott: "Do you have the page number or chapter number for
Kailath's example? I've got the book by my elbow, I'd like to look it
up." No, I don't have Kailath's book and hence unable to quote the
page number in which the example that Porat quotes is present.
Ilmari Karonen: Based on what you have explained, it seems that the
example has more to it than what meets the eye of a less
mathematically sophisticated engineer. Thanks. I still need to
digest what you've said.
--vv
Reply by Ilmari Karonen●January 14, 20112011-01-14
["Followup-To:" header set to sci.math.]
On 2011-01-14, Chris Bore <chris.bore@gmail.com> wrote:
> On Jan 11, 9:17 am, vv <vanam...@netzero.net> wrote:
>> In his book, "A Course in Digital Signal Processing", Porat states
>> that it is a common misconception is that every Linear Time-Invariant
>> system has an impulse response. He then quotes an example from
>> Kailath's book, "Linear Systems". The crux of the argument is to
>> define the class of inputs such that the impulse does not belong to it
>> and then claim the system has no impulse response. This seems like
>> cheating :-). I seem to be missing the point of the following example
>> from Porat (p. 34):
>>
>> Let x(t) be in the input family iff (1) x(t) is continuous, except at
>> a countable number of points t; (2) the discontinuity at each such
>> point is a finite jump, i.e., the limits at both sides of the
>> discontinuity exist; (3) the sum of absolute values of all
>> discontinuity jumps is finite. Let y(t) be the sum of all jumps of
>> x(s) at discontinuity points s < t. This system is linear and time-
>> invariant, but it has no impulse response because delta(t) is not in
>> the input family. Consequently its response cannot be described by a
>> convolution.
>>
>> I guess the last sentence may hold the key, which I need to ponder
>> about more...
>
> The argument is valid in its own terms. If you do not allow an impulse
> input then you can deny there is an impulse response.
There's a bit more to it than that. The input family does contain
sequences of functions that converge to delta, and so one could try to
define the impulse response as the limit of y_i(t) as x_i(t) tends to
delta(t).
Unfortunately, it turns out that this limit is not well defined. For
some obvious choices of x_i(t), like any continuous ones, or
discontinuous ones with even symmetry, the limit is y(t) = 0, which is
problematic enough; a non-trivial linear system certainly should not
have a zero impulse response. Worse yet, though, if you choose x_i(t)
to be continuous on one side of the origin and discontinuous on the
other, y_i(t) will diverge as x_i(t) tends to delta(t).
So I'd say the real moral of the story is that, on input domains which
don't include impulses, you can have linear time-invariant systems
which really don't have a well-defined impulse response in any sense,
and can't be consistently extended to inputs that include impulses.
--
Ilmari Karonen
To reply by e-mail, please replace ".invalid" with ".net" in address.
Reply by Eric Jacobsen●January 14, 20112011-01-14
On Thu, 13 Jan 2011 12:30:23 -0800 (PST), HardySpicer
<gyansorova@gmail.com> wrote:
>On Jan 12, 8:04=A0am, Andreas Tell <li...@brainstream-audio.de> wrote:
>> On 11 Jan., 10:17, vv <vanam...@netzero.net> wrote:
>>
>>
>>
>> > In his book, "A Course in Digital Signal Processing", Porat states
>> > that it is a common misconception is that every Linear Time-Invariant
>> > system has an impulse response. =A0He then quotes an example from
>> > Kailath's book, "Linear Systems". =A0The crux of the argument is to
>> > define the class of inputs such that the impulse does not belong to it
>> > and then claim the system has no impulse response. =A0This seems like
>> > cheating :-). =A0I seem to be missing the point of the following exampl=
>e
>> > from Porat (p. 34):
>>
>> > Let x(t) be in the input family iff (1) x(t) is continuous, except at
>> > a countable number of points t; (2) the discontinuity at each such
>> > point is a finite jump, i.e., the limits at both sides of the
>> > discontinuity exist; (3) the sum of absolute values of all
>> > discontinuity jumps is finite. =A0Let y(t) be the sum of all jumps of
>> > x(s) at discontinuity points s < t. =A0This system is linear and time-
>> > invariant, but it has no impulse response because delta(t) is not in
>> > the input family. =A0Consequently its response cannot be described by a
>> > convolution.
>>
>> This argument is flawed. The "input family" is a vector space, but not
>> a complete one. It doesn't close under Cauchy sequences, and therefore
>> is not a Banach space. Signal Theory practically always starts with
>> either finite dimensional spaces or Banach spaces (or even Hilbert
>> spaces). So the statement is of no practical or theoretical relevance
>> for signal theory. It's a curious counter example and a good reason
>> for why you start with Banach spaces.
>>
>> Cheers,
>>
>> =A0Andreas
>
>Never even heard of a Banach space and been using sig processing for a
>very long time!
I think it's usually a grad-school thing. I encountered it in
grad-school over twenty years ago, but never in industry. So, yeah,
it exists, it may have some utility, but it's not something everyone
will encounter or need.
Eric Jacobsen
Minister of Algorithms
Abineau Communications
http://www.abineau.com
Reply by robert bristow-johnson●January 14, 20112011-01-14
On Jan 13, 3:30�pm, HardySpicer <gyansor...@gmail.com> wrote:
>
>
> Never even heard of a Banach space and been using sig processing for a
> very long time!
you might get to the term Banach space in a course on metric spaces
and functional analysis. as i recall, the important thing to remember
is that Banach spaces are what we call "normed metric spaces" or maybe
"normed vector spaces". the members in the space have to have some
meaningful way of addition (usually that means that the elements or
coordinates of the members add) and that there has to be a "zero
member" or "additive identity member" that when added to any other
member, does not change it. then the *norm* of a member in this
normed vector space is the distance metric from the zero member to
that member. then things like commutativity and the triangle
inequality have to apply.
especially if you go into communications engineering, Metric Spaces
and Functional Analysis is a good math course to take.
r b-j
Reply by Han de Bruijn●January 14, 20112011-01-14
On Jan 13, 9:30�pm, HardySpicer <gyansor...@gmail.com> wrote:
> On Jan 12, 8:04�am, Andreas Tell <li...@brainstream-audio.de> wrote:
>
> > On 11 Jan., 10:17, vv <vanam...@netzero.net> wrote:
>
> > > In his book, "A Course in Digital Signal Processing", Porat states
> > > that it is a common misconception is that every Linear Time-Invariant
> > > system has an impulse response. �He then quotes an example from
> > > Kailath's book, "Linear Systems". �The crux of the argument is to
> > > define the class of inputs such that the impulse does not belong to it
> > > and then claim the system has no impulse response. �This seems like
> > > cheating :-). �I seem to be missing the point of the following example
> > > from Porat (p. 34):
>
> > > Let x(t) be in the input family iff (1) x(t) is continuous, except at
> > > a countable number of points t; (2) the discontinuity at each such
> > > point is a finite jump, i.e., the limits at both sides of the
> > > discontinuity exist; (3) the sum of absolute values of all
> > > discontinuity jumps is finite. �Let y(t) be the sum of all jumps of
> > > x(s) at discontinuity points s < t. �This system is linear and time-
> > > invariant, but it has no impulse response because delta(t) is not in
> > > the input family. �Consequently its response cannot be described by a
> > > convolution.
>
> > This argument is flawed. The "input family" is a vector space, but not
> > a complete one. It doesn't close under Cauchy sequences, and therefore
> > is not a Banach space. Signal Theory practically always starts with
> > either finite dimensional spaces or Banach spaces (or even Hilbert
> > spaces). So the statement is of no practical or theoretical relevance
> > for signal theory. It's a curious counter example and a good reason
> > for why you start with Banach spaces.
>
> > Cheers,
>
> > �Andreas
>
> Never even heard of a Banach space and been using sig processing for a
> very long time!
On Jan 11, 9:17�am, vv <vanam...@netzero.net> wrote:
> In his book, "A Course in Digital Signal Processing", Porat states
> that it is a common misconception is that every Linear Time-Invariant
> system has an impulse response. �He then quotes an example from
> Kailath's book, "Linear Systems". �The crux of the argument is to
> define the class of inputs such that the impulse does not belong to it
> and then claim the system has no impulse response. �This seems like
> cheating :-). �I seem to be missing the point of the following example
> from Porat (p. 34):
>
> Let x(t) be in the input family iff (1) x(t) is continuous, except at
> a countable number of points t; (2) the discontinuity at each such
> point is a finite jump, i.e., the limits at both sides of the
> discontinuity exist; (3) the sum of absolute values of all
> discontinuity jumps is finite. �Let y(t) be the sum of all jumps of
> x(s) at discontinuity points s < t. �This system is linear and time-
> invariant, but it has no impulse response because delta(t) is not in
> the input family. �Consequently its response cannot be described by a
> convolution.
>
> I guess the last sentence may hold the key, which I need to ponder
> about more...
>
> --vv
The argument is valid in its own terms. If you do not allow an impulse
input then you can deny there is an impulse response.
You can formalise the argument by saying that some input domains do
not include impulses. Those are not common spaces for DSP.
I think a useful point is being made, though: that one should at least
be aware that talking about things like the impulse response may hide
an assumption about things like the input domain. There are other
perhaps more useful simlar points: for example the importance of being
clear about the interval over which you define something like an FFT.
You can clarify explicitly by stating the scope: eg "for input domains
that include an impulse" or you can have that in your 'context' (eg
'in my work I assume domains that include impulses') or you can decide
that we can go on like this in ever-decreasing circles like the Giant
Oozzelum Bird that eventually disappears up its own backside and hope
for the best.
Chris
---
Chris Bore
BORES Signal Processing
www.bores.com
Reply by HardySpicer●January 13, 20112011-01-13
On Jan 12, 8:04�am, Andreas Tell <li...@brainstream-audio.de> wrote:
> On 11 Jan., 10:17, vv <vanam...@netzero.net> wrote:
>
>
>
> > In his book, "A Course in Digital Signal Processing", Porat states
> > that it is a common misconception is that every Linear Time-Invariant
> > system has an impulse response. �He then quotes an example from
> > Kailath's book, "Linear Systems". �The crux of the argument is to
> > define the class of inputs such that the impulse does not belong to it
> > and then claim the system has no impulse response. �This seems like
> > cheating :-). �I seem to be missing the point of the following example
> > from Porat (p. 34):
>
> > Let x(t) be in the input family iff (1) x(t) is continuous, except at
> > a countable number of points t; (2) the discontinuity at each such
> > point is a finite jump, i.e., the limits at both sides of the
> > discontinuity exist; (3) the sum of absolute values of all
> > discontinuity jumps is finite. �Let y(t) be the sum of all jumps of
> > x(s) at discontinuity points s < t. �This system is linear and time-
> > invariant, but it has no impulse response because delta(t) is not in
> > the input family. �Consequently its response cannot be described by a
> > convolution.
>
> This argument is flawed. The "input family" is a vector space, but not
> a complete one. It doesn't close under Cauchy sequences, and therefore
> is not a Banach space. Signal Theory practically always starts with
> either finite dimensional spaces or Banach spaces (or even Hilbert
> spaces). So the statement is of no practical or theoretical relevance
> for signal theory. It's a curious counter example and a good reason
> for why you start with Banach spaces.
>
> Cheers,
>
> �Andreas
Never even heard of a Banach space and been using sig processing for a
very long time!
Reply by Andreas Tell●January 11, 20112011-01-11
On 11 Jan., 10:17, vv <vanam...@netzero.net> wrote:
> In his book, "A Course in Digital Signal Processing", Porat states
> that it is a common misconception is that every Linear Time-Invariant
> system has an impulse response. �He then quotes an example from
> Kailath's book, "Linear Systems". �The crux of the argument is to
> define the class of inputs such that the impulse does not belong to it
> and then claim the system has no impulse response. �This seems like
> cheating :-). �I seem to be missing the point of the following example
> from Porat (p. 34):
>
> Let x(t) be in the input family iff (1) x(t) is continuous, except at
> a countable number of points t; (2) the discontinuity at each such
> point is a finite jump, i.e., the limits at both sides of the
> discontinuity exist; (3) the sum of absolute values of all
> discontinuity jumps is finite. �Let y(t) be the sum of all jumps of
> x(s) at discontinuity points s < t. �This system is linear and time-
> invariant, but it has no impulse response because delta(t) is not in
> the input family. �Consequently its response cannot be described by a
> convolution.
This argument is flawed. The "input family" is a vector space, but not
a complete one. It doesn't close under Cauchy sequences, and therefore
is not a Banach space. Signal Theory practically always starts with
either finite dimensional spaces or Banach spaces (or even Hilbert
spaces). So the statement is of no practical or theoretical relevance
for signal theory. It's a curious counter example and a good reason
for why you start with Banach spaces.
Cheers,
Andreas
Reply by Tim Wescott●January 11, 20112011-01-11
On 01/11/2011 01:17 AM, vv wrote:
> In his book, "A Course in Digital Signal Processing", Porat states
> that it is a common misconception is that every Linear Time-Invariant
> system has an impulse response. He then quotes an example from
> Kailath's book, "Linear Systems". The crux of the argument is to
> define the class of inputs such that the impulse does not belong to it
> and then claim the system has no impulse response. This seems like
> cheating :-). I seem to be missing the point of the following example
> from Porat (p. 34):
>
> Let x(t) be in the input family iff (1) x(t) is continuous, except at
> a countable number of points t; (2) the discontinuity at each such
> point is a finite jump, i.e., the limits at both sides of the
> discontinuity exist; (3) the sum of absolute values of all
> discontinuity jumps is finite. Let y(t) be the sum of all jumps of
> x(s) at discontinuity points s< t. This system is linear and time-
> invariant, but it has no impulse response because delta(t) is not in
> the input family. Consequently its response cannot be described by a
> convolution.
>
> I guess the last sentence may hold the key, which I need to ponder
> about more...
Do you have the page number or chapter number for Kailath's example?
I've got the book by my elbow, I'd like to look it up.
While I won't go as far as to say that the system as defined fails to
have _any_ practical application in engineering problems, it certainly
fails to have any _remotely common_ practical application in engineering
problems.
What it does, as far as I'm concerned, is put a constraint on a writer
about saying "if it's LTI then it has an impulse response". Saying "if
it's _sensible_ and LTI then ..." leaves the reader wondering what you
mean. I'd find some way to put in a weasel word in the main text, with
a footnote pointing to an argument much like the one above, then I'd
stress that you're almost certainly not going to see such a system in
the real world, so why worry?
What I would find of more interest is what additional constraints to put
on a linear system such that it _does_ have an impulse response, and try
to get an idea of how many, and how broad, the families of LTI systems
without impulse responses are. I rather suspect the answer is "dunno"
and "not very".
Interestingly enough there _are_ linear systems that aren't "linear"
over the real number line: the two examples I can think of are the
finite fields that are used for error detection and correction, and the
angle of a single-axis rotating shaft. Finite fields are both discrete
and display "wrapping", i.e. in the field {0, 1}, 1 + 1 = 0, 0 + 1 = 1,
etc. Angles aren't discrete, but they _do_ wrap: for all practical
purposes 2*pi = 0, unless you're counting turns.
--
Tim Wescott
Wescott Design Services
http://www.wescottdesign.com
Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html