LTI Systems and Impulse Response

In his book, "A Course in Digital Signal Processing", Porat states
that it is a common misconception is that every Linear Time-Invariant
system has an impulse response.  He then quotes an example from
Kailath's book, "Linear Systems".  The crux of the argument is to
define the class of inputs such that the impulse does not belong to it
and then claim the system has no impulse response.  This seems like
cheating :-).  I seem to be missing the point of the following example
from Porat (p. 34):

Let x(t) be in the input family iff (1) x(t) is continuous, except at
a countable number of points t; (2) the discontinuity at each such
point is a finite jump, i.e., the limits at both sides of the
discontinuity exist; (3) the sum of absolute values of all
discontinuity jumps is finite.  Let y(t) be the sum of all jumps of
x(s) at discontinuity points s < t.  This system is linear and time-
invariant, but it has no impulse response because delta(t) is not in
the input family.  Consequently its response cannot be described by a
convolution.

I guess the last sentence may hold the key, which I need to ponder
about more...

--vv

On Jan 11, 10:17&#4294967295;pm, vv <vanam...@netzero.net> wrote:
> In his book, "A Course in Digital Signal Processing", Porat states
> that it is a common misconception is that every Linear Time-Invariant
> system has an impulse response. &#4294967295;He then quotes an example from
> Kailath's book, "Linear Systems". &#4294967295;The crux of the argument is to
> define the class of inputs such that the impulse does not belong to it
> and then claim the system has no impulse response. &#4294967295;This seems like
> cheating :-). &#4294967295;I seem to be missing the point of the following example
> from Porat (p. 34):
>
> Let x(t) be in the input family iff (1) x(t) is continuous, except at
> a countable number of points t; (2) the discontinuity at each such
> point is a finite jump, i.e., the limits at both sides of the
> discontinuity exist; (3) the sum of absolute values of all
> discontinuity jumps is finite. &#4294967295;Let y(t) be the sum of all jumps of
> x(s) at discontinuity points s < t. &#4294967295;This system is linear and time-
> invariant, but it has no impulse response because delta(t) is not in
> the input family. &#4294967295;Consequently its response cannot be described by a
> convolution.
>
> I guess the last sentence may hold the key, which I need to ponder
> about more...
>
> --vv

That's called masturbation with maths, and is not for engineers.

On Jan 11, 10:17&#4294967295;am, vv <vanam...@netzero.net> wrote:
> In his book, "A Course in Digital Signal Processing", Porat states
> that it is a common misconception is that every Linear Time-Invariant
> system has an impulse response. &#4294967295;He then quotes an example from
> Kailath's book, "Linear Systems". &#4294967295;The crux of the argument is to
> define the class of inputs such that the impulse does not belong to it
> and then claim the system has no impulse response. &#4294967295;This seems like
> cheating :-). &#4294967295;I seem to be missing the point of the following example
> from Porat (p. 34):
>
> Let x(t) be in the input family iff (1) x(t) is continuous, except at
> a countable number of points t; (2) the discontinuity at each such
> point is a finite jump, i.e., the limits at both sides of the
> discontinuity exist; (3) the sum of absolute values of all
> discontinuity jumps is finite. &#4294967295;Let y(t) be the sum of all jumps of
> x(s) at discontinuity points s < t. &#4294967295;This system is linear and time-
> invariant, but it has no impulse response because delta(t) is not in
> the input family. &#4294967295;Consequently its response cannot be described by a
> convolution.
>
> I guess the last sentence may hold the key, which I need to ponder
> about more...
>
> --vv

Yes, it IS cheating. You don't need to ponder, because it can not bear
any relevance to any practical engineering application. BAH !!

Han de Bruijn

On 01/11/2011 01:17 AM, vv wrote:
> In his book, "A Course in Digital Signal Processing", Porat states
> that it is a common misconception is that every Linear Time-Invariant
> system has an impulse response.  He then quotes an example from
> Kailath's book, "Linear Systems".  The crux of the argument is to
> define the class of inputs such that the impulse does not belong to it
> and then claim the system has no impulse response.  This seems like
> cheating :-).  I seem to be missing the point of the following example
> from Porat (p. 34):
>
> Let x(t) be in the input family iff (1) x(t) is continuous, except at
> a countable number of points t; (2) the discontinuity at each such
> point is a finite jump, i.e., the limits at both sides of the
> discontinuity exist; (3) the sum of absolute values of all
> discontinuity jumps is finite.  Let y(t) be the sum of all jumps of
> x(s) at discontinuity points s<  t.  This system is linear and time-
> invariant, but it has no impulse response because delta(t) is not in
> the input family.  Consequently its response cannot be described by a
> convolution.
>
> I guess the last sentence may hold the key, which I need to ponder
> about more...

Do you have the page number or chapter number for Kailath's example? 
I've got the book by my elbow, I'd like to look it up.

While I won't go as far as to say that the system as defined fails to 
have _any_ practical application in engineering problems, it certainly 
fails to have any _remotely common_ practical application in engineering 
problems.

What it does, as far as I'm concerned, is put a constraint on a writer 
about saying "if it's LTI then it has an impulse response".  Saying "if 
it's _sensible_ and LTI then ..." leaves the reader wondering what you 
mean.  I'd find some way to put in a weasel word in the main text, with 
a footnote pointing to an argument much like the one above, then I'd 
stress that you're almost certainly not going to see such a system in 
the real world, so why worry?

What I would find of more interest is what additional constraints to put 
on a linear system such that it _does_ have an impulse response, and try 
to get an idea of how many, and how broad, the families of LTI systems 
without impulse responses are.  I rather suspect the answer is "dunno" 
and "not very".

Interestingly enough there _are_ linear systems that aren't "linear" 
over the real number line: the two examples I can think of are the 
finite fields that are used for error detection and correction, and the 
angle of a single-axis rotating shaft.  Finite fields are both discrete 
and display "wrapping", i.e. in the field {0, 1}, 1 + 1 = 0, 0 + 1 = 1, 
etc.  Angles aren't discrete, but they _do_ wrap: for all practical 
purposes 2*pi = 0, unless you're counting turns.

-- 

Tim Wescott
Wescott Design Services
http://www.wescottdesign.com

Do you need to implement control loops in software?
"Applied Control Theory for Embedded Systems" was written for you.
See details at http://www.wescottdesign.com/actfes/actfes.html

On 11 Jan., 10:17, vv <vanam...@netzero.net> wrote:
> In his book, "A Course in Digital Signal Processing", Porat states
> that it is a common misconception is that every Linear Time-Invariant
> system has an impulse response. &#4294967295;He then quotes an example from
> Kailath's book, "Linear Systems". &#4294967295;The crux of the argument is to
> define the class of inputs such that the impulse does not belong to it
> and then claim the system has no impulse response. &#4294967295;This seems like
> cheating :-). &#4294967295;I seem to be missing the point of the following example
> from Porat (p. 34):
>
> Let x(t) be in the input family iff (1) x(t) is continuous, except at
> a countable number of points t; (2) the discontinuity at each such
> point is a finite jump, i.e., the limits at both sides of the
> discontinuity exist; (3) the sum of absolute values of all
> discontinuity jumps is finite. &#4294967295;Let y(t) be the sum of all jumps of
> x(s) at discontinuity points s < t. &#4294967295;This system is linear and time-
> invariant, but it has no impulse response because delta(t) is not in
> the input family. &#4294967295;Consequently its response cannot be described by a
> convolution.

This argument is flawed. The "input family" is a vector space, but not
a complete one. It doesn't close under Cauchy sequences, and therefore
is not a Banach space. Signal Theory practically always starts with
either finite dimensional spaces or Banach spaces (or even Hilbert
spaces). So the statement is of no practical or theoretical relevance
for signal theory. It's a curious counter example and a good reason
for why you start with Banach spaces.

Cheers,

 Andreas

On Jan 12, 8:04&#4294967295;am, Andreas Tell <li...@brainstream-audio.de> wrote:
> On 11 Jan., 10:17, vv <vanam...@netzero.net> wrote:
>
>
>
> > In his book, "A Course in Digital Signal Processing", Porat states
> > that it is a common misconception is that every Linear Time-Invariant
> > system has an impulse response. &#4294967295;He then quotes an example from
> > Kailath's book, "Linear Systems". &#4294967295;The crux of the argument is to
> > define the class of inputs such that the impulse does not belong to it
> > and then claim the system has no impulse response. &#4294967295;This seems like
> > cheating :-). &#4294967295;I seem to be missing the point of the following example
> > from Porat (p. 34):
>
> > Let x(t) be in the input family iff (1) x(t) is continuous, except at
> > a countable number of points t; (2) the discontinuity at each such
> > point is a finite jump, i.e., the limits at both sides of the
> > discontinuity exist; (3) the sum of absolute values of all
> > discontinuity jumps is finite. &#4294967295;Let y(t) be the sum of all jumps of
> > x(s) at discontinuity points s < t. &#4294967295;This system is linear and time-
> > invariant, but it has no impulse response because delta(t) is not in
> > the input family. &#4294967295;Consequently its response cannot be described by a
> > convolution.
>
> This argument is flawed. The "input family" is a vector space, but not
> a complete one. It doesn't close under Cauchy sequences, and therefore
> is not a Banach space. Signal Theory practically always starts with
> either finite dimensional spaces or Banach spaces (or even Hilbert
> spaces). So the statement is of no practical or theoretical relevance
> for signal theory. It's a curious counter example and a good reason
> for why you start with Banach spaces.
>
> Cheers,
>
> &#4294967295;Andreas

Never even heard of a Banach space and been using sig processing for a
very long time!

On Jan 11, 9:17&#4294967295;am, vv <vanam...@netzero.net> wrote:
> In his book, "A Course in Digital Signal Processing", Porat states
> that it is a common misconception is that every Linear Time-Invariant
> system has an impulse response. &#4294967295;He then quotes an example from
> Kailath's book, "Linear Systems". &#4294967295;The crux of the argument is to
> define the class of inputs such that the impulse does not belong to it
> and then claim the system has no impulse response. &#4294967295;This seems like
> cheating :-). &#4294967295;I seem to be missing the point of the following example
> from Porat (p. 34):
>
> Let x(t) be in the input family iff (1) x(t) is continuous, except at
> a countable number of points t; (2) the discontinuity at each such
> point is a finite jump, i.e., the limits at both sides of the
> discontinuity exist; (3) the sum of absolute values of all
> discontinuity jumps is finite. &#4294967295;Let y(t) be the sum of all jumps of
> x(s) at discontinuity points s < t. &#4294967295;This system is linear and time-
> invariant, but it has no impulse response because delta(t) is not in
> the input family. &#4294967295;Consequently its response cannot be described by a
> convolution.
>
> I guess the last sentence may hold the key, which I need to ponder
> about more...
>
> --vv

The argument is valid in its own terms. If you do not allow an impulse
input then you can deny there is an impulse response.

You can formalise the argument by saying that some input domains do
not include impulses. Those are not common spaces for DSP.

I think a useful point is being made, though: that one should at least
be aware that talking about things like the impulse response may hide
an assumption about things like the input domain. There are other
perhaps more useful simlar points: for example the importance of being
clear about the interval over which you define something like an FFT.
You can clarify explicitly by stating the scope: eg "for input domains
that include an impulse" or you can have that in your 'context' (eg
'in my work I assume domains that include impulses') or you can decide
that we can go on like this in ever-decreasing circles like the Giant
Oozzelum Bird that eventually disappears up its own backside and hope
for the best.

Chris
---
Chris Bore
BORES Signal Processing
www.bores.com

On Jan 13, 9:30&#4294967295;pm, HardySpicer <gyansor...@gmail.com> wrote:
> On Jan 12, 8:04&#4294967295;am, Andreas Tell <li...@brainstream-audio.de> wrote:
>
> > On 11 Jan., 10:17, vv <vanam...@netzero.net> wrote:
>
> > > In his book, "A Course in Digital Signal Processing", Porat states
> > > that it is a common misconception is that every Linear Time-Invariant
> > > system has an impulse response. &#4294967295;He then quotes an example from
> > > Kailath's book, "Linear Systems". &#4294967295;The crux of the argument is to
> > > define the class of inputs such that the impulse does not belong to it
> > > and then claim the system has no impulse response. &#4294967295;This seems like
> > > cheating :-). &#4294967295;I seem to be missing the point of the following example
> > > from Porat (p. 34):
>
> > > Let x(t) be in the input family iff (1) x(t) is continuous, except at
> > > a countable number of points t; (2) the discontinuity at each such
> > > point is a finite jump, i.e., the limits at both sides of the
> > > discontinuity exist; (3) the sum of absolute values of all
> > > discontinuity jumps is finite. &#4294967295;Let y(t) be the sum of all jumps of
> > > x(s) at discontinuity points s < t. &#4294967295;This system is linear and time-
> > > invariant, but it has no impulse response because delta(t) is not in
> > > the input family. &#4294967295;Consequently its response cannot be described by a
> > > convolution.
>
> > This argument is flawed. The "input family" is a vector space, but not
> > a complete one. It doesn't close under Cauchy sequences, and therefore
> > is not a Banach space. Signal Theory practically always starts with
> > either finite dimensional spaces or Banach spaces (or even Hilbert
> > spaces). So the statement is of no practical or theoretical relevance
> > for signal theory. It's a curious counter example and a good reason
> > for why you start with Banach spaces.
>
> > Cheers,
>
> > &#4294967295;Andreas
>
> Never even heard of a Banach space and been using sig processing for a
> very long time!

Never heard of a Sobolev space and been using Finite Element Methods
for a very long time!

http://en.wikipedia.org/wiki/Finite_element_method

Han de Bruijn

On Jan 13, 3:30&#4294967295;pm, HardySpicer <gyansor...@gmail.com> wrote:
>
>
> Never even heard of a Banach space and been using sig processing for a
> very long time!

you might get to the term Banach space in a course on metric spaces
and functional analysis.  as i recall, the important thing to remember
is that Banach spaces are what we call "normed metric spaces" or maybe
"normed vector spaces".  the members in the space have to have some
meaningful way of addition (usually that means that the elements or
coordinates of the members add) and that there has to be a "zero
member" or "additive identity member" that when added to any other
member, does not change it.  then the *norm* of a member in this
normed vector space is the distance metric from the zero member to
that member.  then things like commutativity and the triangle
inequality have to apply.

especially if you go into communications engineering, Metric Spaces
and Functional Analysis is a good math course to take.

r b-j

On Thu, 13 Jan 2011 12:30:23 -0800 (PST), HardySpicer
<gyansorova@gmail.com> wrote:

>On Jan 12, 8:04=A0am, Andreas Tell <li...@brainstream-audio.de> wrote:
>> On 11 Jan., 10:17, vv <vanam...@netzero.net> wrote:
>>
>>
>>
>> > In his book, "A Course in Digital Signal Processing", Porat states
>> > that it is a common misconception is that every Linear Time-Invariant
>> > system has an impulse response. =A0He then quotes an example from
>> > Kailath's book, "Linear Systems". =A0The crux of the argument is to
>> > define the class of inputs such that the impulse does not belong to it
>> > and then claim the system has no impulse response. =A0This seems like
>> > cheating :-). =A0I seem to be missing the point of the following exampl=
>e
>> > from Porat (p. 34):
>>
>> > Let x(t) be in the input family iff (1) x(t) is continuous, except at
>> > a countable number of points t; (2) the discontinuity at each such
>> > point is a finite jump, i.e., the limits at both sides of the
>> > discontinuity exist; (3) the sum of absolute values of all
>> > discontinuity jumps is finite. =A0Let y(t) be the sum of all jumps of
>> > x(s) at discontinuity points s < t. =A0This system is linear and time-
>> > invariant, but it has no impulse response because delta(t) is not in
>> > the input family. =A0Consequently its response cannot be described by a
>> > convolution.
>>
>> This argument is flawed. The "input family" is a vector space, but not
>> a complete one. It doesn't close under Cauchy sequences, and therefore
>> is not a Banach space. Signal Theory practically always starts with
>> either finite dimensional spaces or Banach spaces (or even Hilbert
>> spaces). So the statement is of no practical or theoretical relevance
>> for signal theory. It's a curious counter example and a good reason
>> for why you start with Banach spaces.
>>
>> Cheers,
>>
>> =A0Andreas
>
>Never even heard of a Banach space and been using sig processing for a
>very long time!

I think it's usually a grad-school thing.   I encountered it in
grad-school over twenty years ago, but never in industry.   So, yeah,
it exists, it may have some utility, but it's not something everyone
will encounter or need.


Eric Jacobsen
Minister of Algorithms
Abineau Communications
http://www.abineau.com