questions about superposition and linearity.....

Hello,

if we have a linear time invariant system, then no matter when the delta
input occurs, the impulse response is just a shifted version of response to
the unit delta occurring at t=0. We all know that, right...

A simple doubt: if a unit delta enters the system at time t, and another
unit delta input follows the first delta (some seconds later), such that
the response of the first delta is still forming while the second delta
enters the system and starts stimulating the system. It would seem that the
final output cannot be just the summation (superposition) of two impulse
responses (same response but shifted). The entrance of the second delta in
the system seems to interfere with the formation of the impulse response of
the first delta....

Q: where is the flaw in this description?

I know that it is not the case (linear superposition holds, as if the
delta inputs occurred individually, one in the absence of the other. There
is not "coupling".....

thanks,
fisico32

fisico32 <marcoscipioni1@gmail.com> wrote:
 
< if we have a linear time invariant system, then no matter when the delta
< input occurs, the impulse response is just a shifted version of response to
< the unit delta occurring at t=0. We all know that, right...

There are two important ideas above:

linear (superposition holds)

time invariant (delayed input results in delayed output)

You can have one without the other.

Many physical systems are linear for small signals (or close enough),
but not for larger signals.  The whole field of non-linear optics
describes systems where superposition does not hold in optical
materials, (though usually still time invariant).

-- glen 

On Tue, 06 Oct 2009 20:15:17 -0500, fisico32 wrote:

> Hello,
> 
> if we have a linear time invariant system, then no matter when the delta
> input occurs, the impulse response is just a shifted version of response
> to the unit delta occurring at t=0. We all know that, right...
> 
> A simple doubt: if a unit delta enters the system at time t, and another
> unit delta input follows the first delta (some seconds later), such that
> the response of the first delta is still forming while the second delta
> enters the system and starts stimulating the system. It would seem that
> the final output cannot be just the summation (superposition) of two
> impulse responses (same response but shifted). The entrance of the
> second delta in the system seems to interfere with the formation of the
> impulse response of the first delta....
> 
> Q: where is the flaw in this description?
> 
> I know that it is not the case (linear superposition holds, as if the
> delta inputs occurred individually, one in the absence of the other.
> There is not "coupling".....

You answer your own question with your last statement.  The response to 
the second delta is superimposed upon the response to the first.

I suspect that your intuition is at war with your training -- your 
intuition is telling you that it can't be that easy, but the 
superposition principal is telling you that yes, it is that easy.  To 
resolve this, you probable just need to give your intuition some practice 
-- make some easy linear differential equation, and solve it for just the 
case you give above.  Keep doing it until your intuition gives up and 
stops arguing.

And note, as Glen pointed out, that system linearity and system time 
variance are two different things.  A system can be both linear and time 
invariant, linear and time varying, nonlinear and time invariant, or both 
nonlinear and time varying.  

Your example, with very slight modifications, holds for a linear time 
varying system as well as a linear time invariant system -- the only 
difference is that the impulse response may with changing time; but the 
superposition of the two from two different stimulus impulses does not 
change.

-- 
www.wescottdesign.com

Tim Wescott <tim@seemywebsite.com> wrote:
(someone wrote)
 
<> if we have a linear time invariant system, then no matter when the delta
<> input occurs, the impulse response is just a shifted version of response
<> to the unit delta occurring at t=0. We all know that, right...
< the second delta is superimposed upon the response to the first.
(snip)
 
< I suspect that your intuition is at war with your training -- your 
< intuition is telling you that it can't be that easy, but the 
< superposition principal is telling you that yes, it is that easy.  To 
< resolve this, you probable just need to give your intuition some practice 
< -- make some easy linear differential equation, and solve it for just the 
< case you give above.  Keep doing it until your intuition gives up and 
< stops arguing.

In addition, many real (analog) systems are not perfectly linear.
Among others, non-linearity results in the harmonic distortion
and intermodulation distortion reported for stereo equipment.

It is linearity (at low fields) that allows two light beams to
cross without disturbing each other.  At high enough fields
(close to pulling the electrons off atoms) optical materials
aren't linear anymore, and light beams can affect each other.

It is a good idea, then, to always watch out for the non-linearities.

-- glen

On 7 Okt, 03:15, "fisico32" <marcoscipio...@gmail.com> wrote:
> Hello,
>
> if we have a linear time invariant system, then no matter when the delta
> input occurs, the impulse response is just a shifted version of response to
> the unit delta occurring at t=0. We all know that, right...
>
> A simple doubt: if a unit delta enters the system at time t, and another
> unit delta input follows the first delta (some seconds later), such that
> the response of the first delta is still forming while the second delta
> enters the system and starts stimulating the system. It would seem that the
> final output cannot be just the summation (superposition) of two impulse
> responses (same response but shifted).

Actually, it is. As a matter of fact, you just took the
first step towards deriving the convolution sum formula:

1) Apply one delayed impulse every time instance, d[n-m]
2) Scale the m'th impulse by the m'th number in a sequence
   x[n]: x[m]d[n-m]
3) The shifted, scaled impulse response becomes x[m]h[n-m]
4) Sum all shifted, scaled impulse responses that were
   caused by all shifted scaled impulses, and reaorganize
   the algebra.

If you do all this correctly, you end up with the convolution
sum formula

y[n] = x[n] (*) h[n].

Rune

On Oct 6, 10:41&#4294967295;pm, Tim Wescott <t...@seemywebsite.com> wrote:

> And note, as Glen pointed out, that system linearity and system time
> variance are two different things. &#4294967295;A system can be both linear and time
> invariant, linear and time varying, nonlinear and time invariant, or both

But most real systems are non-linear - its just that we simplify
things so we can get closed-form solutions.
We ignore minor non-linearities. Much of the neural network approaches
assume non-linearity from the outset and can achieve better results  -
say in adaptive filtering. (at the expense of more computation)


Hardy

>On Oct 6, 10:41=A0pm, Tim Wescott <t...@seemywebsite.com> wrote:
>
>> And note, as Glen pointed out, that system linearity and system time
>> variance are two different things. =A0A system can be both linear and
tim=
>e
>> invariant, linear and time varying, nonlinear and time invariant, or
both
>
>But most real systems are non-linear - its just that we simplify
>things so we can get closed-form solutions.
>We ignore minor non-linearities. Much of the neural network approaches
>assume non-linearity from the outset and can achieve better results  -
>say in adaptive filtering. (at the expense of more computation)

Well, a Dirac delta applied to any nonlinearity is fairly difficult to
handle mathematically.  Anyone taken a stab at squaring a delta?  The OP
postulated LTI.  (And don't tell me it's discrete time, since that's
generally linear unless you TRY to make it nonlinear, barring overflow.)  

Sorry, but that juxtaposition of deltas with nonlinearities was just
bugging me every time it was repeated...  If I've missed something
fundamental in handling that, do correct me, but I've never gotten, say,
Mathematica to get past a delta-squared.

Michael Plante <michael.plante@gmail.com> wrote:
(snip)
 
< Well, a Dirac delta applied to any nonlinearity is fairly difficult to
< handle mathematically.  Anyone taken a stab at squaring a delta?  The OP
< postulated LTI.  (And don't tell me it's discrete time, since that's
< generally linear unless you TRY to make it nonlinear, barring overflow.)  

Rereading the OP, it seems possible to read it in either the
continuous or discrete sense.  That is, dirac or kronicker delta.

< Sorry, but that juxtaposition of deltas with nonlinearities was just
< bugging me every time it was repeated...  If I've missed something
< fundamental in handling that, do correct me, but I've never gotten, say,
< Mathematica to get past a delta-squared.

I believe, though, that it doesn't cause so much of a problem.

The question, then, is the response to A delta(x) proportional
to A.  If so, the system is linear.  A system with finite bandwidth
will give a finite amplitude response to such input. (Maybe not
finite time response, though.)

-- glen

>Michael Plante <michael.plante@gmail.com> wrote:
>(snip)
> 
>< Well, a Dirac delta applied to any nonlinearity is fairly difficult to
>< handle mathematically.  Anyone taken a stab at squaring a delta?  The
OP
>< postulated LTI.  (And don't tell me it's discrete time, since that's
>< generally linear unless you TRY to make it nonlinear, barring
overflow.)  
>
>Rereading the OP, it seems possible to read it in either the
>continuous or discrete sense.  That is, dirac or kronicker delta.

Agreed, however my response was not an objection to what the OP wrote. 
People started writing about nonlinearities in the sense of nonideality,
but the only justification for talking nonlinearities when linearity is
postulated is if they're unintentional, which implies analog to me.


>< Sorry, but that juxtaposition of deltas with nonlinearities was just
>< bugging me every time it was repeated...  If I've missed something
>< fundamental in handling that, do correct me, but I've never gotten,
say,
>< Mathematica to get past a delta-squared.
>
>I believe, though, that it doesn't cause so much of a problem.

Could you give an example where it's not a problem to wind up with a
squared delta?  The only thing I can think of is if you're multiplying it
by something that's zero at that time, the trivial example, and almost not
worth mentioning.


>The question, then, is the response to A delta(x) proportional
>to A.  If so, the system is linear.  A system with finite bandwidth
>will give a finite amplitude response to such input. (Maybe not
>finite time response, though.)

If it's linear, I have no problem.  As an approximation, it's probably
also not a problem (just make an approximation to the delta that's valid in
that bandwidth).  But we typically use a delta to approximate reality, not
the other way around.  Or am I wrong?  

On Wed, 07 Oct 2009 14:49:02 -0500, Michael Plante wrote:

>>On Oct 6, 10:41=A0pm, Tim Wescott <t...@seemywebsite.com> wrote:
>>
>>> And note, as Glen pointed out, that system linearity and system time
>>> variance are two different things. =A0A system can be both linear and
> tim=
>>e
>>> invariant, linear and time varying, nonlinear and time invariant, or
> both
>>
>>But most real systems are non-linear - its just that we simplify things
>>so we can get closed-form solutions. We ignore minor non-linearities.
>>Much of the neural network approaches assume non-linearity from the
>>outset and can achieve better results  - say in adaptive filtering. (at
>>the expense of more computation)
> 
> Well, a Dirac delta applied to any nonlinearity is fairly difficult to
> handle mathematically.  Anyone taken a stab at squaring a delta?  The OP
> postulated LTI.  (And don't tell me it's discrete time, since that's
> generally linear unless you TRY to make it nonlinear, barring overflow.)
> 
> Sorry, but that juxtaposition of deltas with nonlinearities was just
> bugging me every time it was repeated...  If I've missed something
> fundamental in handling that, do correct me, but I've never gotten, say,
> Mathematica to get past a delta-squared.

I'm not sure where you see a juxtaposition of deltas and nonlinearities 
-- the OP posited a linear system, I just chimed in with a comment about 
how nonlinearity and time variance aren't necessarily related, and Hardy 
made the quite astute statement that in the real world no system is 
linear.

(No real system is linear, at least if it's over the field of real 
numbers.  Linear shift registers that generate pseudo-random data or 
convolutionally coded data are linear, but only over the field of numbers 
for which they're designed).

-- 
www.wescottdesign.com