LTI vs non-LTI

Rick Lyons <R.Lyons@_bogus_ieee.org> wrote:
> On Sun, 28 Apr 2013 13:57:23 -0500, Tim Wescott
> <tim@seemywebsite.please> wrote:
 
>>On Sun, 28 Apr 2013 09:58:15 -0700, Rick Lyons wrote:

>>> I've always thought of mixers as being nonlinear because a mixer's
>>> output contains spectral components that did not exist at either of its
>>> two inputs.

>>"Have always thought"?  That's wishy-washy.
 
> [Snipped by Lyons]
 
> Whoa Tim.
>  I used the phrase "I've always thought of mixers being 
> nonlinear" instead of "Tim's post can be misleading."

Seems to me that linear (or lack thereof) is easy to define
when there is one input, but not so easy with two:

   f(x)+f(y)=f(x+y),   f(Ax)=Af(x)

among others, it is easy to show that some of our favorite
operators, integral and derivative, are linear operators.

With more than one input, there are more combinations to consider.

Is addition linear?  x+y=x+y, that doesn't say anything.

   define the operator g(x,y)=x+y, is it linear?

   g(a+b,c)=a+g(b,c) g(a,b+c)=b+g(a,c), is that enough?

  g(a,0)+g(b,0)=g(a+b,0) but

  g(a,1)+g(b,1) is not equal to g(a+b,1).

Now, consider h(x,y)=x y, as multiply is close to the operation
of many mixers:

  h(a,c)+h(b,c)=h(a+b,c) and h(c,a)+h(c,b)=h(c,a+b)

Multiply is linear in one input (port) if the other is kept
constant. 

   h(a,b) + h(b,b) is not h(a+b,a+b). 

Now, consider the microwave mixer, possibly defined as:

  f(a+b) = f(a) + f(b) + C a b + ...

first, we are back to a one port (one input) operation again. 
Any linear operator (device) won't have a product term in its output,
but many non-linear ones will. I believe that there are also microwave
mixers satisfying:

  f(a+b) = f(a) + f(b) + 2 a b + ...

such that the LO input needs to be only half frequency. Sometimes
convenient in transistor costs.

-- glen

Tim Wescott <tim@seemywebsite.com> wrote:

(snip, someone wrote)

>>>>>   Upsampling and downsampling are time variant

(snip)
>>> Sampling is time varying because the input is ignored for all time
>>> except the sampling instant.  If you're willing to use impulse
>>> functionals in your analysis then sampling is just a form of mixing, in
>>> the linear, time- varying sense that I was advocating earlier.

(snip, I wrote)
>> Well, assuming that the signal has the appropriate band limiting.
 
> I certainly hope you're off on a tangent, and not saying anything about 
> the signal needing to be bandlimited for my comments to be valid.  I'm 
> talking about _systems_, not the signals that may be running through them.

OK, consider system f(x), a 44.1kHz sampler, and system g(y) an
anti-aliasing filter that cuts off before 44.1kHz.
 
>> Consider starting with a continuous time signal that is perfectly band
>> limited. Now sample it at different sampling rates above the Nyquist
>> rate, with an ideal sampler. All then contain the exact same
>> information, which is that in the original signal.
 
> So?  What does that have to do with a discussion about whether a system 
> is time varying or not?

Are systems f(x) and g(y) time invariant? How about g(f(x))?
 

(snip, I wrote)
>> Maybe I don't understand "time varying" in the sense you use it here,
>> but up sampling by a rational factor should give the same signal you
>> would have seen sampling the original at the higher rate. (Assuming the
>> filters are all ideal, which they usually aren't, except in homework
>> problems.)
 
> I am using "time varying" in the sense that the expression
 
> h(x(t), t) = h(x(t-tau), t - tau)
 
> is not true for all tau if the system h is time varying.  What other 
> definition is there?

-- glen

Rick Lyons <R.Lyons@_bogus_ieee.org> wrote:

(snip, someone wrote)
>>Sir, can you please elaborate?

(snip)
>  Concerning downsampling:
 
> From the very nature of its operation, we know if we 
> delay the input sequence by one sample, a downsampler 
> will generate an entirely different output sequence. 
> For example, if we apply an input sequence 
> x(n) = x(0), x(1), x(2), x(3), x(4), etc., to a 
> downsampler and M = 3, the output y(m) will be the 
> sequence x(0), x(3), x(6), etc. Should we delay the 
> input sequence by one sample, our delayed xd(n) input 
> would be x(1), x(2), x(3), x(4), x(5), etc.
> In this case the downsampled output sequence yd(m) 
> would be x(1), x(4), x(7), etc., which is not a 
> delayed version of y(m). Thus a downsampler is not 
> time invariant. 

OK, but we usually use sampling in the context of reconstruction.
We want to be able to sample a continuous signal, and then 
reconstruct the original, or something close to it.

Consider passing the two downsampled signals through reconstruction
(anti-aliasing) filters? 

If you consider the sampled signal as a container holding a
previously band-limited continuous signal, or if you consider
that aliases don't add any information to a signal, then it seems
to me that the result is different.
 
> What this means is that if a downsampling operation 
> is in cascade with other operations, we are not 
> permitted to swap the order of any of those operations 
> and the downsampling process without modifying those 
> operations in some way.

Yes, you have to be careful with the order of operations.

-- glen

i've been refraining from this thread, sorta.

On Apr 28, 5:36&#4294967295;am, "manishp" <58525@dsprelated> wrote:
>
> In order for me to appreciate a little better, can I get some very simple
> examples of,
>
> 1) a non-LTI system that is linear but is time variant (that is, it is non
> time invariant)

your audio sound system where you ware wiggling the volume or tone
control.

> 2) a non-LTI system that is time invariaint but non linear

a distortion circuit with a diode in it or a transistor driven into
saturation or similar.


now, to glen and Tim and Rick and Eric, about this mixer business...

i think we can derive, purely from the base definitions of "Linear"
and "Time-Invariant", an input-output characteristic that follows the
convolution integral.  if the mixer cannot be described by a
convolution, then it ain't LTI.

we know that for LTI, that the only frequencies that come out are
frequencies that go in (but possibly with their amplitudes and phases
adjusted).

i don't see how heterodyning can be described as LTI.


r b-j

>Seems to me that linear (or lack thereof) is easy to define
>when there is one input, but not so easy with two:
>
>   f(x)+f(y)=f(x+y),   f(Ax)=Af(x)
>
>among others, it is easy to show that some of our favorite
>operators, integral and derivative, are linear operators.
>
>With more than one input, there are more combinations to consider.


I was thinking about that as well.  It seems to me that for a multiple
input system to be linear, one would have to be able to express it in terms
of multiple single input linear systems.  This is from the principle of
superposition.

I.E.  f(x, y) = f(x, 0) + f(0, y) = g(x) + h(y)

It is then obvious that addition would be linear and multiplication would
be non-linear.  Perhaps one could abstract to say that something is linear
with respect to something else.  For example, the system f is linear in x
with respect to y?

I.E.  if f(x, y) = x*y

Then,

f(x1 + x2, y) = f(x1, y) + f(x2, y) for all y.

Anyone familiar with terminology regarding this kind of situation?

On 2013-04-30 20:47, glen herrmannsfeldt wrote:
[...]
> With more than one input, there are more combinations to consider.

I suspect that, if we consider the multi
input as a vector, then:

z=f(V), where V=[x, y, z, ...] is a vector, then:

f(aV1+bV2)=af(V1)+bf(V2) means f(.) is linear.

I wonder if considering a multi-input independently
linear is each variable makes sense too.

bye,

-- 

piergiorgio

On Tue, 30 Apr 2013 10:30:53 -0700, Rick Lyons wrote:

> On Sun, 28 Apr 2013 13:57:23 -0500, Tim Wescott
> <tim@seemywebsite.please> wrote:
> 
>>On Sun, 28 Apr 2013 09:58:15 -0700, Rick Lyons wrote:
>>
>>
>>> I've always thought of mixers as being nonlinear because a mixer's
>>> output contains spectral components that did not exist at either of
>>> its two inputs.
>>
>>"Have always thought"?  That's wishy-washy.
> 
> [Snipped by Lyons]
> 
> Whoa Tim.
>   I used the phrase "I've always thought of mixers being
> nonlinear" instead of "Tim's post can be misleading."
> 
> When I read your 2nd 4/28/13 post it seemed to me you left the
> impression that mixers are linear. What I think you should have written
> was, "Mixers are nonlinear.  But it's possible to operate a mixer in
> such a way that the effects of its nonlinearity can be kept acceptably
> small."
> 
> My phrase "I've always thought" was my clumsy attempt to be diplomatic.
> 
> See Ya',
> [-Rick-]

Diplomacy?  What's that?

I guess it depends on whether you're viewing a mixer as a two-input one 
output device, in which case it is indeed nonlinear, or if you're viewing 
a mixer as a one-input device that autonomously acts in a time-varying 
way (i.e., if you're viewing it as a mixer with attached LO, or just 
modeling it with one line of math).  In the former case it's nonlinear, 
in the latter it is linear but time varying.

In either case one should specify what you mean -- as such, perhaps we 
both erred.

As for the statement on mixers being nonlinear, etc., well, here's my 
guiding light on this (paraphrasing intentional, but meant to educate 
rather than offend): "Any physical system, device, or whatever is 
nonlinear.  But it is possible to operate many such systems in such a way 
that the effects of nonlinearity can be kept acceptably small".

Mixers fall into the category of "systems that can be usefully considered 
linear", but so do amplifiers, digital signal processors running DSP 
software (even though the innards of said processor is HORRIBLY 
nonlinear), FPGAs doing DSP (ditto on the nonlinearity), at times motors, 
heaters, and some chemical processes, the vacuum, semiconductor devices, 
vacuum tubes, flywheels, shock absorbers (kinda), airplanes, cars, 
trucks, busses, trains, ad infinitum.  So it's not like mixers are unique 
to that description.

Certainly the criteria that a mixer is nonlinear because its output 
contains different spectral components than its input is not valid: that 
could be a sign of nonlinearity, but it can also be a sign of time 
variance.  In the sense of an ideal mixer-and-LO that I was invoking, the 
output contains spectral components that were not in the input -- but 
said ideal mixer is still a perfectly linear system (also wholly 
imaginary, but it's a useful approximation).

-- 
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
Why am I not happy that they have found common ground?

Tim Wescott, Communications, Control, Circuits & Software
http://www.wescottdesign.com

On Tue, 30 Apr 2013 15:11:32 -0500, dszabo wrote:

>>Seems to me that linear (or lack thereof) is easy to define when there
>>is one input, but not so easy with two:
>>
>>   f(x)+f(y)=f(x+y),   f(Ax)=Af(x)
>>
>>among others, it is easy to show that some of our favorite operators,
>>integral and derivative, are linear operators.
>>
>>With more than one input, there are more combinations to consider.
> 
> 
> I was thinking about that as well.  It seems to me that for a multiple
> input system to be linear, one would have to be able to express it in
> terms of multiple single input linear systems.  This is from the
> principle of superposition.
> 
> I.E.  f(x, y) = f(x, 0) + f(0, y) = g(x) + h(y)
> 
> It is then obvious that addition would be linear and multiplication
> would be non-linear.  Perhaps one could abstract to say that something
> is linear with respect to something else.  For example, the system f is
> linear in x with respect to y?
> 
> I.E.  if f(x, y) = x*y
> 
> Then,
> 
> f(x1 + x2, y) = f(x1, y) + f(x2, y) for all y.
> 
> Anyone familiar with terminology regarding this kind of situation?

The notation that I see in the control systems literature is

dx/dt = F(x, u, t), y = H(x, u, t)

where x, y, and u are vectors and t is plain ol' scalar time.

In this case then you can fairly directly figure out how superposition 
would work (and, in fact, you can chase your tail a bit and see that for 
a lumped-state, linear, time-invariant system, the equations boil down to 
the old state-space representation:

dx/dt = A * x + B * u, y = C * x + D * u

)

You don't see that much in signal processing literature, possibly because 
the need to have a general systems representation is not so severe, and 
possibly because most communications systems either have specific well-
known nonlinearities or they are very close to linear by design.

-- 
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
Why am I not happy that they have found common ground?

Tim Wescott, Communications, Control, Circuits & Software
http://www.wescottdesign.com

On Tue, 30 Apr 2013 18:58:18 +0000, glen herrmannsfeldt wrote:

> Tim Wescott <tim@seemywebsite.com> wrote:
> 
> (snip, someone wrote)
> 
>>>>>>   Upsampling and downsampling are time variant
> 
> (snip)
>>>> Sampling is time varying because the input is ignored for all time
>>>> except the sampling instant.  If you're willing to use impulse
>>>> functionals in your analysis then sampling is just a form of mixing,
>>>> in the linear, time- varying sense that I was advocating earlier.
> 
> (snip, I wrote)
>>> Well, assuming that the signal has the appropriate band limiting.
>  
>> I certainly hope you're off on a tangent, and not saying anything about
>> the signal needing to be bandlimited for my comments to be valid.  I'm
>> talking about _systems_, not the signals that may be running through
>> them.
> 
> OK, consider system f(x), a 44.1kHz sampler, and system g(y) an
> anti-aliasing filter that cuts off before 44.1kHz.
>  
>>> Consider starting with a continuous time signal that is perfectly band
>>> limited. Now sample it at different sampling rates above the Nyquist
>>> rate, with an ideal sampler. All then contain the exact same
>>> information, which is that in the original signal.
>  
>> So?  What does that have to do with a discussion about whether a system
>> is time varying or not?
> 
> Are systems f(x) and g(y) time invariant? How about g(f(x))?

I'm going to take your statement as meaning z = f(g(y)), as that puts the 
anti-alias filter before the sampling -- I think that must be what you 
meant.  (Either that or you're failing to differentiate reconstruction 
from aliasing, which I very much doubt).

Given that:

g(y) is (or darn well should be!) time invariant.

f(x) is time varying.

Even without knowing that f(x) is time varying, you know that g(f(x)) is 
time varying because its input is in continuous-time and its output is in 
discrete time.  The action of the sampler loses the detail of what's 
going on between the sampling instants of x.  It happens that in your 
example that detail is redundant -- but f(x) still throws that detail 
away, changing it from redundant to nonexistent.

Had you really wanted to flummox me you'd then follow this by system h
(z), which is a perfect reconstructor such that the action of

x_p = h(f(g(y))) 

is indistinguishable but for delay from x = g(y).  At this point I would 
say that h(f(g(y))) certainly _acts like_ a linear time invariant system, 
and in many cases should be treated as such -- but with a great deal of 
caution, always keeping in mind that there's some pretty oddball stuff 
going on in the middle of that there seemingly-innocent system.

-- 
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
Why am I not happy that they have found common ground?

Tim Wescott, Communications, Control, Circuits & Software
http://www.wescottdesign.com

On Tue, 30 Apr 2013 19:06:18 +0000, glen herrmannsfeldt wrote:

> Rick Lyons <R.Lyons@_bogus_ieee.org> wrote:
> 
> (snip, someone wrote)
>>>Sir, can you please elaborate?
> 
> (snip)
>>  Concerning downsampling:
>  
>> From the very nature of its operation, we know if we delay the input
>> sequence by one sample, a downsampler will generate an entirely
>> different output sequence. For example, if we apply an input sequence
>> x(n) = x(0), x(1), x(2), x(3), x(4), etc., to a downsampler and M = 3,
>> the output y(m) will be the sequence x(0), x(3), x(6), etc. Should we
>> delay the input sequence by one sample, our delayed xd(n) input would
>> be x(1), x(2), x(3), x(4), x(5), etc. In this case the downsampled
>> output sequence yd(m) would be x(1), x(4), x(7), etc., which is not a
>> delayed version of y(m). Thus a downsampler is not time invariant.
> 
> OK, but we usually use sampling in the context of reconstruction. We
> want to be able to sample a continuous signal, and then reconstruct the
> original, or something close to it.

Ah ha!  This is why you and I are out of step in this whole discussion.

When you're doing control systems work, you use sampling in the context 
of doing a good-enough job of control.  Anti-aliasing and reconstruction 
are actually your enemy in this case, because they only make signal 
fidelity better if you ignore delay, whereas in a control system signal 
fidelity must take delay into account -- which means that an aliased, but 
unfiltered, signal is better than one that's had the snot filtered out of 
it and 720 degrees of delay added at the edges of the anti-alias filter.

> Consider passing the two downsampled signals through reconstruction
> (anti-aliasing) filters?
> 
> If you consider the sampled signal as a container holding a previously
> band-limited continuous signal, or if you consider that aliases don't
> add any information to a signal, then it seems to me that the result is
> different.

I think you're allowing yourself to confuse your signals with your 
systems -- for the whole screed see my response to your other post vis-a-
vis anti-alias filters followed by sampling.

>> What this means is that if a downsampling operation is in cascade with
>> other operations, we are not permitted to swap the order of any of
>> those operations and the downsampling process without modifying those
>> operations in some way.
> 
> Yes, you have to be careful with the order of operations.

And careful consideration of which signal processing stages are linear 
and/or time invariant is a big help in doing this.  It's one of the 
reasons you want to keep track.

-- 
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
Why am I not happy that they have found common ground?

Tim Wescott, Communications, Control, Circuits & Software
http://www.wescottdesign.com