Linear System Properties?

Hi,
I have some simple questions about the properties of linear systems. In a
linear System we have to assume the properties of homogeneity, additivity
and shift invariance.
In a DSP book I read that from these properties you can conclude that a
sine wave as input will always produce a sine wave on the systems output
with the same frequency. I can't really see what speciality of sine waves
should account for this special behaviour towards sine wave in linear
systems. I tried to prove this idea mathematically, but I did not get very
far.
I hope this question has not been asked befor too many times, if so, I
would thank you for pointing me in the right direction (WWW-pages, FAQ,
etc). I tried searching google, but i was not able to find anything
relevant.
Thanks in advance
Till

-- 
Please add "Salt and Peper" to the subject line to bypass my spam filter

Hey Till,

You're right - it isn't just sine waves that a linear system will not
produce new frequencies of - ANY waveform will be unaltered in
frequency by a linear system. 

The special thing about sine waves are that they provide a fundamental
set of functions from which ANY other waveform can be constructed, so
if they aren't moved in frequency by a linear system, neither are any 
other waveforms. 

--Randy

> Hi,
> I have some simple questions about the properties of linear systems. In a
> linear System we have to assume the properties of homogeneity, additivity
> and shift invariance.
> In a DSP book I read that from these properties you can conclude that a
> sine wave as input will always produce a sine wave on the systems output
> with the same frequency. I can't really see what speciality of sine waves
> should account for this special behaviour towards sine wave in linear
> systems. I tried to prove this idea mathematically, but I did not get very
> far.
> I hope this question has not been asked befor too many times, if so, I
> would thank you for pointing me in the right direction (WWW-pages, FAQ,
> etc). I tried searching google, but i was not able to find anything
> relevant.
> Thanks in advance
> Till
>
> -- 
> Please add "Salt and Peper" to the subject line to bypass my spam filter
>

-- 
%  Randy Yates                  % "Though you ride on the wheels of tomorrow,
%% Fuquay-Varina, NC            %  you still wander the fields of your
%%% 919-577-9882                %  sorrow."
%%%% <yates@ieee.org>           % '21st Century Man', *Time*, ELO
http://home.earthlink.net/~yatescr

In article <c41ho7$13u4$1@f1node01.rhrz.uni-bonn.de>,
"Till Crueger" <TillFC@gmx.net> wrote:

>Subject: Linear System Properties?
>From: "Till Crueger" <TillFC@gmx.net>
>Organization: HRZ - University of Bonn (Germany)
>Date: Fri, 26 Mar 2004 16:20:38 +0100
>Newsgroups: comp.dsp
>
>Hi,
>I have some simple questions about the properties of linear systems. In a
>linear System we have to assume the properties of homogeneity, additivity
>and shift invariance.
>In a DSP book I read that from these properties you can conclude that a
>sine wave as input will always produce a sine wave on the systems output
>with the same frequency. I can't really see what speciality of sine waves
>should account for this special behaviour towards sine wave in linear
>systems. I tried to prove this idea mathematically, but I did not get very
>far.
>I hope this question has not been asked befor too many times, if so, I
>would thank you for pointing me in the right direction (WWW-pages, FAQ,
>etc). I tried searching google, but i was not able to find anything
>relevant.
>Thanks in advance
>Till

Ask what happens if you look at the diferential, for continuous time, or
difference, for discrete time, equation that such a special testing 
function satisfies. I like Tukey's Math 596 Notes, from his collected 
works, becasue they use the fewest assumptions but there are many other 
sources.

>-- 
>Please add "Salt and Peper" to the subject line to bypass my spam filter
>

On Fri, 26 Mar 2004 15:28:12 +0000, Randy Yates wrote:

> Hey Till,
> 
> You're right - it isn't just sine waves that a linear system will not
> produce new frequencies of - ANY waveform will be unaltered in frequency
> by a linear system.
> 
> The special thing about sine waves are that they provide a fundamental
> set of functions from which ANY other waveform can be constructed, so if
> they aren't moved in frequency by a linear system, neither are any other
> waveforms.

Hmm, I thought about that too.
However shouldn't it also be possible to construct any waveform from
square waves? If this is so, how come the shape of square waves is altered
in a linear system?
Furthermore I thought about a system which only would double any frequency
component present in a given Signal, and I fail to see how this system is
non-linear.
Till

-- 
Please add "Salt and Peper" to the subject line to bypass my spam filter

Till Crueger wrote:

> Hi,
> I have some simple questions about the properties of linear systems. In a
> linear System we have to assume the properties of homogeneity, additivity
> and shift invariance.
> In a DSP book I read that from these properties you can conclude that a
> sine wave as input will always produce a sine wave on the systems output
> with the same frequency. I can't really see what speciality of sine waves
> should account for this special behaviour towards sine wave in linear
> systems. I tried to prove this idea mathematically, but I did not get very
> far.
> I hope this question has not been asked befor too many times, if so, I
> would thank you for pointing me in the right direction (WWW-pages, FAQ,
> etc). I tried searching google, but i was not able to find anything
> relevant.
> Thanks in advance
> Till
> 

The output of a linear (time-invariant) system consists of its inputs
modified in some way. The additive property lets us separate the input
into various parts, compute (or otherwise determine) how the system
modifies each of those parts at its output, and reassemble ("superpose")
the individual parts to get the entire output.

Arbitrary waveforms can be created by assembling sinusoids of
appropriate frequencies and phases. You can think of sinusoids as the
basic building blocks of waveforms. A physical manifestation of this is
that the signal that emerges from a very sharp filter (or one that has
been passed through a moderately selective filter many times) is a
sinusoid; just one of the many components that may have been part of the
input.

Since a LTIV system adds no new frequency components to a signal passed
through it, and a sinusoid is a single frequency, then when a single
sinusoid passes through such a system, the output must be a sinusoid of
the same frequency. There may be delay, which amounts to a phase shift,
but if there is any output at all, it must have the same frequency as
the input.

If you prefer a mathematical analysis, consider that a linear
time-invariant system can be described by a homogenous linear
differential equation with constant coefficients (practically a
tautology!). Now consider the outcome when applying a sinusoidal
forcing function to such an equation.

Jerry
--
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

"Till Crueger" <TillFC@gmx.net> wrote in message
news:c41ho7$13u4$1@f1node01.rhrz.uni-bonn.de...
> Hi,
> I have some simple questions about the properties of linear systems. In a
> linear System we have to assume the properties of homogeneity, additivity
> and shift invariance.
> In a DSP book I read that from these properties you can conclude that a
> sine wave as input will always produce a sine wave on the systems output
> with the same frequency. I can't really see what speciality of sine waves
> should account for this special behaviour towards sine wave in linear
> systems. I tried to prove this idea mathematically, but I did not get very
> far.
> I hope this question has not been asked befor too many times, if so, I
> would thank you for pointing me in the right direction (WWW-pages, FAQ,
> etc). I tried searching google, but i was not able to find anything
> relevant.
> Thanks in advance
> Till

Till,

First, let's stick to linear (continuous) systems and leave DSP to another
post or discussion.
You will find that linear systems are described with ordinary linear
differential equations with constant coefficients.
(You will find that the analysis you refer to is steady-state analysis.  The
sine wave you refer to is of infinite extent).
The equations allow for formulation with the Laplace Transform - so looking
at Laplace Transform theory or proofs in linear systems texts would probably
be the thing to do.

At a more arm-waving intuitive level, an electrical linear system is made up
of resistors, capacitors and inductors plus idealized scale changers
(amplifiers), voltage and current sources, transformers, etc.  It's pretty
easy to see that these behave differently at different frequencies and that
combinations of them will behave very differently at different frequencies.

A single sine wave as an excitation to a linear system results in a single
sine wave as a response.
A square wave can be represented as an infinite Fourier Series - a
combination of harmonically related sine waves.
So, if a square wave were to be used as a system input then the single input
/ single output idea would no longer be possible.  The output would have to
be represented as something other than square waves - and, an infinite set
of harmonically related sine waves is the natural result.  That's why square
waves don't form a useful basis for linear system analysis.

The form of sine waves is preserved in linear systems - which admits to
scale and phase representations.  The form of other waveforms generally
isn't preserved.

Fred

Randy Yates wrote:

> Hey Till,
> 
> You're right - it isn't just sine waves that a linear system will not
> produce new frequencies of - ANY waveform will be unaltered in
> frequency by a linear system. 

Randy,

This is true in a sense, but misleading. For example, you can't expect
the output of a linear system to be a square wave just because the input
is excited by one. The output may contain all the component frequencies
of the input, but shape isn't necessarily maintained. Sinusoids are
exceptional in that shape is retained. Sometimes it's hard for us to
remember how special that property really is.
> 
> The special thing about sine waves are that they provide a fundamental
> set of functions from which ANY other waveform can be constructed, so
> if they aren't moved in frequency by a linear system, neither are any 
> other waveforms. 

That says what I mean, but not as explicitly as I think is needed. I
hope I'm not treading on sensitive toes.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

Till Crueger wrote:

   ...

> Furthermore I thought about a system which only would double any frequency
> component present in a given Signal, and I fail to see how this system is
> non-linear.
> Till

Describe your system, and we'll show you how it's nonlinear. You can
show that yourself if you like. Write an equation that relates output to
input. Only equations with linear terms describe linear circuits.
(Piecewise linear won't do.)

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

"Till Crueger" <TillFC@gmx.net> wrote in message
news:c41ho7$13u4$1@f1node01.rhrz.uni-bonn.de...
> I have some simple questions about the properties of linear systems. In a
> linear System we have to assume the properties of homogeneity, additivity
> and shift invariance.
> In a DSP book I read that from these properties you can conclude that a
> sine wave as input will always produce a sine wave on the systems output
> with the same frequency. I can't really see what speciality of sine waves
> should account for this special behaviour towards sine wave in linear
> systems. I tried to prove this idea mathematically, but I did not get very
> far.

Till,

If you are looking for a textbook treatment, I would recommend "Signals &
Systems" by Oppenheim & Willsky. I don't know of a good web site, but
perhaps if you search for some university classes you'll find some notes.

I can give you a condensed version of one possible mathematical
interpretation. Keep in mind that there are currently no DSP algorithms
named after me though. ;-)

1.) For linear time-invariant (LTI) systems, the system response is given by
the convolution of the impulse response with the input. (If you think of the
input signal as being composed of a sum of scaled and shifted impulse
functions, it is possible to see how you can characterize the output using
only the impulse response of the system. A picture is worth a thousand words
here.) Convolution takes the form of an integral for the continuous time
case and a summation for the discrete time case. I personally find it easier
to understand what you are after by convincing myself of this first and
using the mathematical formulation of the convolution.
2.) For input signals that have the form of a complex exponential (exp(st)
for continuous time signals or z**n for discrete time signals, where s and z
are complex), the output of an LTI system will be the same complex
exponential multiplied by a value that is a function of s (or z). You can
prove this mathematically by evaluating the convolution integral (or
summation) for a complex exponential input.

Of course, since the system is linear, the response to a sum of any number
of inputs will be the sum of the individual outputs. You can create a sine
wave using Euler's relation

exp(jw) = cos(w) + jsin(w)

by subtracting

exp(-jw) = cos(w) - jsin(w)

and dividing both sides by 2j to obtain

sin(w) = 1/2j(exp(jw) - exp(-jw)).

In general, if you can represent a periodic signal as a sum of complex
exponentials (i.e. a Fourier Series), the output can be obtained by summing
the system response of the individual terms.

Hope this helps,

Jeremy Furtek
Delphi Research
jeremy at delphiresearch dot com

Jerry Avins <jya@ieee.org> writes:

> Randy Yates wrote:
>
>> Hey Till,
>> You're right - it isn't just sine waves that a linear system will not
>> produce new frequencies of - ANY waveform will be unaltered in
>> frequency by a linear system.
>
> Randy,

Jerry,

I hear you (somewhat) but I'm going to play the opposite side here to
the hilt.

> This is true in a sense, 

This is true absolutely. No "sense" or interpretation required.

> but misleading. 

How can the truth be misleading? 

> For example, you can't expect
> the output of a linear system to be a square wave just because the input
> is excited by one. 

I do not expect that. What I do expect is that the output
will not contain any frequencies that weren't in the
input. 

> The output may contain all the component frequencies
> of the input, but shape isn't necessarily maintained. 

Did I say or imply the _shape_ was maintained? In fact I did not.

But beyond the question of what I said or didn't say, your comments seem
to be aimed at how one should explain something, and THAT depends on 
style and technique. This is my style. Making the truth sharp, in my
experience, usually dispells bad conclusions and sheds light on wrong
thinking.
-- 
%  Randy Yates                  % "So now it's getting late,
%% Fuquay-Varina, NC            %    and those who hesitate
%%% 919-577-9882                %    got no one..."
%%%% <yates@ieee.org>           % 'Waterfall', *Face The Music*, ELO
http://home.earthlink.net/~yatescr