LTI vs non-LTI

>Tim Wescott <tim@seemywebsite.com> wrote:
>> On Mon, 29 Apr 2013 07:50:10 -0500, manishp wrote:
> 
>>>>   Upsampling and downsampling are time variant
> 
>>> Sir, can you please elaborate?
>
>(snip)
>> Sampling is time varying because the input is ignored for all time
except 
>> the sampling instant.  If you're willing to use impulse functionals in 
>> your analysis then sampling is just a form of mixing, in the linear,
time-
>> varying sense that I was advocating earlier.
> 
>> Downsampling in the discrete-time world is much like sampling from 
>> continuous-time to discrete time, and can be modeled similarly.  If 
>> you're willing to view continuous-time as the ultimate in discrete-time

>> domains, then it's exactly the same (but taking that view is a hell of a

>> leap, and not to be taken lightly without really thinking through the 
>> mathematical implications).
>
>Well, assuming that the signal has the appropriate band limiting.
>
>Consider starting with a continuous time signal that is perfectly
>band limited. Now sample it at different sampling rates above the
>Nyquist rate, with an ideal sampler. All then contain the exact same
>information, which is that in the original signal. 
>

While it would contain all the information required to reconstruct the
original signal, the sampled signal would be dependent on when the sampling
occured, hence time variant.
 
>> I seem to be disagreeing with Rick a lot these days, but upsampling by
an 
>> integer amount (or, for that matter reconstructing to continuous time),

>> when expressed the way I normally view it, is not time-varying.  Rather,

>> it is just taking the "train-of-impulses" and running them through a
hold 
>> filter.
> 
>> If you view the discrete-time domain as entirely separate from the 
>> continuous-time domain, or if you view two different discrete-time 
>> domains as being disjoint and not related, then upsampling by an integer

>> amount either is time varying, or the concept is invalid -- I'm not sure

>> which.
> 
>> Upsampling by a non-integer amount _is_ time varying.  The easiest way
to 
>> determine this is by noting that upsampling by any rational but non-
>> integer ratio is tantamount to upsampling by an integer ratio, then 
>> downsampling by some other integer ratio -- and that downsampling step
is 
>> time-varying, which "poisons" the whole process.
>
>Maybe I don't understand "time varying" in the sense you use it here,
>but up sampling by a rational factor should give the same signal you
>would have seen sampling the original at the higher rate. (Assuming the
>filters are all ideal, which they usually aren't, except in homework
>problems.)
>
>-- glen
>

Agreed.  The upsampled signal would even be identical to the original
sampled signal in the time domain.  Bandlimiting this signal would recreate
the original signal sampled at the original sampling frequency times the
upsampling ratio.  The process thus far is time invariant.  But if
downsampling occured to obtain a non-integer upsampling ratio, then it
would still be dependent on when sampling occurs, which would again be time
variant.

Time invariance means y(t+t0) == H{ x(t+t0) } for all t0.  Where x is the
input, H is the system and y is the output.  It doesn't necissarily mean
that the input signal x can or cannot be reconstructed from the output y.

On Mon, 29 Apr 2013 18:32:15 +0000 (UTC), glen herrmannsfeldt
<gah@ugcs.caltech.edu> wrote:

>Piergiorgio Sartor <piergiorgio.sartor.this.should.not.be.used@nexgo.removethis.de> wrote:
>> On 2013-04-29 04:14, glen herrmannsfeldt wrote:
>> [...]
>>> Also, median is supposed to be better than mean, but again
>>> the math is easier for mean.
> 
>> Better for what? In which sense?
>
>Better in the statistical, or statistician sense.

Mean and Median are different measures that provide different
information.   Whether one is "better" than the other depends on what
the user is looking for.   I think the question was what criterion you
were using to determine the contexts for when median is "better".

It certainly isn't better when you need the mean.


>> OF course, median returns (if odd) a value
>> belonging to the set, but this does not mean
>> this value is better. It depends.
>
>If you know that the data samples a Guassian distribution,
>then mean and standard deviation should have the appropriate
>statistical properties. 
>
>Often, though, they are used where the data is known not to be
>Gaussian, but where they are easier to compute.
>
>The distribution of home sale prices is not Gaussian, and you will 
>more often see median used in economic reports on home prices.
>
>-- glen

That's because the information that the median provides for that
situation is useful, and it is distinct precisely because it is
different than the mean.

The mean is a useful measure in many cases when the distribution isn't
gaussian.   So is the median.   They're different, so I think "better"
is in the eye of the beholder.


Eric Jacobsen
Anchor Hill Communications
http://www.anchorhill.com

Eric Jacobsen <eric.jacobsen@ieee.org> wrote:

(snip, I wrote)
>>>> Also, median is supposed to be better than mean, but again
>>>> the math is easier for mean.
 
>>> Better for what? In which sense?

>>Better in the statistical, or statistician sense.
 
> Mean and Median are different measures that provide different
> information.   Whether one is "better" than the other depends on what
> the user is looking for.   I think the question was what criterion you
> were using to determine the contexts for when median is "better".
 
> It certainly isn't better when you need the mean.

But why do you think you need the mean? 

If someone gives you some sets of numbers, and asks for
one number to best represent each set, without telling you
were the numbers came from, or what will the resulting
number will be used for, what do you choose?

Some possibilities are mean, median, mode, geometric
mean, harmonic mean, and roll the dice.
Not claiming to be a statistician, but statisticians
that I know like median better.

So, why do people like mean? Not because it is better
statstically, but because it is easier analytically.

Same with least squares fits. Well, mean is the result
of a 0th order least squares fit. That is, the number
that minimizes the sum of the squares between it and
all the numbers in the set. 

So, why least squares instead of least absolute value?
Because the math is easier. You can take the derivative of
the sum of squares, set it to zero, and find the mimimum.
You can do that for any function that is linear in its
coefficients, polynomials being a popular choice.
 
>>> OF course, median returns (if odd) a value
>>> belonging to the set, but this does not mean
>>> this value is better. It depends.

(snip regarding mean)
>>Often, though, they are used where the data is known not to be
>>Gaussian, but where they are easier to compute.

>>The distribution of home sale prices is not Gaussian, and you will 
>>more often see median used in economic reports on home prices.

> That's because the information that the median provides for that
> situation is useful, and it is distinct precisely because it is
> different than the mean.
 
> The mean is a useful measure in many cases when the distribution isn't
> gaussian.   So is the median.   They're different, so I think "better"
> is in the eye of the beholder.

The problem with mean is that it overrepresents the outliers.
In house prices, one expensive sale can make a hugh change in the mean.

-- glen

Tim Wescott <tim@seemywebsite.please> wrote:
> On Sun, 28 Apr 2013 09:58:15 -0700, Rick Lyons wrote:
 
>> I've always thought of mixers as being nonlinear because a mixer's
>> output contains spectral components that did not exist at either of its
>> two inputs.
 
> "Have always thought"?  That's wishy-washy.
 
> Most physical mixers are not linear from the LO port -- generally the 
> local oscillator signal gets badly distorted.  But decent mixers _are_ 
> linear going from the RF port to the IF port, and there's a lot of profit 
> in treating them as such in analysis.

Maybe it depends on what you think of as a mixer. I first got interested
in mixers considering the generation of the stereo subcarrier in FM
broadcasting, and the chrominance subcarrier in NTSC TV signals.(*)
Both uses doubly balanced mixer, which, as well as I understand it,
should be linear from either port to the output. The mixing process
itself is still non-linear, but with one port constant, the output
is linear in the other port. 

For singly balanced mixers, I suppose it only needs to be linear for
one port to the output. 
 
> (Note, too, that mixers are often the first point of distortion in a 
> radio circuit, because you _want_ them to be perfectly linear, but you 
> have to _work_ to make them so.)

(*) In about 1978 I went to the school library to find a book on the 
generation of the NTSC subcarrier. I found in the card catalog (yes, we
still had card catalogs then) a book called "Television Today and
Tomorrow." Sounded like it should cover it, but I didn't notice in the
card catalog the copyright year of 1942. (**) The book was all about 
spinning disks and neon lamps. No CRT and no NTSC.

(**) It seems that there are now at least four books written since 1976
with "Television Today and Tomorrow" in the title. 

-- glen

On Tue, 30 Apr 2013 00:42:40 +0000 (UTC), glen herrmannsfeldt
<gah@ugcs.caltech.edu> wrote:

>Eric Jacobsen <eric.jacobsen@ieee.org> wrote:
>
>(snip, I wrote)
>>>>> Also, median is supposed to be better than mean, but again
>>>>> the math is easier for mean.
> 
>>>> Better for what? In which sense?
>
>>>Better in the statistical, or statistician sense.
> 
>> Mean and Median are different measures that provide different
>> information.   Whether one is "better" than the other depends on what
>> the user is looking for.   I think the question was what criterion you
>> were using to determine the contexts for when median is "better".
> 
>> It certainly isn't better when you need the mean.
>
>But why do you think you need the mean? 

Why would I think I need the median instead?

Next time you go to remove the DC offset of a signal, do you think it
will be "better" to remove the mean or the median?

Compute Expected Value?

Implement a boxcar filter?

Compute power?

RMS?

The median has its place, but when you're trying to work within the
framework of established algorithms and processes I don't see making
things "better" by using something different.   

You haven't addressed what is "better" about the median.


>If someone gives you some sets of numbers, and asks for
>one number to best represent each set, without telling you
>were the numbers came from, or what will the resulting
>number will be used for, what do you choose?

That depends entirely on what "best represent each set" means.   What
does "best" mean here?

>Some possibilities are mean, median, mode, geometric
>mean, harmonic mean, and roll the dice.
>Not claiming to be a statistician, but statisticians
>that I know like median better.

For cases where the median is useful, it is useful.   That doesn't
mean that mean isn't useful.  On average statisticians may not have
the same opinions about a lot of things, but I wouldn't put any weight
on it at all if the mean is what I'm looking for.

>So, why do people like mean? Not because it is better
>statstically, but because it is easier analytically.

Again, "better" without any definition or context.   It's like saying
"optimal" without specifying the optimality criterion.

>Same with least squares fits. Well, mean is the result
>of a 0th order least squares fit. That is, the number
>that minimizes the sum of the squares between it and
>all the numbers in the set. 
>
>So, why least squares instead of least absolute value?
>Because the math is easier. You can take the derivative of
>the sum of squares, set it to zero, and find the mimimum.
>You can do that for any function that is linear in its
>coefficients, polynomials being a popular choice.
> 
>>>> OF course, median returns (if odd) a value
>>>> belonging to the set, but this does not mean
>>>> this value is better. It depends.
>
>(snip regarding mean)
>>>Often, though, they are used where the data is known not to be
>>>Gaussian, but where they are easier to compute.
>
>>>The distribution of home sale prices is not Gaussian, and you will 
>>>more often see median used in economic reports on home prices.
>
>> That's because the information that the median provides for that
>> situation is useful, and it is distinct precisely because it is
>> different than the mean.
> 
>> The mean is a useful measure in many cases when the distribution isn't
>> gaussian.   So is the median.   They're different, so I think "better"
>> is in the eye of the beholder.
>
>The problem with mean is that it overrepresents the outliers.
>In house prices, one expensive sale can make a hugh change in the mean.

When you care about the outliers, because they may have a big affect
on steering a control loop, managing dynamic range, or monitoring the
excursions of a process, or a host of other reasons, the median may
not be a good choice.   The mean and median are different.  Median
shows up where it is useful, mean shows up where it is useful.   You
haven't made much of a case that I can see why median is "better",
which seems to be your argument.   I think it's just "different",
because it is, and people will use whichever works best for them by
their own judgement of what "best" means.

It doesn't hurt that mean is generally easier to compute.   That alone
may make it "better" in many applications.

Median is also a bit less tractable mathematically.   e.g., what is
bin 0 of a DFT?   That's a pretty easy and common computation, so it's
pretty easy to use in analysis.  Median not so much.   That is
probably another reason why it might be "better" for a lot of people
in their judgement of what "better" means for whatever they're doing
at the time.


Eric Jacobsen
Anchor Hill Communications
http://www.anchorhill.com

No love for the mode, eh?

On Apr 29, 9:27�pm, eric.jacob...@ieee.org (Eric Jacobsen) wrote:
>  what is bin 0 of a DFT?

MATLAB user: "whazza 'bin 0'?"

> That's a pretty easy and common computation,

and non-existent in MATLAB (at least with that label).

grumble, grumble...

r b-j

On Mon, 29 Apr 2013 18:45:25 +0000, glen herrmannsfeldt wrote:

> Tim Wescott <tim@seemywebsite.com> wrote:
>> On Mon, 29 Apr 2013 07:50:10 -0500, manishp wrote:
>  
>>>>   Upsampling and downsampling are time variant
>  
>>> Sir, can you please elaborate?
> 
> (snip)
>> Sampling is time varying because the input is ignored for all time
>> except the sampling instant.  If you're willing to use impulse
>> functionals in your analysis then sampling is just a form of mixing, in
>> the linear, time- varying sense that I was advocating earlier.
>  
>> Downsampling in the discrete-time world is much like sampling from
>> continuous-time to discrete time, and can be modeled similarly.  If
>> you're willing to view continuous-time as the ultimate in discrete-time
>> domains, then it's exactly the same (but taking that view is a hell of
>> a leap, and not to be taken lightly without really thinking through the
>> mathematical implications).
> 
> Well, assuming that the signal has the appropriate band limiting.

I certainly hope you're off on a tangent, and not saying anything about 
the signal needing to be bandlimited for my comments to be valid.  I'm 
talking about _systems_, not the signals that may be running through them.

> Consider starting with a continuous time signal that is perfectly band
> limited. Now sample it at different sampling rates above the Nyquist
> rate, with an ideal sampler. All then contain the exact same
> information, which is that in the original signal.

So?  What does that have to do with a discussion about whether a system 
is time varying or not?

>> I seem to be disagreeing with Rick a lot these days, but upsampling by
>> an integer amount (or, for that matter reconstructing to continuous
>> time), when expressed the way I normally view it, is not time-varying. 
>> Rather, it is just taking the "train-of-impulses" and running them
>> through a hold filter.
>  
>> If you view the discrete-time domain as entirely separate from the
>> continuous-time domain, or if you view two different discrete-time
>> domains as being disjoint and not related, then upsampling by an
>> integer amount either is time varying, or the concept is invalid -- I'm
>> not sure which.
>  
>> Upsampling by a non-integer amount _is_ time varying.  The easiest way
>> to determine this is by noting that upsampling by any rational but non-
>> integer ratio is tantamount to upsampling by an integer ratio, then
>> downsampling by some other integer ratio -- and that downsampling step
>> is time-varying, which "poisons" the whole process.
> 
> Maybe I don't understand "time varying" in the sense you use it here,
> but up sampling by a rational factor should give the same signal you
> would have seen sampling the original at the higher rate. (Assuming the
> filters are all ideal, which they usually aren't, except in homework
> problems.)

I am using "time varying" in the sense that the expression

h(x(t), t) = h(x(t-tau), t - tau)

is not true for all tau if the system h is time varying.  What other 
definition is there?

-- 
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
Why am I not happy that they have found common ground?

Tim Wescott, Communications, Control, Circuits & Software
http://www.wescottdesign.com

On Sun, 28 Apr 2013 13:57:23 -0500, Tim Wescott
<tim@seemywebsite.please> wrote:

>On Sun, 28 Apr 2013 09:58:15 -0700, Rick Lyons wrote:
>
>>
>> I've always thought of mixers as being nonlinear because a mixer's
>> output contains spectral components that did not exist at either of its
>> two inputs.
>
>"Have always thought"?  That's wishy-washy.

[Snipped by Lyons]

Whoa Tim.
  I used the phrase "I've always thought of mixers being 
nonlinear" instead of "Tim's post can be misleading."

When I read your 2nd 4/28/13 post it seemed to me 
you left the impression that mixers are linear.  
What I think you should have written was, "Mixers are 
nonlinear.  But it's possible to operate a mixer in 
such a way that the effects of its nonlinearity 
can be kept acceptably small."

My phrase "I've always thought" was my clumsy attempt
to be diplomatic.

See Ya',
[-Rick-]
 


  

On Mon, 29 Apr 2013 07:50:10 -0500, "manishp" <58525@dsprelated>
wrote:

>>   Upsampling and downsampling are time variant 
>
>Sir, can you please elaborate?
>Essentially, you are saying that if I take a sequence, send it to 
>downsampler at different times, I would see different output.
>
>Isn't this an issue in practical applications
>
>Thanks, manish

Hi,
  Concerning downsampling:

From the very nature of its operation, we know if we 
delay the input sequence by one sample, a downsampler 
will generate an entirely different output sequence. 
For example, if we apply an input sequence 
x(n) = x(0), x(1), x(2), x(3), x(4), etc., to a 
downsampler and M = 3, the output y(m) will be the 
sequence x(0), x(3), x(6), etc. Should we delay the 
input sequence by one sample, our delayed xd(n) input 
would be x(1), x(2), x(3), x(4), x(5), etc.
In this case the downsampled output sequence yd(m) 
would be x(1), x(4), x(7), etc., which is not a 
delayed version of y(m). Thus a downsampler is not 
time invariant. 

What this means is that if a downsampling operation 
is in cascade with other operations, we are not 
permitted to swap the order of any of those operations 
and the downsampling process without modifying those 
operations in some way.

Hope that helps.

See Ya',
[-Rick-]