Low freq "analog" of Nyquist? ( possibly naive question )

"Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in message
news:a3iOa.2454$Jk5.1463040@feed2.centurytel.net...
> "Glen Herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message
> news:qSaOa.58648$fG.41271@sccrnsc01...

(snip)

> > The reason sin() and cos() are popular basis functions is because they
are
> > solutions for the simplest second order differential equation, and that
> > equation comes up in many places.
> >
> > Different differential equations have different solutions and different
> > basis functions will be used.  Problems with cylindrical symmetry have
> > Bessel function solutions, for example.  Sometimes it is easier to build
> > something cylindrical, and whether one likes Bessel functions or not (I
> > don't especially like them), one is stuck with them.
> >
> > > I tried to define f(t) but made an error.  The objective was to define
a
> > > function that would be zero at t=0 and at t=T
> > > and in order to respectfully
> > > create a counter example to your assertion about things
> > > zero at these points.  I should have said:
> > >
> > > f(t)=cos(pi*t/T) - cos(3*pi*t/T) + sin(3*pi*t/T)
> > > Using your notation with more definition:
> > > w=pi/T
> > > f(t)=sum of A(n)cos(nwt) + B(n)sin(nwt); n=0 to inf
> > > A(0) is 0; A(1)=1.0, A(2)=0, A(3)=-1.0 all other A(n)=0
> > > B(3)=1.0
> > >
> > > OK, I think I have it right this time....
> > > The sum is zero at t=0 and t=T.
> > >
> > > But, it seems that you proscribe a function with this structure
because
> it
> > > isn't the solution to a differential equation?  Do I understand?
> > > The function above meets the "zero at the ends" criteria even
> > > with cosines in
> > > the sum because the two cosines sum to zero at the ends.
> > > The added requirement about being able to remove one
> > >  of the terms and still have zero is a new requirement...
> > > and is restrictive for what we're talking about.
> > >
> > > Being the solution to a differential equation isn't necessary in this
> > > discussion or I'm really going to learn something interesting!!  I
think
> > > this because *any* sinusoid is fair game as a signal component - not

As far as cos(x)-cos(3 x) being a solution for the differential equation
with f(0)=f(pi)=0, I believe it can't be because f'(0)=0.  The only solution
with f(0)=0 and f'(0)=0 is f(x)=0.

The question I was trying to ask was solutions to a differential equation.
That is the reason for all these problems.  A signal running down a coax
cable, a sound wave travelling through air, a light wave travelling though
space, or a vibration travelling down a violin string, are all solutions to
the same differential equation, and the solutions are sums of sin() and
cos() satisfying the boundary conditions.

Now, consider the vibrations of a drum head.  This turns out to be a
different equation, and the solutions are not sin() and cos().  The
different modes are not integer multiples of each other, which makes the
music of drums much different than of string or wind instruments.

Now, sometimes it is an interesting question, why do such problems turn out
to be solutions of simple differential equations?  Maybe it is just that we
are lucky.  I think also, though, that a world where some of these things
weren't true wouldn't be very livable.

I had hoped that some others would get into this discussion, but no-one else
did.

-- glen

"Glen Herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message
news:CKnOa.1544$N7.405@sccrnsc03...

(snip related to solutions of f ''(x) + w**2*f(x)=0 )

> As far as cos(x)-cos(3 x) being a solution for the differential equation
> with f(0)=f(pi)=0, I believe it can't be because f'(0)=0.  The only
solution
> with f(0)=0 and f'(0)=0 is f(x)=0.
>
> The question I was trying to ask was solutions to a differential equation.
> That is the reason for all these problems.  A signal running down a coax
> cable, a sound wave travelling through air, a light wave travelling though
> space, or a vibration travelling down a violin string, are all solutions
to
> the same differential equation, and the solutions are sums of sin() and
> cos() satisfying the boundary conditions.

Continuing on, there can only be two conditions for a second order equation.

The equation itself says that f''(x)=0 when f(x)=0, but your solution
doesn't satisfy that.

Otherwise, if you go back to the partial differential equation and
separation of variables, the w**2 term comes when the separation of
variables is done.   The solution to the partial differential equation can
have cos(x) and cos(3x) terms, but they will each have different time
dependencies.    If there isn't any dispersion (a different equation if
there is), all waves move at the same velocity.  Higher spatial frequency
(higher k in sin(k*x)) have higher time frequency (higher w).  Terms can't
mix frequencies that aren't proportional.

-- glen

"Glen Herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message news:<1rRMa.2466$NW6.2669@rwcrnsc51.ops.asp.att.net>...
> "Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in message
> news:%cQMa.2283$Jk5.1212136@feed2.centurytel.net...

> > 1) You can take a Fourier transform of any set of samples however they may
> > be spaced, but it would be a regular, continuous, Fourier Transform - not
>  a
> > DFT.
> 
> Maybe I was mixing too many things into the discussion.  That was a claim
> that Nyquist doesn't require uniform sampling.   The appropriate number of
> sample points are mathematically enough, though as an engineering solution
> they aren't very good.   So lets consider a DFT.  (I haven't tried this
> before.)  The Fourier series tranforms a periodic function into a discrete
> series of frequencies.  If the source function represents sampled data
> instead of a continuous function, even with non-uniform sampling, we can
> still ask what series of sin() and cos() satisfy such samples.  If we have N
> sample points there is no use in asking for more than N frequency
> components.  There is also no restriction to choose the N lowest
> frequencies.  Given N frequencies, periodic with the required period, and N
> sampling points we can reconstruct the amplitude at those points.   Hmm, I
> don't see any restriction that the sample points be uniformly spaced.   The
> sin() and cos() must be harmonics of the appropriate fundamental, but I can
> use any N such that I desire.
> 
> Now, FFT has some uniform requirements, and it is certainly a better way to
> do it, but I don't see such a restriction on DFT.

The DFT is a mapping of a signal to a vector basis that is orthogonal. 
Messing with the sampling interval *may* mess up the orthogonality of the 
basis vectors. I am not at all sure about that, I only know that the 
integer-fraction phase increments between samples are very convenient for 
proving that the Fourier basis is indeed orthogonal.

Rune

"Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in message news:<vT%Ma.2287$Jk5.1243587@feed2.centurytel.net>...

> Consider the continuous Fourier Transform of such a function - as you have
> done.
> You said:
> "Consider a signal, f, sampled over time T, from t=0 to t=T.  Assume that
> f(0)=f(T)=0 for now.  All the components must be sine with periods that are
> multiples of 2T."
>
> Hmmmm.... I'm not so sure about this.  Here's a counter example:
> This is a waveform, that has zeros at 0 and T.
> cos(pi*T) - cos(3*pi*T) + sin(3*pi*T)
> The sine has period 3*pi*T/2*pi = (3/2)*T which is not a multiple of 2*T.

With clamped end points as with the string, the period must be a multiple 
of T. The length of the string must be expressed in terms of half 
wavelengths. 

> and there are cosines.

It may look like it does, but no. In my table of maths formulas, I find  

     sin(A)*sin(B)=1/2*(cos(A-B)-cos(A+B)).

I think your example will pop out from A=2*pi*T, B=pi*T.

Rune

"Rune Allnor" <allnor@tele.ntnu.no> wrote in message
news:f56893ae.0307071924.58c6909e@posting.google.com...
> "Glen Herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message
news:<1rRMa.2466$NW6.2669@rwcrnsc51.ops.asp.att.net>...

(snip)

> > Now, FFT has some uniform requirements, and it is certainly a better way
to
> > do it, but I don't see such a restriction on DFT.
>
> The DFT is a mapping of a signal to a vector basis that is orthogonal.
> Messing with the sampling interval *may* mess up the orthogonality of the
> basis vectors. I am not at all sure about that, I only know that the
> integer-fraction phase increments between samples are very convenient for
> proving that the Fourier basis is indeed orthogonal.

The basis functions must be linearly independent, and the set of basis
functions should be complete.  It is not required of basis functions that
they be orthogonal, or orthonormal.  Both are conveniences that make the
rest of the math easier.

Harmonics of sin() and cos() are orthonormal basis functions so that part is
met.

One can, however, evaluate a sum of sin() and cos() at arbitrary points.

That is, F(k)=Sum(j=1 to N)  f(x(j))*exp(i k x(j))    where x(j) are a set
of non-uniformly spaced points, though the sum could be evaluated for
uniformly distributed k.

So one question was whether this was still a DFT, or not.  (It is discrete,
and is still a Fourier transform, as far as I know.)

The inverse transform can be performed, and evaluated at the desired
positions.   The FFT algorithm cannot be used for cases like this, which is
one reason not to use it.

-- glen

"Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in message news:<a3iOa.2454$Jk5.1463040@feed2.centurytel.net>...

> When you say:
> >A complete set of basis functions is sufficient to solve the equations.
> > That is why things like Fourier transform work.
> 
> I'm not really sure which "equations" you're referring to here.
> Now, I know that the Fourier Transform is a nice tool to use for solving
> differential equations - as is the Laplace Transform.
> At the same time, I believe that the Fourier Transform is useful in
> analyzing and manipulating waveforms of a very general variety that go
> beyond situations of differential equations with nice boundary conditions.

The differential equations need boundary conditions to be completely 
specified. As with the string, one can say that the solutions are 
"on the form s(t)=Acos(wt)+Bsin(wt)" where A, B and w must be computed 
from initial and boundary conditions. Real Analysis, the "generalized 
Fourier theory" if you like, investigates what properties solutions
to some "simple" differential equations have, and it turns out that 
those basis functions that also are solutions to diff.eqs. (sinusoidals,
Bessel functions, legendre polynomials,...) have very convenient 
properties.  

> I don't think you'll find trig identities that will work because the
> functions I provided have infinite derivatives at the edges - well, I think
> that's the correct way to put it.  There is 1/2 cycle of a sine in one term
> and 3/2 cycle of a sine in another term.
> Now, if I relate these terms to a vibrating string with nodes at the ends,
> then there must be an "image" vibrating string that moves in "opposition"
> (?) that takes up another string length in order for the entire span of the
> real string and the image string to make up an entire "period".  And, these
> are standing waves.  If there's an excitation that is not at a resonant
> frequency then there will be a travelling wave, right?  Not my field of
> expertise to express or manipulate these equations but the analogy seems OK.

You will get a somewhat more messy expression for the solution, but 
the basic technique of expressing what's going on in terms of Fourier 
series is still valid. 
 
> So, one might double the frequency span by doubling the temporal epoch and
> encompass the sin(t) and sin(3*t/2) types of terms and have continuous
> derivatives at the edges of the newly defined "period".  But there's still
> the possibility of sin(pi*t) which will surely not fit in any period you
> might define if it includes the other two terms.....
> 
> So, I'm still stuck on the assertion that time-limiting a function or,
> equivalently, having a time-limited function has nothing to do with
> frequency domain sampling - whether the time function is zero at the ends or
> not.

Sampling and differential equations are two different discussions. 

> The frequency domain sampling can be viewed as a result of considering the
> time-limited function to be a single period of a periodic waveform.  That
> should cause no problems but the Fourier Series *can* be of infinite extent.

If the periodic function is piecewise continuous.
 
> If the Fourier Series is indeed infinite, then subsequent temporal sampling
> will cause frequency domain aliasing and the character of the function is
> (usually irrevocably) changed. 

Thats what the sampling theorem says.

> If this change is accepted, then there is
> generally a periodic time / periodic frequency - discrete frequency /
> discrete time representation that's accepted.  However, accepting the
> aliasing is necessary  to be able to assert that an arbitrary time-limited
> function can be expressed as a finite discrete sequence or periodic sequence
> in frequency.

Accepting sampling of piecewise contnuous functoins is accepting aliasing.

Rune

"Glen Herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message news:<CKnOa.1544$N7.405@sccrnsc03>...
> As far as cos(x)-cos(3 x) being a solution for the differential equation
> with f(0)=f(pi)=0, I believe it can't be because f'(0)=0.  The only solution
> with f(0)=0 and f'(0)=0 is f(x)=0.

This is the Wronskian, right? It doesn't it help to reformulate as 
f(x)=sin(x)sin(2x). I get the derivative to be 

   f'(x)=cos(x)sin(2x)+2sin(x)cos(2x)

which also yields a singular Wronskian. Which means 

   f(x)=cos(x)-cos(3x)=2sin(x)sin(2x)

is invalid as solution to the differential equation.

> The question I was trying to ask was solutions to a differential equation.
> That is the reason for all these problems.  A signal running down a coax
> cable, a sound wave travelling through air, a light wave travelling though
> space, or a vibration travelling down a violin string, are all solutions to
> the same differential equation, and the solutions are sums of sin() and
> cos() satisfying the boundary conditions.

Which is a very good argument for using Fourier series to analyze 
those kinds of signals.

> Now, consider the vibrations of a drum head.  This turns out to be a
> different equation, and the solutions are not sin() and cos().  The
> different modes are not integer multiples of each other, which makes the
> music of drums much different than of string or wind instruments.

Those who deal with underwater acoustics may have heard about "Normal 
Modes". These very useful functions appear as solutions to the depth 
equation in the separable partial differential equation that describe 
the acoustic propagation in the shallow sea. The link between the 
propagation properties of the recorded signal and the poles of the 
"medium filter" (the propagation Green's function) is almost eerie.

While there usually doesn't exist any analytical expression for the 
modes, the fact that they originate as solutions in a PDE ensures 
that they have certain properties like orthogonality (which means 
that power estimates can be done, since Parseval's theorem is valid), 
and completeness (any physically reasonable signal can be expressed 
in terms of modes and propagated in a channel).
 
Once you know the mode functions (from models or measurements), all you 
need to do is to map the signal you want to study onto those basis 
functions by means of inner products. Orthogonality between the modes 
ensure that you don't even have to worry about cross terms. 

There are some problems that you need to look out for, though. While 
the normal modes do come in countable sets, they aren't always counted
quite the way one expects. A few years ago I saw a study published by 
somebody who worked with long-range propagation problems in the northern 
Pacific Ocean. The source was at Hawaii and the reciever outside Alaska.
Somewhere along the propagation path there was an oceanographic front
where the sound speed profile in the water changed from "temperate" to 
"arctic". The authors reported that they had experienced great difficulties 
getting their results to make sense at first. When they examined the data 
and the propagation models on both sides of the fronts, they found that
some very subtle changes in the models made the modes "switch places"
from one side to the other. Mode 0 on the south side of the front didn't 
look anything like mode 0 in the north, but was virtually identical to 
mode 3. An almost trivial observation, but just about enough to play havoc
with the data if you aren't aware of it. These effects were reported in

Shang and Wang: "Subarctic frontal effects on long-range acoustic 
     propagation in the Northern Pacific Ocean", 
     Journ. Ac. Soc. Am., Vol 105 no 3, pp 1592, March 1999. 

> Now, sometimes it is an interesting question, why do such problems turn out
> to be solutions of simple differential equations?  Maybe it is just that we
> are lucky.  I think also, though, that a world where some of these things
> weren't true wouldn't be very livable.
> 
> I had hoped that some others would get into this discussion, but no-one else
> did.
 
If this isn't trolling, nothing is...  ;)

Rune

"Rune Allnor" <allnor@tele.ntnu.no> wrote in message
news:f56893ae.0307072142.59e3861f@posting.google.com...
> "Glen Herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message
news:<CKnOa.1544$N7.405@sccrnsc03>...
> > As far as cos(x)-cos(3 x) being a solution for the differential equation
> > with f(0)=f(pi)=0, I believe it can't be because f'(0)=0.  The only
solution
> > with f(0)=0 and f'(0)=0 is f(x)=0.
>
> This is the Wronskian, right? It doesn't it help to reformulate as
> f(x)=sin(x)sin(2x). I get the derivative to be
>
>    f'(x)=cos(x)sin(2x)+2sin(x)cos(2x)
>
> which also yields a singular Wronskian. Which means
>
>    f(x)=cos(x)-cos(3x)=2sin(x)sin(2x)
>
> is invalid as solution to the differential equation.

I almost forgot about the Wronskian.   Almost...

> > The question I was trying to ask was solutions to a differential
equation.
> > That is the reason for all these problems.  A signal running down a coax
> > cable, a sound wave travelling through air, a light wave travelling
though
> > space, or a vibration travelling down a violin string, are all solutions
to
> > the same differential equation, and the solutions are sums of sin() and
> > cos() satisfying the boundary conditions.
>
> Which is a very good argument for using Fourier series to analyze
> those kinds of signals.

(snip)

> Those who deal with underwater acoustics may have heard about "Normal
> Modes". These very useful functions appear as solutions to the depth
> equation in the separable partial differential equation that describe
> the acoustic propagation in the shallow sea. The link between the
> propagation properties of the recorded signal and the poles of the
> "medium filter" (the propagation Green's function) is almost eerie.

I almost got to normal modes, but I wasn't sure that it would help the
argument any.   Also, I was trying to do it in the direction of separable
PDE's which, as far as I remember, is the way Fourier did it, but yes, they
are normal modes.

> While there usually doesn't exist any analytical expression for the
> modes, the fact that they originate as solutions in a PDE ensures
> that they have certain properties like orthogonality (which means
> that power estimates can be done, since Parseval's theorem is valid),
> and completeness (any physically reasonable signal can be expressed
> in terms of modes and propagated in a channel).
>
> Once you know the mode functions (from models or measurements), all you
> need to do is to map the signal you want to study onto those basis
> functions by means of inner products. Orthogonality between the modes
> ensure that you don't even have to worry about cross terms.

(snip)

> > Now, sometimes it is an interesting question, why do such problems turn
out
> > to be solutions of simple differential equations?  Maybe it is just that
we
> > are lucky.  I think also, though, that a world where some of these
things
> > weren't true wouldn't be very livable.
> >
> > I had hoped that some others would get into this discussion, but no-one
else
> > did.
>
> If this isn't trolling, nothing is...  ;)

Well, I was hoping for constructive discussion.  I usually consider trolling
as non-constructive discussion.  Though sometimes I manage to make an
interesting discussion out of a troll, anyway.

Thanks for the Wronskian reminder, though.

-- glen

"Rune Allnor" <allnor@tele.ntnu.no> wrote in message
news:f56893ae.0307071953.6603dfa8@posting.google.com...
> "Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in message
news:<a3iOa.2454$Jk5.1463040@feed2.centurytel.net>...
>
> > When you say:
> > >A complete set of basis functions is sufficient to solve the equations.
> > > That is why things like Fourier transform work.
> >
> > I'm not really sure which "equations" you're referring to here.
> > Now, I know that the Fourier Transform is a nice tool to use for solving
> > differential equations - as is the Laplace Transform.
> > At the same time, I believe that the Fourier Transform is useful in
> > analyzing and manipulating waveforms of a very general variety that go
> > beyond situations of differential equations with nice boundary
conditions.
>
> The differential equations need boundary conditions to be completely
> specified. As with the string, one can say that the solutions are
> "on the form s(t)=Acos(wt)+Bsin(wt)" where A, B and w must be computed
> from initial and boundary conditions. Real Analysis, the "generalized
> Fourier theory" if you like, investigates what properties solutions
> to some "simple" differential equations have, and it turns out that
> those basis functions that also are solutions to diff.eqs. (sinusoidals,
> Bessel functions, legendre polynomials,...) have very convenient
> properties.
>
> > I don't think you'll find trig identities that will work because the
> > functions I provided have infinite derivatives at the edges - well, I
think
> > that's the correct way to put it.  There is 1/2 cycle of a sine in one
term
> > and 3/2 cycle of a sine in another term.
> > Now, if I relate these terms to a vibrating string with nodes at the
ends,
> > then there must be an "image" vibrating string that moves in
"opposition"
> > (?) that takes up another string length in order for the entire span of
the
> > real string and the image string to make up an entire "period".  And,
these
> > are standing waves.  If there's an excitation that is not at a resonant
> > frequency then there will be a travelling wave, right?  Not my field of
> > expertise to express or manipulate these equations but the analogy seems
OK.
>
> You will get a somewhat more messy expression for the solution, but
> the basic technique of expressing what's going on in terms of Fourier
> series is still valid.
>
> > So, one might double the frequency span by doubling the temporal epoch
and
> > encompass the sin(t) and sin(3*t/2) types of terms and have continuous
> > derivatives at the edges of the newly defined "period".  But there's
still
> > the possibility of sin(pi*t) which will surely not fit in any period you
> > might define if it includes the other two terms.....
> >
> > So, I'm still stuck on the assertion that time-limiting a function or,
> > equivalently, having a time-limited function has nothing to do with
> > frequency domain sampling - whether the time function is zero at the
ends or
> > not.
>
> Sampling and differential equations are two different discussions.
>
> > The frequency domain sampling can be viewed as a result of considering
the
> > time-limited function to be a single period of a periodic waveform.
That
> > should cause no problems but the Fourier Series *can* be of infinite
extent.
>
> If the periodic function is piecewise continuous.
>
> > If the Fourier Series is indeed infinite, then subsequent temporal
sampling
> > will cause frequency domain aliasing and the character of the function
is
> > (usually irrevocably) changed.
>
> Thats what the sampling theorem says.
>
> > If this change is accepted, then there is
> > generally a periodic time / periodic frequency - discrete frequency /
> > discrete time representation that's accepted.  However, accepting the
> > aliasing is necessary  to be able to assert that an arbitrary
time-limited
> > function can be expressed as a finite discrete sequence or periodic
sequence
> > in frequency.
>
> Accepting sampling of piecewise contnuous functoins is accepting aliasing.

Rune,

I feel we've been sucked into a long discussion about nothing.

Glen's assertion:

"Consider a signal, f, sampled over time T, from t=0 to t=T.  Assume that
f(0)=f(T)=0 for now.  All the components must be sine with periods that are
multiples of 2T. "

was what started this lengthy discussion.  What preceded this was a question
about what happens to spectral analysis if you take a relatively short
temporal sample length.

I disagreed with the assertion above and mostly ignored the introduction of
solutions of differential equations.  I disagree that "all of the components
must be sine with periods that are multiples of 2T" - don't you?  I would
rather say that f can be "anything reasonable" and unrelated to T in
general.  That's the situation one has in normal DSP practice.

There are counter examples given earlier in the thread ....... including
sin(t) + sin(pi*t)

It's a circular discussion to now change and say: but gee, we were talking
about solutions to differential equations in the first place - because we
weren't.  So, defending the assertion on the basis that it matches up with
solutions to differential equations is not to the point.

Perhaps we're simply talking about two different things.....

Bye.

Fred

"Glen Herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message news:<g4tOa.5154$ye4.9486@sccrnsc01>...
> "Rune Allnor" <allnor@tele.ntnu.no> wrote in message
> news:f56893ae.0307072142.59e3861f@posting.google.com...
> > "Glen Herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message
>  news:<CKnOa.1544$N7.405@sccrnsc03>...
> > > I had hoped that some others would get into this discussion, but no-one
>  else
> > > did.
> >
> > If this isn't trolling, nothing is...  ;)
> 
> Well, I was hoping for constructive discussion.  I usually consider trolling
> as non-constructive discussion.  Though sometimes I manage to make an
> interesting discussion out of a troll, anyway.

You are, of course, right. I was merely failing to make a joke over the 
fact that your sigh over no-one joining in was exactly what made me
review the tread and fill in here and there. 

> Thanks for the Wronskian reminder, though.

Y'welcome.

Rune

> -- glen