Rigorous definition of the Spectral Density of a random signal?

Carlos Moreno <moreno_at_mochima_dot_com@x.xxx> wrote in message news:<uxzmb.64090$nP4.1108315@weber.videotron.net>...
 
> So, I'm looking for a *rigurous* definition of what the
> values of the PSD really represent.  (notice that I don't
> want the PSD defined as the Fourier transform of the
> auto-correlation of the signal -- what I need is a
> definition of what the value of the PSD at a specific
> frequency means/represents)
> 
>..... material snipped .... 
> 
> What I would want:  "The value of the PDF at x is the
> limit as dx approaches to zero of the probability that
> the variable takes values in (x, x+dx), divided by dx"
> Or:  "The pdf function is such that the probability
> that the variable takes values within a region R is
> given by the integral of the pdf over the region R"
> 
> These are both valid examples of what I would call an
> actual/rigurous definition of what the meaning of the
> value of the PDF is. 



Would you accept the following? 

Consider an ideal narrowbandband filter with center frequency w,
passband gain 1 and bandwidth dw whose input happens to be the
random process in question.  Then, the value of the PSD of the
input process at frequency  w is the limit as dw approaches to
zero of the ratio of the average power of the filter output signal
to the filter bandwidth dw.

or

The PSD is a function S(w) such that the power of the output
signal from a filter (with transfer function H(w)) whose input is
the process is given by the integral of S(w)|H)w)|^2 over the real
line.  If H(w) in an ideal bandpass filter with unit passband gain,
then the power is the integral of S(w) over the passband.

Tom Loredo wrote:
> How about going to the library and looking at any one of many good books
> on the statistics of time series?  Vol. 1 of Priestly is a good one for
> this question.

Well, one thing that promted me to bring the subject for a
"live" discussion with people that really know the subject,
is that a couple of books (ironically, with a common co-
author) give interpretations that, the way I see, are
contradictory.  They do not give a definition of what the
PSD is, but just spit a bunch of blah-blah about what I
would call "the consequences from an intuitive point of
view of what PSD is"  :-\  You see how one can get pretty
frustrated  :-(

I guess I'll look up Priestly's book, to see if I can get
a better perspective to the subject.

Thanks!

Carlos
--

Rune Allnor wrote:

> In my humble opinion, the terms "rigorous" and "statistics" are almost
> contradictions in terms. While the formalities of maths also applies 
> when used in statistsics, I think the interpretations may not be as 
> strict as with other mathemathical diciplines.

That does not mean that probabilities/statistics is not a rigurous
subject -- it simply means that a lot of people do not understand
them and therefore misinterpret it.

I can imagine Kolmogorov right now desperately rolling in his grave
if he heard your comment!!  :-)

>>Is it the average of the total engergy of the 60Hz component
>>over different realizations of the random signal?
> 
> Nope. It's the "Power Spectral Density" we are talking about. If there 
> was a nonzero *energy* component in random noise, electrical energy 
> would be available for free: Just mount a reciever and drain energy out 
> of the blue.

???

Electrical energy *is* available for free -- not in whatever
amount that we may want.  But if you mount a receiver, you *can*
indeed drain energy from it.  You're not "creating" energy out
of the blue -- you're transforming energy from whatever physical
source of the noise, into electrical energy that you now extract;
there's definitely no magic in that.

>>Is it the average of energy density at 60Hz over different
>>realizations of the random signal?
> 
> Nope, due to the energy/density thing commented in the previous point.

Actually, I (doubly) disagree on this point -- there is non-zero
energy component *in a frequency band of non-zero length*.  Notice
that the term I used, "energy density", refers to (informally
speaking) the "amount of energy per unit of frequency  (i.e.,
density in the above definition would be defined as the limit as
delta-omega approaches 0 of the energy contained in the band
(omega, omega + delta-omega) divided by delta-omega).

Notice that with this definition, the energy of a particular
frequency component is zero.

>>Is it the average of the power of the 60Hz component over
>>different realizations of the random signal?
> 
> Almost: It's the average power density of all realizations. 

Ok, here we go...  This can not be right.  I know the name
says "power density", but this simply can not refer to power.

First of all, power is a function of time (the power is an
instantaneous measure of the signal, that takes different
values over time).

But even assuming that you're talking about power as the
average power over the entire realization (i.e., the average
over time), I still disagree!

The Fourier transform of a signal represents *energy* density
(unless we're talking about periodic signals, but that's a
different thing, and it's obviously not the case with random
signals in general).

My interpretation of PSD was always the average of the
Fourier transforms of all possible realizations of the random
signal -- if that's the case, then the values of PSD represent
the average energy density (over all possible realizations).

The notion of "power density" has counter-examples, the way I
see it:  if the PSD of white noise, which is a constant,
represents power per unit of frequency, then the power would
be infinite; that's certainly not the case for Gaussian white
noise;  or white noise where, say, the value at each time is
an independent random variable uniformly distributed between
-1 and 1 (or whatever values).  The power of such signals is
definitely not infinite;  however, they're both white noise,
and they do have a PSD that is a constant for all frequencies.

If we are talking about energy density, then the above leads
to no contradiction:  we integrate with respect to frequency,
and of course we obtain infinity:  the signal has infinite
energy (the signal taken as a whole, from t = -infinity to
+infinity)

Does it make sense what I'm saying?

Carlos
--

Carlos Moreno <moreno_at_mochima_dot_com@x.xxx> wrote in message news:<yPHnb.61923$He4.1598924@wagner.videotron.net>...
> Rune Allnor wrote:
> 
> > In my humble opinion, the terms "rigorous" and "statistics" are almost
> > contradictions in terms. While the formalities of maths also applies 
> > when used in statistsics, I think the interpretations may not be as 
> > strict as with other mathemathical diciplines.
> 
> That does not mean that probabilities/statistics is not a rigurous
> subject -- it simply means that a lot of people do not understand
> them and therefore misinterpret it.
> 
> I can imagine Kolmogorov right now desperately rolling in his grave
> if he heard your comment!!  :-)

My point is only that if you use statistics and rigorous mathemathics 
to describe the probablility distribution function of a fair dice, it
will show a PDF as 

    P(x=n) = 1/6, n=1,2,3,4,5,6                        [1]

i.e. the dice will *on average* show one eye every 6'th time it is rolled. 
A flawed ("strict") interpretation is to say that "it will show one eye 
exactly once in the next six throws". I've heard that kind of statements 
made. 

> >>Is it the average of the total engergy of the 60Hz component
> >>over different realizations of the random signal?
> > 
> > Nope. It's the "Power Spectral Density" we are talking about. If there 
> > was a nonzero *energy* component in random noise, electrical energy 
> > would be available for free: Just mount a reciever and drain energy out 
> > of the blue.
> 
> ???
> 
> Electrical energy *is* available for free -- not in whatever
> amount that we may want.  But if you mount a receiver, you *can*
> indeed drain energy from it.  You're not "creating" energy out
> of the blue -- you're transforming energy from whatever physical
> source of the noise, into electrical energy that you now extract;
> there's definitely no magic in that.

Could you provide examples? Admittedly, a solar panel or a hydroelectric 
generator are examples on types of recievers that actually generate
energy. I was thinking of radio recievers, where the measured signal 
is used to modulate a flow of energy internal to the reciever, energy 
that was provided by the reciever's power supply.

Ok, you got me wondering if I confuse "power" and "energy"... my point 
is that you can't charge a battery by just connecting it to a recieving 
antenna, and drayn energy/power from the static noise the antenna 
recieves from the surroundings.

> >>Is it the average of energy density at 60Hz over different
> >>realizations of the random signal?
> > 
> > Nope, due to the energy/density thing commented in the previous point.
> 
> Actually, I (doubly) disagree on this point -- there is non-zero
> energy component *in a frequency band of non-zero length*.  Notice
> that the term I used, "energy density", refers to (informally
> speaking) the "amount of energy per unit of frequency  (i.e.,
> density in the above definition would be defined as the limit as
> delta-omega approaches 0 of the energy contained in the band
> (omega, omega + delta-omega) divided by delta-omega).
> 
> Notice that with this definition, the energy of a particular
> frequency component is zero.
> 
> >>Is it the average of the power of the 60Hz component over
> >>different realizations of the random signal?
> > 
> > Almost: It's the average power density of all realizations. 
> 
> Ok, here we go...  This can not be right.  I know the name
> says "power density", but this simply can not refer to power.
> 
> First of all, power is a function of time (the power is an
> instantaneous measure of the signal, that takes different
> values over time).
> 
> But even assuming that you're talking about power as the
> average power over the entire realization (i.e., the average
> over time), I still disagree!
> 
> The Fourier transform of a signal represents *energy* density
> (unless we're talking about periodic signals, but that's a
> different thing, and it's obviously not the case with random
> signals in general).

The difference is in the domain of the function. Most of the 
discussions about "random signals" in DSP is based on processes 
that are at least "stationary", and most are also assumed to 
be "ergodic". 

Stationary signals are, by definition, assumed to have infinite 
support, i.e. 

   E{x(t)} = m_x,      -infinite < t < infinite        [2]

and are power signals. A pulsed random signal where the pulse 
exists between T_start and T_stop would in general be described as 

               0                    t < T_start
   E{x(t)} =  m_x(t)     T_start <= t < T_stop         [3]
               0         T_stop  <= t

Since the contributions to the Fourier integral comes from a limited 
time domain [T_start, T_stop> this is an energy signal.

> My interpretation of PSD was always the average of the
> Fourier transforms of all possible realizations of the random
> signal -- if that's the case, then the values of PSD represent
> the average energy density (over all possible realizations).
> 
> The notion of "power density" has counter-examples, the way I
> see it:  if the PSD of white noise, which is a constant,
> represents power per unit of frequency, then the power would
> be infinite; that's certainly not the case for Gaussian white
> noise;  or white noise where, say, the value at each time is
> an independent random variable uniformly distributed between
> -1 and 1 (or whatever values).  The power of such signals is
> definitely not infinite;  however, they're both white noise,
> and they do have a PSD that is a constant for all frequencies.
> 
> If we are talking about energy density, then the above leads
> to no contradiction:  we integrate with respect to frequency,
> and of course we obtain infinity:  the signal has infinite
> energy (the signal taken as a whole, from t = -infinity to
> +infinity)
 
The Fourier transform talks about integrable(?) functions, that is, 
functions where the integral over the definition domain satisfies

   integral |x(t)|^2 dt < infinite                     [4]

where the integration limits must be specified in each case. If the 
integral goes between plus and minus infinity and you want to integrate 
a sine or cosine, you have to modify the integral to 

                   inf    1
     lim         integral - |x(t)|^2 dt < infinite     [5]
   T-> infinite   -inf    T

or the maths doesn't work. That's what is known as a "Power Signal". 
As I commented above, if the contributions come from a limited 
time domain, you can integrate between finite limits and the signal 
is an energy signal.

As for the example of white noise/constant spectrum, my very crude 
interpretation is that the Fourier integral yelds infinite energy 
but finite power, since white noise is stationary and therefore must 
be regarded a power signal. 

> Does it make sense what I'm saying?

Yes, sort of. I would be a bit happier if we could agree on exactly 
what a "random signal" is. The fundamental assumption that forms the 
basis of my discussion, is that the signal is an infinite-domain, 
stationary power signal that has to be Fourier transformed under the 
constraint [5] above. 

Rune

>Could you provide examples? Admittedly, a solar panel or a hydroelectric 
>generator are examples on types of recievers that actually generate
>energy. I was thinking of radio recievers, where the measured signal 
>is used to modulate a flow of energy internal to the reciever, energy 
>that was provided by the reciever's power supply.

>Ok, you got me wondering if I confuse "power" and "energy"... my point 
>is that you can't charge a battery by just connecting it to a recieving 
>antenna, and drayn energy/power from the static noise the antenna 
>recieves from the surroundings.

Actually one of the abuses in the early days of radio were the folks who
set of "tranformers" to sap power from the transmitters. So yes you can 
charge a battery from an attenna, and whether you want to think of loud 
music as noise is left to you as an exercise.

Almost any good book on stochastic processes will give you good rigourous
definitions. You will have to take the trouble to understand them and
stop being confused by those around you who "simplify" things. 

Finding bad drivers in an old VWs does not mean that there are no good 
drivers in Porsches. But you have to be able to tell the difference.

Dilip V. Sarwate wrote:

> Would you accept the following? 
> 
> Consider an ideal narrowbandband filter with center frequency w,
> passband gain 1 and bandwidth dw whose input happens to be the
> random process in question.  Then, the value of the PSD of the
> input process at frequency  w is the limit as dw approaches to
> zero of the ratio of the average power of the filter output signal
> to the filter bandwidth dw.

Well, there is a tiny detail that is not clear.  When you say
"average power", do you mean the average over time of the output
power in one realization?  Or do you mean the average over all
possible realizations of the "mean power"?  (where mean power
is the average over time of the power)

(I think this distintcion has to do with ergodicity, but I'd
prefer to avoid that issue, unless it is necessary)

Either way, there is something that I can not understand:  how
can the PSD represent density of *power*?  (I know the name
includes the terms "power" and "density" :-))

The spectrum of a deterministic signal represent the density
of *energy* (modulo a square root somewhere in there) of the
particular frequency component (i.e., the total energy, from
t = -infinity to +infinity).  I figure the PSD should be the
probabilistic equivalent to the spectrum, right? (something
like the average of the spectra of all possible realizations
of the random signal).

If the PSD represents density of power, one contradiction that
I find is:  suppose that you define a signal as the following:
the value of x(t) at each time t is an independent random
variable uniformly distributed in the interval (-1,1)  (when I
say independent, I mean independent of x at *any other time*).

The PSD of this signal is necessarily a constant value for all
frequencies.

If the PSD represents *power* density, then the above signal
would be found to be infinity (which is not -- the average
power of it is 1/3).

Can you help sort this out?

> The PSD is a function S(w) such that the power of the output
> signal from a filter (with transfer function H(w)) whose input is
> the process is given by the integral of S(w)|H)w)|^2 over the real
> line.  If H(w) in an ideal bandpass filter with unit passband gain,
> then the power is the integral of S(w) over the passband.

Yes, but this does not define what PSD represents -- this simply
states a characteristic of it.  Of course the output power is
given by S(w) |H(w)|^2, because the output *energy* follows that
relationship  (what I'm saying is that this property could hold
for more than one definition of what PSD is)

Cheers,

Carlos
--

Carlos Moreno wrote:

> If the PSD represents density of power, one contradiction that
> I find is:  suppose that you define a signal as the following:
> the value of x(t) at each time t is an independent random
> variable uniformly distributed in the interval (-1,1)  (when I
> say independent, I mean independent of x at *any other time*).
> 
> The PSD of this signal is necessarily a constant value for all
> frequencies.
> 
> If the PSD represents *power* density, then the above signal
> would be found to be infinity (which is not -- the average
> power of it is 1/3).

Oops.  Seems like my brain was too slow compared to my fingers :-)

The last paragraph above should say:  "then the power of the
above signal would be found to be infinity"

Carlos
--

g.sande@worldnet.att.net (Gordon Sande) wrote in message news:<BBC550E5966885935@24.222.34.2>...
> >Could you provide examples? Admittedly, a solar panel or a hydroelectric 
> >generator are examples on types of recievers that actually generate
> >energy. I was thinking of radio recievers, where the measured signal 
> >is used to modulate a flow of energy internal to the reciever, energy 
> >that was provided by the reciever's power supply.
>  
> >Ok, you got me wondering if I confuse "power" and "energy"... my point 
> >is that you can't charge a battery by just connecting it to a recieving 
> >antenna, and drayn energy/power from the static noise the antenna 
> >recieves from the surroundings.
> 
> Actually one of the abuses in the early days of radio were the folks who
> set of "tranformers" to sap power from the transmitters. So yes you can 
> charge a battery from an attenna, and whether you want to think of loud 
> music as noise is left to you as an exercise.

So what you're saying is that the transmitted EM wave would transmit 
energy as in a long-distance transformer? The efficiency would be 
ludicrously small but non-zero... It makes sense... I'll have to check 
my sources(!), the first signal analysis book I ever used in the first 
class I ever took on the subject. I thought I understood the difference 
but apparently I didn't.

> Almost any good book on stochastic processes will give you good rigourous
> definitions. You will have to take the trouble to understand them and
> stop being confused by those around you who "simplify" things. 
> 
> Finding bad drivers in an old VWs does not mean that there are no good 
> drivers in Porsches. But you have to be able to tell the difference.

Getting challenged at comp.dsp is good training in that respect... ;)

Rune

allnor@tele.ntnu.no (Rune Allnor) wrote in message news:<f56893ae.0310290616.2d551bf@posting.google.com>...
> If the 
> integral goes between plus and minus infinity and you want to integrate 
> a sine or cosine, you have to modify the integral to 
> 
>                    inf    1
>      lim         integral - |x(t)|^2 dt < infinite     [5]
>    T-> infinite   -inf    T
> 
> or the maths doesn't work. That's what is known as a "Power Signal". 

I think the correct Fourier integral for power signals should be

                    T/2    1
      lim         integral - |x(t)|^2 dt < infinite     [5]
    T-> infinite   -T/2    T
 
(integration limits are changed).

Rune

Carlos Moreno <moreno_at_mochima_dot_com@x.xxx> wrote in message news:<SRQnb.12130$zB4.68273@wagner.videotron.net>...
> If the PSD represents density of power, one contradiction that
> I find is:  suppose that you define a signal as the following:
> the value of x(t) at each time t is an independent random
> variable uniformly distributed in the interval (-1,1)  (when I
> say independent, I mean independent of x at *any other time*).
> 
> The PSD of this signal is necessarily a constant value for all
> frequencies.

No, it's not. The infinite sequence comprising only unit elements,

  ....., 1, 1, 1, 1, 1,......

is a valid (thoght almost improbable) realization of this process. 
How does its spectrum look like?

Rune