Forums

Rigorous definition of the Spectral Density of a random signal?

Started by Carlos Moreno October 25, 2003
Carlos Moreno <moreno_at_mochima_dot_com@x.xxx> wrote in message news:<uxzmb.64090$nP4.1108315@weber.videotron.net>...
 
> So, I'm looking for a *rigurous* definition of what the > values of the PSD really represent. (notice that I don't > want the PSD defined as the Fourier transform of the > auto-correlation of the signal -- what I need is a > definition of what the value of the PSD at a specific > frequency means/represents) > >..... material snipped .... > > What I would want: "The value of the PDF at x is the > limit as dx approaches to zero of the probability that > the variable takes values in (x, x+dx), divided by dx" > Or: "The pdf function is such that the probability > that the variable takes values within a region R is > given by the integral of the pdf over the region R" > > These are both valid examples of what I would call an > actual/rigurous definition of what the meaning of the > value of the PDF is.
Would you accept the following? Consider an ideal narrowbandband filter with center frequency w, passband gain 1 and bandwidth dw whose input happens to be the random process in question. Then, the value of the PSD of the input process at frequency w is the limit as dw approaches to zero of the ratio of the average power of the filter output signal to the filter bandwidth dw. or The PSD is a function S(w) such that the power of the output signal from a filter (with transfer function H(w)) whose input is the process is given by the integral of S(w)|H)w)|^2 over the real line. If H(w) in an ideal bandpass filter with unit passband gain, then the power is the integral of S(w) over the passband.
Tom Loredo wrote:
> How about going to the library and looking at any one of many good books > on the statistics of time series? Vol. 1 of Priestly is a good one for > this question.
Well, one thing that promted me to bring the subject for a "live" discussion with people that really know the subject, is that a couple of books (ironically, with a common co- author) give interpretations that, the way I see, are contradictory. They do not give a definition of what the PSD is, but just spit a bunch of blah-blah about what I would call "the consequences from an intuitive point of view of what PSD is" :-\ You see how one can get pretty frustrated :-( I guess I'll look up Priestly's book, to see if I can get a better perspective to the subject. Thanks! Carlos --
Rune Allnor wrote:

> In my humble opinion, the terms "rigorous" and "statistics" are almost > contradictions in terms. While the formalities of maths also applies > when used in statistsics, I think the interpretations may not be as > strict as with other mathemathical diciplines.
That does not mean that probabilities/statistics is not a rigurous subject -- it simply means that a lot of people do not understand them and therefore misinterpret it. I can imagine Kolmogorov right now desperately rolling in his grave if he heard your comment!! :-)
>>Is it the average of the total engergy of the 60Hz component >>over different realizations of the random signal? > > Nope. It's the "Power Spectral Density" we are talking about. If there > was a nonzero *energy* component in random noise, electrical energy > would be available for free: Just mount a reciever and drain energy out > of the blue.
??? Electrical energy *is* available for free -- not in whatever amount that we may want. But if you mount a receiver, you *can* indeed drain energy from it. You're not "creating" energy out of the blue -- you're transforming energy from whatever physical source of the noise, into electrical energy that you now extract; there's definitely no magic in that.
>>Is it the average of energy density at 60Hz over different >>realizations of the random signal? > > Nope, due to the energy/density thing commented in the previous point.
Actually, I (doubly) disagree on this point -- there is non-zero energy component *in a frequency band of non-zero length*. Notice that the term I used, "energy density", refers to (informally speaking) the "amount of energy per unit of frequency (i.e., density in the above definition would be defined as the limit as delta-omega approaches 0 of the energy contained in the band (omega, omega + delta-omega) divided by delta-omega). Notice that with this definition, the energy of a particular frequency component is zero.
>>Is it the average of the power of the 60Hz component over >>different realizations of the random signal? > > Almost: It's the average power density of all realizations.
Ok, here we go... This can not be right. I know the name says "power density", but this simply can not refer to power. First of all, power is a function of time (the power is an instantaneous measure of the signal, that takes different values over time). But even assuming that you're talking about power as the average power over the entire realization (i.e., the average over time), I still disagree! The Fourier transform of a signal represents *energy* density (unless we're talking about periodic signals, but that's a different thing, and it's obviously not the case with random signals in general). My interpretation of PSD was always the average of the Fourier transforms of all possible realizations of the random signal -- if that's the case, then the values of PSD represent the average energy density (over all possible realizations). The notion of "power density" has counter-examples, the way I see it: if the PSD of white noise, which is a constant, represents power per unit of frequency, then the power would be infinite; that's certainly not the case for Gaussian white noise; or white noise where, say, the value at each time is an independent random variable uniformly distributed between -1 and 1 (or whatever values). The power of such signals is definitely not infinite; however, they're both white noise, and they do have a PSD that is a constant for all frequencies. If we are talking about energy density, then the above leads to no contradiction: we integrate with respect to frequency, and of course we obtain infinity: the signal has infinite energy (the signal taken as a whole, from t = -infinity to +infinity) Does it make sense what I'm saying? Carlos --
Carlos Moreno <moreno_at_mochima_dot_com@x.xxx> wrote in message news:<yPHnb.61923$He4.1598924@wagner.videotron.net>...
> Rune Allnor wrote: > > > In my humble opinion, the terms "rigorous" and "statistics" are almost > > contradictions in terms. While the formalities of maths also applies > > when used in statistsics, I think the interpretations may not be as > > strict as with other mathemathical diciplines. > > That does not mean that probabilities/statistics is not a rigurous > subject -- it simply means that a lot of people do not understand > them and therefore misinterpret it. > > I can imagine Kolmogorov right now desperately rolling in his grave > if he heard your comment!! :-)
My point is only that if you use statistics and rigorous mathemathics to describe the probablility distribution function of a fair dice, it will show a PDF as P(x=n) = 1/6, n=1,2,3,4,5,6 [1] i.e. the dice will *on average* show one eye every 6'th time it is rolled. A flawed ("strict") interpretation is to say that "it will show one eye exactly once in the next six throws". I've heard that kind of statements made.
> >>Is it the average of the total engergy of the 60Hz component > >>over different realizations of the random signal? > > > > Nope. It's the "Power Spectral Density" we are talking about. If there > > was a nonzero *energy* component in random noise, electrical energy > > would be available for free: Just mount a reciever and drain energy out > > of the blue. > > ??? > > Electrical energy *is* available for free -- not in whatever > amount that we may want. But if you mount a receiver, you *can* > indeed drain energy from it. You're not "creating" energy out > of the blue -- you're transforming energy from whatever physical > source of the noise, into electrical energy that you now extract; > there's definitely no magic in that.
Could you provide examples? Admittedly, a solar panel or a hydroelectric generator are examples on types of recievers that actually generate energy. I was thinking of radio recievers, where the measured signal is used to modulate a flow of energy internal to the reciever, energy that was provided by the reciever's power supply. Ok, you got me wondering if I confuse "power" and "energy"... my point is that you can't charge a battery by just connecting it to a recieving antenna, and drayn energy/power from the static noise the antenna recieves from the surroundings.
> >>Is it the average of energy density at 60Hz over different > >>realizations of the random signal? > > > > Nope, due to the energy/density thing commented in the previous point. > > Actually, I (doubly) disagree on this point -- there is non-zero > energy component *in a frequency band of non-zero length*. Notice > that the term I used, "energy density", refers to (informally > speaking) the "amount of energy per unit of frequency (i.e., > density in the above definition would be defined as the limit as > delta-omega approaches 0 of the energy contained in the band > (omega, omega + delta-omega) divided by delta-omega). > > Notice that with this definition, the energy of a particular > frequency component is zero. > > >>Is it the average of the power of the 60Hz component over > >>different realizations of the random signal? > > > > Almost: It's the average power density of all realizations. > > Ok, here we go... This can not be right. I know the name > says "power density", but this simply can not refer to power. > > First of all, power is a function of time (the power is an > instantaneous measure of the signal, that takes different > values over time). > > But even assuming that you're talking about power as the > average power over the entire realization (i.e., the average > over time), I still disagree! > > The Fourier transform of a signal represents *energy* density > (unless we're talking about periodic signals, but that's a > different thing, and it's obviously not the case with random > signals in general).
The difference is in the domain of the function. Most of the discussions about "random signals" in DSP is based on processes that are at least "stationary", and most are also assumed to be "ergodic". Stationary signals are, by definition, assumed to have infinite support, i.e. E{x(t)} = m_x, -infinite < t < infinite [2] and are power signals. A pulsed random signal where the pulse exists between T_start and T_stop would in general be described as 0 t < T_start E{x(t)} = m_x(t) T_start <= t < T_stop [3] 0 T_stop <= t Since the contributions to the Fourier integral comes from a limited time domain [T_start, T_stop> this is an energy signal.
> My interpretation of PSD was always the average of the > Fourier transforms of all possible realizations of the random > signal -- if that's the case, then the values of PSD represent > the average energy density (over all possible realizations). > > The notion of "power density" has counter-examples, the way I > see it: if the PSD of white noise, which is a constant, > represents power per unit of frequency, then the power would > be infinite; that's certainly not the case for Gaussian white > noise; or white noise where, say, the value at each time is > an independent random variable uniformly distributed between > -1 and 1 (or whatever values). The power of such signals is > definitely not infinite; however, they're both white noise, > and they do have a PSD that is a constant for all frequencies. > > If we are talking about energy density, then the above leads > to no contradiction: we integrate with respect to frequency, > and of course we obtain infinity: the signal has infinite > energy (the signal taken as a whole, from t = -infinity to > +infinity)
The Fourier transform talks about integrable(?) functions, that is, functions where the integral over the definition domain satisfies integral |x(t)|^2 dt < infinite [4] where the integration limits must be specified in each case. If the integral goes between plus and minus infinity and you want to integrate a sine or cosine, you have to modify the integral to inf 1 lim integral - |x(t)|^2 dt < infinite [5] T-> infinite -inf T or the maths doesn't work. That's what is known as a "Power Signal". As I commented above, if the contributions come from a limited time domain, you can integrate between finite limits and the signal is an energy signal. As for the example of white noise/constant spectrum, my very crude interpretation is that the Fourier integral yelds infinite energy but finite power, since white noise is stationary and therefore must be regarded a power signal.
> Does it make sense what I'm saying?
Yes, sort of. I would be a bit happier if we could agree on exactly what a "random signal" is. The fundamental assumption that forms the basis of my discussion, is that the signal is an infinite-domain, stationary power signal that has to be Fourier transformed under the constraint [5] above. Rune
>Could you provide examples? Admittedly, a solar panel or a hydroelectric >generator are examples on types of recievers that actually generate >energy. I was thinking of radio recievers, where the measured signal >is used to modulate a flow of energy internal to the reciever, energy >that was provided by the reciever's power supply.
>Ok, you got me wondering if I confuse "power" and "energy"... my point >is that you can't charge a battery by just connecting it to a recieving >antenna, and drayn energy/power from the static noise the antenna >recieves from the surroundings.
Actually one of the abuses in the early days of radio were the folks who set of "tranformers" to sap power from the transmitters. So yes you can charge a battery from an attenna, and whether you want to think of loud music as noise is left to you as an exercise. Almost any good book on stochastic processes will give you good rigourous definitions. You will have to take the trouble to understand them and stop being confused by those around you who "simplify" things. Finding bad drivers in an old VWs does not mean that there are no good drivers in Porsches. But you have to be able to tell the difference.
Dilip V. Sarwate wrote:

> Would you accept the following? > > Consider an ideal narrowbandband filter with center frequency w, > passband gain 1 and bandwidth dw whose input happens to be the > random process in question. Then, the value of the PSD of the > input process at frequency w is the limit as dw approaches to > zero of the ratio of the average power of the filter output signal > to the filter bandwidth dw.
Well, there is a tiny detail that is not clear. When you say "average power", do you mean the average over time of the output power in one realization? Or do you mean the average over all possible realizations of the "mean power"? (where mean power is the average over time of the power) (I think this distintcion has to do with ergodicity, but I'd prefer to avoid that issue, unless it is necessary) Either way, there is something that I can not understand: how can the PSD represent density of *power*? (I know the name includes the terms "power" and "density" :-)) The spectrum of a deterministic signal represent the density of *energy* (modulo a square root somewhere in there) of the particular frequency component (i.e., the total energy, from t = -infinity to +infinity). I figure the PSD should be the probabilistic equivalent to the spectrum, right? (something like the average of the spectra of all possible realizations of the random signal). If the PSD represents density of power, one contradiction that I find is: suppose that you define a signal as the following: the value of x(t) at each time t is an independent random variable uniformly distributed in the interval (-1,1) (when I say independent, I mean independent of x at *any other time*). The PSD of this signal is necessarily a constant value for all frequencies. If the PSD represents *power* density, then the above signal would be found to be infinity (which is not -- the average power of it is 1/3). Can you help sort this out?
> The PSD is a function S(w) such that the power of the output > signal from a filter (with transfer function H(w)) whose input is > the process is given by the integral of S(w)|H)w)|^2 over the real > line. If H(w) in an ideal bandpass filter with unit passband gain, > then the power is the integral of S(w) over the passband.
Yes, but this does not define what PSD represents -- this simply states a characteristic of it. Of course the output power is given by S(w) |H(w)|^2, because the output *energy* follows that relationship (what I'm saying is that this property could hold for more than one definition of what PSD is) Cheers, Carlos --
Carlos Moreno wrote:

> If the PSD represents density of power, one contradiction that > I find is: suppose that you define a signal as the following: > the value of x(t) at each time t is an independent random > variable uniformly distributed in the interval (-1,1) (when I > say independent, I mean independent of x at *any other time*). > > The PSD of this signal is necessarily a constant value for all > frequencies. > > If the PSD represents *power* density, then the above signal > would be found to be infinity (which is not -- the average > power of it is 1/3).
Oops. Seems like my brain was too slow compared to my fingers :-) The last paragraph above should say: "then the power of the above signal would be found to be infinity" Carlos --
g.sande@worldnet.att.net (Gordon Sande) wrote in message news:<BBC550E5966885935@24.222.34.2>...
> >Could you provide examples? Admittedly, a solar panel or a hydroelectric > >generator are examples on types of recievers that actually generate > >energy. I was thinking of radio recievers, where the measured signal > >is used to modulate a flow of energy internal to the reciever, energy > >that was provided by the reciever's power supply. > > >Ok, you got me wondering if I confuse "power" and "energy"... my point > >is that you can't charge a battery by just connecting it to a recieving > >antenna, and drayn energy/power from the static noise the antenna > >recieves from the surroundings. > > Actually one of the abuses in the early days of radio were the folks who > set of "tranformers" to sap power from the transmitters. So yes you can > charge a battery from an attenna, and whether you want to think of loud > music as noise is left to you as an exercise.
So what you're saying is that the transmitted EM wave would transmit energy as in a long-distance transformer? The efficiency would be ludicrously small but non-zero... It makes sense... I'll have to check my sources(!), the first signal analysis book I ever used in the first class I ever took on the subject. I thought I understood the difference but apparently I didn't.
> Almost any good book on stochastic processes will give you good rigourous > definitions. You will have to take the trouble to understand them and > stop being confused by those around you who "simplify" things. > > Finding bad drivers in an old VWs does not mean that there are no good > drivers in Porsches. But you have to be able to tell the difference.
Getting challenged at comp.dsp is good training in that respect... ;) Rune
allnor@tele.ntnu.no (Rune Allnor) wrote in message news:<f56893ae.0310290616.2d551bf@posting.google.com>...
> If the > integral goes between plus and minus infinity and you want to integrate > a sine or cosine, you have to modify the integral to > > inf 1 > lim integral - |x(t)|^2 dt < infinite [5] > T-> infinite -inf T > > or the maths doesn't work. That's what is known as a "Power Signal".
I think the correct Fourier integral for power signals should be T/2 1 lim integral - |x(t)|^2 dt < infinite [5] T-> infinite -T/2 T (integration limits are changed). Rune
Carlos Moreno <moreno_at_mochima_dot_com@x.xxx> wrote in message news:<SRQnb.12130$zB4.68273@wagner.videotron.net>...
> If the PSD represents density of power, one contradiction that > I find is: suppose that you define a signal as the following: > the value of x(t) at each time t is an independent random > variable uniformly distributed in the interval (-1,1) (when I > say independent, I mean independent of x at *any other time*). > > The PSD of this signal is necessarily a constant value for all > frequencies.
No, it's not. The infinite sequence comprising only unit elements, ....., 1, 1, 1, 1, 1,...... is a valid (thoght almost improbable) realization of this process. How does its spectrum look like? Rune