Rigorous definition of the Spectral Density of a random signal?

Carlos Moreno wrote:
> They do not give a definition of what the
> PSD is, but just spit a bunch of blah-blah about what I
> would call "the consequences from an intuitive point of
> view of what PSD is"  :-\  You see how one can get pretty
> frustrated  :-(

Yes.  Look at Priestly; it focuses on the statistical properties
and is very rigorous and technical.  It's one of the top few
references on statistical properties of periodograms and
related statistics.

Part of the issue is context.  For example, if you believe your
data contains an unknown single sinusoid and Gaussian noise, then
there is a simple interpretation of the PSD as the log of the
(marginal) posterior probability density for the frequency of
the sinusoid (this is from the Bayesian viewpoint; there is an
analogous frequentist result in terms of least squares estimation
of the frequency).  If instead you believe the signal has a smooth
spectrum, things get more complicated.  For example, Whittle showed
that, under certain conditions, you can use the PSD of the data to 
easily construct an approximate likelihood function for the PSD of the
underlying signal (look for "Whittle likelihood" to learn more about
this).  So just from these two examples perhaps you
can see that there is likely to be more than one "correct" answer
to your question unless you get more specific about the actual
problem you are trying to solve.  Priestly discusses much of this
in great detail (though not any of the Bayesian stuff; for that
look at bayes.wustl.edu for Jaynes's article on Bayesian spectrum
and chirp analysis, and at Bretthorst's book).

-Tom

-- 

To respond by email, replace "somewhere" with "astro" in the
return address.

Rune Allnor wrote:

>>If the PSD represents density of power, one contradiction that
>>I find is:  suppose that you define a signal as the following:
>>the value of x(t) at each time t is an independent random
>>variable uniformly distributed in the interval (-1,1)  (when I
>>say independent, I mean independent of x at *any other time*).
>>
>>The PSD of this signal is necessarily a constant value for all
>>frequencies.
> 
> 
> No, it's not. 

Sorry, yes it is!

> The infinite sequence comprising only unit elements,
> 
>   ....., 1, 1, 1, 1, 1,......
> 
> is a valid (thoght almost improbable) realization of this process. 

Except that I was talking about the PSD of the random signal that
I defined.  You are talking about the spectrum of one particular
realization of that random signal -- those are two entirely different 
things.

Carlos
--

Carlos, I think you are abusing terminology and thus missing Rune's point.

> > If the PSD represents density of power, one contradiction that
> > I find is:  suppose that you define a signal as the following:
> > the value of x(t) at each time t is an independent random
> > variable uniformly distributed in the interval (-1,1)  (when I
> > say independent, I mean independent of x at *any other time*).

This does not define a signal; it defines a *distribution* of signals.

Carlos Moreno wrote:
> 
> Except that I was talking about the PSD of the random signal that
> I defined.  You are talking about the spectrum of one particular
> realization of that random signal -- those are two entirely different
> things.

Indeed they are---and only a particular realization of the process
you defined has a PSD (and Rune did correctly identify one possible
realization with a nonconstant PSD).  The process as a whole produces
a distribution of PSD functions.

-Tom

-- 

To respond by email, replace "somewhere" with "astro" in the
return address.

Tom Loredo wrote:
> Carlos, I think you are abusing terminology and thus missing Rune's point.
> 
>>>If the PSD represents density of power, one contradiction that
>>>I find is:  suppose that you define a signal as the following:
>>>the value of x(t) at each time t is an independent random
>>>variable uniformly distributed in the interval (-1,1)  (when I
>>>say independent, I mean independent of x at *any other time*).
> 
> This does not define a signal; it defines a *distribution* of signals.

Actually, I disagree.  Although I see that I wasn't rigorous
enough with my phrasing.

The above (my previous post) defines a *random signal*, or a
*random process*  (maybe the latter is a more appropriate
term).  But I got used to talk about "random signals",
understanding that when we say "the (random) signal" we
are talking about an experiment with outcomes being the
realizations of that random signal.

The PSD that I'm talking about is the PSD of random signals,
and not the Spectrum of a particular realization of a
random signal.  That's why I contested Rune's point; the
spectrum of any specific realization of the (random) signal
I defined is not relevant.

White noise has a PSD that is constant for all values of
frequency (from -infinity to infinity).

The random signal (or random process) that I described is
white noise;  it has a PSD that is constant for all values
of frequency.  Yet its average power is not infinite (as
we would conclude if the PSD represented average density
of power).

So, I'm hoping that I'm clarifying the notation and what
I meant.

What troubles me is that there are a couple of posts in
this thread that claim that PSD is the average (over all
possible realizations of the random signal) of *power*
density, and no-one has contested that notion -- except
me, claiming that it represents density of *energy*.
The funny detail is that no-one has explicitly contested
my claim either!!  Are you guys under the impression
that I'm just trolling?  Or that maybe I'm cheating in
some homework.  I can truthfully tell you that neither
of these options is true.  As I said in my first post,
something that our teacher said in class triggered a
disagreement that I think goes back to different
understandings of what PSD is:  they claim that white
noise has infinite variance, and I disagree  (their
argument is that the power is obtained by integrating
the PSD with respect to frequency, and since the PSD
of white noise is constant, then the integral gives
infinite power).

Cheers,

Carlos
--

Rune Allnor wrote:

> ...
> Could you provide examples? Admittedly, a solar panel or a
> hydroelectric generator are examples on types of recievers that
> actually generate energy. I was thinking of radio recievers, where
> the measured signal is used to modulate a flow of energy internal
> to the reciever, energy that was provided by the reciever's power
> supply.

In Berlin (was it in the early 30th?), people took lamp bulbs, 
connected two wires to them, one of them was grounded.
By raising the other wire into the air, they could switch on the 
lamp.

Surely, this was only possible because of the short distance of the 
transmitting station.

This became common usage in an increasing number of garden houses, 
garages, ... until in the end a law was released which forbade the 
"abuse" of radio energy.

Even today, the "Deutsche Welle" short-wave broadcasting station,
causes lamps (fluorescent tubes) to shine without being switched on,
as soon as the antenna is directed to Africa.
This happens in a village some 3km south of the station.
(I've been told by somebody who lived there.)

Bernhard 

Carlos Moreno <moreno_at_mochima_dot_com@x.xxx> wrote in message news:<eEXnb.5927$_G2.147481@wagner.videotron.net>...
> Rune Allnor wrote:
> 
> >>If the PSD represents density of power, one contradiction that
> >>I find is:  suppose that you define a signal as the following:
> >>the value of x(t) at each time t is an independent random
> >>variable uniformly distributed in the interval (-1,1)  (when I
> >>say independent, I mean independent of x at *any other time*).
> >>
> >>The PSD of this signal is necessarily a constant value for all
> >>frequencies.
> > 
> > 
> > No, it's not. 
> 
> Sorry, yes it is!
> 
> > The infinite sequence comprising only unit elements,
> > 
> >   ....., 1, 1, 1, 1, 1,......
> > 
> > is a valid (thoght almost improbable) realization of this process. 
> 
> Except that I was talking about the PSD of the random signal that
> I defined.  You are talking about the spectrum of one particular
> realization of that random signal -- those are two entirely different 
> things.

Could you please define what a "realization of a random signal" is?
I can't make any sense of that. If you said "realization of a random 
process" it would make sense. There is a vast difference between 
"signal" and "process". The PSD is a property of the process, the 
squared magnitude of the frequency spectrum of the signal is an 
estimate of that spectrum. See below.

I did ask in a different post that you define more clearly what you mean 
by "random signal". I can't find a post of yours where you do that. 
I will make an attempt to clarify the basis of my argumentation:

If you study one sequence of numbers or one signal (one particular 
realization), then use the standard techniques like DFTs, and do so. 
However, once we start talking about "random signals" and "random 
processes" it is implied that we regard the observed signal (the 
"realization") as one glimpse into a world where some governing 
rule, of which we know little (the "process"), influences the 
behaviour of the observed data. The objective of the study is to 
use the observed data to infer something about these governing rules, 
not to describe the data themselves in any particular detail. 

The statistician admits (by using the very term "random") that he can 
not (and will not) attemt a complete, deterministic description of the 
system. He will, however, make the best use he can of what data he has, 
in such a way that the inherent uncertainty of the analysis is vclear, 
and also such that he can use any new data that may become available 
to him.

So we *assume* some property of the data, i.e. that they are a 
"realization" of some "random process". We can the impose some more 
assumption of this model, ("stationary", "ergodic", "zero mean", 
"Gaussian",...) and try to characterize it or estimate some parameters. 
You may want to look into the difference between "true" parameters and 
"estimators" for those parameters. As in the example cited above, the 
estimated PSD of the (constant) data series of one realization is as far 
from the "true" PSD of the random process as possible, but the statistical 
arguments still hold. 

Of course, attempting to estimate the true PSD on the basis of that 
realization is doomed to fail, but that's just the inherent peril of 
statistical signal processing or statistics in general.

I'll just remind you about a point I made in an earlier post about 
interpreting the maths of the statistics in a somewhat different manner 
than "only maths". 

Rune

Carlos Moreno <moreno_at_mochima_dot_com@x.xxx> wrote in message news:<KW_nb.7503$_G2.329389@wagner.videotron.net>...
> Tom Loredo wrote:
> > Carlos, I think you are abusing terminology and thus missing Rune's point.
> > 
> >>>If the PSD represents density of power, one contradiction that
> >>>I find is:  suppose that you define a signal as the following:
> >>>the value of x(t) at each time t is an independent random
> >>>variable uniformly distributed in the interval (-1,1)  (when I
> >>>say independent, I mean independent of x at *any other time*).
> > 
> > This does not define a signal; it defines a *distribution* of signals.
> 
> Actually, I disagree.  Although I see that I wasn't rigorous
> enough with my phrasing.
> 
> The above (my previous post) defines a *random signal*, or a
> *random process*  (maybe the latter is a more appropriate
> term).  But I got used to talk about "random signals",
> understanding that when we say "the (random) signal" we
> are talking about an experiment with outcomes being the
> realizations of that random signal.

Then it may be useful to check out the terminology. The "random 
process" usually denotes this distribution of "realizations", 
and the term "signal" usually refers to one particular outcome 
or observation in one particular realization or experiment.

> The PSD that I'm talking about is the PSD of random signals,
> and not the Spectrum of a particular realization of a
> random signal. That's why I contested Rune's point; the
> spectrum of any specific realization of the (random) signal
> I defined is not relevant.

The spectral densities of realizations are most certainly relevant, 
they are what you observe, they are what you have to work with.
Ref the connection between the probability density function of 
your dice, and the outcome of actually rolling it.

> White noise has a PSD that is constant for all values of
> frequency (from -infinity to infinity).

The generating process does. One particular realization need not have.

> The random signal (or random process) that I described is
> white noise;  it has a PSD that is constant for all values
> of frequency.  Yet its average power is not infinite (as
> we would conclude if the PSD represented average density
> of power).
> 
> So, I'm hoping that I'm clarifying the notation and what
> I meant.

You have clarified you notation, but not necessarily what you meant.
Humpty Dumpty may have been a charming guy, but his linguistics
is not an example to follow.

> What troubles me is that there are a couple of posts in
> this thread that claim that PSD is the average (over all
> possible realizations of the random signal) of *power*
> density, and no-one has contested that notion -- except
> me, claiming that it represents density of *energy*.
> The funny detail is that no-one has explicitly contested
> my claim either!! 

I *think* I wrote a post a couple of days ago, where I commented 
on the difference between Energy Signals and Power Signals.
There need not be any contradictions between energy and power, 
it's a matter of using the correct tool in each case. 

Coming to think of it, I actually asked in that very same post if 
you could make an effort to check wether the signals you are concerned 
about were best classified as Energy Signal or Power Signals...

> Are you guys under the impression
> that I'm just trolling? 

The thought has occured to me...

> Or that maybe I'm cheating in
> some homework. I can truthfully tell you that neither
> of these options is true.  As I said in my first post,
> something that our teacher said in class triggered a
> disagreement that I think goes back to different
> understandings of what PSD is:  they claim that white
> noise has infinite variance, and I disagree  (their
> argument is that the power is obtained by integrating
> the PSD with respect to frequency, and since the PSD
> of white noise is constant, then the integral gives
> infinite power).

I don't know what your teacher said, so I can't comment on any 
remarks you heard in class. But it seems more and more clear to me 
that this is an issue of getting the different definitions right, 
understanding those differences, and to know that different analysis 
tools exist for different types of signals. You might find it useful 
to go to your teacher and ask him or her about the details.

Rune

Carlos Moreno wrote:

   ...

> White noise has a PSD that is constant for all values of
> frequency (from -infinity to infinity).
> 
> The random signal (or random process) that I described is
> white noise;  it has a PSD that is constant for all values
> of frequency.  Yet its average power is not infinite (as
> we would conclude if the PSD represented average density
> of power).

   ...

The average power that that model implies is indeed infinite. The
argument is a demonstration that ideal white noise -- constant energy
per unit bandwidth for all frequencies -- can't exist. Like the rigid
bodies of mechanics, white noise is no more than a useful abstraction.

Just a the notion of a rigid body provides an easy attack on statically
determinate systems, the notion of pure white noise is usefully applied
to bandlimited systems. Properties of the noise outside the system's
bandwidth are irrelevant. We deal regularly with abstractions that don't
represent something we can build or touch. Impulses and other
singularities are examples.

Jerry
-- 
Engineering is the art of making what you want from things you can get.
�����������������������������������������������������������������������

Jerry Avins wrote:

>> White noise has a PSD that is constant for all values of
>> frequency (from -infinity to infinity).
>>
>> The random signal (or random process) that I described is
>> white noise;  it has a PSD that is constant for all values
>> of frequency.  Yet its average power is not infinite (as
>> we would conclude if the PSD represented average density
>> of power).
> 
> The average power that that model implies is indeed infinite. 

But it can not be!  The power of something that has values
bounded in absolute value can not be infinite!

> The argument is a demonstration that ideal white noise -- constant energy
> per unit bandwidth for all frequencies -- can't exist.

If it can exist in nature or not, is irrelevant  (I mean, any
argument could be taken to philosophical grounds, claiming
that numbers do not exist in nature and that they're only
a model, an abstraction that only exist in our minds....
With that sort of argument one could dismiss basically
*any* mathematical result, valid or not).

My point is that, the white noise as I defined it, as a
mathematical abstraction, has properties that can be
rigorously defined according to the laws of mathematics.

The way I see it, the power of the white noise that I
described *can not* be infinite.  In fact, calculating
the variance is a trivial exercise:  E{x^2} = integral
from -1 to 1 of (1/2) x^2 dx.  That gives you the average
power (assuming a normalized 1-ohm resistor, etc. etc.),
which is definitely not infinite.

Yes, white noise is a mathematical abstraction;  the
disagreement is on the mathematical properties of this
abstraction.

You mention also the delta.  As an example, if someone
tells you that the value of the delta at t = 0 is 1??
You will undoubtedly tell them that they're wrong!!  It
doesn't matter that a delta can not exist in practice;
the mathematical properties that define a delta are
specific and rigurous.

Same thing (IMO) about white noise.  We talk about
*Gaussian* white noise with zero mean and variance
sigma^2.  Well, if the variance is sigma^2, then the
average power is not infinite!

Carlos
--

Carlos Moreno wrote:
 > [...]
> So, can someone help me with finding a rigurous definition
> of what the value at a given frequency of the PSD of a
> random signal means?

Carlos,

I think you mean to ask is how we derive the units
of the PSD. If so, that's not too hard.

For this discussion, let's assume that the units of our
signal is volts, which I will denote by "[V]" ([anything]
denotes the units of "anything"). Let's also say the
voltage signal is identically distributed, and that the
pdf of the voltage (the ensemble pdf) at any time t
is simply f(x). Let's also denote the voltage signal
X(t).

First of all we need to convince ourselves that the pdf
of a random variable representing voltage has units of
[1/V]. Reason like this: The integral of the pdf gives
a probability, which is unitless. Since the integral
of a pdf is of the form \int f(x) dx, and the "dx" has
units of [V], then f(x) must be [1/V] to make the result
unitless.

OK, now what are the units of the autocorrelation function
Rxx(tau) of X(t)? All time lags have the same units, then for
simplicity let's examine the units of Rxx(0) = E[X^2]. This
is

   \int x^2 f(x) dx,

which has units of [V^2] * [1/V] * [V], or [V^2]. Of course you
know the units [V^2] are directly proportional to power.

Finally, what are the units of the PSD?

   \int Rxx(tau) e^{j*omega*tau) dtau

has units of [V^2] * [s] ("s" = seconds). And that is
directly proportional to watts/Hz (since [Hz] = [1/s]).
-- 
%  Randy Yates                  % "...the answer lies within your soul
%% Fuquay-Varina, NC            %       'cause no one knows which side
%%% 919-577-9882                %                   the coin will fall."
%%%% <yates@ieee.org>           %  'Big Wheels', *Out of the Blue*, ELO
http://home.earthlink.net/~yatescr