Generalizations of Weierstrass' approximation theorem

Hi Folks.

Weierstrass' theorem of approximations says (at least in the form I 
learned it in an intro course on real analysis) that a function f(x) 
defined on  a <= x <= b, can be approximated with arbitrary presicion 
by a certain polynomial p(x):

   |f(x) - p(x)| < e,   for some e > 0,   a <= x <=b.        [1]

Now, I have had the misfortune of getting myself mixed up with people 
who claim that given some physical measurement g(t), there exist one 
and only one physical (computational) model m(t) that explains the 
measurement g(t). This "self evident" fact is then used as a basis 
for remote sensing of just about anything. The actual algorithm 
is based on "searching over all possible models, and all possible 
parametrizations of each model", where one claims that the "true"
physical model is the model (with parameters) that minimizes 
the residue

   |g(t) - m(t)| < e,   for some e > 0,   a <= t <=b.        [2]  

No, I'm not kidding. There are people out there who actually 
do these kinds of things, and expect to be taken seriously.
The fact that one basically searches a space of infinte x infinite
dimension [models x parameters] is not seen as a relevant hurdle.
Reckognizing the "true" model m(t) is seen as trivial, in that 
it minimizes the residue e in [2]. Such "trivial" details as
observation and instrumentation noise are, "of course", "not 
relevant" in the measurement and are seen as "insignificant" 
technical details. The task of actually dealing with such boring 
technicalities are left as an excersise to the reader.

Of course, I don't agree with such methods. What I am concerned, 
any data set can be approximated with arbitrary presicion of a 
polynomial according to [1] above. A polynomial p needs not 
have anything whatsoever to do with whatever physical process 
dominated the measurement. Similarly, it is possible to find 
a model m(t) with parameters, that approximates the data g(t) 
with arbitrary presicion, without m(t) having any physical 
significance whatsoever.

So my question is whether something like Weierstrass' approximation 
theorem, dealing with polynomials in [1], can be adapted to the 
case [2] where the approximation m(t) is a general model and not 
a polynomial?

Rune

"Rune Allnor" <allnor@tele.ntnu.no> wrote in message 
news:f56893ae.0411152337.2d56e216@posting.google.com...
>
> So my question is whether something like Weierstrass' approximation
> theorem, dealing with polynomials in [1], can be adapted to the
> case [2] where the approximation m(t) is a general model and not
> a polynomial?
>
> Rune

The Stone-Weierstrass theorem probably is what you want.

-Frederick Umminger 

Sounds like you've got yourself mixed up with a
load of religious loonies.

"Rune Allnor" <allnor@tele.ntnu.no> wrote in message
news:f56893ae.0411152337.2d56e216@posting.google.com...

> Now, I have had the misfortune of getting myself mixed up with people
> who claim that given some physical measurement g(t), there exist one
> and only one physical (computational) model m(t) that explains the
> measurement g(t).

The use of infinities to represent real-life samples of the order
of single-figure volts cannot, of course, be termed "approximates"
by any stretch of the imagination!

"Rune Allnor" <allnor@tele.ntnu.no> wrote in message
news:f56893ae.0411152337.2d56e216@posting.google.com...
> ...... A polynomial p needs not
> have anything whatsoever to do with whatever physical process
> dominated the measurement. Similarly, it is possible to find
> a model m(t) with parameters, that approximates the data g(t)
> with arbitrary presicion, without m(t) having any physical
> significance whatsoever.

Rune Allnor wrote:
> Hi Folks.
> 
> Weierstrass' theorem of approximations says (at least in the form I 
> learned it in an intro course on real analysis) that a function f(x) 
> defined on  a <= x <= b, can be approximated with arbitrary presicion 
> by a certain polynomial p(x):
> 
>    |f(x) - p(x)| < e,   for some e > 0,   a <= x <=b.        [1]

The slightly more abstract version is called Stone-Weierstrass for
"general" functions which satisfy some conditions, that polynomials
automatically satisfy. S-W is OK for proofs but is not a very practical
method of doing approximations. Consult a reasonable numerical analysis
source.

> Now, I have had the misfortune of getting myself mixed up with people 
> who claim that given some physical measurement g(t), there exist one 
> and only one physical (computational) model m(t) that explains the 
> measurement g(t). This "self evident" fact is then used as a basis 
> for remote sensing of just about anything. The actual algorithm 
> is based on "searching over all possible models, and all possible 
> parametrizations of each model", where one claims that the "true"

Sounds like freshman rhetoric. Rather than waste time "disproving"
such nonsense one would be better off discovering what the true
statement should have been. Technical jargon, and its short cut
versions, can appear to be wildly improbable if it is taken out
of context. You sound like you are getting it out of context and
repeated, with editing for "improvements" and "clarifications",
out of context yet again.

> physical model is the model (with parameters) that minimizes 
> the residue
> 
>    |g(t) - m(t)| < e,   for some e > 0,   a <= t <=b.        [2]  
> 
> No, I'm not kidding. There are people out there who actually 
> do these kinds of things, and expect to be taken seriously.
> The fact that one basically searches a space of infinte x infinite
> dimension [models x parameters] is not seen as a relevant hurdle.
> Reckognizing the "true" model m(t) is seen as trivial, in that 
> it minimizes the residue e in [2]. Such "trivial" details as
> observation and instrumentation noise are, "of course", "not 
> relevant" in the measurement and are seen as "insignificant" 
> technical details. The task of actually dealing with such boring 
> technicalities are left as an excersise to the reader.

There is a major field of studies called inverse problems (with
applications like CAT scanning) that does quite well. But they tend
to have good models. In CAT scanning the source image may be "all
possible images" but material densities are bounded above and below,
objects have minimal sizes, etc, so "all possible" is less imposing
that it sounds. By the time the source function and the noise sources
are tamed (regularized in the jargon) and required to be self consistent
(satisfy the inverse problems conditions) it is surprising how little
information is lost. Getting it back is often not easy but that is
why there are inverse problem methods.

> Of course, I don't agree with such methods. What I am concerned, 
> any data set can be approximated with arbitrary presicion of a 
> polynomial according to [1] above. A polynomial p needs not 
> have anything whatsoever to do with whatever physical process 
> dominated the measurement. Similarly, it is possible to find 
> a model m(t) with parameters, that approximates the data g(t) 
> with arbitrary presicion, without m(t) having any physical 
> significance whatsoever.
> 
> So my question is whether something like Weierstrass' approximation 
> theorem, dealing with polynomials in [1], can be adapted to the 
> case [2] where the approximation m(t) is a general model and not 
> a polynomial?

Read the literature on statistical fitting and on inverse problems
and you will find that poorly stated, otherwise known a silly, problems
provide many solutions like the ones you imagine. It is all the nasty
technical conditions to keep the problem well stated that get lost in
the retelling, editing and clarification of jargon filled claims by the
technical folks. These fields are already capable of producing curious
results when minor problems arise so it is hardly necessary to invent
new problems by miscommunication.

> Rune

Hi Rune,

They sound like a dangerous bunch of people, be careful!

There is a method known as 'Levenberg Marquardt' which can
fit families of 'functions' to data in a Least Mean Squared Error
sense. It will work with polynomials as well as non-linear and
rational functions. It does however need some basic definitions of
functions to look up. It can rank goodness of fit by correlation
and standard error. 

Also, Pade approximation might have some application here
particularly if the data is asymptotic or has apparent finite
discontinuities etc. Fitting ratios of polynomials can
better approximate some data than just a linear polynomial
fit. 

This all sounds like brute force empiricism, i.e. the theory
coming after the experimental data but as Donald Knuth 
said "The purpose of computing is insight, not numbers".

Hope This helps.

David.

Rune Allnor wrote:

> Hi Folks.
> 
> Weierstrass' theorem of approximations says (at least in the form I 
> learned it in an intro course on real analysis) that a function f(x) 
> defined on  a <= x <= b, can be approximated with arbitrary presicion 
> by a certain polynomial p(x):
> 
>    |f(x) - p(x)| < e,   for some e > 0,   a <= x <=b.        [1]
> 
> Now, I have had the misfortune of getting myself mixed up with people 
> who claim that given some physical measurement g(t), there exist one 
> and only one physical (computational) model m(t) that explains the 
> measurement g(t). ...

Tell your friends that the temperature in my house is 21.2C and it was
20.0 yesterday. Ask them where I live.

Consider the sequence 1, 2, 3, 4, (not known). Offer several polynomials
that produce that sequence, each producing a different fifth member of
the sequence. (If you need help generating them, just ask.)

What do they smoke?

Jerry
-- 
Engineering is the art of making what you want from things you can get.
&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;&#4294967295;

"Rune Allnor" schrieb 
> 
> Now, I have had the misfortune of getting myself mixed up with
> people who claim that given some physical measurement g(t), 
> there exist one and only one physical (computational) model 
> m(t) that explains the measurement g(t). 
>

Err, isn't that true? There is only one physically true 
explanation of how the world works. We may not know the 
explanation, we may have only imperfect models explaining 
reality only partially, but nevertheless theoretically there 
is only one model.

Anyway you may be interested in the "Buckingham Pi theorem"
(a google will turn up many references, including :)
http://test.wikipedia.org/wiki/Buckingham_Pi_theorem
   or
http://scienceworld.wolfram.com/physics/BuckinghamsPiTheorem.html
which allows to draw up models of physics without necessarily 
knowing in-depth what's going on.

Basically, you make up a list of variables, that you think are
relevant to the problem and mingle them up to get dimensionless
variables (what Buckingham called the (capital) Pi's, hence the
name, it has nothing to do with 3.1415...). Then you can 
measurements, draw curves and approximate them with functions.

Example of an application: Heat transfer
variables: 
l = length of body [m]
v = velocity of air [m/s]
Ta = air temeperature [K]
Tb = body temp [K]
rho = body heat capacity [J/kgK]
nu = viscosity of air [m2/s]
...

and you end up with formulas like:

Nu = c1 * (Re * Pr)^c2

where
c1,c2 = empirically determined constants
Nu = Nusselt number = dimensionless surface temperature gradient
Re = Reynolds number = ratio of the inertia and viscous forces
             (dimensionless velocity)
Pr = Prandtl number = ratio of momentum and thermal diffusivities

This allows you to perform experiments on a model (e.g. a scaled
down version of a Space Shuttle) and then apply the results
to the real thing.

You have to take care that the dimensionless parameters are 
similar in the model and in reality and you have to take care 
that you do not violate the underlying (perhaps unknown :-()
physical model, e.g. normally you design such that 
  Reynolds(model) = Reynolds(reality)
but at higer velocities, you have to make sure that
  Mach(model) = Mach(reality)
(Mach number = ratio of velocity to velocity of sound)
because the effects of the compressibility of air are much more
important than the others.


I may be way off and this may be not what you are looking for,
but I hope this helps.

Regards
Martin

Thanks, folks, for agreeing with me with respect to the very 
uncomfortable situation I'm in. I'll try to answer as much as 
poosible of all responses I've read so far, in this post.

Gordon Sande <g.sande@worldnet.att.net> wrote in message news:<kInmd.163715$df2.154937@edtnps89>...
> Rune Allnor wrote:
> > Hi Folks.
> > 
> > Weierstrass' theorem of approximations says (at least in the form I 
> > learned it in an intro course on real analysis) that a function f(x) 
> > defined on  a <= x <= b, can be approximated with arbitrary presicion 
> > by a certain polynomial p(x):
> > 
> >    |f(x) - p(x)| < e,   for some e > 0,   a <= x <=b.        [1]
> 
> The slightly more abstract version is called Stone-Weierstrass for
> "general" functions which satisfy some conditions, that polynomials
> automatically satisfy. S-W is OK for proofs but is not a very practical
> method of doing approximations. Consult a reasonable numerical analysis
> source.

Thanks. Soembody else suggested it, and I found some on-line papers. 
I couldn't find the Stone-Weierstrass mentioned in any of my own 
books, so I guess I don't have a decent text on numerical analysis...
or real analysis, for that matter.

> > Now, I have had the misfortune of getting myself mixed up with people 
> > who claim that given some physical measurement g(t), there exist one 
> > and only one physical (computational) model m(t) that explains the 
> > measurement g(t). This "self evident" fact is then used as a basis 
> > for remote sensing of just about anything. The actual algorithm 
> > is based on "searching over all possible models, and all possible 
> > parametrizations of each model", where one claims that the "true"
> 
> Sounds like freshman rhetoric. Rather than waste time "disproving"
> such nonsense one would be better off discovering what the true
> statement should have been. Technical jargon, and its short cut
> versions, can appear to be wildly improbable if it is taken out
> of context. You sound like you are getting it out of context and
> repeated, with editing for "improvements" and "clarifications",
> out of context yet again.

Unfortunately, this is very in context. The technical jargon is 
"model-based signal processing" which is something of a buzzword 
in underwater acoustics, these days. 

The problem is that one does some sort of sonar survey at sea, where 
one locates objects partially or fully buried in the sea floor. Once 
such a buried object is detected and located (which can be done), 
the problem is to determine what kind of object this is. Is it a rock?
A mine? A munition shell? A box containing munition shells? Something 
completely different? The problem is to answer these types of
questions.

Lots of people have done lots of research on developing forward models
for acoustical scattering from buried and partially buried data. The 
forward problem is the problem where all aspects of the sea floor, the
object and the sonar system is specified in advance, and the resulting
measurements of the acoustic field are simulated. The inverse problem,
the analysis problem, is generally seen as 'easy' once the forward 
modelling program is available: Just run all possible models with 
all possible parameters through the modelling routine, and see 
what fits best. 

The really disturbing part is that the people who suggest these kinds 
of things, are professors with 20, 30, 40 years of seniority. It's 
not just one who have lost it, it's a whole community who base their 
research on these methods. 

If you want me to, I can dig up some more recent published papers, 
but you could review my rant from a couple of years back, to get an 
impression of how these people think:

http://groups.google.com/groups?hl=no&lr=&threadm=f56893ae.0207290839.5ba1659b%40posting.google.com&rnum=3&prev=/groups%3Fq%3D%2522MFP/SL%2522%26hl%3Dno%26lr%3D%26selm%3Df56893ae.0207290839.5ba1659b%2540posting.google.com%26rnum%3D3

> > physical model is the model (with parameters) that minimizes 
> > the residue
> > 
> >    |g(t) - m(t)| < e,   for some e > 0,   a <= t <=b.        [2]  
> > 
> > No, I'm not kidding. There are people out there who actually 
> > do these kinds of things, and expect to be taken seriously.
> > The fact that one basically searches a space of infinte x infinite
> > dimension [models x parameters] is not seen as a relevant hurdle.
> > Reckognizing the "true" model m(t) is seen as trivial, in that 
> > it minimizes the residue e in [2]. Such "trivial" details as
> > observation and instrumentation noise are, "of course", "not 
> > relevant" in the measurement and are seen as "insignificant" 
> > technical details. The task of actually dealing with such boring 
> > technicalities are left as an excersise to the reader.
> 
> There is a major field of studies called inverse problems (with
> applications like CAT scanning) that does quite well. But they tend
> to have good models. In CAT scanning the source image may be "all
> possible images" but material densities are bounded above and below,
> objects have minimal sizes, etc, so "all possible" is less imposing
> that it sounds. By the time the source function and the noise sources
> are tamed (regularized in the jargon) and required to be self consistent
> (satisfy the inverse problems conditions) it is surprising how little
> information is lost. Getting it back is often not easy but that is
> why there are inverse problem methods.

Well, I have some experience from processing and analysis of seismic
data.
In that context, inverse problems are not automated at all, but are 
a highly interactive excersise. The best written account I have found 
for how seismologists work, is the first couple of paragraphs in 
the article

Robinson: "Model-driven predictive deconvolution"
          Geophysics, v63 no2, March 1998, pp 713:

  "INTRODUCTION

   In his perceptive and captivating book, Clay (1990) observes 
   that geophysicists use essentially the same approach to finding 
   oil that the fictional Sherlock Holmes used in solving mysteries. 
   He points out that the subsurface structure of the earth represents
   and unknown, a mystery. The geophysicist acts like a detective in 
   identifying a problem worthy of attention, collecting data, and 
   then (like Holmes) deducing an explanation. Geophysicists identify 
   a potential oil-bearing area, collect information in the form of 
   reflected seismic waves, and then deduce the underground structure 
   in order to delineate a possible oil reservoir. Some of the
recorded
   seismic information is useful, and is called a signal. However, 
   much of the recorded seismic information is made up of noise and
   interference that mask the useful signals. One of the most 
   troublesome forms of interfernce is that caused by multiple 
   reflections.
   
   Sheriff (1991) defines a model as a concept from which one can
deduce
   effects for comparisions with observations. He states that a model 
   can be used to develop a better understanding of the observations. 
   However, he carefully points out that agreement between
observations
   and effects derived from the model does not establish that the 
   particular models represents the actual situation. Geophysical 
   interpretation problems almost always lack uniqueness."

Clay: 1990, "Elementary Exploration Seismology", Prentice-Hall.
Sheriff: 1991, "Encyclopedic dictionary of exploration geophysics",
      3rd. ed., Soc. Expl. Geophys.

This is the tradition I'm trained in, this is the philosophy I apply 
to all my work. If you get as far as to come up with a base model 
that can be fed to a Levenberg-Marquart type of alhgorithm, all the 
hard work is done:

1) You have successfully(?) identified certain key effects in the data
2) You have used these effects to predict derived effects
3) You have found these secondary effects in the data and thus 
  "verified" (or at least not contradicted) your first model
4) You have repeated the steps 2)-3) above until you either 
   find a contradicting effect, or are comfortable with letting 
   the computer take over the drudgery.

One can use various types of inversion schemes for doing some of the 
drudgery, but all the detective work must still be done in advance. 
And whatever method is used to produca the end model, it can still 
be plain wrong. I have been involved with projects where three years 
of hard analysis work was abandoned because of the results of a small 
analysis job that took no more than half an hour to do. But the
results
contradicted the then prevailing hypothesis.

> > Of course, I don't agree with such methods. What I am concerned, 
> > any data set can be approximated with arbitrary presicion of a 
> > polynomial according to [1] above. A polynomial p needs not 
> > have anything whatsoever to do with whatever physical process 
> > dominated the measurement. Similarly, it is possible to find 
> > a model m(t) with parameters, that approximates the data g(t) 
> > with arbitrary presicion, without m(t) having any physical 
> > significance whatsoever.
> > 
> > So my question is whether something like Weierstrass' approximation 
> > theorem, dealing with polynomials in [1], can be adapted to the 
> > case [2] where the approximation m(t) is a general model and not 
> > a polynomial?
> 
> Read the literature on statistical fitting and on inverse problems
> and you will find that poorly stated, otherwise known a silly, problems
> provide many solutions like the ones you imagine. It is all the nasty
> technical conditions to keep the problem well stated that get lost in
> the retelling, editing and clarification of jargon filled claims by the
> technical folks. These fields are already capable of producing curious
> results when minor problems arise so it is hardly necessary to invent
> new problems by miscommunication.

Oh, how I wish mere miscommunication was the causes! It is not. People
catually think data analysis is as simple as I described. I don't know
what, if anything these people have smoked. What I do know, is that
they
have never been challenged on technical issues for decades. Thes guys 
can base an assumption on that the Earth is flat, and demand to be
taken
seriously. No one have ever had them stop and think twice about what 
they actually present under the cloak of "science" and "research".
Again, check out the thread I mentioned above, to get an impression 
of what's going on.

Rune

Jerry Avins <jya@ieee.org> wrote in message news:<2vv9dkF2nr1otU1@uni-berlin.de>...
> Rune Allnor wrote:
> 
> > Hi Folks.
> > 
> > Weierstrass' theorem of approximations says (at least in the form I 
> > learned it in an intro course on real analysis) that a function f(x) 
> > defined on  a <= x <= b, can be approximated with arbitrary presicion 
> > by a certain polynomial p(x):
> > 
> >    |f(x) - p(x)| < e,   for some e > 0,   a <= x <=b.        [1]
> > 
> > Now, I have had the misfortune of getting myself mixed up with people 
> > who claim that given some physical measurement g(t), there exist one 
> > and only one physical (computational) model m(t) that explains the 
> > measurement g(t). ...
> 
> Tell your friends that the temperature in my house is 21.2C and it was
> 20.0 yesterday. Ask them where I live.

Heh, I've tried that. My favourite example is based on a technique known 
as "focalization" which is very "hot" these days. The idea behind 
"focalization" is that one makes one measurement of a propagating wave
in a medium where the speed of sound is unknown, and estimate *both*
the speed of sound in the medium *and* the distance between the 
source and the reciever. This is equivalent to make a radar system 
where the speed of light, c, is unknown: "The 2 way time of flight is 
1 second. Find bot the slant range r and the speed of sound c."

No, I'm not kidding. People actually publish such ideas and expect to 
be taken seriously:

  http://www.fysel.ntnu.no/~allnor/pdfs/paper1.pdf

Note the first sentence under the heading "I. Focalization":

"Focalization is a generalization of MFP in which both source parameters 
and environmental parameters are unknown or partially unknown."

These guys don't waste any time in checking whether there are conditions 
to whether their general ideas work. The statement that the method works 
even if all relevant parameters are "unknown" is pretty unequivocal.

Now, I am not clairvoyant so I don't know what the authors actually 
mean to say. I do know how lots of people interpret this paper, and, I 
guess, the particular statement about "everything being unknown": 
Any measurement goes, and if inserted into this inversion scheme, 
the "true" model (i.e. the "true" environmental and source parameters) 
pop out in the other end as if by magic.

It may be due to my mental abilities, but I am not capable of 
making sense of the focalization paper. Interestingly, it is registered 
with 81 citations in the literature, when I checked today.

> Consider the sequence 1, 2, 3, 4, (not known). Offer several polynomials
> that produce that sequence, each producing a different fifth member of
> the sequence. (If you need help generating them, just ask.)

Yeah, right. Have you ever tried to ask a professor to actually think 
through what he says? It's not recommended. When I tried, it cost me 
first a job and later a full year of sick leave.
 
> What do they smoke?

I don't have the faintest idea. I'm not sure I want to know.

Rune