Evaluating the "smoothness" of a vector

Maybe I missed it but:

It seems to me that one of the issues here (there are a few) is "what 
does smoothness mean?".
The intuitive notion seems clear enough.
An algorithmic or terminology definition of smoothness immediately 
suggests things like: *variance* doesn't it?

As I understand the problem:

Start with a sequence of data points which we are referring to as a 
"vector".  No mention of temporal, spatial, any kind of regularity.

Convolve this sequence with a gaussian sequence or, perhaps, a 
continuous gaussian.  So the outcome is either discrete or continuous.
I suppose one could treat the original sequence as being equally-spaced 
in order to do a discrete convolution with the gaussian.

Next (and I'm not sure I'm clear on this):

"I want an operation that
for each point in the vector gives me zero if the vector was
originally convolved with a gaussian kernel of width W."

Well, this makes no sense to me.....

Convolving with [1 2 -6 2 1] is one of many weightings (i.e. short FIR 
filters) that will determine if the current sample is above or below the 
*local* mean of adjacent samples.

This is a common method of "thresholding".  It's commonly associated 
with a follow-in test to see if the outcome is above (and/or maybe 
below) some threshold.

Reading on from the OP:

First "prune" the data.  aka "throw out the outliers" as Tim said

Next "smooth" the data.  Apply a low pass filter, short term averager. 
Apparently the objective is to remove local noise.
I note that the thresholder prefilter above is a *differentiator*. 
Differentiators are notoriously noisy.  That is their outputs have lots 
of fluctuations.  Because of that, it's common practice in some 
situations to lowpass filter the data first / i.e. short-term average.
The trick in doing this is to not filter so narrowly that the wanted 
data is perturbed and to not filter so broadly so that there is 
inadequate noise suppression.  In my experience it's a tough trade.

"The ideal operator I'm looking for gives me 0 at each point when
evaluated on V_sm. And gives me an increasing value the more my points
deviates from it."

I will risk paraphrasing this back
"The ideal operator yields a measure "0" at each point in the original 
sequence when evaluated over V_sm.
- The measure "O" increases as the current point deviates from the local 
mean??  Which suggests that the question being asked is:
"How far away from the local mean is the current data point?"
But, somehow that doesn't seem to fit the words I'm reading.
- The measure "O" increases as the variance of the sequence increases 
using the current sample as the mean??  This suggests that the question 
being asked is:
"How much variation in the data is there local to the current sample?"

I'm left confused about "what is ground truth?  i.e. what is the 
reference for comparison here?  Are we trying to take a measure of the 
current sample?  Or are we trying to take a measure of the surrounding 
samples?  Or are we trying to take a measure of all the local samples 
centered somehow or emphasizing somehow the current sample?

Then we have:

"I could compute the distance between my vector and a smoothed version
of itself"

Now this makes some sense to me.  It suggests asking: how noisy is the 
local data relative to its own mean.  The only thing about the current 
sample is its location in the sequence being measured and that's all.

I'm back to suggesting that the variance of the short-term sequence is 
such a measure.  There; now do we have a method and language to go with it?

Fred

On Fri, 22 Jun 2012 02:35:54 -0700 (PDT), Luca85 <l.pres8@gmail.com>
wrote:
>
> I feel that what I want to do is a very basic thing, so there should be a
>predefined operation.
>
I'm not really paying any attention to this entire thread.  So don't
blast me for anything related to it beyond this one statement.

But,

Hahhahahahhhhahhahah.  I love it.  I want my epitaph to read,

"Everyone said that what I want to do is a very basic thing, so there
should be a predefined operation."

On 23 Giu, 10:53, Mac Decman <dearman.m...@gmail.com> wrote:

> Hahhahahahhhhahhahah. &#4294967295;I love it. &#4294967295;I want my epitaph to read,
>
> "Everyone said that what I want to do is a very basic thing, so there
> should be a predefined operation."

Maybe I said that in the wrong may :-)
What I meant was: "I don't want to re-invent the wheel". Which I know
that happens a lot when somebody not expert in a field has to deal
with it. In these situations it's extremely common to waste a huge
amount of time trying to figure out things that somebody else already
did and, most of all, did it much better.

So I wasn't asking for someone to do my job, I wanted to make sure
that I wasn't trying to do something already existent, something
stupid or something wrong.