Adaptive Filter | ADPCM | ADSP | ADSP-2181 | Aliasing | AMR | Anti-Aliasing | ARMA | Autocorrelation | AutoCovariance | Beamforming | Bessel | Blackfin | Butterworth | C6713 | CCS | Chebyshev | CIC Filter | Circular Convolution | Code Composer Studio | Comb Filter | Compression | Convolution | Cross Correlation | DCT | Decimation | Deconvolution | Demodulation | DM642 | DSP Boards | DSP/BIOS | DTMF | Echo Cancellation | Equalization | Equalizer | ETSI | EZLITE (Ez-kit Lite) | FFT | FFTW | FIR Filter | Fixed Point | FSK | G.711 | G.723 | G.729 | Gaussian Noise | Goertzel | GPIO | Hilbert Transform | IFFT | IIR Filter | Interpolation | Invariance | JTAG | Kalman | Laplace Transform | Levinson | LPC | McBSP | MIPS | Modulation | MPEG | Multirate | Notch Filter | Nyquist | OFDM | Oversampling | Pink Noise | Pitch | PLL | Polyphase | QAM | QDMA | Quantization | Quantizer | Radar | Random Noise | Reed Solomon | Remez | Resampling | RTDX | Sampling | Sharc | TI C6711 | Undersampling | Viterbi | Wavelets | White Noise | Wiener Filter | Windowing | XDS510PP | Z Transform

Dear all,

I am trying to estimate parameters from a set of data. For example let
"y" be received symbols, "x" input symbol, "h" channel (constants) and
"n" be random Gaussian distributed, where "k" represents index of a
array varies from k = 0 to 100. We can consider x(-1) = 0 for k = 0.

    y(k) = x(k)*h(0) + x(k-1)*h(1) + n(k)

 y(k) and x(k) are assumed known at the receiver, and I want to
estimate "h(0) and h(1)".

I can use ML/LS or least mean squares (LMS) to find estimate h(0) or
h(1).

If for minute I assume at the receiver I don't know how many "h" are
there to generate "y" and I estimate assuming that there is only one h
for under-estimate i.e. "h(0)" or I over estimate take "h(0), h(1),
and h(2)" instead of two i.e. "h(0) and h(1)".

Does anyone has any idea if there is any study material on
maximum-lilelihood estimation in case of over or under-estimation.

Thanks,

Cheers,
Vimal

______________________________
New DSP Code Snippets Section now Live.   Learn more about the reward program for contributors here.

v...@yahoo.com (Vimal) wrote in message news:<b...@posting.google.com>...
> Dear all,
> 
> I am trying to estimate parameters from a set of data. For example let
> "y" be received symbols, "x" input symbol, "h" channel (constants) and
> "n" be random Gaussian distributed, where "k" represents index of a
> array varies from k = 0 to 100. We can consider x(-1) = 0 for k = 0.
> 
>     y(k) = x(k)*h(0) + x(k-1)*h(1) + n(k)
> 
>  y(k) and x(k) are assumed known at the receiver, and I want to
> estimate "h(0) and h(1)".
> 
> I can use ML/LS or least mean squares (LMS) to find estimate h(0) or
> h(1).
> 
> If for minute I assume at the receiver I don't know how many "h" are
> there to generate "y" and I estimate assuming that there is only one h
> for under-estimate i.e. "h(0)" or I over estimate take "h(0), h(1),
> and h(2)" instead of two i.e. "h(0) and h(1)".
> 
> Does anyone has any idea if there is any study material on
> maximum-lilelihood estimation in case of over or under-estimation.

This is the classical problem of order estimation of AR models.

What you cold try, is to decide on a maximum order of your model and 
use your ML estimator to find estimates for the reflection coefficients
of your AR process. Once you have them, you could apply some sort of 
order estimator (AIC, MDL,...) on those reflection coefficients to 
estimate the order. Or you could use an ML method for that step too, 
see

Kay: Conditional model order estimation
     IEEE Trans. Sig. Proc., v49 n9, September 2001.

You chould also check whether order estimators can be incorporated 
into the recursive algorithm in Ch. 6.5.2 of 

Kay: Modern Spectral Estimation, Theory and Applications.
     Prentice-Hall, 1988.

Rune

______________________________
New DSP Code Snippets Section now Live.   Learn more about the reward program for contributors here.

Hi.

>> Dear all,
>> 
>> I am trying to estimate parameters from a set of data. For example
>> let "y" be received symbols, "x" input symbol, "h" channel
>> (constants) and "n" be random Gaussian distributed, where "k"
>> represents index of a array varies from k = 0 to 100. We can
>> consider x(-1) = 0 for k = 0.
>> 
>> y(k) = x(k)*h(0) + x(k-1)*h(1) + n(k)
>> 
>> y(k) and x(k) are assumed known at the receiver, and I want to
>> estimate "h(0) and h(1)".
>> 
>> I can use ML/LS or least mean squares (LMS) to find estimate h(0)
>> or h(1).
>> 
>> If for minute I assume at the receiver I don't know how many "h"
>> are there to generate "y" and I estimate assuming that there is
>> only one h for under-estimate i.e. "h(0)" or I over estimate take
>> "h(0), h(1), and h(2)" instead of two i.e. "h(0) and h(1)".
>> 
>> Does anyone has any idea if there is any study material on
>> maximum-lilelihood estimation in case of over or under-estimation.

Rune> This is the classical problem of order estimation of AR models.

I just wanted to point out that recently there was a tutorial on order 
estimation in  the IEEE Signal Processing Magazine.

Model-order selection
Stoica, P.   Selen, Y.   
Uppsala University in Sweden;
This paper appears in: Signal Processing Magazine, IEEE

Publication Date: July 2004
On page(s): 36- 47
Volume: 21,   Issue: 4  

-- 
/Mads (http://kom.aau.dk/~mgc)

______________________________
New DSP Code Snippets Section now Live.   Learn more about the reward program for contributors here.

Hello,

Thanks for valuable suggestions and references. I found Signal
Processing Magazine article interesting, however it seems that AIC/BIC
are for parametric models (the article's introduction).

Does (over or under) parameter estimation for non-parameteric
estimation techniques make any sense!!, Pl. suggest...

Cheers,
Vimal

______________________________
New DSP Code Snippets Section now Live.   Learn more about the reward program for contributors here.

v...@yahoo.com (Vimal) wrote in message news:<b...@posting.google.com>...

> Does (over or under) parameter estimation for non-parameteric
> estimation techniques make any sense!!

"Find the parameters of a non-parametric model"... I am sure you'll 
find a clue to your answer if you take a few seconds to contemplate 
your question. 

Rune

______________________________
New DSP Code Snippets Section now Live.   Learn more about the reward program for contributors here.

Rune Allnor wrote:

> v...@yahoo.com (Vimal) wrote in message news:<b...@posting.google.com>...
> 
> 
>>Does (over or under) parameter estimation for non-parameteric
>>estimation techniques make any sense!!
> 
> 
> "Find the parameters of a non-parametric model"... I am sure you'll 
> find a clue to your answer if you take a few seconds to contemplate 
> your question. 
> 
> Rune

Would it work with a parametric non-model?

Jerry
-- 
... the worst possible design that just meets the specification - almost
a definition of practical engineering.                     .. Chris Bore
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

______________________________
New DSP Code Snippets Section now Live.   Learn more about the reward program for contributors here.

Thanks Rune... :-)

Now one small question, can I assume different tap weights H = [h(0),
..., h(N)] as different parameters.

From literature it seems that mean (modes), variance etc. are taken as
parameters which can define a density.

Would H form a parameter or each element of vector H can be considered
as parameter, as the mean in the earlier example will be linear
combination of elements of vector-H.

Cheers,
Vimal

______________________________
New DSP Code Snippets Section now Live.   Learn more about the reward program for contributors here.

v...@yahoo.com (Vimal) wrote in message news:<b...@posting.google.com>...
> Thanks Rune... :-)
> 
> Now one small question, can I assume different tap weights H = [h(0),
> ..., h(N)] as different parameters.
> 
> From literature it seems that mean (modes), variance etc. are taken as
> parameters which can define a density.
> 
> Would H form a parameter or each element of vector H can be considered
> as parameter, as the mean in the earlier example will be linear
> combination of elements of vector-H.

Well, technically speaking you are right: The filter H is a model that 
has N+1 parameter, that can be estimated or computed or chosen. Any 
number that in any way can be taken as a description of some data or 
process, can be viewed as a "parameter" in a "model".

However, the term "model" is used where some basic structure is used to 
analyze the data at hand. The analyst could, for instance *choose* to 
use the sum-of-sines model as basis for analyzing a data set, or he 
could *choose* to use the AR model or the ARMA model. In each case one 
would find that a different number of parameters are needed to describe 
the same data. 

So for the FIR filter, the terms "model" and "parameters" don't apply, 
since a FIR filter is a FIR filter, and once you decide to build a FIR 
filter with certain spec, all the numbers fall straight out of the 
machinery. The statistical moments are not "parameters" since they are 
not based on any assumptions of the basic structure of the data. Once 
you *choose* to view the amplitude PDF of signal amplitude as, say, 
Gaussia, the mean and variance become parameters of that PDF. But it 
is *your* choise to use the Gaussian model. There is nothing in the 
measured data as such that tells you to analyze them in this way or that.

The AR, ARMA, sum-of-sines, etc are "models" because the analyst must 
make an active choise between them, prior to analysis. The analyst 
is faced with a couple of tough choises, including

- How to choose the correct class of models (AR, ARMA, Gaussian PDF,...) 
  prior to analyzing any particular type of data.
- Once this is done, how to estimate the number of parameters to 
  be used for any particular set of data. This is the order estimation 
  problem, and amounts to choosing what order p is "best" among, say, 
  all AR(p) processes.
- How to handle observation noise.
- How to handle model errors.
- What type of implementation to use in the estimation.

Mind you that there are people out there, holding positions and tenure, 
that apparently beleive that the "correct" model and the "correct" set 
of parameters will manifest themselves if "analyzing" any particular set 
of data by means of computer software, as if by divine revelation. I can 
not prove these people wrong, just as little as I can prove the 
(non)existence of a God. I see one disturbing aspect in these people's 
"research", though, in that all "proofs" and "demonstrations" of their 
hypotheses are based on people "analyzing" data they simulated (never 
measured!) themselves, and where they make sure that the same computer 
models are used for "analysis" of data, as was used for synthesis. 
Naturally, they also make sure that they search for the same parameters 
and in the same parameters ranges that were used during synthesis. 

So be a bit careful with how you deal with "research results" in 
parametric or "model based" data analysis.

Rune

______________________________
New DSP Code Snippets Section now Live.   Learn more about the reward program for contributors here.

Thanks again Rune for enlightenment. I should have used 'non-model'
rather than 'non-parameteric' to describe my problem.

> So for the FIR filter, the terms "model" and "parameters" don't apply, 
> since a FIR filter is a FIR filter, and once you decide to build a FIR 
> filter with certain spec, all the numbers fall straight out of the 
> machinery. 

 Rune, I will use Jerry's comment and elaborate. 

What if the model is actually a non-model, like for FIR of (N-1)-order
instead of correct being N-order (as I generate my data from N-order).
Then can AIB/BIC thing help...

 Your right as most (or all) of the times in simulations we generate
the data with known values and then estimate those parameters using
some criterion. I guess doing that was a start and now we can move to
the next level of uncertainty.

______________________________
New DSP Code Snippets Section now Live.   Learn more about the reward program for contributors here.

v...@yahoo.com (Vimal) wrote in message news:<b...@posting.google.com>...
> Thanks again Rune for enlightenment. I should have used 'non-model'
> rather than 'non-parameteric' to describe my problem.
> 
> > So for the FIR filter, the terms "model" and "parameters" don't apply, 
> > since a FIR filter is a FIR filter, and once you decide to build a FIR 
> > filter with certain spec, all the numbers fall straight out of the 
> > machinery. 
> 
>  Rune, I will use Jerry's comment and elaborate. 
> 
> What if the model is actually a non-model, like for FIR of (N-1)-order
> instead of correct being N-order (as I generate my data from N-order).
> Then can AIB/BIC thing help...
> 
>  Your right as most (or all) of the times in simulations we generate
> the data with known values and then estimate those parameters using
> some criterion. I guess doing that was a start and now we can move to
> the next level of uncertainty.

Well, I've certainly transcended to the next level of uncertainty. I have 
no idea what you mean. A "parameter" is intimately related to a "model", 
that you as user has to choose. Without the prior choise of a model, I 
am not able to make sense of what a parameter is.

Rune

______________________________
New DSP Code Snippets Section now Live.   Learn more about the reward program for contributors here.