Dear all,
I am trying to estimate parameters from a set of data. For example let
"y" be received symbols, "x" input symbol, "h" channel (constants) and
"n" be random Gaussian distributed, where "k" represents index of a
array varies from k = 0 to 100. We can consider x(-1) = 0 for k = 0.
y(k) = x(k)*h(0) + x(k-1)*h(1) + n(k)
y(k) and x(k) are assumed known at the receiver, and I want to
estimate "h(0) and h(1)".
I can use ML/LS or least mean squares (LMS) to find estimate h(0) or
h(1).
If for minute I assume at the receiver I don't know how many "h" are
there to generate "y" and I estimate assuming that there is only one h
for under-estimate i.e. "h(0)" or I over estimate take "h(0), h(1),
and h(2)" instead of two i.e. "h(0) and h(1)".
Does anyone has any idea if there is any study material on
maximum-lilelihood estimation in case of over or under-estimation.
Thanks,
Cheers,
Vimal
(over or under) Parameter Estimation
Started by ●August 12, 2004
Reply by ●August 13, 20042004-08-13
vimal_bhatia2@yahoo.com (Vimal) wrote in message news:<b6fc6dda.0408121649.6cccfb28x@posting.google.com>...> Dear all, > > I am trying to estimate parameters from a set of data. For example let > "y" be received symbols, "x" input symbol, "h" channel (constants) and > "n" be random Gaussian distributed, where "k" represents index of a > array varies from k = 0 to 100. We can consider x(-1) = 0 for k = 0. > > y(k) = x(k)*h(0) + x(k-1)*h(1) + n(k) > > y(k) and x(k) are assumed known at the receiver, and I want to > estimate "h(0) and h(1)". > > I can use ML/LS or least mean squares (LMS) to find estimate h(0) or > h(1). > > If for minute I assume at the receiver I don't know how many "h" are > there to generate "y" and I estimate assuming that there is only one h > for under-estimate i.e. "h(0)" or I over estimate take "h(0), h(1), > and h(2)" instead of two i.e. "h(0) and h(1)". > > Does anyone has any idea if there is any study material on > maximum-lilelihood estimation in case of over or under-estimation.This is the classical problem of order estimation of AR models. What you cold try, is to decide on a maximum order of your model and use your ML estimator to find estimates for the reflection coefficients of your AR process. Once you have them, you could apply some sort of order estimator (AIC, MDL,...) on those reflection coefficients to estimate the order. Or you could use an ML method for that step too, see Kay: Conditional model order estimation IEEE Trans. Sig. Proc., v49 n9, September 2001. You chould also check whether order estimators can be incorporated into the recursive algorithm in Ch. 6.5.2 of Kay: Modern Spectral Estimation, Theory and Applications. Prentice-Hall, 1988. Rune
Reply by ●August 13, 20042004-08-13
Hi.>> Dear all, >> >> I am trying to estimate parameters from a set of data. For example >> let "y" be received symbols, "x" input symbol, "h" channel >> (constants) and "n" be random Gaussian distributed, where "k" >> represents index of a array varies from k = 0 to 100. We can >> consider x(-1) = 0 for k = 0. >> >> y(k) = x(k)*h(0) + x(k-1)*h(1) + n(k) >> >> y(k) and x(k) are assumed known at the receiver, and I want to >> estimate "h(0) and h(1)". >> >> I can use ML/LS or least mean squares (LMS) to find estimate h(0) >> or h(1). >> >> If for minute I assume at the receiver I don't know how many "h" >> are there to generate "y" and I estimate assuming that there is >> only one h for under-estimate i.e. "h(0)" or I over estimate take >> "h(0), h(1), and h(2)" instead of two i.e. "h(0) and h(1)". >> >> Does anyone has any idea if there is any study material on >> maximum-lilelihood estimation in case of over or under-estimation.Rune> This is the classical problem of order estimation of AR models. I just wanted to point out that recently there was a tutorial on order estimation in the IEEE Signal Processing Magazine. Model-order selection Stoica, P. Selen, Y. Uppsala University in Sweden; This paper appears in: Signal Processing Magazine, IEEE Publication Date: July 2004 On page(s): 36- 47 Volume: 21, Issue: 4 -- /Mads (http://kom.aau.dk/~mgc)
Reply by ●August 18, 20042004-08-18
Hello, Thanks for valuable suggestions and references. I found Signal Processing Magazine article interesting, however it seems that AIC/BIC are for parametric models (the article's introduction). Does (over or under) parameter estimation for non-parameteric estimation techniques make any sense!!, Pl. suggest... Cheers, Vimal
Reply by ●August 19, 20042004-08-19
vimal_bhatia2@yahoo.com (Vimal) wrote in message news:<b6fc6dda.0408180832.3b7fb210@posting.google.com>...> Does (over or under) parameter estimation for non-parameteric > estimation techniques make any sense!!"Find the parameters of a non-parametric model"... I am sure you'll find a clue to your answer if you take a few seconds to contemplate your question. Rune
Reply by ●August 19, 20042004-08-19
Rune Allnor wrote:> vimal_bhatia2@yahoo.com (Vimal) wrote in message news:<b6fc6dda.0408180832.3b7fb210@posting.google.com>... > > >>Does (over or under) parameter estimation for non-parameteric >>estimation techniques make any sense!! > > > "Find the parameters of a non-parametric model"... I am sure you'll > find a clue to your answer if you take a few seconds to contemplate > your question. > > RuneWould it work with a parametric non-model? Jerry -- ... the worst possible design that just meets the specification - almost a definition of practical engineering. .. Chris Bore ������������������������������������������������������������������������
Reply by ●August 19, 20042004-08-19
Thanks Rune... :-) Now one small question, can I assume different tap weights H = [h(0), ..., h(N)] as different parameters. From literature it seems that mean (modes), variance etc. are taken as parameters which can define a density. Would H form a parameter or each element of vector H can be considered as parameter, as the mean in the earlier example will be linear combination of elements of vector-H. Cheers, Vimal
Reply by ●August 20, 20042004-08-20
vimal_bhatia2@yahoo.com (Vimal) wrote in message news:<b6fc6dda.0408190956.238ed354@posting.google.com>...> Thanks Rune... :-) > > Now one small question, can I assume different tap weights H = [h(0), > ..., h(N)] as different parameters. > > From literature it seems that mean (modes), variance etc. are taken as > parameters which can define a density. > > Would H form a parameter or each element of vector H can be considered > as parameter, as the mean in the earlier example will be linear > combination of elements of vector-H.Well, technically speaking you are right: The filter H is a model that has N+1 parameter, that can be estimated or computed or chosen. Any number that in any way can be taken as a description of some data or process, can be viewed as a "parameter" in a "model". However, the term "model" is used where some basic structure is used to analyze the data at hand. The analyst could, for instance *choose* to use the sum-of-sines model as basis for analyzing a data set, or he could *choose* to use the AR model or the ARMA model. In each case one would find that a different number of parameters are needed to describe the same data. So for the FIR filter, the terms "model" and "parameters" don't apply, since a FIR filter is a FIR filter, and once you decide to build a FIR filter with certain spec, all the numbers fall straight out of the machinery. The statistical moments are not "parameters" since they are not based on any assumptions of the basic structure of the data. Once you *choose* to view the amplitude PDF of signal amplitude as, say, Gaussia, the mean and variance become parameters of that PDF. But it is *your* choise to use the Gaussian model. There is nothing in the measured data as such that tells you to analyze them in this way or that. The AR, ARMA, sum-of-sines, etc are "models" because the analyst must make an active choise between them, prior to analysis. The analyst is faced with a couple of tough choises, including - How to choose the correct class of models (AR, ARMA, Gaussian PDF,...) prior to analyzing any particular type of data. - Once this is done, how to estimate the number of parameters to be used for any particular set of data. This is the order estimation problem, and amounts to choosing what order p is "best" among, say, all AR(p) processes. - How to handle observation noise. - How to handle model errors. - What type of implementation to use in the estimation. Mind you that there are people out there, holding positions and tenure, that apparently beleive that the "correct" model and the "correct" set of parameters will manifest themselves if "analyzing" any particular set of data by means of computer software, as if by divine revelation. I can not prove these people wrong, just as little as I can prove the (non)existence of a God. I see one disturbing aspect in these people's "research", though, in that all "proofs" and "demonstrations" of their hypotheses are based on people "analyzing" data they simulated (never measured!) themselves, and where they make sure that the same computer models are used for "analysis" of data, as was used for synthesis. Naturally, they also make sure that they search for the same parameters and in the same parameters ranges that were used during synthesis. So be a bit careful with how you deal with "research results" in parametric or "model based" data analysis. Rune
Reply by ●August 20, 20042004-08-20
Thanks again Rune for enlightenment. I should have used 'non-model' rather than 'non-parameteric' to describe my problem.> So for the FIR filter, the terms "model" and "parameters" don't apply, > since a FIR filter is a FIR filter, and once you decide to build a FIR > filter with certain spec, all the numbers fall straight out of the > machinery.Rune, I will use Jerry's comment and elaborate. What if the model is actually a non-model, like for FIR of (N-1)-order instead of correct being N-order (as I generate my data from N-order). Then can AIB/BIC thing help... Your right as most (or all) of the times in simulations we generate the data with known values and then estimate those parameters using some criterion. I guess doing that was a start and now we can move to the next level of uncertainty.
Reply by ●August 21, 20042004-08-21
vimal_bhatia2@yahoo.com (Vimal) wrote in message news:<b6fc6dda.0408200918.3be5197f@posting.google.com>...> Thanks again Rune for enlightenment. I should have used 'non-model' > rather than 'non-parameteric' to describe my problem. > > > So for the FIR filter, the terms "model" and "parameters" don't apply, > > since a FIR filter is a FIR filter, and once you decide to build a FIR > > filter with certain spec, all the numbers fall straight out of the > > machinery. > > Rune, I will use Jerry's comment and elaborate. > > What if the model is actually a non-model, like for FIR of (N-1)-order > instead of correct being N-order (as I generate my data from N-order). > Then can AIB/BIC thing help... > > Your right as most (or all) of the times in simulations we generate > the data with known values and then estimate those parameters using > some criterion. I guess doing that was a start and now we can move to > the next level of uncertainty.Well, I've certainly transcended to the next level of uncertainty. I have no idea what you mean. A "parameter" is intimately related to a "model", that you as user has to choose. Without the prior choise of a model, I am not able to make sense of what a parameter is. Rune
