Kalman filter - getting biased estimates: for multi parameter estimation

I am a beginner & am using Kalman Filter for multiple parameter estimation
in processing of dual freq. carrier phase navigation receiver.(position,
velocity, Ionospheric error, receiver bias(units of length) and
transmitter(satellite) bias (again units of length) are the five parameters
I am estimating together). When I start with a very poor estimate to begin
the filter (meaning when I give higher initial value of process noise
covariance), the final biases are large for estimation of parameter 4 and
5. And in some cases the algorithm never converges. (cannot observe a
steady state behavior). I cannot explain this behavior. I am expecting
Kalman filter to behave like nearly unbiased estimator.
Upon starting with low process noise covariance, the algorithm gives
residuals within acceptable limits. Am I missing any fundamental limitation
in multi-parameter estimation? 

On 2 Sep, 13:46, "lgganesh" <ganes...@gmail.com> wrote:
> I am a beginner & am using Kalman Filter for multiple parameter estimation
> in processing of dual freq. carrier phase navigation receiver.(position,
> velocity, Ionospheric error, receiver bias(units of length) and
> transmitter(satellite) bias (again units of length) are the five parameters
> I am estimating together). When I start with a very poor estimate to begin
> the filter (meaning when I give higher initial value of process noise
> covariance), the final biases are large for estimation of parameter 4 and
> 5. And in some cases the algorithm never converges. (cannot observe a
> steady state behavior). I cannot explain this behavior. I am expecting
> Kalman filter to behave like nearly unbiased estimator.
> Upon starting with low process noise covariance, the algorithm gives
> residuals within acceptable limits. Am I missing any fundamental limitation
> in multi-parameter estimation?

Yep. The nonlinear Kalman filter is known to be 'itchy', and
not guaranteed to converge. Read Dan Simon's book.

Rune

On Sep 2, 12:36&#4294967295;pm, "Dr. Haddad" <tariq.hadda...@gmail.com> wrote:
> On Sep 2, 7:46&#4294967295;am, "lgganesh" <ganes...@gmail.com> wrote:
...
> Try to search for Particle filters, they provide a more robust
> estimation approach especially in physical non-Gaussian processes.

With the caveat of being painfully compute intensive.

Chris

>On Sep 2, 12:36=A0pm, "Dr. Haddad" <tariq.hadda...@gmail.com> wrote:
>> On Sep 2, 7:46=A0am, "lgganesh" <ganes...@gmail.com> wrote:
>...
>> Try to search for Particle filters, they provide a more robust
>> estimation approach especially in physical non-Gaussian processes.
>
>With the caveat of being painfully compute intensive.
>
>Chris
>

Thank you for your replies.
 I am convinced to live with the biases but want to know what exactly is
the reason for this bias. What makes the estimate converge to a biased true
value? I would also appreciate if you could bring to my notice any good
research paper or litreture on this specific issue.
Many thanks

Ganesh

lgganesh schrieb:
>> On Sep 2, 12:36=A0pm, "Dr. Haddad" <tariq.hadda...@gmail.com> wrote:
>>> On Sep 2, 7:46=A0am, "lgganesh" <ganes...@gmail.com> wrote:
>> ...
>>> Try to search for Particle filters, they provide a more robust
>>> estimation approach especially in physical non-Gaussian processes.
>> With the caveat of being painfully compute intensive.
>>
>> Chris
>>
> 
> Thank you for your replies.
>  I am convinced to live with the biases but want to know what exactly is
> the reason for this bias. What makes the estimate converge to a biased true
> value? I would also appreciate if you could bring to my notice any good
> research paper or litreture on this specific issue.
> Many thanks
> 
> Ganesh

Try to get hands on some basic book about estimation and tracking 
problems (like Estimation and Tracking from Bar-Shalom)
I can think of at least three reasons for a kalman filter not to 
converge to the true estimate:

1. Your system is nonlinear. The EKF always produces a small bias, check 
for some book if you want the mathematical derivation of that.
2. Your state initialization is far off from the ideal one. For the 
kalman filter to be optimal prior knowledge about the states at t=0 is 
assumed
3. You have states in your model which are not observable at every 
cycle, which leads to a divergence in the process noise of the state. 
(You can pretend it is zero as a dirty workaround)

Since I do not know your state model this is just happy guessing. But 
seriously there are some assumptions that have to be full filled for the 
kalman filter being an unbiased estimator and unbiased in the 
least-squares sense. So you better have a look at some textbook and 
check your state model.

Greetz,
Sebastian

On Sep 10, 11:47&#4294967295;pm, Sebastian Doht <seb_d...@lycos.com> wrote:
> lgganesh schrieb:
>
>
>
> >> On Sep 2, 12:36=A0pm, "Dr. Haddad" <tariq.hadda...@gmail.com> wrote:
> >>> On Sep 2, 7:46=A0am, "lgganesh" <ganes...@gmail.com> wrote:
> >> ...
> >>> Try to search for Particle filters, they provide a more robust
> >>> estimation approach especially in physical non-Gaussian processes.
> >> With the caveat of being painfully compute intensive.
>
> >> Chris
>
> > Thank you for your replies.
> > &#4294967295;I am convinced to live with the biases but want to know what exactly is
> > the reason for this bias. What makes the estimate converge to a biased true
> > value? I would also appreciate if you could bring to my notice any good
> > research paper or litreture on this specific issue.
> > Many thanks
>
> > Ganesh
>
> Try to get hands on some basic book about estimation and tracking
> problems (like Estimation and Tracking from Bar-Shalom)
> I can think of at least three reasons for a kalman filter not to
> converge to the true estimate:
>
> 1. Your system is nonlinear. The EKF always produces a small bias, check
> for some book if you want the mathematical derivation of that.
> 2. Your state initialization is far off from the ideal one. For the
> kalman filter to be optimal prior knowledge about the states at t=0 is
> assumed
> 3. You have states in your model which are not observable at every
> cycle, which leads to a divergence in the process noise of the state.
> (You can pretend it is zero as a dirty workaround)
>
> Since I do not know your state model this is just happy guessing. But
> seriously there are some assumptions that have to be full filled for the
> kalman filter being an unbiased estimator and unbiased in the
> least-squares sense. So you better have a look at some textbook and
> check your state model.
>
> Greetz,
> Sebastian

Thank you Sebastian,
I am just using a simple kalman filter ( and not EKF). My model is a
simple navigation (GNSS) observational equation to solve for the
following states position, velocity, iono error, receiver bias and
transmitter bias.

I observe that as the number of states to be estimated increases, the
kalman filter progressively converges to a biased estimate (which I
observe by plotting the difference between true value and estimated
value).
The filter converges to zero bias when we estimate only 3 states but
on estimation of 5 states we get an bias convergence.
Is there any inference I can derive from this fact?

I also tried to initialize the filter with different values of process
noise and there was no apparent improvement.
ofocurse a very low process noise reduces the bias significantly but I
want to initialize it to a large value in begining.

Your reason # 3 looks like one possible source of error to me. How can
I verify whether the states are observable or not.

Thanks you

Ganesh L G schrieb:
> On Sep 10, 11:47 pm, Sebastian Doht <seb_d...@lycos.com> wrote:
>> lgganesh schrieb:
>>
>>
>>
>>>> On Sep 2, 12:36=A0pm, "Dr. Haddad" <tariq.hadda...@gmail.com> wrote:
>>>>> On Sep 2, 7:46=A0am, "lgganesh" <ganes...@gmail.com> wrote:
>>>> ...
>>>>> Try to search for Particle filters, they provide a more robust
>>>>> estimation approach especially in physical non-Gaussian processes.
>>>> With the caveat of being painfully compute intensive.
>>>> Chris
>>> Thank you for your replies.
>>>  I am convinced to live with the biases but want to know what exactly is
>>> the reason for this bias. What makes the estimate converge to a biased true
>>> value? I would also appreciate if you could bring to my notice any good
>>> research paper or litreture on this specific issue.
>>> Many thanks
>>> Ganesh
>> Try to get hands on some basic book about estimation and tracking
>> problems (like Estimation and Tracking from Bar-Shalom)
>> I can think of at least three reasons for a kalman filter not to
>> converge to the true estimate:
>>
>> 1. Your system is nonlinear. The EKF always produces a small bias, check
>> for some book if you want the mathematical derivation of that.
>> 2. Your state initialization is far off from the ideal one. For the
>> kalman filter to be optimal prior knowledge about the states at t=0 is
>> assumed
>> 3. You have states in your model which are not observable at every
>> cycle, which leads to a divergence in the process noise of the state.
>> (You can pretend it is zero as a dirty workaround)
>>
>> Since I do not know your state model this is just happy guessing. But
>> seriously there are some assumptions that have to be full filled for the
>> kalman filter being an unbiased estimator and unbiased in the
>> least-squares sense. So you better have a look at some textbook and
>> check your state model.
>>
>> Greetz,
>> Sebastian
> 
> Thank you Sebastian,
> I am just using a simple kalman filter ( and not EKF). My model is a
> simple navigation (GNSS) observational equation to solve for the
> following states position, velocity, iono error, receiver bias and
> transmitter bias.
> 
> I observe that as the number of states to be estimated increases, the
> kalman filter progressively converges to a biased estimate (which I
> observe by plotting the difference between true value and estimated
> value).
> The filter converges to zero bias when we estimate only 3 states but
> on estimation of 5 states we get an bias convergence.
> Is there any inference I can derive from this fact?
> 
> I also tried to initialize the filter with different values of process
> noise and there was no apparent improvement.
> ofocurse a very low process noise reduces the bias significantly but I
> want to initialize it to a large value in begining.
> 
> Your reason # 3 looks like one possible source of error to me. How can
> I verify whether the states are observable or not.
> 
> Thanks you

I do not think # 3 applies but I do not know much about GPS. Reason # 3 
is usally something which has to be dealt with when estimating the 
dimension or the orientation of an object. But states like position and 
velocity should be measurable/computable all the time. Maybe something 
with the modelling of the biases and the iono error is wrong but that is 
something you better discuss with somebody with more background about 
GPS. The process noise might be derived from the uncertainity of your 
model itself instead of arbitrary choosen. In your case your process 
noise could be derived by linear error propagation from the maximum 
acceleration that could occur. But since I do not have a book right here 
to verify what I am saying and can not remember all the details you 
should better check at your nearest library.


Sebastian