Kalman filter tuning for hand tracking

Hello

I am working on tracking objects (specifically a hand) in video sequences.
My question is: how to find the process noise Q and measurement noise R
covariance matrices? I am using a state model: [x,y,vx,vy]^T, that is x and
y positions and velocities. The process matrix is simply
A= [I, deltaT*I]
      [0,      I     ]
(I is 2 x 2) and the measurement matrix is H = [I, 0]

To find Q, I have tried a following method:
1. Guess Q and run Kalman filter
2. record xhat(k/k) and find the difference w(k)=xhat(k/k)-A*xhat(k-1,k-1)
3. find new Q = covariance of w(k), and go to 1 until Q converges

This method, however, will not give good results if the velocity components
are not measured.

I will appreciate any advice

Regards,

Piotr

"gutek" <g@c.pl> writes:

> I am working on tracking objects (specifically a hand) in video sequences.
> My question is: how to find the process noise Q and measurement noise R
> covariance matrices? I am using a state model: [x,y,vx,vy]^T, that is x and
> y positions and velocities. The process matrix is simply
> A= [I, deltaT*I]
>       [0,      I     ]
> (I is 2 x 2) and the measurement matrix is H = [I, 0]
> 
> To find Q, I have tried a following method:
> 1. Guess Q and run Kalman filter
> 2. record xhat(k/k) and find the difference w(k)=xhat(k/k)-A*xhat(k-1,k-1)
> 3. find new Q = covariance of w(k), and go to 1 until Q converges
> 
> This method, however, will not give good results if the velocity components
> are not measured.

Hi Piotr,

One interpretation of the process noise covariance (Q) is that it is a
measure of how much you expect your positions and velocities to
change.

If you really believe your model, then you probably mainly expect your
positions to be incremented by the associated velocity, rather than
some other influence. This means the size of the Q entries affecting
the x and y positions can be small (though zero is almost certainly
too small).

It's probably OK to make the assumption that the process noises on the
positions and velocities are independent.  This makes your Q matrix
diagonal (so you only have, at most, four parameters to choose).

So: how much do you expect your velocities to change over the running
of the filter?

Ciao,

Peter K.

-- 
Peter J. Kootsookos

"I will ignore all ideas for new works [..], the invention of which
 has reached its limits and for whose improvement I see no further
 hope."

- Julius Frontinus, c. AD 84

U&#4294967295;ytkownik "Peter J. Kootsookos" <p.kootsookos@remove.ieee.org> napisa&#4294967295; w
wiadomo&#4294967295;ci news:s68llkjvusg.fsf@mango.itee.uq.edu.au...
> "gutek" <g@c.pl> writes:
>
/cut
>
> If you really believe your model, then you probably mainly expect your
> positions to be incremented by the associated velocity, rather than
> some other influence.

No, I do not believe in my model too much ;). But its ok for me. In fact I
want Kalman filter only to discriminate pixels that are too far from the
predicted position.

> It's probably OK to make the assumption that the process noises on the
> positions and velocities are independent.  This makes your Q matrix
> diagonal (so you only have, at most, four parameters to choose).
>
Here I use a different approach. I assume that the process noise is due to
accelerations ax, ay normally distributed around 0. So the process noise
vector w is:
| 0.5 deltaT.^2*ax|
| 0.5 deltaT.^2*ay|
| deltaT*ax           |
| deltaT*ay           |

and Q =cov(ww^T);

So if i put
C = | Var(ax)        0|
       | 0        Var(ay)|
then
Q= |0.25C, 0.5C|
      |0.5C  ,   C   |

I checked this for a generated sequence of points with known Var(ax) and
Var(ay). I measured the difference between true and predicted values and the
data covariance matched this model very well. But when I am trying to
estimate Q using only updated state vector values, the covariance matrix
does not fit this model at all.
I tried also to simulate velocity measurements by taking the difference
between the old position x(k-1/k-1) and new measured position z(k). But the
results were still bad.

> So: how much do you expect your velocities to change over the running
> of the filter?

You can imagine that hand motion performing rapid gestures is not linearly
predictible. So the process noise will present high covariance values. Its
ok, I need only rough estimate of the hand position, but I'd rather not
guess what the covariance is but estimate it from the training data.

regards

Piotr

"gutek" <g@c.pl> writes:

> No, I do not believe in my model too much ;). But its ok for me. In fact I
> want Kalman filter only to discriminate pixels that are too far from the
> predicted position.

OK.

> Here I use a different approach. I assume that the process noise is due to
> accelerations ax, ay normally distributed around 0. 

So why not include the accelerations in the state vector, and make
"jerk" the noise "driving" the system?

> So the process noise
> vector w is:
> | 0.5 deltaT.^2*ax|
> | 0.5 deltaT.^2*ay|
> | deltaT*ax           |
> | deltaT*ay           |
> 
> and Q =cov(ww^T);
> 
> So if i put
> C = | Var(ax)        0|
>        | 0        Var(ay)|
> then
> Q= |0.25C, 0.5C|
>       |0.5C  ,   C   |
> 
> I checked this for a generated sequence of points with known Var(ax) and
> Var(ay). I measured the difference between true and predicted values and the
> data covariance matched this model very well. But when I am trying to
> estimate Q using only updated state vector values, the covariance matrix
> does not fit this model at all.

Do the innovations (errors) in your Kalman filter equations end up
being white (i.e. uncorrelated) ?  If not, then there is probably a
problem somewhere.

> I tried also to simulate velocity measurements by taking the difference
> between the old position x(k-1/k-1) and new measured position z(k). But the
> results were still bad.

Try the innovations and see what they tell you.

> > So: how much do you expect your velocities to change over the running
> > of the filter?
> 
> You can imagine that hand motion performing rapid gestures is not linearly
> predictible. So the process noise will present high covariance values. Its
> ok, I need only rough estimate of the hand position, but I'd rather not
> guess what the covariance is but estimate it from the training data.

OK.  The problem is that different gestures will give you different
estimates of the process noise covariance. I could imagine a straight
up-and-down gesture would give one value for the y variance and a very
small value for the x variance, and vice versa for a simple left-right
gesture.

Unless your gestures are all pretty similar, I don't think your
training data will tell you much.

Ciao,

Peter K.

-- 
Peter J. Kootsookos

"I will ignore all ideas for new works [..], the invention of which
 has reached its limits and for whose improvement I see no further
 hope."

- Julius Frontinus, c. AD 84

Hello Peter

For some reasons your last post has not appeared in my news browser, so I
cannot reply it directly. But I've read this in google's archives.

I'd rather not introduce additional variables in my state vector. I assume
that the system has a consant velocity and I am using accelerations as noise
just to predict what Q should look like and see whether I can trust the
results I get from the data.

If by innovation you mean z-H*x(k/k-1), its covariance is mainly diagonal.
Its 2 x 2, because measurement z contains only positions, not velocities.
If you are asking about corelation of w1 = x(k/k)-x(k/k-1), then position
and velocity along one coordinate (e.g position x and vx) are corelated, but
not in the same way as:
w2 = xtrue - x(k/k-1) (here I know the true values because I check this
method for a generated sequence). the latter, cov(w2) is more or less
similar to:

Q= |0.25C, 0.5C|
      |0.5C  ,   C   |

but the former, cov(w1), is sth like:

[3*C1     2*C1]
[2*C1       C1   ]

with C1 being sth like:

[ a   0]
[ 0   b]

where a is generally greater than b. (More or less agrees with standard
deviations I introduced on ax and ay to generate the process noise)
So the correlation tendency seems to be inverted in some way. If I use the
result as my initial Q, it does not converge, but oscilates with these
proportions maintained.

Concerning gestures, I am aware, that various gestures produce different
covariance. My intention is to work on sign language gestures, so I think it
is possible to determine a covariance matrix which will embrace most signs.
But as I mentioned, let Q determined from the predictions be large, its ok
until it agrees with the process noise covariance found using true position
values.

Regards

Piotr

"gutek" <ja@nospam.pl> writes:

> For some reasons your last post has not appeared in my news browser, so I
> cannot reply it directly. But I've read this in google's archives.

Yes, UQ's news server has been a little dodgy lately.

> I'd rather not introduce additional variables in my state vector. I assume
> that the system has a consant velocity and I am using accelerations as noise
> just to predict what Q should look like and see whether I can trust the
> results I get from the data.

OK. You seem to understand things; I was just trying to "rock the
boat" a little to see what degrees of freedom we have.

> If by innovation you mean z-H*x(k/k-1), its covariance is mainly diagonal.

Yes, that's the one.  It should be white (i.e. its covariance should
be diagonal)... so that means you shouldn't have a real problem with
your

> Its 2 x 2, because measurement z contains only positions, not velocities.

Yup, the innovations are the errors between the measurements (z) and
your estimate (Hx(k|k-1)).

> If you are asking about corelation of w1 = x(k/k)-x(k/k-1), then position
> and velocity along one coordinate (e.g position x and vx) are corelated, 

Yup, as they should be: your model makes x an integration (well,
accumulation) of previous vxs, as well as a noise component.

> but not in the same way as: w2 = xtrue - x(k/k-1) (here I know the
> true values because I check this method for a generated
> sequence). 

Do you know the true velocities?

> the latter, cov(w2) is more or less similar to:
> 
> Q= |0.25C, 0.5C|
>       |0.5C  ,   C   |
> 
> but the former, cov(w1), is sth like:
> 
> [3*C1     2*C1]
> [2*C1       C1   ]
> 
> with C1 being sth like:
> 
> [ a   0]
> [ 0   b]
> 
> where a is generally greater than b. (More or less agrees with standard
> deviations I introduced on ax and ay to generate the process noise)

Is this saying something about x being a preferred direction?  If a is
larger than b in the top-right part of cov(w1), then that seems odd.
Shouldn't they be (roughly) the same?

> So the correlation tendency seems to be inverted in some way. If I use the
> result as my initial Q, it does not converge, but oscilates with these
> proportions maintained.

Hmm.

> Concerning gestures, I am aware, that various gestures produce different
> covariance. My intention is to work on sign language gestures, so I think it
> is possible to determine a covariance matrix which will embrace most signs.
> But as I mentioned, let Q determined from the predictions be large, its ok
> until it agrees with the process noise covariance found using true position
> values.

OK.

Sorry I haven't been much help; I can't see why the covariances above
are coming out like that.  Could it be some interaction between Q and
your choice of R?  I assume R (measurement noise covariance) is chosen as sigma^2 I?

Ciao,

Peter K.

-- 
Peter J. Kootsookos

"I will ignore all ideas for new works [..], the invention of which
 has reached its limits and for whose improvement I see no further
 hope."

- Julius Frontinus, c. AD 84

U&#4294967295;ytkownik "Peter J. Kootsookos" <p.kootsookos@remove.ieee.org> napisa&#4294967295; w
wiadomo&#4294967295;ci news:s68wu40lmjm.fsf@mango.itee.uq.edu.au...
> "gutek" <ja@nospam.pl> writes:
>
> OK. You seem to understand things; I was just trying to "rock the
> boat" a little to see what degrees of freedom we have.
>

glad to hear that :)

>
> Do you know the true velocities?
>

Yes.

> Is this saying something about x being a preferred direction?  If a is
> larger than b in the top-right part of cov(w1), then that seems odd.
> Shouldn't they be (roughly) the same?

In the true point generation process I used:
ax = normrnd(0,4.5); // random point from normal zero mean , 4.5 sigma
distribution
ay = normrnd(0,1.7);
(Matlab code). So a and b should correspond to 4.5^2 and 1.7^2, respectively

>
> Sorry I haven't been much help; I can't see why the covariances above
> are coming out like that.  Could it be some interaction between Q and
> your choice of R?  I assume R (measurement noise covariance) is chosen as
sigma^2 I?
>

Yes. My measurement noise in the quantization of a floating point number to
the nearest int. This in fact is a uniformly distributed noise, not
gaussian, but i used its sigma^2 = 0.0833 to scale R.

I noticed that you have been answering for questions on KF for some time, so
it must be one of your interests. If you would like to spare some time, you
could repeat my experiment, and see if you get similar results. This might
be interesting and all together it takes about 50 lines of code. Just to
make sure that I do not make a big mistake, here is what I exactly do
1) generate x_true = [x,y,vx,vy]
2) add noise process of known parameters
3) predict x_true based on the previous x_update value
4) measure x,y
5) find x_update

and then compare cov(xtrue-x_pred) and cov(x_update-x_pred)

Regards

Piotr