With a tactical grade IMU updated by GPS the EKF (extended Kalman filter) approach gave me very small residuals (less than 1 meter RMS from pseudoranges and 1 cm/sec from 1-sec carrier phase changes (table near the bottom of http://jameslfarrell.com/wp-content/uploads/2012/03/GPSINS.pdf ), while airborne under severe vibration. The inertial instrument errors weren't anything like biases nor white noise; they could be modeled as "slowly-varying" (band-limited) for very short durations ONLY. The reason is, as I noted once before in this discussion, they come from a diverse collection of sources, many of them motion-dependent. Those motions include vibrations as well as profile dynamics, generating error constituents proportional to rates and acceleration components -- and to their squares and products, both translational and rotational, with in-phase and quadrature correlations across axes. After a lifetime of dealing with those effects, including thousands of hours with in-flight data, I can assure you that a simple characterization for them is a very short-term phenomenon while they're in motion. The errors change within less than a minute.
Reply by navaide●August 14, 20172017-08-14
On Friday, July 21, 2017 at 10:21:51 PM UTC-4, gyans...@gmail.com wrote:
> I am using a fairly standard approach to estimating angular pitch using a KF. It uses both accelerometer and Gyro angle data. Now it estimated the angle fine enough and I implement the steady-state KF. Never tried this before but then put a PID or lag-lead controller on this measurement. I find that the Kalman filter bandwidth is stuff all and severely reduces the bandwidth of my closed-loop system - which is using classical control methods. Since the bandwidth is obviously proportional is some way to the additive noise covariances I am thinking to lower their value in the algorithm so that the filter bandwidth increases.
>
> Anyway, the KF, although useful does not appear to have any measure of its own bandwidth so we can at least get an idea of the limitations. Obviously the more filtering in a control loop the more phase lag and less stability.
=============================================================================
With a tactical grade IMU updated by GPS the EKF (extended Kalman filter) approach gave me very small residuals (less than 1 meter RMS from pseudoranges and 1 cm/sec from 1- sec carrier phase changes -- table near the bottom of http://jameslfarrell.com/wp-content/uploads/2012/03/GPSINS.pdf ), while airborne under severe vibration. The inertial instrument errors weren't anything like biases nor white noise; they could be modeled as "slowly-varying" (band-limited) for very short durations ONLY. The reason is, as I noted once before in this discussion, they come from a diverse collection of sources, many of which are motion-dependent. Those motions include vibrations as well as profile dynamics, generating error constituents proportional to rates and acceleration components, -- as well as their squares and
products, both translational and rotational, with in-phase and quadrature correlations across axes.
After a lifetime of dealing with those effects, including thousands of hours with in-flight data, I can assure you that a simple characterization for them is a very short-term phenomenon while they're in motion. The errors change within less than a minute.
Reply by ●August 13, 20172017-08-13
On Sunday, August 13, 2017 at 12:18:32 PM UTC+12, Tim Wescott wrote:
> On Thu, 10 Aug 2017 14:18:04 -0400, Randy Yates wrote:
>
> > Tim Wescott <tim@seemywebsite.really> writes:
> >> [...]
> >> I believe it's been done, but you probably won't find it driving down
> >> Kalman Road.
> >
> > Maybe Wiener Circle? Hopf Hwy?
>
> Could be. I don't know if there's an extension of the Wiener filter that
> works in multiple dimensions.
>
> I _do_ know that the Wiener spits out an answer that's based on the
> problem statement, not on any preconditions that you may think to
> impose. And I'm 99.44% sure that the Wiener filter, where it can be
> formulated at all, has an identical transfer function to the steady-state
> Kalman.
>
> --
> www.wescottdesign.com
Yes I am in fact using the steady-state Kalman filter or the Wiener filter (same thing).
Reply by Tim Wescott●August 12, 20172017-08-12
On Thu, 10 Aug 2017 14:18:04 -0400, Randy Yates wrote:
> Tim Wescott <tim@seemywebsite.really> writes:
>> [...]
>> I believe it's been done, but you probably won't find it driving down
>> Kalman Road.
>
> Maybe Wiener Circle? Hopf Hwy?
Could be. I don't know if there's an extension of the Wiener filter that
works in multiple dimensions.
I _do_ know that the Wiener spits out an answer that's based on the
problem statement, not on any preconditions that you may think to
impose. And I'm 99.44% sure that the Wiener filter, where it can be
formulated at all, has an identical transfer function to the steady-state
Kalman.
--
www.wescottdesign.com
Reply by Randy Yates●August 10, 20172017-08-10
Tim Wescott <tim@seemywebsite.really> writes:
> [...]
> I believe it's been done, but you probably won't find it driving down
> Kalman Road.
Maybe Wiener Circle? Hopf Hwy?
--
Randy Yates, Embedded Linux Developer
Garner Underground, Inc.
866-260-9040, x3901
http://www.garnerundergroundinc.com
Reply by Tim Wescott●August 6, 20172017-08-06
On Tue, 25 Jul 2017 13:57:15 -0700, gyansorova wrote:
> On Monday, July 24, 2017 at 10:02:03 AM UTC+12, Tim Wescott wrote:
>> On Sat, 22 Jul 2017 20:05:19 -0700, gyansorova wrote:
>>
>> > On Sunday, July 23, 2017 at 9:26:58 AM UTC+12, Tim Wescott wrote:
>> >> On Fri, 21 Jul 2017 19:21:46 -0700, gyansorova wrote:
>> >>
>> >> > I am using a fairly standard approach to estimating angular pitch
>> >> > using a KF. It uses both accelerometer and Gyro angle data. Now it
>> >> > estimated the angle fine enough and I implement the steady-state
>> >> > KF.
>> >> > Never tried this before but then put a PID or lag-lead controller
>> >> > on this measurement. I find that the Kalman filter bandwidth is
>> >> > stuff all and severely reduces the bandwidth of my closed-loop
>> >> > system - which is using classical control methods. Since the
>> >> > bandwidth is obviously proportional is some way to the additive
>> >> > noise covariances I am thinking to lower their value in the
>> >> > algorithm so that the filter bandwidth increases.
>> >>
>> >> What is your KF equation? Do you have any independent measurement
>> >> that you can use to correct your state estimates? Because without
>> >> some independent measurement (e.g. of position) then I'm not sure
>> >> that a KF isn't just a fancy name stuck on a fairly pedestrian
>> >> filter.
>> >>
>> >> And I hope you're aware that you simply cannot use a straight KF
>> >> when angles are involved -- at best you can use an EKF, but only if
>> >> the variance between your estimated angles and real angles are
>> >> small.
>> >>
>> >> > Anyway, the KF, although useful does not appear to have any
>> >> > measure of its own bandwidth so we can at least get an idea of the
>> >> > limitations. Obviously the more filtering in a control loop the
>> >> > more phase lag and less stability.
>> >>
>> >> The Kalman Filter, by itself, optimizes for a specific thing (sum of
>> >> squares of error) if and only if the model you bake into the KF
>> >> exactly matches reality. If the model matches reality enough -- and
>> >> there are no easy guides for answering "how big of a mismatch are we
>> >> talking of here?"
>> >> -- the filter goes unstable. This is a particular problem with the
>> >> extended Kalman filter (EKF), although (thankfully) less so with the
>> >> unscented Kalman filter (UKF).
>> >>
>> >> The Kalman Filter does NOT, in any way, shape, or form, promise to
>> >> be the best observer in a closed-loop controller, _unless_ your
>> >> model _exactly_ matches reality. And models never match reality.
>> >> And if the observer is already unstable, are things going to go well
>> >> in closed loop?
>> >>
>> >> I've had pretty good luck with the EKF for estimating orientation,
>> >> position, and velocity of a vehicle when the IMU was backed up with
>> >> GPS data -- but the UKF worked even better.
>> >>
>> >> Without knowing the details I can't give you any specific advise,
>> >> but here are some things to consider:
>> >>
>> >> * Get a good book. I have "Optimal State Estimation" by Simon, but
>> >> at ten years old it's already dated. I paid the bills for about two
>> >> years just by propping that book and Titterton's inertial navigation
>> >> book next to my computer screen and fitting what they said to my
>> >> customer's problem.
>> >>
>> >> If he's got a second or third edition out, get it -- hell, get the
>> >> first one, and just be aware that Progress Has Been Made in the area
>> >> of choosing constellations for UKFs.
>> >>
>> >> * Consider using an H-infinity filter. I haven't used one. They're
>> >> supposed to work much better when the model doesn't match reality,
>> >> and they have some easy-to-use "kludge" parameters. The downside is
>> >> that the computation at each step is intensive enough that it's
>> >> generally not done on-line -- which means that you could only use it
>> >> for the simplest of EKF formulations, and an unscented-ish version
>> >> would be straight out. You could probably use it to come up with
>> >> your steady-state gains, however.
>> >>
>> >> * If you're using this as an observer in a control system, consider
>> >> using a KF formulation to get an idea of the structure of the
>> >> observer,
>> >> then toss the actual gains in the trash and cook up your own.
>> >> You're smart enough to do the stability analysis at that point.
>> >>
>> >> --
>> >>
>> >> Tim Wescott Wescott Design Services http://www.wescottdesign.com
>> >
>> > Thanks, it's not the theory that's the problem so much as how do I
>> > know or define the bandwidth? Seems to me that as you say, I am only
>> > considering mse so I have no control over the bandwidth. So I could
>> > control something that is perfectly stable but only has 0.1Hz
>> > bandwidth.
>> > Usually in classical control we move the bandwidth up so we can get
>> > better tracking and faster response.
>> > here it is invisible.
>>
>> Yup.
>>
>> You may do better framing the problem as a linear quadratic problem.
>> You'll still be at the mercy of the algorithm for the resultant gains,
>> but you'll have more knobs to tweak (basically the control cost and the
>> error cost, IIRC).
>>
>> I have a -- probably excessively -- dim view of a lot of "optimal"
>> problems in control theory, because they often seem to fall apart upon
>> meeting with reality.
>>
>> --
>>
>> Tim Wescott Wescott Design Services http://www.wescottdesign.com
>
> Thanks yes I agree. I think we need an algorithm that works on bandwidth
> rather than nebulous concepts like mse.
I believe it's been done, but you probably won't find it driving down
Kalman Road.
--
www.wescottdesign.com
Reply by ●July 25, 20172017-07-25
On Monday, July 24, 2017 at 10:02:03 AM UTC+12, Tim Wescott wrote:
> On Sat, 22 Jul 2017 20:05:19 -0700, gyansorova wrote:
>
> > On Sunday, July 23, 2017 at 9:26:58 AM UTC+12, Tim Wescott wrote:
> >> On Fri, 21 Jul 2017 19:21:46 -0700, gyansorova wrote:
> >>
> >> > I am using a fairly standard approach to estimating angular pitch
> >> > using a KF. It uses both accelerometer and Gyro angle data. Now it
> >> > estimated the angle fine enough and I implement the steady-state KF.
> >> > Never tried this before but then put a PID or lag-lead controller on
> >> > this measurement. I find that the Kalman filter bandwidth is stuff
> >> > all and severely reduces the bandwidth of my closed-loop system -
> >> > which is using classical control methods. Since the bandwidth is
> >> > obviously proportional is some way to the additive noise covariances
> >> > I am thinking to lower their value in the algorithm so that the
> >> > filter bandwidth increases.
> >>
> >> What is your KF equation? Do you have any independent measurement that
> >> you can use to correct your state estimates? Because without some
> >> independent measurement (e.g. of position) then I'm not sure that a KF
> >> isn't just a fancy name stuck on a fairly pedestrian filter.
> >>
> >> And I hope you're aware that you simply cannot use a straight KF when
> >> angles are involved -- at best you can use an EKF, but only if the
> >> variance between your estimated angles and real angles are small.
> >>
> >> > Anyway, the KF, although useful does not appear to have any measure
> >> > of its own bandwidth so we can at least get an idea of the
> >> > limitations. Obviously the more filtering in a control loop the more
> >> > phase lag and less stability.
> >>
> >> The Kalman Filter, by itself, optimizes for a specific thing (sum of
> >> squares of error) if and only if the model you bake into the KF exactly
> >> matches reality. If the model matches reality enough -- and there are
> >> no easy guides for answering "how big of a mismatch are we talking of
> >> here?"
> >> -- the filter goes unstable. This is a particular problem with the
> >> extended Kalman filter (EKF), although (thankfully) less so with the
> >> unscented Kalman filter (UKF).
> >>
> >> The Kalman Filter does NOT, in any way, shape, or form, promise to be
> >> the best observer in a closed-loop controller, _unless_ your model
> >> _exactly_ matches reality. And models never match reality. And if the
> >> observer is already unstable, are things going to go well in closed
> >> loop?
> >>
> >> I've had pretty good luck with the EKF for estimating orientation,
> >> position, and velocity of a vehicle when the IMU was backed up with GPS
> >> data -- but the UKF worked even better.
> >>
> >> Without knowing the details I can't give you any specific advise, but
> >> here are some things to consider:
> >>
> >> * Get a good book. I have "Optimal State Estimation" by Simon, but at
> >> ten years old it's already dated. I paid the bills for about two years
> >> just by propping that book and Titterton's inertial navigation book
> >> next to my computer screen and fitting what they said to my customer's
> >> problem.
> >>
> >> If he's got a second or third edition out, get it -- hell, get the
> >> first one, and just be aware that Progress Has Been Made in the area of
> >> choosing constellations for UKFs.
> >>
> >> * Consider using an H-infinity filter. I haven't used one. They're
> >> supposed to work much better when the model doesn't match reality, and
> >> they have some easy-to-use "kludge" parameters. The downside is that
> >> the computation at each step is intensive enough that it's generally
> >> not done on-line -- which means that you could only use it for the
> >> simplest of EKF formulations, and an unscented-ish version would be
> >> straight out. You could probably use it to come up with your
> >> steady-state gains, however.
> >>
> >> * If you're using this as an observer in a control system, consider
> >> using a KF formulation to get an idea of the structure of the observer,
> >> then toss the actual gains in the trash and cook up your own. You're
> >> smart enough to do the stability analysis at that point.
> >>
> >> --
> >>
> >> Tim Wescott Wescott Design Services http://www.wescottdesign.com
> >
> > Thanks, it's not the theory that's the problem so much as how do I know
> > or define the bandwidth? Seems to me that as you say, I am only
> > considering mse so I have no control over the bandwidth. So I could
> > control something that is perfectly stable but only has 0.1Hz bandwidth.
> > Usually in classical control we move the bandwidth up so we can get
> > better tracking and faster response.
> > here it is invisible.
>
> Yup.
>
> You may do better framing the problem as a linear quadratic problem.
> You'll still be at the mercy of the algorithm for the resultant gains,
> but you'll have more knobs to tweak (basically the control cost and the
> error cost, IIRC).
>
> I have a -- probably excessively -- dim view of a lot of "optimal"
> problems in control theory, because they often seem to fall apart upon
> meeting with reality.
>
> --
>
> Tim Wescott
> Wescott Design Services
> http://www.wescottdesign.com
Thanks yes I agree. I think we need an algorithm that works on bandwidth rather than nebulous concepts like mse.
Reply by Tim Wescott●July 23, 20172017-07-23
On Sat, 22 Jul 2017 20:05:19 -0700, gyansorova wrote:
> On Sunday, July 23, 2017 at 9:26:58 AM UTC+12, Tim Wescott wrote:
>> On Fri, 21 Jul 2017 19:21:46 -0700, gyansorova wrote:
>>
>> > I am using a fairly standard approach to estimating angular pitch
>> > using a KF. It uses both accelerometer and Gyro angle data. Now it
>> > estimated the angle fine enough and I implement the steady-state KF.
>> > Never tried this before but then put a PID or lag-lead controller on
>> > this measurement. I find that the Kalman filter bandwidth is stuff
>> > all and severely reduces the bandwidth of my closed-loop system -
>> > which is using classical control methods. Since the bandwidth is
>> > obviously proportional is some way to the additive noise covariances
>> > I am thinking to lower their value in the algorithm so that the
>> > filter bandwidth increases.
>>
>> What is your KF equation? Do you have any independent measurement that
>> you can use to correct your state estimates? Because without some
>> independent measurement (e.g. of position) then I'm not sure that a KF
>> isn't just a fancy name stuck on a fairly pedestrian filter.
>>
>> And I hope you're aware that you simply cannot use a straight KF when
>> angles are involved -- at best you can use an EKF, but only if the
>> variance between your estimated angles and real angles are small.
>>
>> > Anyway, the KF, although useful does not appear to have any measure
>> > of its own bandwidth so we can at least get an idea of the
>> > limitations. Obviously the more filtering in a control loop the more
>> > phase lag and less stability.
>>
>> The Kalman Filter, by itself, optimizes for a specific thing (sum of
>> squares of error) if and only if the model you bake into the KF exactly
>> matches reality. If the model matches reality enough -- and there are
>> no easy guides for answering "how big of a mismatch are we talking of
>> here?"
>> -- the filter goes unstable. This is a particular problem with the
>> extended Kalman filter (EKF), although (thankfully) less so with the
>> unscented Kalman filter (UKF).
>>
>> The Kalman Filter does NOT, in any way, shape, or form, promise to be
>> the best observer in a closed-loop controller, _unless_ your model
>> _exactly_ matches reality. And models never match reality. And if the
>> observer is already unstable, are things going to go well in closed
>> loop?
>>
>> I've had pretty good luck with the EKF for estimating orientation,
>> position, and velocity of a vehicle when the IMU was backed up with GPS
>> data -- but the UKF worked even better.
>>
>> Without knowing the details I can't give you any specific advise, but
>> here are some things to consider:
>>
>> * Get a good book. I have "Optimal State Estimation" by Simon, but at
>> ten years old it's already dated. I paid the bills for about two years
>> just by propping that book and Titterton's inertial navigation book
>> next to my computer screen and fitting what they said to my customer's
>> problem.
>>
>> If he's got a second or third edition out, get it -- hell, get the
>> first one, and just be aware that Progress Has Been Made in the area of
>> choosing constellations for UKFs.
>>
>> * Consider using an H-infinity filter. I haven't used one. They're
>> supposed to work much better when the model doesn't match reality, and
>> they have some easy-to-use "kludge" parameters. The downside is that
>> the computation at each step is intensive enough that it's generally
>> not done on-line -- which means that you could only use it for the
>> simplest of EKF formulations, and an unscented-ish version would be
>> straight out. You could probably use it to come up with your
>> steady-state gains, however.
>>
>> * If you're using this as an observer in a control system, consider
>> using a KF formulation to get an idea of the structure of the observer,
>> then toss the actual gains in the trash and cook up your own. You're
>> smart enough to do the stability analysis at that point.
>>
>> --
>>
>> Tim Wescott Wescott Design Services http://www.wescottdesign.com
>
> Thanks, it's not the theory that's the problem so much as how do I know
> or define the bandwidth? Seems to me that as you say, I am only
> considering mse so I have no control over the bandwidth. So I could
> control something that is perfectly stable but only has 0.1Hz bandwidth.
> Usually in classical control we move the bandwidth up so we can get
> better tracking and faster response.
> here it is invisible.
Yup.
You may do better framing the problem as a linear quadratic problem.
You'll still be at the mercy of the algorithm for the resultant gains,
but you'll have more knobs to tweak (basically the control cost and the
error cost, IIRC).
I have a -- probably excessively -- dim view of a lot of "optimal"
problems in control theory, because they often seem to fall apart upon
meeting with reality.
--
Tim Wescott
Wescott Design Services
http://www.wescottdesign.com
Reply by ●July 23, 20172017-07-23
On Sunday, July 23, 2017 at 9:26:58 AM UTC+12, Tim Wescott wrote:
> On Fri, 21 Jul 2017 19:21:46 -0700, gyansorova wrote:
>
> > I am using a fairly standard approach to estimating angular pitch using
> > a KF. It uses both accelerometer and Gyro angle data. Now it estimated
> > the angle fine enough and I implement the steady-state KF. Never tried
> > this before but then put a PID or lag-lead controller on this
> > measurement. I find that the Kalman filter bandwidth is stuff all and
> > severely reduces the bandwidth of my closed-loop system - which is using
> > classical control methods. Since the bandwidth is obviously proportional
> > is some way to the additive noise covariances I am thinking to lower
> > their value in the algorithm so that the filter bandwidth increases.
>
> What is your KF equation? Do you have any independent measurement that
> you can use to correct your state estimates? Because without some
> independent measurement (e.g. of position) then I'm not sure that a KF
> isn't just a fancy name stuck on a fairly pedestrian filter.
>
> And I hope you're aware that you simply cannot use a straight KF when
> angles are involved -- at best you can use an EKF, but only if the
> variance between your estimated angles and real angles are small.
>
> > Anyway, the KF, although useful does not appear to have any measure of
> > its own bandwidth so we can at least get an idea of the limitations.
> > Obviously the more filtering in a control loop the more phase lag and
> > less stability.
>
> The Kalman Filter, by itself, optimizes for a specific thing (sum of
> squares of error) if and only if the model you bake into the KF exactly
> matches reality. If the model matches reality enough -- and there are no
> easy guides for answering "how big of a mismatch are we talking of here?"
> -- the filter goes unstable. This is a particular problem with the
> extended Kalman filter (EKF), although (thankfully) less so with the
> unscented Kalman filter (UKF).
>
> The Kalman Filter does NOT, in any way, shape, or form, promise to be the
> best observer in a closed-loop controller, _unless_ your model _exactly_
> matches reality. And models never match reality. And if the observer is
> already unstable, are things going to go well in closed loop?
>
> I've had pretty good luck with the EKF for estimating orientation,
> position, and velocity of a vehicle when the IMU was backed up with GPS
> data -- but the UKF worked even better.
>
> Without knowing the details I can't give you any specific advise, but
> here are some things to consider:
>
> * Get a good book. I have "Optimal State Estimation" by Simon, but at
> ten years old it's already dated. I paid the bills for about two years
> just by propping that book and Titterton's inertial navigation book next
> to my computer screen and fitting what they said to my customer's
> problem.
>
> If he's got a second or third edition out, get it -- hell, get the first
> one, and just be aware that Progress Has Been Made in the area of
> choosing constellations for UKFs.
>
> * Consider using an H-infinity filter. I haven't used one. They're
> supposed to work much better when the model doesn't match reality, and
> they have some easy-to-use "kludge" parameters. The downside is that the
> computation at each step is intensive enough that it's generally not done
> on-line -- which means that you could only use it for the simplest of EKF
> formulations, and an unscented-ish version would be straight out. You
> could probably use it to come up with your steady-state gains, however.
>
> * If you're using this as an observer in a control system, consider using
> a KF formulation to get an idea of the structure of the observer, then
> toss the actual gains in the trash and cook up your own. You're smart
> enough to do the stability analysis at that point.
>
> --
>
> Tim Wescott
> Wescott Design Services
> http://www.wescottdesign.com
Thanks, it's not the theory that's the problem so much as how do I know or define the bandwidth? Seems to me that as you say, I am only considering mse so I have no control over the bandwidth. So I could control something that is perfectly stable but only has 0.1Hz bandwidth. Usually in classical control we move the bandwidth up so we can get better tracking and faster response.
here it is invisible.
Reply by navaide●July 22, 20172017-07-22
On Friday, July 21, 2017 at 10:21:51 PM UTC-4, gyans...@gmail.com wrote:
> I am using a fairly standard approach to estimating angular pitch using a KF. It uses both accelerometer and Gyro angle data. Now it estimated the angle fine enough and I implement the steady-state KF. Never tried this before but then put a PID or lag-lead controller on this measurement. I find that the Kalman filter bandwidth is stuff all and severely reduces the bandwidth of my closed-loop system - which is using classical control methods. Since the bandwidth is obviously proportional is some way to the additive noise covariances I am thinking to lower their value in the algorithm so that the filter bandwidth increases.
>
> Anyway, the KF, although useful does not appear to have any measure of its own bandwidth so we can at least get an idea of the limitations. Obviously the more filtering in a control loop the more phase lag and less stability.
I'm glad that the subject of modeling error was broached in connection with this effort. Limitations due to imperfect models are in fact pivotal, especially for inertial instruments. At this point I'll take just enough time to mention the tip-of-the-iceberg for gyros http://jameslfarrell.com/gyro-mounting-misalignment/
and accelerometers https://www.youtube.com/watch?v=4H5M92OWr5Y . This won't hit the nail on the head for your immediate need, but awareness will be raised.