This is probably going to sound at least vague and possibly
stupid as well (to those well-versed in the field) but I
ask anyway in hopes of learning something.
[brogan], in discussing the "concept of state" (section 9.2),
states
Knowledge of the state at some initial time t_0, plus knowledge of
the system inputs after t_0, allows the determination of the state
at a later time t_1.
This sounds to me a lot like the definition of a Markov process, which
[viniotis] describes as:
Intuitively, the random process is a Markov process if "its future
is independent of the past, when the present is known."
I'm having a hard time quantifying exactly what the relationship
is (if any) between the concept of state and a Markov process.
The questions that come to mind are:
1. In a state space control system, is the input,
the system, or the output a MP?
2. Whatever the answer to 1 is, it isn't random at all, is it,
since, e.g., both the state x(k) and the output y(k) are a
deterministic function of the input u[k]:
x[k+1] = f(x[k], u[k], k)
y[k] = h(x[k], u[k], k)
Any insight would be appreciated.
--Randy
@book{brogan,
title = "Modern Control Theory",
author = "William L. Brogan",
publisher = "Prentice-Hall",
edition = "second",
year = "1985"}
@BOOK{viniotis,
title = "{Probability and Random Processes for Electrical Engineers}",
author = "{Yannis~Viniotis}",
publisher = "WCB McGraw-Hill",
year = "1998"}
--
% Randy Yates % "Bird, on the wing,
%% Fuquay-Varina, NC % goes floating by
%%% 919-577-9882 % but there's a teardrop in his eye..."
%%%% <yates@ieee.org> % 'One Summer Dream', *Face The Music*, ELO
http://home.earthlink.net/~yatescr
Relationship Between State Space and Markov Process
Started by ●March 7, 2007
Reply by ●March 7, 20072007-03-07
Randy Yates wrote:> This is probably going to sound at least vague and possibly > stupid as well (to those well-versed in the field) but I > ask anyway in hopes of learning something. > > [brogan], in discussing the "concept of state" (section 9.2), > states > > Knowledge of the state at some initial time t_0, plus knowledge of > the system inputs after t_0, allows the determination of the state > at a later time t_1. > > This sounds to me a lot like the definition of a Markov process, which > [viniotis] describes as: > > Intuitively, the random process is a Markov process if "its future > is independent of the past, when the present is known." > > I'm having a hard time quantifying exactly what the relationship > is (if any) between the concept of state and a Markov process.At first glance, [even] "its future is independent of the past, when the present is known" ^^^^^^^^^^^ ^ seems to directly contradict Knowledge of the state at some initial time t_0, plus knowledge of the system inputs after t_0, allows the determination of the state at a later time t_1. Where is the problem? Jerry -- Engineering is the art of making what you want from things you can get. ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Reply by ●March 7, 20072007-03-07
Jerry Avins <jya@ieee.org> writes:> Randy Yates wrote: >> This is probably going to sound at least vague and possibly >> stupid as well (to those well-versed in the field) but I >> ask anyway in hopes of learning something. >> [brogan], in discussing the "concept of state" (section 9.2), >> states >> Knowledge of the state at some initial time t_0, plus knowledge of >> the system inputs after t_0, allows the determination of the state >> at a later time t_1. >> This sounds to me a lot like the definition of a Markov process, >> which >> [viniotis] describes as: >> Intuitively, the random process is a Markov process if "its future >> is independent of the past, when the present is known." >> I'm having a hard time quantifying exactly what the relationship >> is (if any) between the concept of state and a Markov process. > > At first glance, > [even] > "its future is independent of the past, when the present is known" > ^^^^^^^^^^^ ^ > > seems to directly contradict > > Knowledge of the state at some initial time t_0, plus knowledge of > the system inputs after t_0, allows the determination of the state > at a later time t_1.In both cases, the future is "independent" of the past given the present. I don't see a contradiction. It doesn't say that the future is *unconditionally* independent of the past, but rather that, given the prresent (or conditioned on the present), the future is independent of the past. I took the Markov process definition somewhat out of context. If X(t) is a random process, then at any time t=tau, X(tau) is a random variable, and that random variable can be conditioned on some event, e.g., X(t) is the temperature and A is the season is "summer" implies the cumulative distribution function F_{X(tau)|A}(x) = Pr(X(tau) <= x | A) would be the probability that the temperature at time tau is less than x given that it is summer. If we denote the event B = x(tau) we can write this as P(B | A), and then use the property of conditional probability that P(B | A) = P(B,A) / P(A). If A and B are "independent," then P(B,A) = P(B)P(A) (by definition), and thus P(B | A) = P(B). So in this sense, the term "independent" has a specific, probabilistic meaning here. So I guess you could say that a Markov process has the property that f((X(tau) | X(t_b)) | X(t_a)) = f(X(tau) | X(t_b)) when t_a < t_b < tau. -- % Randy Yates % "Though you ride on the wheels of tomorrow, %% Fuquay-Varina, NC % you still wander the fields of your %%% 919-577-9882 % sorrow." %%%% <yates@ieee.org> % '21st Century Man', *Time*, ELO http://home.earthlink.net/~yatescr
Reply by ●March 8, 20072007-03-08
Randy Yates wrote:> Jerry Avins <jya@ieee.org> writes:...>> At first glance, >> [even] >> "its future is independent of the past, when the present is known" >> ^^^^^^^^^^^ ^ >> >> seems to directly contradict >> >> Knowledge of the state at some initial time t_0, plus knowledge of >> the system inputs after t_0, allows the determination of the state >> at a later time t_1. > > In both cases, the future is "independent" of the past given the > present. I don't see a contradiction. > > It doesn't say that the future is *unconditionally* independent of > the past, but rather that, given the prresent (or conditioned on the > present), the future is independent of the past.How does Knowledge of the state at some initial time t_0, plus knowledge of the system inputs after t_0, allows the determination of the state at a later time t_1. ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯ smack of independence? What am I missing? Jerry -- Engineering is the art of making what you want from things you can get. ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Reply by ●March 8, 20072007-03-08
Jerry Avins <jya@ieee.org> writes:> Randy Yates wrote: >> Jerry Avins <jya@ieee.org> writes: > > ... > >>> At first glance, >>> [even] >>> "its future is independent of the past, when the present is known" >>> ^^^^^^^^^^^ ^ >>> >>> seems to directly contradict >>> >>> Knowledge of the state at some initial time t_0, plus knowledge of >>> the system inputs after t_0, allows the determination of the state >>> at a later time t_1. >> In both cases, the future is "independent" of the past given the >> present. I don't see a contradiction. It doesn't say that the future >> is *unconditionally* independent of >> the past, but rather that, given the prresent (or conditioned on the >> present), the future is independent of the past. > > How does > > Knowledge of the state at some initial time t_0, plus knowledge of > the system inputs after t_0, allows the determination of the state > at a later time t_1. ������������������������������������� > > smack of independence? What am I missing?It is independent in the sense that, if you had the state at some initial time t_b (to be consistent with my previous notation), then it *doesn't matter* what the state was at time t_a. That, I would say, is pretty near to "independent." 7. (Math.) Not dependent upon another quantity in respect to value or rate of variation; -- said of quantities or functions. 1913 Webster -- % Randy Yates % "And all that I can do %% Fuquay-Varina, NC % is say I'm sorry, %%% 919-577-9882 % that's the way it goes..." %%%% <yates@ieee.org> % Getting To The Point', *Balance of Power*, ELO http://home.earthlink.net/~yatescr
Reply by ●March 8, 20072007-03-08
On Thu, 08 Mar 2007 01:12:47 -0000, Randy Yates <yates@ieee.org> wrote:> This is probably going to sound at least vague and possibly > stupid as well (to those well-versed in the field) but I > ask anyway in hopes of learning something. > > [brogan], in discussing the "concept of state" (section 9.2), > states > > Knowledge of the state at some initial time t_0, plus knowledge of > the system inputs after t_0, allows the determination of the state > at a later time t_1. > > This sounds to me a lot like the definition of a Markov process, which=> [viniotis] describes as: > > Intuitively, the random process is a Markov process if "its future > is independent of the past, when the present is known." > > I'm having a hard time quantifying exactly what the relationship > is (if any) between the concept of state and a Markov process.If the input to a state-space model can be described by a probability = distribution, then I would say that the state is indeed a (first-order) = = Markov process, in that it satisfies independence from the past: p(x[k+1] | x[k],x[k-1],...,x[0]) =3D p(x[k+1] | x[k]) If instead we treat the input sequence to the model as deterministic, th= en = the relationship above becomes meaningless. I don't know whether Markov= = processes can be treated in a deterministic framework, but Googling for = = 'deterministic "Markov chain"' brings up plenty of results. -- = Oli
Reply by ●March 8, 20072007-03-08
On 8 mar, 02:12, Randy Yates <y...@ieee.org> wrote:> This is probably going to sound at least vague and possibly > stupid as well (to those well-versed in the field) but I > ask anyway in hopes of learning something. > > [brogan], in discussing the "concept of state" (section 9.2), > states > > Knowledge of the state at some initial time t_0, plus knowledge of > the system inputs after t_0, allows the determination of the state > at a later time t_1. > > This sounds to me a lot like the definition of a Markov process, which > [viniotis] describes as: > > Intuitively, the random process is a Markov process if "its future > is independent of the past, when the present is known." > > I'm having a hard time quantifying exactly what the relationship > is (if any) between the concept of state and a Markov process. > The questions that come to mind are: > > 1. In a state space control system, is the input, > the system, or the output a MP? > > 2. Whatever the answer to 1 is, it isn't random at all, is it, > since, e.g., both the state x(k) and the output y(k) are a > deterministic function of the input u[k]: > > x[k+1] = f(x[k], u[k], k) > y[k] = h(x[k], u[k], k) > > Any insight would be appreciated. > > --Randy > > @book{brogan, > title = "Modern Control Theory", > author = "William L. Brogan", > publisher = "Prentice-Hall", > edition = "second", > year = "1985"} > @BOOK{viniotis, > title = "{Probability and Random Processes for Electrical Engineers}", > author = "{Yannis~Viniotis}", > publisher = "WCB McGraw-Hill", > year = "1998"} > > -- > % Randy Yates % "Bird, on the wing, > %% Fuquay-Varina, NC % goes floating by > %%% 919-577-9882 % but there's a teardrop in his eye..." > %%%% <y...@ieee.org> % 'One Summer Dream', *Face The Music*, ELOhttp://home.earthlink.net/~yatescrhi, if the output of your system is a deterministic function it is not, i think a markov process. For exemple for a 2 state markov process, you determine the probability to be in state one and the probability to be in state 2. Then the probability to be in state 1 at T0 and to be in state 1 at T1. the probabilities to change 1 to 2 and 2 to 1. There is no deterministic function because the probability should be 1 in that case Frederic
Reply by ●March 8, 20072007-03-08
On Thu, 08 Mar 2007 04:15:34 -0000, Jerry Avins <jya@ieee.org> wrote:> Randy Yates wrote: >> Jerry Avins <jya@ieee.org> writes: > > ... > >>> At first glance, >>> [even] >>> "its future is independent of the past, when the present is known" >>> ^^^^^^^^^^^ ^ >>> >>> seems to directly contradict >>> >>> Knowledge of the state at some initial time t_0, plus knowledge of >>> the system inputs after t_0, allows the determination of the state >>> at a later time t_1. >> In both cases, the future is "independent" of the past given the >> present. I don't see a contradiction. It doesn't say that the future >> is *unconditionally* independent of >> the past, but rather that, given the prresent (or conditioned on the >> present), the future is independent of the past. > > How does > > Knowledge of the state at some initial time t_0, plus knowledge of > the system inputs after t_0, allows the determination of the state > at a later time t_1. ������������������������������������� > > smack of independence? What am I missing? >Assuming a first-order state-space model, then if you know x[k], knowledge of x[m] for any m < k gives you no more information about what x[k+1] might be. So conditioned on x[k], x[k+1] is indeed independent of x[m]. -- Oli
Reply by ●March 8, 20072007-03-08
Randy Yates wrote: ...> 7. (Math.) Not dependent upon another quantity in respect to > value or rate of variation; -- said of quantities or > functions. > 1913 WebsterLet's start over with Knowledge of the state at some initial time t_0, plus knowledge of the system inputs after t_0, allows the determination of the state at a later time t_1. Paraphrase: From the knowledge the state of a system at some initial time and all inputs to that system after that time, then its current state can be determined. What is indeterminate? Jerry -- Engineering is the art of making what you want from things you can get. ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Reply by ●March 8, 20072007-03-08
"Oli Charlesworth" <catch@olifilth.co.uk> writes:> On Thu, 08 Mar 2007 01:12:47 -0000, Randy Yates <yates@ieee.org> wrote: > >> This is probably going to sound at least vague and possibly >> stupid as well (to those well-versed in the field) but I >> ask anyway in hopes of learning something. >> >> [brogan], in discussing the "concept of state" (section 9.2), >> states >> >> Knowledge of the state at some initial time t_0, plus knowledge of >> the system inputs after t_0, allows the determination of the state >> at a later time t_1. >> >> This sounds to me a lot like the definition of a Markov process, which >> [viniotis] describes as: >> >> Intuitively, the random process is a Markov process if "its future >> is independent of the past, when the present is known." >> >> I'm having a hard time quantifying exactly what the relationship >> is (if any) between the concept of state and a Markov process. > > If the input to a state-space model can be described by a probability > distribution, then I would say that the state is indeed a > (first-order) Markov process, in that it satisfies independence from > the past: > > p(x[k+1] | x[k],x[k-1],...,x[0]) = p(x[k+1] | x[k])Hi Oli, Is this because for any MP X(t), f(X(t)) is also an MP, and we consider each of the state variables a function of the input? If so, then how do you justify X(t) MP ==> f(X(t)) MP? -- % Randy Yates % "How's life on earth? %% Fuquay-Varina, NC % ... What is it worth?" %%% 919-577-9882 % 'Mission (A World Record)', %%%% <yates@ieee.org> % *A New World Record*, ELO http://home.earthlink.net/~yatescr
