Assume that if sampling DOES occur it will be more frequent than once/minute. Assume also that underlying phenomena have periods of days, months, or years. Assume data recording outages on order of hours to days. What can be done? what questions should I be asking?
Analyzing irregularly sampled data -- a neophyte question
Started by ●June 6, 2005
Reply by ●June 6, 20052005-06-06
Richard Owlett wrote:> Assume that if sampling DOES occur it will be more frequent than > once/minute. > > Assume also that underlying phenomena have periods of days, months, or > years. > > Assume data recording outages on order of hours to days. > > What can be done? > what questions should I be asking? > > >At this point, assume that you are offered a pickle or a motorcycle. Which do you prefer?
Reply by ●June 6, 20052005-06-06
Reply by ●June 6, 20052005-06-06
"Richard Owlett" <rowlett@atlascomm.net> wrote in message news:11a9f08hv1v19fd@corp.supernews.com...> Assume that if sampling DOES occur it will be more frequent than > once/minute. > > Assume also that underlying phenomena have periods of days, months, or > years. > > Assume data recording outages on order of hours to days. > > What can be done? > what questions should I be asking?The underlying phenomena have periods of days (or more) So, one question you should ask is: "what is the smallest number of days possible?" Data recording outages occur on the order of hours to days. So, another question you should ask is: "what is the maximum outage?" If the maximum outage is greater than 1/2 the smallest number of days possible in the underlying phenomena being sampled then you have a problem because the sample rate is inadequate. There is no good way to interpolate or extrapolate through an epoch that lacks needed data. That the sampling can occur more frequently as once per minute is probably inconsequential overkill - not terribly useful information except to assure that the sample rate *might* be high enough. What's more important is the minimum sample rate defined by the outages. Fred
Reply by ●June 6, 20052005-06-06
Stan Pawlukiewicz wrote:> Richard Owlett wrote: > >> Assume that if sampling DOES occur it will be more frequent than >> once/minute. >> >> Assume also that underlying phenomena have periods of days, months, or >> years. >> >> Assume data recording outages on order of hours to days. >> >> What can be done? >> what questions should I be asking? >> >> >> > At this point, assume that you are offered a pickle or a motorcycle. > > Which do you prefer? >a well done steak ;)
Reply by ●June 6, 20052005-06-06
Fred Marshall wrote:> "Richard Owlett" <rowlett@atlascomm.net> wrote in message > news:11a9f08hv1v19fd@corp.supernews.com... > >>Assume that if sampling DOES occur it will be more frequent than >>once/minute. >> >>Assume also that underlying phenomena have periods of days, months, or >>years. >> >>Assume data recording outages on order of hours to days. >> >>What can be done? >>what questions should I be asking? > > > The underlying phenomena have periods of days (or more) > So, one question you should ask is: "what is the smallest number of days > possible?" > > [snip]> What's more important is the> minimum sample rate defined by the outages. >I was suspecting that. Does it make any difference that I have some 'a priori' knowledge of the rep rate of underlying phenomena?
Reply by ●June 6, 20052005-06-06
Richard Owlett wrote:> Assume that if sampling DOES occur it will be more frequent than > once/minute. > > Assume also that underlying phenomena have periods of days, months, or > years. > > Assume data recording outages on order of hours to days. > > What can be done? > what questions should I be asking?"How much does a good UPS cost?" Jerry -- Engineering is the art of making what you want from things you can get. �����������������������������������������������������������������������
Reply by ●June 7, 20052005-06-07
Richard Owlett <rowlett@atlascomm.net> writes:> Fred Marshall wrote: > > > "Richard Owlett" <rowlett@atlascomm.net> wrote in message > > news:11a9f08hv1v19fd@corp.supernews.com... > > > >>Assume that if sampling DOES occur it will be more frequent than > >>once/minute. > >> > >>Assume also that underlying phenomena have periods of days, months, or > >>years. > >> > >>Assume data recording outages on order of hours to days. > >> > >>What can be done? > >>what questions should I be asking? > > > > > > The underlying phenomena have periods of days (or more) > > So, one question you should ask is: "what is the smallest number of days > > possible?" > > > > [snip] > > What's more important is the > > minimum sample rate defined by the outages. > > > > I was suspecting that. Does it make any difference that I have some 'a > priori' knowledge of the rep rate of underlying phenomena?Yes. It would help a lot if you would give some specifics. For example, one problem that fits your abstract description is tide prediction. The tidal constituents have periods that vary considerably--the more important ones have periods from about 3 hours to 0.5 years. Outages don't affect analysis of tidal data much because the underlying phenomena are well understood (and periodic!). The preferred technique for analysis is LSHA, or Least Squares Harmonic Analysis. This works because each constituent "j" is known to have the form: a_j cos(sum[k=1,5; n_{jk} omega_k t] + theta_j) where the a_j's represent amplitudes, the theta_j,s represent phases, the n_{jk}'s are small integers and the omega_k's are five fundamental angular velocities resulting from the earth's and the moon's orbits. The n_{jk}'s identify the constituent, and a least squares analysis solves for the a_j and theta_j which best fit the data. Scott -- Scott Hemphill hemphill@alumni.caltech.edu "This isn't flying. This is falling, with style." -- Buzz Lightyear
Reply by ●June 7, 20052005-06-07
Scott Hemphill wrote:> Richard Owlett <rowlett@atlascomm.net> writes: > > >>Fred Marshall wrote: >> >> >>>"Richard Owlett" <rowlett@atlascomm.net> wrote in message >>>news:11a9f08hv1v19fd@corp.supernews.com... >>> >>> >>>>Assume that if sampling DOES occur it will be more frequent than >>>>once/minute. >>>> >>>>Assume also that underlying phenomena have periods of days, months, or >>>>years. >>>> >>>>Assume data recording outages on order of hours to days. >>>> >>>>What can be done? >>>>what questions should I be asking? >>> >>> >>>The underlying phenomena have periods of days (or more) >>>So, one question you should ask is: "what is the smallest number of days >>>possible?" >>> >>>[snip] >> >> > What's more important is the >> >>>minimum sample rate defined by the outages. >>> >> >>I was suspecting that. Does it make any difference that I have some 'a >>priori' knowledge of the rep rate of underlying phenomena? > > > Yes. It would help a lot if you would give some specifics. For example, > one problem that fits your abstract description is tide prediction.I didn't have anything specific in mind. I had just been trying to think out some "what if" scenarios. Tides would come close to where my mind had wandered. The> tidal constituents have periods that vary considerably--the more important > ones have periods from about 3 hours to 0.5 years. Outages don't affect > analysis of tidal data much because the underlying phenomena are well > understood (and periodic!). The preferred technique for analysis is LSHA, > or Least Squares Harmonic Analysis. This works because each constituent > "j" is known to have the form: > > a_j cos(sum[k=1,5; n_{jk} omega_k t] + theta_j) > > where the a_j's represent amplitudes, the theta_j,s represent phases, the > n_{jk}'s are small integers and the omega_k's are five fundamental > angular velocities resulting from the earth's and the moon's orbits. > The n_{jk}'s identify the constituent, and a least squares analysis solves > for the a_j and theta_j which best fit the data. > > Scott
Reply by ●June 7, 20052005-06-07
"Richard Owlett" <rowlett@atlascomm.net> wrote in message news:11a9nhin5bin8ab@corp.supernews.com...> Fred Marshall wrote: > >> "Richard Owlett" <rowlett@atlascomm.net> wrote in message >> news:11a9f08hv1v19fd@corp.supernews.com... >> >>>Assume that if sampling DOES occur it will be more frequent than >>>once/minute. >>> >>>Assume also that underlying phenomena have periods of days, months, or >>>years. >>> >>>Assume data recording outages on order of hours to days. >>> >>>What can be done? >>>what questions should I be asking? >> >> >> The underlying phenomena have periods of days (or more) >> So, one question you should ask is: "what is the smallest number of days >> possible?" >> >> [snip] > > What's more important is the >> minimum sample rate defined by the outages. >> > > I was suspecting that. Does it make any difference that I have some 'a > priori' knowledge of the rep rate of underlying phenomena? > >"rep rate"??? That implies that the data is periodic. If it's really periodic over all, then all you need to do is capture one period at some suitable rate to get the bandwidth covered. After that, it's all repetitious. Somehow I don't think that's what you really meant. If the SNR is less than great then perhaps some number of periods would help. Fred
