ecco wrote:> Ok, everyone, please allow me to abort my original question and > describe my real problem/question: > > I am trying to implement an FIR predictive filter (linear predictive > filter) for a particular kind of waveform. When I calculate (using > Matlab) the filter taps which correspond to the least-mean-square (LMS) > optimization, I find that the filter performs (in terms of entropy) > worst than [0 1], a simple d/dt.For a first order predictor, this filter is not bad choice, and even optimal in some cases (random walks). However, instead of LSE why don't you use maximum entropy methods directly? Look up the Burg predictor, which is available if you have the Signal Processing Toolbox. An adaptive version that also minimizes the Burg error is the Gradient Adaptive Lattice algorithm.> > This tells me that the LMS optimization is not appropriate (optimum) > for my waveforms. I guess I'm looking for alternative optimizations on > which to determine taps.What is you waveform?> > I realize that this is a "classic" problem and that there is no silver > bullet, but can anybody point me in the direction of alternative > optimizations which I might try (other than LMS)? I also realize that > I could also try opening the door to non-linear predictive filters. I > am exploring this option. As it turns out, samples in my waveform only > depend on the previous 5-10 samples; a search for a good non-linear > filter may be tractable.Linear prediction can go a long way, and non-linear predictors is a can of worms. The class of linearly predictable "waveforms" is very large (it includes exponentially damped sinusoids and polynomials), and could well be sufficient in your case. I still don't get why you want the flat magnitude and the linear phase response for the prediction filter. Such filter constraints will obviously increase the prediction error, for any given order. Apart from that, I wouldn't know how to compute a prediction filter with constraints. You might want to have a look at DE filter design. Regards, Andor
FIR filter with a flat transfer function
Started by ●December 20, 2006
Reply by ●December 21, 20062006-12-21
Reply by ●December 21, 20062006-12-21
I wrote:> ....DE filter design ...Sorry, I hate it when people do that (drop acronyms without reference): http://www.icsi.berkeley.edu/~storn/fiwiz.html
Reply by ●December 21, 20062006-12-21
ecco skrev:> Ok, everyone, please allow me to abort my original question and > describe my real problem/question: > > I am trying to implement an FIR predictive filter (linear predictive > filter) for a particular kind of waveform. When I calculate (using > Matlab) the filter taps which correspond to the least-mean-square (LMS) > optimization, I find that the filter performs (in terms of entropy) > worst than [0 1], a simple d/dt.How do you find that h[n]=[0 1] is a d/dt operator? As far as I can tell, it's a bure delay of exactly one sample.> This tells me that the LMS optimization is not appropriate (optimum) > for my waveforms. I guess I'm looking for alternative optimizations on > which to determine taps.It's impossible to see from your description what you might be looking for. What have you done to exclude the usual AR predictors?> I realize that this is a "classic" problem and that there is no silver > bullet, but can anybody point me in the direction of alternative > optimizations which I might try (other than LMS)?Genetic Algorithms or neural nets seem to be considered the answers of all hard questions these days... they might not work but they keep you in tune with "current research".> As it turns out, samples in my waveform only > depend on the previous 5-10 samples;How did you establish this? What method did you use? What numbers popped out of your analysis? Rune
Reply by ●December 21, 20062006-12-21
Thanks, Andor I'll give Burg a shot. How about RLS (recursive lease squares)? My waveform is of biological origin. ec Andor wrote:> ecco wrote: > > Ok, everyone, please allow me to abort my original question and > > describe my real problem/question: > > > > I am trying to implement an FIR predictive filter (linear predictive > > filter) for a particular kind of waveform. When I calculate (using > > Matlab) the filter taps which correspond to the least-mean-square (LMS) > > optimization, I find that the filter performs (in terms of entropy) > > worst than [0 1], a simple d/dt. > > For a first order predictor, this filter is not bad choice, and even > optimal in some cases (random walks). > > However, instead of LSE why don't you use maximum entropy methods > directly? Look up the Burg predictor, which is available if you have > the Signal Processing Toolbox. An adaptive version that also minimizes > the Burg error is the Gradient Adaptive Lattice algorithm. > > > > > This tells me that the LMS optimization is not appropriate (optimum) > > for my waveforms. I guess I'm looking for alternative optimizations on > > which to determine taps. > > What is you waveform? > > > > > I realize that this is a "classic" problem and that there is no silver > > bullet, but can anybody point me in the direction of alternative > > optimizations which I might try (other than LMS)? I also realize that > > I could also try opening the door to non-linear predictive filters. I > > am exploring this option. As it turns out, samples in my waveform only > > depend on the previous 5-10 samples; a search for a good non-linear > > filter may be tractable. > > Linear prediction can go a long way, and non-linear predictors is a can > of worms. The class of linearly predictable "waveforms" is very large > (it includes exponentially damped sinusoids and polynomials), and could > well be sufficient in your case. > > I still don't get why you want the flat magnitude and the linear phase > response for the prediction filter. Such filter constraints will > obviously increase the prediction error, for any given order. Apart > from that, I wouldn't know how to compute a prediction filter with > constraints. You might want to have a look at DE filter design. > > Regards, > Andor
Reply by ●December 21, 20062006-12-21
ecco wrote:> Thanks, Andor > > I'll give Burg a shot. How about RLS (recursive lease squares)? My > waveform is of biological origin.I can't help but think that you made an error somewhere in your Matlab analysis ... any predictor of order >= 3 drastically reduces the prediction error compared to your [0 1] predictor on almost any type of signal. Note that for computing predictor filters, you never actually have the 0 coefficient tap included. Ie. your prediction filter is actually [1] and not [0 1]. See the help files for the AR modeling techniques in Matlab. Regards, Andor
Reply by ●December 21, 20062006-12-21
Andor wrote:> > ecco wrote: >> Thanks, Andor >> >> I'll give Burg a shot. How about RLS (recursive lease squares)? My >> waveform is of biological origin. > > I can't help but think that you made an error somewhere in your Matlab > analysis ... any predictor of order >= 3 drastically reduces the > prediction error compared to your [0 1] predictor on almost any type of > signal.Andor, would you please explain to me why one would want to "predict" a sample already in hand?> Note that for computing predictor filters, you never actually have the > 0 coefficient tap included. Ie. your prediction filter is actually [1] > and not [0 1]. See the help files for the AR modeling techniques in > Matlab.In that case, I could predict the *next* sample, which might have some value if the sample rate is low. Jerry -- Engineering is the art of making what you want from things you can get. ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Reply by ●December 21, 20062006-12-21
Fred Marshall wrote:> "Jerry Avins" <jya@ieee.org> wrote in message > news:2IOdnYwOptwCYhTYnZ2dnUVZ_h63nZ2d@rcn.net... >> ecco wrote to me privately:ecco wrote to me privately: >> >>> Hi, Jerry >>> What I'm building is a predictive filter. The idea is to form an >>> estimate of the current sample based on previous samples. The >>> difference between the estimate and the real signal is what is stored, >>> ergo, the 0th order tap of the filter must be 0. Thanks for spending >>> time thinking about this! >> And I answered him so: >> >> I don't understand the reasoning that leads up to that condition. >> >> Let's keep this in the newsgroup, please. >> >> Jerry >> -- > > I think the time base in this discussion is not well described. Here is how > I understand a predictor: > > 1) You *design* a predictor by delaying the input and then making the best > estimator (filter) that generates the future sample(s) that are already > known. I suppose this equates to having the first or more filter taps equal > to zero to form that delay. The optimization is done by comparing with the > current input and minimizing the error (difference). > > 2) You *apply* the predictor (filter) to the current samples so that the > future samples are predicted. That is, you remove the delay and apply the > same filter to the current input stream. > > In the first case (design) the filter may appear to be noncausal if you > ignore the zero-valued taps at the beginning of the filter and you may > define time=0 as a time in the future. > > In the second case (application), the filter can't be noncausal. Well, > unless you store the data and "predict" later.... so then it's not real time > and it's (like totally) not necessary to predict.Thanks Fred. That makes a lot of sense. I did that sort of thing when all I knew about digital signals was A/D and D/A conversion. It was for the purpose of predicting -- estimating if you like -- what the signal would be if only I could measure it without all that damned noise. It worked, and looking back at what I did, it was a least-squares approximation implemented as an FIR. Linear regression is a powerful tool, apparently even in the hands of the ignorant. Jerry -- Engineering is the art of making what you want from things you can get. ¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
Reply by ●December 21, 20062006-12-21
Jerry Avins wrote:> Carlos Moreno wrote: > >> Jerry Avins wrote: >> >>>>>> [...] >>>>> >>>>> Moreover, with the first tap set to zero, the filter can't >>>>> incorporate the newest data into its output. >>>> >>>> >>>> ????? >>>> >>>> Care to elaborate on what exactly do you mean by that?? >>>> >>>> A *prediction* filter by definition can not include the *current* >>>> sample (which is the one being predicted). >>>> >>>> Sure, the filter that computes the prediction error (perhaps to >>>> be transmitted) does use the current sample --- to subtract it >>>> from the output of the predictor filter (which only takes into >>>> account past samples). >>> >>> >>> A filter can't effectively use the current sample if its coefficient >>> is zero. >> >> >> In the example of the LPC *encoding filter* (not *prediction* filter), >> the coefficient for the current sample is not zero, it's 1. >> >> If P(z) is the *prediction* filter (which has the coefficient for >> the current sample = 0), then the *encoding* filter has transfer >> function 1 - P(z) (the encoding filter simply produces the >> prediction error). >> >> I brought this up as a way of trying to find some applicability to >> your comment, but pointing out that you may have been confusing >> a *prediction* filter with the *encoding* filter in an LPC system >> (apparently, that's not the example that you had in mind?) >> >>>> Am I misunderstanding, or missing something? >>> >>> >>> Not necessarily; maybe I am. Why would want to to "predict" the value >>> of a sample already taken? >> >> >> What's going on here??? Some major confusion must be happening, >> because the above should be precisely my argument to your comment. >> >> You said (quoted above): "with the first tap set to zero, the >> filter can't incorporate the newest data into its output" >> >> But then, if the filter is a *predictor*, then you don't >> incorporate the newest data --- if you have the data, then >> no need to predict it (as I said, what you ask in this last >> message is precisely my argument to your initial comment, >> which I understood as you were implying that it doesn't make >> sense to have the first tap of the filter = 0 because you >> need to incorporate the newest sample into the output and >> with the first tap being 0 you can't ... Did I misread >> something?) > > > I don't know. Maybe I misread something. What is the aim if prediction? > Surely it can't be the past (except in the special case of validating a > model and then we pretend it's the future) and it makes no sense for it > to be the present, so it must be the future. (That's usually what > predictions are for, no?) What's wrong with accounting for the present > in an attempt to predict the future? Is there any advantage to > discounting it?OH!!! Ok, I see now what you meant !!!! You're absolutely right in your concern!!! I guess it all boils down to the "frame of reference" that we use, along with the intent (what we want the system to do). I mean, the present sample was the future one sample ago; in LPC systems, the idea is that the *receiver* has to reconstruct the signal based on the past samples *and the prediction error* --- you have the receiver predict what the current sample is going to be, based on the *past samples only*; that way, you don't have to transmit the (spacious, in terms of amount of bits) current sample, and instead we only transmit the prediction error (so that the receiver can adjust and reconstruct the signal perfectly --- if the prediction error is transmitted intact). Then, of course, the filter is a predictor of the current sample pretending that we have not seen it (because the receiver has not, and will not, see it --- well, it will see it after it *predicts it* and we tell it exactly what as its error in the prediction). Having worked recently in LPC encoding systems for speech, I immediately linked the OP's description to linear prediction filters for this sort of situation, and that's why it didn't surprise me that he wanted the 0th order tap of the filter = 0. I hope this clears up the misunderstanding! Carlos --
Reply by ●December 21, 20062006-12-21
Fred Marshall wrote:> In the second case (application), the filter can't be noncausal. Well, > unless you store the data and "predict" later.... so then it's not real timeBut then it's not noncausal --- it's only using data that has occured in the past. (yeah, I know it's borderline a philosophical debate --- I understand what you mean; conceptually, it is a non-causal filter; a filter with a mathematical formulation that makes it non-causal) Carlos --
Reply by ●December 21, 20062006-12-21
Jerry Avins wrote:> Andor wrote: > > > ecco wrote: > >> Thanks, Andor > > >> I'll give Burg a shot. How about RLS (recursive lease squares)? My > >> waveform is of biological origin. > > > I can't help but think that you made an error somewhere in your Matlab > > analysis ... any predictor of order >= 3 drastically reduces the > > prediction error compared to your [0 1] predictor on almost any type of > > signal. > Andor, would you please explain to me why one would want to "predict" a > sample already in hand?I think ecco is just confusing the notation. A p-th order (one-step) linear predictor does exactly what its name implies: it estimates x[n] from a linear combination of the past samles x[n-1], ..., x[n-p]. Where in time you take the samples from is irrelevant. For example, in linear predictive coding you estimate the current sample x[n], and only transmit the prediction error and the prediction coefficients (which remain constant for some amount of time). For prediction, you estimtate the value x[n+1] (the future) from the current and the past.> > > Note that for computing predictor filters, you never actually have the > > 0 coefficient tap included. Ie. your prediction filter is actually [1] > > and not [0 1]. See the help files for the AR modeling techniques in > > Matlab. > In that case, I could predict the *next* sample, which might have some > value if the sample rate is low.Why would that depend on the sample rate? The ability to linearly predict the next sample given the last p samples in a row only depends only how much the samples are linearly correlated, and not at all on the sample rate. Regards, Andor
