Hi all, I am to design a special adaptive filter in which the error signal is hihgly correlated with the input signal (in fact, the latter is just a delayed version of the error...) Usual resolution of the Wiener equations is therefore not obvious. Of course, since the orthogonality principle is not respected anymore, even the simplest algorithm (stochastic gradient) can not be applied. Has anyone already had to face such a problem ? Where can i get help ? Any website ? Thanks in advance -herbert-
Adaptative filter
Started by ●February 2, 2007
Reply by ●February 2, 20072007-02-02
Tom, I do thank you for answering... but i'm afraid you misread the question. Have a nice day anyway (and rest a bit ;) ) In article <1170517011.267819.13230@l53g2000cwa.googlegroups.com>, tdillon@dilloneng.com says...> Arthur Herbert wrote: > > Hi all, > > I am to design a special adaptive filter in which the error signal is > > hihgly correlated with the input signal (in fact, the latter is just a > > delayed version of the error...) > > Usual resolution of the Wiener equations is therefore not obvious. > > Of course, since the orthogonality principle is not respected anymore, > > even the simplest algorithm (stochastic gradient) can not be applied. > > Has anyone already had to face such a problem ? > > Where can i get help ? Any website ? > > Thanks in advance > > -herbert- > > Arthur, > > Seems like you could use a CORDIC, giving it the angle as an input and > getting the coordinates of the circle out. > > Check out: > > http://my.execpc.com/~geezer/embed/cordic.htm > > There are several decent free sources for CORDIC IP out there. > > Tom > >
Reply by ●February 2, 20072007-02-02
Fred,
The system i have to work with rather looks like this :
+-----------+
+----| Delay |<---+--------------------+
| +-----------+ | |
| | |
input------|---------------------|------------>(+)----+----> e[n]
| | ^
| | |
| v |
+------------------>[LMS]------------+----------> o[n]
As i stated in my first message, the input of 'the adaptative filter'
also is a delayed version of the error signal (maybe i didn't make
myself clear enough)
This is a interference canceller...
This structure is quite different from the one we learn at school in
which the input of the adaptative filter always is the system input
signal or a delayed version of it.
In my case, the input of the adaptative filter is the error signal used
to adapt the filter's coefficient, just delayed.
The thing is, here, the adaptative filter is used to compensate for an
external unknown filter whose input is what you denoted 'o(n)' and the
output adds with the useful signal to form 'input' on the diagram.
(This external unknown filter is not sketched)
After the system has converged, one should have LMS(k)=ExternalFilter(k)
A study of such a system shows that in that case the error signal e(n)
is (obviously) highly correlated with the adaptative filter's input.
Orthogonality principle is not true anymore and the system converges but
with a slight bias. Of course, the 'delay' is roughly equal to the
external filter's delay. If one make the 'delay' much longer than the
external filter's impulse response, the bias vanishes and the system
finds the right solution, i.e. o(n)=useful input signal.
Anyway, this is not my case since the delay can not be considerated as
infinite.
I hope i made myself clear enough. I would be very happy if one could
help !
PS : Fred, thanks again for answering and please excuse my poor english
:)
In article <2fqdnYAs8eB6j1jYnZ2dnUVZ_u6rnZ2d@centurytel.net>,
fmarshallx@remove_the_x.acm.org says...
>
> "Arthur Herbert" <me@here.com> wrote in message
> news:MPG.202d1633d1d3d3c2989680@news.orange.fr...
> > Hi all,
> > I am to design a special adaptive filter in which the error signal is
> > hihgly correlated with the input signal (in fact, the latter is just a
> > delayed version of the error...)
> > Usual resolution of the Wiener equations is therefore not obvious.
> > Of course, since the orthogonality principle is not respected anymore,
> > even the simplest algorithm (stochastic gradient) can not be applied.
> > Has anyone already had to face such a problem ?
> > Where can i get help ? Any website ?
> > Thanks in advance
> > -herbert-
>
>
> +--------------------+
> | |
> | |
> | |
> input------+---------------------|------------>(+)----+----> e[n]
> | | ^
> | | |
> | +-----------+ v |
> +---| Delay |-->[LMS]------------+----------> o[n]
> +-----------+
>
> Adaptive enhancer
> LMS adaptive filter adjusts to minimize e[n] which cancels signal
> o[n] ideally contains only signal
>
> The output of the LMS filter is highly correlated with the input.
> Does that satisfy your requirement?
>
> Fred
>
>
>
Reply by ●February 3, 20072007-02-03
In article <T6ydnb5_MIyi4VjYnZ2dnUVZ_hynnZ2d@centurytel.net>, fmarshallx@remove_the_x.acm.org says... Fred, i'm not good at 'character drawings', do you mind if i email you a .jpeg drawing of the system ? It will therefore be much clearer :) Once again, thanks for your patience :) Hope to read you soon.> >I don't understand your diagram. Part of the confusion is where does >>the >output of the delay go? Added to the input or does it only go to the >>signal >input of the LMS filter? Neither one makes sense to me. It's worth >checking.....>The diagram I provided is for an Adaptive Line Enhancer which attempts >to >output that part of the input that is correlated with itself after a >>>delay - >thus sinusoids and sums thereof. >An alternate version of it might be:+--------------------+ | | | | +-----------+ | | input------+---| Delay |-----|------------>(+)----+----> e[n] | +-----------+ | ^ | | | | v | +------------------>[LMS]------------+----------> o[n]
Reply by ●February 3, 20072007-02-03
Arthur Herbert wrote:> Hi all, > I am to design a special adaptive filter in which the error signal is > hihgly correlated with the input signal (in fact, the latter is just a > delayed version of the error...) > Usual resolution of the Wiener equations is therefore not obvious. > Of course, since the orthogonality principle is not respected anymore, > even the simplest algorithm (stochastic gradient) can not be applied. > Has anyone already had to face such a problem ? > Where can i get help ? Any website ? > Thanks in advance > -herbert-Arthur, Seems like you could use a CORDIC, giving it the angle as an input and getting the coordinates of the circle out. Check out: http://my.execpc.com/~geezer/embed/cordic.htm There are several decent free sources for CORDIC IP out there. Tom
Reply by ●February 3, 20072007-02-03
Sorry about that, seems I read one post and answered another. Arthur Herbert wrote:> Tom, > I do thank you for answering... but i'm afraid you misread the question. > Have a nice day anyway (and rest a bit ;) ) > > In article <1170517011.267819.13230@l53g2000cwa.googlegroups.com>, > tdillon@dilloneng.com says... > > Arthur Herbert wrote: > > > Hi all, > > > I am to design a special adaptive filter in which the error signal is > > > hihgly correlated with the input signal (in fact, the latter is just a > > > delayed version of the error...) > > > Usual resolution of the Wiener equations is therefore not obvious. > > > Of course, since the orthogonality principle is not respected anymore, > > > even the simplest algorithm (stochastic gradient) can not be applied. > > > Has anyone already had to face such a problem ? > > > Where can i get help ? Any website ? > > > Thanks in advance > > > -herbert- > > > > Arthur, > > > > Seems like you could use a CORDIC, giving it the angle as an input and > > getting the coordinates of the circle out. > > > > Check out: > > > > http://my.execpc.com/~geezer/embed/cordic.htm > > > > There are several decent free sources for CORDIC IP out there. > > > > Tom > > > >
Reply by ●February 3, 20072007-02-03
"Arthur Herbert" <me@here.com> wrote in message news:MPG.202d1633d1d3d3c2989680@news.orange.fr...> Hi all, > I am to design a special adaptive filter in which the error signal is > hihgly correlated with the input signal (in fact, the latter is just a > delayed version of the error...) > Usual resolution of the Wiener equations is therefore not obvious. > Of course, since the orthogonality principle is not respected anymore, > even the simplest algorithm (stochastic gradient) can not be applied. > Has anyone already had to face such a problem ? > Where can i get help ? Any website ? > Thanks in advance > -herbert-+--------------------+ | | | | | | input------+---------------------|------------>(+)----+----> e[n] | | ^ | | | | +-----------+ v | +---| Delay |-->[LMS]------------+----------> o[n] +-----------+ Adaptive enhancer LMS adaptive filter adjusts to minimize e[n] which cancels signal o[n] ideally contains only signal The output of the LMS filter is highly correlated with the input. Does that satisfy your requirement? Fred
Reply by ●February 4, 20072007-02-04
"Arthur Herbert" <webmaster@ayapos.homelinux.net> wrote in message news:MPG.202dff13ab1ba19a989682@news.orange.fr...> Fred, > The system i have to work with rather looks like this : > > > +-----------+ > +----| Delay |<---+--------------------+ > | +-----------+ | | > | | | > input------|---------------------|------------>(+)----+----> e[n] > | | ^ > | | | > | v | > +------------------>[LMS]------------+----------> o[n] > > > As i stated in my first message, the input of 'the adaptative filter' > also is a delayed version of the error signal (maybe i didn't make > myself clear enough) > This is a interference canceller... > This structure is quite different from the one we learn at school in > which the input of the adaptative filter always is the system input > signal or a delayed version of it. > In my case, the input of the adaptative filter is the error signal used > to adapt the filter's coefficient, just delayed. > The thing is, here, the adaptative filter is used to compensate for an > external unknown filter whose input is what you denoted 'o(n)' and the > output adds with the useful signal to form 'input' on the diagram. > (This external unknown filter is not sketched) > After the system has converged, one should have LMS(k)=ExternalFilter(k) > A study of such a system shows that in that case the error signal e(n) > is (obviously) highly correlated with the adaptative filter's input. > Orthogonality principle is not true anymore and the system converges but > with a slight bias. Of course, the 'delay' is roughly equal to the > external filter's delay. If one make the 'delay' much longer than the > external filter's impulse response, the bias vanishes and the system > finds the right solution, i.e. o(n)=useful input signal. > Anyway, this is not my case since the delay can not be considerated as > infinite. > I hope i made myself clear enough. I would be very happy if one could > help ! > > PS : Fred, thanks again for answering and please excuse my poor english > :)I don't understand your diagram. Part of the confusion is where does the output of the delay go? Added to the input or does it only go to the signal input of the LMS filter? Neither one makes sense to me. It's worth checking..... The diagram I provided is for an Adaptive Line Enhancer which attempts to output that part of the input that is correlated with itself after a delay - thus sinusoids and sums thereof. An alternate version of it might be: +--------------------+ | | | | +-----------+ | | input------+---| Delay |-----|------------>(+)----+----> e[n] | +-----------+ | ^ | | | | v | +------------------>[LMS]------------+----------> o[n] Adaptive enhancer LMS adaptive filter adjusts to minimize e[n] which cancels signal o[n] ideally contains only signal The "input" to the LMS filter at the top is the "control" input that causes the coefficients to adapt. The input at the left side of the LMS filter is the signal input to be filtered. The delay can either be in the signal path to the summing junction to yield the error, or, the delay can be in the signal path input to the LMS filter. It doesn't matter. The purpose of the delay is so that random noise in the input is uncorrelated between the two paths at the summing junction. If the delay is introduced as above, then it has to be bigger than the filter delay and adequate to decorrelate the noise. If the delay is introduced at the input to the LMS filter as I showed before, then it only needs to be adequate to decorrelate the noise. If the delay is too small or if there isn't a delay, then the LMS filter would reach a trivial solution with coefficients [-1 0 0 0 0 0 ....] and the resulting error would be zero. In that case, the output would be the same as the input - so it's useless. I hope this helps. Fred
Reply by ●February 4, 20072007-02-04
OK. In the mean time, here is one of the fundamental "truths" about adaptive filters: In the noise canceller case, the filter removes only periodic components. That's under the assumption that the reference source and the signal source have uncorrelated noises. Therefore, for any frequency where there is only uncorrelated noise, the filter will become a band stop filter. Otherwise, noise + noise = more noise which increases the output. So, the noise canceller removes any tones that are in the reference that are also in the signal input, by setting appropriate amplitude and phase, and attenuates all those frequencies that only contain random noise. In the line enhancer case, the filter does the same thing except now the reference input and the actual input are the same thing (thus the delay added to decorrelate the noise between the two paths). Then, the filter works to do the same thing as the noise canceller but we use the output of the adaptive filter rather than the error / sum as the useful output. Any uncorrelated noise in the filter's input is attenutated so that the filter's output is mostly the correlated parts - the tones. Maybe that helps... ? Fred
Reply by ●February 5, 20072007-02-05
Herbert wrote:
The system (within the dashed lines) gets as its input the sum of :
-> The actual useful signal, denoted 'i'
-> Part of its own output filtered by an external unknown filter,
repectively denoted 'o' and 'ef'.
The goal of the system is to recover the useful signal, i.e. be able
to
adapt the filter (denoted 'af') so that o=i.
Well, o=i... not exactly, i meant o=i(t-delay) of course... and the aim
of the system is 'just' to cancel the effects of the external filter ;
Fred's interpretation of information:
*********************************
+-----------+
+---------------| EF |<-------------+
| f +-----------+ |
| |
| |
| |
| +---[LMS]<---------------+ |
| | ^ | |
| | ^ | |
| | ^ | |
| | ^ | |
+V -V e +-----------+ | |
i --->(+)---->(+)----+--->| DELAY |-+---------+----> o
+ + +-----------+
Adaptive compensator
Removes correlated effects of external filter "EF" at the output
LMS adaptive filter adjusts to minimize the correlated difference
between "i" and "o"
**********************************
As you can see, the main difference between your diagram and mine is
that in yours, the error signal e(n) is only used to update the LMS
filter. In mine, the error signal (denoted 'e') is also used to
update the filter coefficients but is the actual output of the system
as
well (o).
The problem i'm facing is therefore quite different from an adaptative
line enhancer.
Any idea of the algorithm that can be used ? (knowing that in that
case
'epsilon' and 'o' are correlated and the Wiener equations are not
trivial to solve...)
"Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in message
news:vKqdnRYyv6FglFvYnZ2dnUVZ_qmpnZ2d@centurytel.net...
> OK.
>
> In the mean time, here is one of the fundamental "truths" about adaptive
> filters:
>
> In the noise canceller case, the filter removes only periodic components.
> That's under the assumption that the reference source and the signal
> source have uncorrelated noises. Therefore, for any frequency where there
> is only uncorrelated noise, the filter will become a band stop filter.
> Otherwise, noise + noise = more noise which increases the output. So, the
> noise canceller removes any tones that are in the reference that are also
> in the signal input, by setting appropriate amplitude and phase, and
> attenuates all those frequencies that only contain random noise.
>
> In the line enhancer case, the filter does the same thing except now the
> reference input and the actual input are the same thing (thus the delay
> added to decorrelate the noise between the two paths). Then, the filter
> works to do the same thing as the noise canceller but we use the output of
> the adaptive filter rather than the error / sum as the useful output. Any
> uncorrelated noise in the filter's input is attenutated so that the
> filter's output is mostly the correlated parts - the tones.
>
> Maybe that helps... ?
>
> Fred
OK. Well, I'm having a hard time imagining how "e" is a measure of anything
useful for the purposes of adapting the LMS filter. But, I'm still
pondering.....
I think it would help a lot if you identified *which* nodes in the sytem are
measurable/observable.
For example, I presume that "i" is observable and, because of the block
diagram, that "e" is observable.
This means that f=e-i so that is easily computed.
If "f" is easily computed, then it seems that it can be easily subtracted
out.
Also, if the objective is to make o=i[delayed] then a somewhat different
block diagram makes sense to me:
+-----------+
+---------------| EF |<-------------+
| f +-----------+ |
| |
| |
| |
| +---[LMS]<---------------+ |
| | ^ | |
| | ^ | |
| | ^ | |
| | ^ | |
V V ^ +-----------+ | |
i -+->(+)---->(+)-------->| DELAY |-+---------+----> o
| +-----------+ |
| ^ |
| ^ |
| e ^ |
+--------------->(+)<--------------------------+
+ -
Adaptive compensator II
Removes correlated effects of external filter "EF" at the output
LMS adaptive filter adjusts to minimize the correlated difference
between "i" and "o"
So, assuming that "o" is observable, it seems much clearer to me that the
"error" is the difference between i and o.
Fred
