Forums

Adaptative filter

Started by Arthur Herbert February 2, 2007
Hi all,
I am to design a special adaptive filter in which the error signal is 
hihgly correlated with the input signal (in fact, the latter is just a 
delayed version of the error...)
Usual resolution of the Wiener equations is therefore not obvious.
Of course, since the orthogonality principle is not respected anymore, 
even the simplest algorithm (stochastic gradient) can not be applied.
Has anyone already had to face such a problem ?
Where can i get help ? Any website ?
Thanks in advance
-herbert-
Tom,
I do thank you for answering... but i'm afraid you misread the question.
Have a nice day anyway (and rest a bit ;) )

In article <1170517011.267819.13230@l53g2000cwa.googlegroups.com>, 
tdillon@dilloneng.com says...
> Arthur Herbert wrote: > > Hi all, > > I am to design a special adaptive filter in which the error signal is > > hihgly correlated with the input signal (in fact, the latter is just a > > delayed version of the error...) > > Usual resolution of the Wiener equations is therefore not obvious. > > Of course, since the orthogonality principle is not respected anymore, > > even the simplest algorithm (stochastic gradient) can not be applied. > > Has anyone already had to face such a problem ? > > Where can i get help ? Any website ? > > Thanks in advance > > -herbert- > > Arthur, > > Seems like you could use a CORDIC, giving it the angle as an input and > getting the coordinates of the circle out. > > Check out: > > http://my.execpc.com/~geezer/embed/cordic.htm > > There are several decent free sources for CORDIC IP out there. > > Tom > >
Fred,
The system i have to work with rather looks like this :


                  +-----------+ 
             +----|   Delay   |<---+--------------------+
             |    +-----------+    |                    |
             |                     |                    |
  input------|---------------------|------------>(+)----+----> e[n]
             |                     |              ^
             |                     |              |
             |                     v              |
             +------------------>[LMS]------------+----------> o[n]
                

As i stated in my first message, the input of 'the adaptative filter' 
also is a delayed version of the error signal (maybe i didn't make 
myself clear enough)
This is a interference canceller...
This structure is quite different from the one we learn at school in 
which the input of the adaptative filter always is the system input 
signal or a delayed version of it.
In my case, the input of the adaptative filter is the error signal used 
to adapt the filter's coefficient, just delayed.
The thing is, here, the adaptative filter is used to compensate for an 
external unknown filter whose input is what you denoted 'o(n)' and the 
output adds with the useful signal to form 'input' on the diagram.
(This external unknown filter is not sketched)
After the system has converged, one should have LMS(k)=ExternalFilter(k)
A study of such a system shows that in that case the error signal e(n) 
is (obviously) highly correlated with the adaptative filter's input.
Orthogonality principle is not true anymore and the system converges but 
with a slight bias. Of course, the 'delay' is roughly equal to the 
external filter's delay. If one make the 'delay' much longer than the 
external filter's impulse response, the bias vanishes and the system 
finds the right solution, i.e. o(n)=useful input signal.
Anyway, this is not my case since the delay can not be considerated as 
infinite.
I hope i made myself clear enough. I would be very happy if one could 
help !

PS : Fred, thanks again for answering and please excuse my poor english 
:)



In article <2fqdnYAs8eB6j1jYnZ2dnUVZ_u6rnZ2d@centurytel.net>, 
fmarshallx@remove_the_x.acm.org says...
> > "Arthur Herbert" <me@here.com> wrote in message
> news:MPG.202d1633d1d3d3c2989680@news.orange.fr... > > Hi all, > > I am to design a special adaptive filter in which the error signal is > > hihgly correlated with the input signal (in fact, the latter is just a > > delayed version of the error...) > > Usual resolution of the Wiener equations is therefore not obvious. > > Of course, since the orthogonality principle is not respected anymore, > > even the simplest algorithm (stochastic gradient) can not be applied. > > Has anyone already had to face such a problem ? > > Where can i get help ? Any website ? > > Thanks in advance > > -herbert- > > > +--------------------+ > | | > | | > | | > input------+---------------------|------------>(+)----+----> e[n] > | | ^ > | | | > | +-----------+ v | > +---| Delay |-->[LMS]------------+----------> o[n] > +-----------+ > > Adaptive enhancer > LMS adaptive filter adjusts to minimize e[n] which cancels signal > o[n] ideally contains only signal > > The output of the LMS filter is highly correlated with the input. > Does that satisfy your requirement? > > Fred > > >
In article <T6ydnb5_MIyi4VjYnZ2dnUVZ_hynnZ2d@centurytel.net>, 
fmarshallx@remove_the_x.acm.org says...

Fred, i'm not good at 'character drawings', do you mind if i email you a 
.jpeg drawing of the system ? It will therefore be much clearer :)

Once again, thanks for your patience :)
Hope to read you soon.

> >I don't understand your diagram. Part of the confusion is where does >>the >output of the delay go? Added to the input or does it only go to the >>signal >input of the LMS filter? Neither one makes sense to me. It's worth >checking.....
>The diagram I provided is for an Adaptive Line Enhancer which attempts >to >output that part of the input that is correlated with itself after a >>>delay - >thus sinusoids and sums thereof. >An alternate version of it might be:
+--------------------+ | | | | +-----------+ | | input------+---| Delay |-----|------------>(+)----+----> e[n] | +-----------+ | ^ | | | | v | +------------------>[LMS]------------+----------> o[n]
Arthur Herbert wrote:
> Hi all, > I am to design a special adaptive filter in which the error signal is > hihgly correlated with the input signal (in fact, the latter is just a > delayed version of the error...) > Usual resolution of the Wiener equations is therefore not obvious. > Of course, since the orthogonality principle is not respected anymore, > even the simplest algorithm (stochastic gradient) can not be applied. > Has anyone already had to face such a problem ? > Where can i get help ? Any website ? > Thanks in advance > -herbert-
Arthur, Seems like you could use a CORDIC, giving it the angle as an input and getting the coordinates of the circle out. Check out: http://my.execpc.com/~geezer/embed/cordic.htm There are several decent free sources for CORDIC IP out there. Tom
Sorry about that, seems I read one post and answered another.


Arthur Herbert wrote:
> Tom, > I do thank you for answering... but i'm afraid you misread the question. > Have a nice day anyway (and rest a bit ;) ) > > In article <1170517011.267819.13230@l53g2000cwa.googlegroups.com>, > tdillon@dilloneng.com says... > > Arthur Herbert wrote: > > > Hi all, > > > I am to design a special adaptive filter in which the error signal is > > > hihgly correlated with the input signal (in fact, the latter is just a > > > delayed version of the error...) > > > Usual resolution of the Wiener equations is therefore not obvious. > > > Of course, since the orthogonality principle is not respected anymore, > > > even the simplest algorithm (stochastic gradient) can not be applied. > > > Has anyone already had to face such a problem ? > > > Where can i get help ? Any website ? > > > Thanks in advance > > > -herbert- > > > > Arthur, > > > > Seems like you could use a CORDIC, giving it the angle as an input and > > getting the coordinates of the circle out. > > > > Check out: > > > > http://my.execpc.com/~geezer/embed/cordic.htm > > > > There are several decent free sources for CORDIC IP out there. > > > > Tom > > > >
"Arthur Herbert" <me@here.com> wrote in message 
news:MPG.202d1633d1d3d3c2989680@news.orange.fr...
> Hi all, > I am to design a special adaptive filter in which the error signal is > hihgly correlated with the input signal (in fact, the latter is just a > delayed version of the error...) > Usual resolution of the Wiener equations is therefore not obvious. > Of course, since the orthogonality principle is not respected anymore, > even the simplest algorithm (stochastic gradient) can not be applied. > Has anyone already had to face such a problem ? > Where can i get help ? Any website ? > Thanks in advance > -herbert-
+--------------------+ | | | | | | input------+---------------------|------------>(+)----+----> e[n] | | ^ | | | | +-----------+ v | +---| Delay |-->[LMS]------------+----------> o[n] +-----------+ Adaptive enhancer LMS adaptive filter adjusts to minimize e[n] which cancels signal o[n] ideally contains only signal The output of the LMS filter is highly correlated with the input. Does that satisfy your requirement? Fred
"Arthur Herbert" <webmaster@ayapos.homelinux.net> wrote in message 
news:MPG.202dff13ab1ba19a989682@news.orange.fr...
> Fred, > The system i have to work with rather looks like this : > > > +-----------+ > +----| Delay |<---+--------------------+ > | +-----------+ | | > | | | > input------|---------------------|------------>(+)----+----> e[n] > | | ^ > | | | > | v | > +------------------>[LMS]------------+----------> o[n] > > > As i stated in my first message, the input of 'the adaptative filter' > also is a delayed version of the error signal (maybe i didn't make > myself clear enough) > This is a interference canceller... > This structure is quite different from the one we learn at school in > which the input of the adaptative filter always is the system input > signal or a delayed version of it. > In my case, the input of the adaptative filter is the error signal used > to adapt the filter's coefficient, just delayed. > The thing is, here, the adaptative filter is used to compensate for an > external unknown filter whose input is what you denoted 'o(n)' and the > output adds with the useful signal to form 'input' on the diagram. > (This external unknown filter is not sketched) > After the system has converged, one should have LMS(k)=ExternalFilter(k) > A study of such a system shows that in that case the error signal e(n) > is (obviously) highly correlated with the adaptative filter's input. > Orthogonality principle is not true anymore and the system converges but > with a slight bias. Of course, the 'delay' is roughly equal to the > external filter's delay. If one make the 'delay' much longer than the > external filter's impulse response, the bias vanishes and the system > finds the right solution, i.e. o(n)=useful input signal. > Anyway, this is not my case since the delay can not be considerated as > infinite. > I hope i made myself clear enough. I would be very happy if one could > help ! > > PS : Fred, thanks again for answering and please excuse my poor english > :)
I don't understand your diagram. Part of the confusion is where does the output of the delay go? Added to the input or does it only go to the signal input of the LMS filter? Neither one makes sense to me. It's worth checking..... The diagram I provided is for an Adaptive Line Enhancer which attempts to output that part of the input that is correlated with itself after a delay - thus sinusoids and sums thereof. An alternate version of it might be: +--------------------+ | | | | +-----------+ | | input------+---| Delay |-----|------------>(+)----+----> e[n] | +-----------+ | ^ | | | | v | +------------------>[LMS]------------+----------> o[n] Adaptive enhancer LMS adaptive filter adjusts to minimize e[n] which cancels signal o[n] ideally contains only signal The "input" to the LMS filter at the top is the "control" input that causes the coefficients to adapt. The input at the left side of the LMS filter is the signal input to be filtered. The delay can either be in the signal path to the summing junction to yield the error, or, the delay can be in the signal path input to the LMS filter. It doesn't matter. The purpose of the delay is so that random noise in the input is uncorrelated between the two paths at the summing junction. If the delay is introduced as above, then it has to be bigger than the filter delay and adequate to decorrelate the noise. If the delay is introduced at the input to the LMS filter as I showed before, then it only needs to be adequate to decorrelate the noise. If the delay is too small or if there isn't a delay, then the LMS filter would reach a trivial solution with coefficients [-1 0 0 0 0 0 ....] and the resulting error would be zero. In that case, the output would be the same as the input - so it's useless. I hope this helps. Fred
OK.

In the mean time, here is one of the fundamental "truths" about adaptive 
filters:

In the noise canceller case, the filter removes only periodic components. 
That's under the assumption that the reference source and the signal source 
have uncorrelated noises.  Therefore, for any frequency where there is only 
uncorrelated noise, the filter will become a band stop filter.  Otherwise, 
noise + noise = more noise which increases the output.  So, the noise 
canceller removes any tones that are in the reference that are also in the 
signal input, by setting appropriate amplitude and phase, and attenuates all 
those frequencies that only contain random noise.

In the line enhancer case, the filter does the same thing except now the 
reference input and the actual input are the same thing (thus the delay 
added to decorrelate the noise between the two paths).  Then, the filter 
works to do the same thing as the noise canceller but we use the output of 
the adaptive filter rather than the error / sum as the useful output.  Any 
uncorrelated noise in the filter's input is attenutated so that the filter's 
output is mostly the correlated parts - the tones.

Maybe that helps... ?

Fred



Herbert wrote:

The system (within the dashed lines) gets as its input the sum of :
-> The actual useful signal, denoted 'i'
-> Part of its own output filtered by an external unknown filter,
repectively denoted 'o' and 'ef'.
The goal of the system is to recover the useful signal, i.e. be able
to
adapt the filter (denoted 'af') so that o=i.

Well, o=i... not exactly, i meant o=i(t-delay) of course... and the aim
of the system is 'just' to cancel the effects of the external filter ;

Fred's interpretation of information:
*********************************
                            +-----------+
            +---------------|    EF     |<-------------+
            |           f   +-----------+              |
            |                                          |
            |                                          |
            |                                          |
            |       +---[LMS]<---------------+         |
            |       |     ^                  |         |
            |       |     ^                  |         |
            |       |     ^                  |         |
            |       |     ^                  |         |
           +V      -V     e    +-----------+ |         |
     i --->(+)---->(+)----+--->|  DELAY    |-+---------+----> o
          +        +           +-----------+

    Adaptive compensator
    Removes correlated effects of external filter "EF" at the output


    LMS adaptive filter adjusts to minimize the correlated difference

    between "i" and "o"
**********************************


As you can see, the main difference between your diagram and mine is
that in yours, the error signal e(n) is only used to update the LMS
filter. In mine, the error signal (denoted 'e') is also used to
update the filter coefficients but is the actual output of the system
as
well (o).
The problem i'm facing is therefore quite different from an adaptative
line enhancer.
Any idea of the algorithm that can be used ? (knowing that in that
case
'epsilon' and 'o' are correlated and the Wiener equations are not
trivial to solve...)

"Fred Marshall" <fmarshallx@remove_the_x.acm.org> wrote in message 
news:vKqdnRYyv6FglFvYnZ2dnUVZ_qmpnZ2d@centurytel.net...
> OK. > > In the mean time, here is one of the fundamental "truths" about adaptive > filters: > > In the noise canceller case, the filter removes only periodic components. > That's under the assumption that the reference source and the signal > source have uncorrelated noises. Therefore, for any frequency where there > is only uncorrelated noise, the filter will become a band stop filter. > Otherwise, noise + noise = more noise which increases the output. So, the > noise canceller removes any tones that are in the reference that are also > in the signal input, by setting appropriate amplitude and phase, and > attenuates all those frequencies that only contain random noise. > > In the line enhancer case, the filter does the same thing except now the > reference input and the actual input are the same thing (thus the delay > added to decorrelate the noise between the two paths). Then, the filter > works to do the same thing as the noise canceller but we use the output of > the adaptive filter rather than the error / sum as the useful output. Any > uncorrelated noise in the filter's input is attenutated so that the > filter's output is mostly the correlated parts - the tones. > > Maybe that helps... ? > > Fred
OK. Well, I'm having a hard time imagining how "e" is a measure of anything useful for the purposes of adapting the LMS filter. But, I'm still pondering..... I think it would help a lot if you identified *which* nodes in the sytem are measurable/observable. For example, I presume that "i" is observable and, because of the block diagram, that "e" is observable. This means that f=e-i so that is easily computed. If "f" is easily computed, then it seems that it can be easily subtracted out. Also, if the objective is to make o=i[delayed] then a somewhat different block diagram makes sense to me: +-----------+ +---------------| EF |<-------------+ | f +-----------+ | | | | | | | | +---[LMS]<---------------+ | | | ^ | | | | ^ | | | | ^ | | | | ^ | | V V ^ +-----------+ | | i -+->(+)---->(+)-------->| DELAY |-+---------+----> o | +-----------+ | | ^ | | ^ | | e ^ | +--------------->(+)<--------------------------+ + - Adaptive compensator II Removes correlated effects of external filter "EF" at the output LMS adaptive filter adjusts to minimize the correlated difference between "i" and "o" So, assuming that "o" is observable, it seems much clearer to me that the "error" is the difference between i and o. Fred