Reply by robert bristow-johnson June 26, 20112011-06-26
On Jun 24, 3:35&#4294967295;am, "basgoossen" <bas@n_o_s_p_a_m.2dive.nl> wrote:
> Thanks everybody for the answers, > > The matlab code mentionned is not something i use but an example of > overlap-save progressing found on this forum. I implement the source in > Java so i can't use the matlab's filter function ;-). > > For now i have got rid of 99% of the filter artifacts, my current approach > is to use a buffer size that is twice the requested buffer size and use > only the middle portion of the processed buffer to write the output. Since > the chunks are only 20ms that adds a delay of roughly 5ms, which is > acceptable in the application. >
Bas, is the impulse response or transfer function of your filter static or is it changing? FYI, there are two, slightly different, techniques both called "overlap-add" or OLA. one is for use with more esoteric algorithms where the spectrum is often modified in a more sophisticated manner than just a static FIR filter. could be time-varying filtering or compression or a phase vocoder or some kinda sinusoidal modeling. in this technique, you will likely use some kind of overlapping and complementary windows, like the Hann window. depending on what you are doing, there may be zero padding in addition to the windowing. the frame hop is half of the width of the complementary window. there *could* be artifacts because of framing (and whatever nasty thing your algorithm is doing to the spectrum), even if you employ this OLA properly. the other OLA technique is for implementing "fast convolution" and is the alternative to "overlap-save" or OLS. in both of these cases, your window is rectangular. in OLS, it is essentially no-window (the rectangular window width is the entire frame of data) and in OLA, the rectangular window width is no larger than N-L+1 where N is the FFT length and L is the FIR length. in both cases, the frame hop is no more than N-L+1 samples. in both cases, the transfer function, H[k], must be the DFT of an impulse response, h[n], that is finite (as in "FIR") in length and with length L which should be quite a bit smaller than N. it can't be any ol' H[k] if you want to avoid artifacts at the frame boundaries. if h[n] is non-zero only for 0 <= n < L, and if you do your OLS or OLA correctly, there should be *no* artifacts at the frame boundaries (assuming no numerical issues because of insufficient word width). but if you dial in whatever H[k] your heart desires (without guaranteeing that the length of h[n] is limited to L) and use this technique, *even* in the static case, you can get artifacts at the frontier between frames. as with the time-variant case of OLA, there is no reason you cannot change the H[k] between frames (but h[n] still must be limited in length to L samples) in the "fast convolution" case, but you may very well get artifacts at the frame boundaries. you can get *click* artifacts using OLS and changing H[k] (between adjacent frames) and a somewhat better transition between frames using OLA. r b-j
Reply by robert bristow-johnson June 25, 20112011-06-25
On Jun 23, 3:59&#4294967295;pm, niarn <niar...@gmail.com> wrote:
> > There are 3 things to be aware of > > 1. As already suggested you need to read about OLA and/or OLS. > 2. You need a larger overlap. > 3. You need to make sure that your circular convolution as implemented > by the multiplication of the fourier transforms either corrresponds to > a linear convolution or at least gets close to it.
fundamentally, if Bas is using OLA or OLS, he/she needs to make sure that the H[k] sequence that is used in the frequency domain, of length N, is the DFT of an h[n] that is a *finite* impulse response of length L and zero padded with N-L zeros at the end. for each frame, Bas gets only N-L+1 valid output samples, so the overlap must be L-1. and no windowing (other than the built-in rectangular window) is used for OLS or for this kind of OLA. i would suggest reading O&S. r b-j
Reply by Alexander Petrov June 24, 20112011-06-24
basgoossen

Use oversampled filter banks for frequency domain filtering and avoid
aliasing artifacts.

For example

http://electronix.ru/forum/index.php?s=&showtopic=23652&view=findpost&p=929325
Reply by basgoossen June 24, 20112011-06-24
Thanks everybody for the answers,

The matlab code mentionned is not something i use but an example of
overlap-save progressing found on this forum. I implement the source in
Java so i can't use the matlab's filter function ;-).

For now i have got rid of 99% of the filter artifacts, my current approach
is to use a buffer size that is twice the requested buffer size and use
only the middle portion of the processed buffer to write the output. Since
the chunks are only 20ms that adds a delay of roughly 5ms, which is
acceptable in the application.

I've also tested whith brick-wall filters as named this way earlyer in this
topic. Those now also give quite satisfying results, Only very high amp
noise on the endge of the brick ;-) give artifacts, but for normal band
filtering it seems to work.

At this moment i am not using windows to prevent spectral leakage, but it
seems not to have a bad effect on these chunks. When i draw the spectrum i
see sharp peaks without leakage... I don't know why but it just seems to
work.

If anyone wants to warn me of possible side effects of this filter please
let me know. For now i think the preformance is good enough for the
application (Sip communication).
Reply by niarn June 23, 20112011-06-23
There are some problems with your code. I suggest you try this.
You have a scenario where you are filtering a tone with a static low-
pass filter.
In this case you can create a reference quite easily
yref = filter(h,1,x);
Now you can compare the output of your frequency-domain processing
routine with this reference.

There are 3 things to be aware of

1. As already suggested you need to read about OLA and/or OLS.
2. You need a larger overlap.
3. You need to make sure that your circular convolution as implemented
by the multiplication of the fourier transforms either corrresponds to
a linear convolution or at least gets close to it.
Cheers
Reply by taks June 23, 20112011-06-23
>Instead of implementing the OLS or OLA functionality manually, you >could just use MATLAB's filter() function, providing the "h" vector >above as your filter coefficients. > >Jason
That's always true, unless there's a specific reason for doing the filtering via the frequency domain, such as very large vectors, or a homework assignment, hehe. Mark
Reply by Jason June 23, 20112011-06-23
On Jun 23, 12:55&#4294967295;pm, "basgoossen" <bas@n_o_s_p_a_m.2dive.nl> wrote:
> >Are you just zeroing out the frequencies you want to filter? > > No i use an inverse hamming window and multiply this with the frequency > value's at the frequency bands i whant to filter. So a gradualy lower the > frequency's around the center frequency. > > I've read a lot about this overlap-add and overlap save, i just can't seem > to get it to work... > > i have this example (also from this forum): > N = 1024; > n = (1:N)'-1; %'# define the time index > x = sin(2*pi*1.5*n/N); %# input with 1.5 cycles per 1024 points > h = hanning(129) .* sinc(0.25*(-64:1:64)'); %'# windowed sinc LPF, Fc = > pi/4 > h = [h./sum(h)]; %# normalize DC gain > > Nproc = 512; > xproc = zeros(2*Nproc,1); %# initialize temp buffer > idx = 1:Nproc; %# initialize half-buffer index > ycorrect = zeros(2*Nproc,1); %# initialize destination > for ctr = 1:(length(x)/Nproc) %# iterate over x 512 points at a time > &#4294967295; &#4294967295; xproc(1:Nproc) = xproc((Nproc+1):end); %# shift 2nd half of last > iteration to 1st half of this iteration > &#4294967295; &#4294967295; xproc((Nproc+1):end) = x(idx); %# fill 2nd half of this iteration with > new data > &#4294967295; &#4294967295; yproc = ifft(fft(xproc) .* fft(h,2*Nproc)); %# calculate new buffer > &#4294967295; &#4294967295; ycorrect(idx) = real(yproc((Nproc+1):end)); %# keep 2nd half of new > buffer > &#4294967295; &#4294967295; idx = idx + Nproc; %# step half-buffer index > end > > but for someone that has no experience with matlab it's not that self > explenatory. i tryed to implement it in java but now all i get is very > faint sound mostly plops and klicks and nothing more, so it is further off > from what i want. Could somebody try to explain in laymans terms how this > works?
Instead of implementing the OLS or OLA functionality manually, you could just use MATLAB's filter() function, providing the "h" vector above as your filter coefficients. Jason
Reply by taks June 23, 20112011-06-23
>>Are you just zeroing out the frequencies you want to filter? >No i use an inverse hamming window and multiply this with the frequency >value's at the frequency bands i whant to filter. So a gradualy lower the >frequency's around the center frequency.
OK, so that shouldn't be much of a problem.
>I've read a lot about this overlap-add and overlap save, i just can't
seem
>to get it to work...
Sorry, I can't be of much help. Graychip (I think) used to have a good tutorial on both, though since they were bought out by TI I don't know if it still exists. Either way, it may be worthwhile searching TI's website to see if it does. My guess now is that you're getting edge effects from stacked IFFTs. Note, too, that the filter method you are employing still imparts the equivalent filter delay on the start of the signal. Overlapping removes this as well as far as I know. Mark
Reply by basgoossen June 23, 20112011-06-23
>Are you just zeroing out the frequencies you want to filter?
No i use an inverse hamming window and multiply this with the frequency value's at the frequency bands i whant to filter. So a gradualy lower the frequency's around the center frequency. I've read a lot about this overlap-add and overlap save, i just can't seem to get it to work... i have this example (also from this forum): N = 1024; n = (1:N)'-1; %'# define the time index x = sin(2*pi*1.5*n/N); %# input with 1.5 cycles per 1024 points h = hanning(129) .* sinc(0.25*(-64:1:64)'); %'# windowed sinc LPF, Fc = pi/4 h = [h./sum(h)]; %# normalize DC gain Nproc = 512; xproc = zeros(2*Nproc,1); %# initialize temp buffer idx = 1:Nproc; %# initialize half-buffer index ycorrect = zeros(2*Nproc,1); %# initialize destination for ctr = 1:(length(x)/Nproc) %# iterate over x 512 points at a time xproc(1:Nproc) = xproc((Nproc+1):end); %# shift 2nd half of last iteration to 1st half of this iteration xproc((Nproc+1):end) = x(idx); %# fill 2nd half of this iteration with new data yproc = ifft(fft(xproc) .* fft(h,2*Nproc)); %# calculate new buffer ycorrect(idx) = real(yproc((Nproc+1):end)); %# keep 2nd half of new buffer idx = idx + Nproc; %# step half-buffer index end but for someone that has no experience with matlab it's not that self explenatory. i tryed to implement it in java but now all i get is very faint sound mostly plops and klicks and nothing more, so it is further off from what i want. Could somebody try to explain in laymans terms how this works?
Reply by taks June 23, 20112011-06-23
Are you just zeroing out the frequencies you want to filter?  If so, then
that is your problem.  Brick-wall filters in the frequency-domain are
essentially "infinite" bandwidth which translate to ringing in the
time-domain.  There's also an issue with doing consecutive block IFFTs then
converting back to the time domain - there will be start/stop transients at
the end of each block, which, coupled with the filtering issue, will likely
cause the pops you are getting.

Look up overlap/save and overlap/add (I don't use either so I can't be of
much help) for the IFFT issue.  For the filter issue, figure out what your
time-domain impulse response should be then compute the FFT of the filter
(zero-padding to get to the length of your IFFT) and use that as a
multiplier in the frequency-domain.

Mark