Hello, We have made a transient voltage recording. Sampling rate approx. 10000 Hz. The data file is available in ASCII-format. At present we used MS Excel to present the recorded data graphically. However some of the individual recordings are disturbed by noise and too many to remove manually. We would like to "pass" the data file through a simple low pass filter, to remove (smoothen) the noise. I this possible to do this in Excel without to much trouble? How? Best Regards L.L
Low-pass filtering data file in MS Excel or other program
Started by ●November 23, 2004
Reply by ●November 24, 20042004-11-24
On Tue, 23 Nov 2004 21:20:29 +0100, "L.L." <bruges@ikke.dk> wrote:>Hello, > >We have made a transient voltage recording. Sampling rate approx. 10000 Hz. >The data file is available in ASCII-format. At present we used MS Excel to >present the recorded data graphically. However some of the individual >recordings are disturbed by noise and too many to remove manually. We would >like to "pass" the data file through a simple low pass filter, to remove >(smoothen) the noise. >I this possible to do this in Excel without to much trouble? How? > >Best Regards >L.LHi, yes, it certainly *is* possible. To implement a 5-tap nonrecursive lowpass FIR filter (whose coefficients are: B0,B1,B2,B3,& B4), put your unfiltered samples in column A. Then define the contents of Cell B5 to be equal to that shown below. Next, copy Cell B5 and paste it down Column B, clear down to the row containing your last xn sample. If you've pasted it properly, then Cell B6 should look like that shown below. After the "paste" operation, your filtered-data samples will then be in Column B. Column Column A B 1 x0 2 x1 3 x2 4 x3 5 x4 =B0*A1+B1*A2+B2*A3+B3*A4+B4*A5 6 x5 =B0*A2+B1*A3+B2*A4+B3*A5+B4*A6 7 x6 " 8 x7 9 x8 10 x9 11 x10 12 x11 13 x12 14 x13 " " That's the easy part. The hard part is determining how many Bn filter coefficients you need and what their values will be. This is a "filter design" problem. Do you know how, and have the software, to design (determine the coefficients of) nonrecursive lowpass FIR filters? If not, let us know, 'cause we can design one for you if you're able to tell us the frequency-domain characteristics of your desired lowpass filter. To quote Susan B. Anthony, "Failure is impossible." [-Rick-]
Reply by ●November 24, 20042004-11-24
Rick Lyons wrote:> On Tue, 23 Nov 2004 21:20:29 +0100, "L.L." <bruges@ikke.dk> wrote: >=20 >=20 >>Hello, >> >>We have made a transient voltage recording. Sampling rate approx. 10000=Hz.=20>>The data file is available in ASCII-format. At present we used MS Excel=to=20>>present the recorded data graphically. However some of the individual=20 >>recordings are disturbed by noise and too many to remove manually. We w=ould=20>>like to "pass" the data file through a simple low pass filter, to remov=e=20>>(smoothen) the noise. >>I this possible to do this in Excel without to much trouble? How? >> >>Best Regards >>L.L >=20 >=20 >=20 > Hi, >=20 > yes, it certainly *is* possible. To implement a=20 > 5-tap nonrecursive lowpass FIR filter (whose coefficients=20 > are: B0,B1,B2,B3,& B4), put your unfiltered=20 > samples in column A. Then define the contents of Cell B5=20 > to be equal to that shown below. Next, copy Cell B5 and=20 > paste it down Column B, clear down to the row containing your=20 > last xn sample. If you've pasted it properly, then Cell B6=20 > should look like that shown below. After the "paste"=20 > operation, your filtered-data samples will then be in Column B.=20 >=20 >=20 > Column Column > A B >=20 > 1 x0 > 2 x1 > 3 x2 > 4 x3 > 5 x4 =3DB0*A1+B1*A2+B2*A3+B3*A4+B4*A5 > 6 x5 =3DB0*A2+B1*A3+B2*A4+B3*A5+B4*A6 > 7 x6 " > 8 x7 > 9 x8 > 10 x9 > 11 x10 > 12 x11 > 13 x12 > 14 x13 > " " >=20 > That's the easy part. The hard part is determining=20 > how many Bn filter coefficients you need and what=20 > their values will be. This is a "filter design"=20 > problem. Do you know how, and have the software,=20 > to design (determine the coefficients of) nonrecursive=20 > lowpass FIR filters? If not, let us know, 'cause=20 > we can design one for you if you're able to tell=20 > us the frequency-domain characteristics of your=20 > desired lowpass filter. >=20 > To quote Susan B. Anthony, "Failure is impossible." >=20 > [-Rick-] >=20The easiest: select a graph line click right mouse button Add Trendline moving average choose period OK Ren=E9
Reply by ●November 24, 20042004-11-24
"Rick Lyons" <r.lyons@_BOGUS_ieee.org> a �crit dans le message de news: 41a476db.463298406@news.sf.sbcglobal.net...> yes, it certainly *is* possible. To implement a > 5-tap nonrecursive lowpass FIR filter (whose coefficients > are: B0,B1,B2,B3,& B4), put your unfiltered > samples in column A. Then define the contents of Cell B5 > to be equal to that shown below. Next, copy Cell B5 and > paste it down Column B, clear down to the row containing your > last xn sample. If you've pasted it properly, then Cell B6 > should look like that shown below. After the "paste" > operation, your filtered-data samples will then be in Column B.Another quite effective method is to do it in the frequency domain : - Ask Excel to calculate the FFT of your data vector (yes, there is an FFT routine in Excel, see tools/data analysis) - Cut the FFT spectrum at your desired cutoff frequency (ie put 0s in the frequencies above your cutting frequency) - Ask Excel to calculate the inverse FFT of the result... Just take care that the FFT result is from -F to +F... Cheers, -- Robert Lacoste ALCIOM - The mixed signal experts www.alciom.com To contact us, thanks to click on the antispam link below : http://www.cerbermail.com/?dCSHUxvwpw
Reply by ●November 24, 20042004-11-24
"L.L." <bruges@ikke.dk> skrev i en meddelelse news:41a39b8c$0$240$edfadb0f@dread11.news.tele.dk...> Hello, > > We have made a transient voltage recording. Sampling rate approx. 10000 > Hz. The data file is available in ASCII-format. At present we used MS > Excel to present the recorded data graphically. However some of the > individual recordings are disturbed by noise and too many to remove > manually. We would like to "pass" the data file through a simple low pass > filter, to remove (smoothen) the noise. > I this possible to do this in Excel without to much trouble? How? > > Best Regards > L.L > >Thank you all, for your fast response. You let me in the right direction! Will try to recall my old digital filter design theory, and calculate the filter coefficients. Should be pretty easy to implement in Excel afterwards. Also the FFT solution will be tested. Thanks again. L.L
Reply by ●November 24, 20042004-11-24
"Robert Lacoste" <see-www-alciom-com-for-email-adress> skrev i en meddelelse news:41a496eb$0$10782$7a628cd7@news.club-internet.fr...> > "Rick Lyons" <r.lyons@_BOGUS_ieee.org> a �crit dans le message de news: > 41a476db.463298406@news.sf.sbcglobal.net... >> yes, it certainly *is* possible. To implement a >> 5-tap nonrecursive lowpass FIR filter (whose coefficients >> are: B0,B1,B2,B3,& B4), put your unfiltered >> samples in column A. Then define the contents of Cell B5 >> to be equal to that shown below. Next, copy Cell B5 and >> paste it down Column B, clear down to the row containing your >> last xn sample. If you've pasted it properly, then Cell B6 >> should look like that shown below. After the "paste" >> operation, your filtered-data samples will then be in Column B. > > Another quite effective method is to do it in the frequency domain : > - Ask Excel to calculate the FFT of your data vector (yes, there is an FFT > routine in Excel, see tools/data analysis) > - Cut the FFT spectrum at your desired cutoff frequency (ie put 0s in the > frequencies above your cutting frequency) > - Ask Excel to calculate the inverse FFT of the result... > > Just take care that the FFT result is from -F to +F... > > Cheers, > > -- > Robert Lacoste > ALCIOM - The mixed signal experts > www.alciom.com > > To contact us, thanks to click on the antispam link below : > http://www.cerbermail.com/?dCSHUxvwpw > > >FFT in Excel seems only able to handle 4096 data points.....we got 65000 :-o L.L.
Reply by ●November 24, 20042004-11-24
L.L. wrote:> "L.L." <bruges@ikke.dk> skrev i en meddelelse > news:41a39b8c$0$240$edfadb0f@dread11.news.tele.dk... > >>Hello, >> >>We have made a transient voltage recording. Sampling rate approx. 10000 >>Hz. The data file is available in ASCII-format. At present we used MS >>Excel to present the recorded data graphically. However some of the >>individual recordings are disturbed by noise and too many to remove >>manually. We would like to "pass" the data file through a simple low pass >>filter, to remove (smoothen) the noise. >>I this possible to do this in Excel without to much trouble? How? >> >>Best Regards >>L.L >> >> > > Thank you all, for your fast response. You let me in the right direction! > Will try to recall my old digital filter design theory, and calculate the > filter coefficients. Should be pretty easy to implement in Excel afterwards. > Also the FFT solution will be tested. Thanks again. > L.L > >You might also want to check out Scilab [ http://scilabsoft.inria.fr/ ] The price is right - free. Has FFT, plotting and some smoothing routines built-in. Can accept formatted ASCII data ala FORTRAN or C.
Reply by ●November 26, 20042004-11-26
SNIP>> Hi, > > yes, it certainly *is* possible. To implement a > 5-tap nonrecursive lowpass FIR filter (whose coefficients > are: B0,B1,B2,B3,& B4), put your unfiltered > samples in column A. Then define the contents of Cell B5 > to be equal to that shown below. Next, copy Cell B5 and > paste it down Column B, clear down to the row containing your > last xn sample. If you've pasted it properly, then Cell B6 > should look like that shown below. After the "paste" > operation, your filtered-data samples will then be in Column B. > > > Column Column > A B > > 1 x0 > 2 x1 > 3 x2 > 4 x3 > 5 x4 =B0*A1+B1*A2+B2*A3+B3*A4+B4*A5 > 6 x5 =B0*A2+B1*A3+B2*A4+B3*A5+B4*A6 > 7 x6 " > 8 x7 > 9 x8 > 10 x9 > 11 x10 > 12 x11 > 13 x12 > 14 x13 > " " >SNIP Write B$0, B$1, B$2, B$3, B$4 for the filter coefficents, or they will be increased on copying them down. Wim
Reply by ●November 26, 20042004-11-26
Richard Owlett wrote:> L.L. wrote: > >> "L.L." <bruges@ikke.dk> skrev i en meddelelse >> news:41a39b8c$0$240$edfadb0f@dread11.news.tele.dk... >> >>> Hello, >>> >>> We have made a transient voltage recording. Sampling rate approx. >>> 10000 Hz. The data file is available in ASCII-format. At present we >>> used MS Excel to present the recorded data graphically. However some >>> of the individual recordings are disturbed by noise and too many to >>> remove manually. We would like to "pass" the data file through a >>> simple low pass filter, to remove (smoothen) the noise. >>> I this possible to do this in Excel without to much trouble? How? >>> >>> Best Regards >>> L.L >>> >>> >> >> Thank you all, for your fast response. You let me in the right direction! >> Will try to recall my old digital filter design theory, and calculate >> the filter coefficients. Should be pretty easy to implement in Excel >> afterwards. Also the FFT solution will be tested. Thanks again. >> L.L >> > > You might also want to check out Scilab [ http://scilabsoft.inria.fr/ ] > The price is right - free. Has FFT, plotting and some smoothing routines > built-in. Can accept formatted ASCII data ala FORTRAN or C. >Another excellent free tool for statistical analysis of data (not necessarily numerical) is R, available at: http://www.r-project.org Regards, Andor
Reply by ●November 27, 20042004-11-27
"L.L." <bruges@ikke.dk> wrote in message news:<41a39b8c$0$240$edfadb0f@dread11.news.tele.dk>...> Hello, > > We have made a transient voltage recording. Sampling rate approx. 10000 Hz. > The data file is available in ASCII-format. At present we used MS Excel to > present the recorded data graphically. However some of the individual > recordings are disturbed by noise and too many to remove manually. We would > like to "pass" the data file through a simple low pass filter, to remove > (smoothen) the noise. > I this possible to do this in Excel without to much trouble? How? > > Best Regards > L.LConsider using Excel's built in function median. Use a sliding window of 5 or possibly more cells. This works well if the signal is really noisy as the median function rejects big spikes with out affecting the average signal too much. You may also find beneficial results from applying he median function before applying the low pass filter. It depends on the noise. Peter Nachtwey
