Hi mates, I need to know somehow, the parts of overall structure setup for removing background noise and determining background noise sources. I am working on a problem to solve background noise emitted from test objects. Here's written all on the document, I have https://www.dropbox.com/sh/38hbzpufqiajm7n/D_vTPEP9mb What all I am looking to know , What these mathematical modules could be? What these algorithms could serve for? Are they contains some logic to remove noise? If so, what could be that logic? Once I heard , it might be Sqrt(), I don't know what it has to do with removal of noise in this structure. I am sorry if this question is not elaborated at all, I am Informatics student , but somehow I have to solve this problem and build architecture to develop a software in .NET. But before that I have to remove noise as it is an objective of my thesis part. Does these math algorithms needs to be replaces from other algorithms? If some body helps me out, I would be too much obliged for this. Regards Muhammad Usman Khalil
How can I remove Background Noise Removal after FFT?
Started by ●July 16, 2013
Reply by ●July 17, 20132013-07-17
On 7/16/13 7:09 PM, muhammadusman.khalil@gmail.com wrote:> Hi mates, > > I need to know somehow, the parts of overall structure setup for removing background noise and determining background noise sources....> Once I heard , it might be Sqrt(),better get your flak jacket on. -- r b-j rbj@audioimagination.com "Imagination is more important than knowledge."
Reply by ●July 17, 20132013-07-17
On 7/16/2013 11:22 PM, robert bristow-johnson wrote:> On 7/16/13 7:09 PM, muhammadusman.khalil@gmail.com wrote: >> Hi mates, >> >> I need to know somehow, the parts of overall structure setup for >> removing background noise and determining background noise sources. > ... >> Once I heard , it might be Sqrt(), > > better get your flak jacket on.Lol. Reminds me of a story a friend told about the stock trading forums on Yahoo. A new guy makes a first post saying something innocuous like "I'm thinking of buying this stock, what do you guys think?" He received a barrage of insults and put-downs... the sort of stuff you only get on the Internet or in freshman dorms. Someone finally makes a post saying, "So, how do you like the welcome wagon?" -- Rick
Reply by ●July 17, 20132013-07-17
Am Mittwoch, 17. Juli 2013 04:09:30 UTC+2 schrieb muhammadus...@gmail.com:> Hi mates, > > > > I need to know somehow, the parts of overall structure setup for removing background noise and determining background noise sources. > > > > I am working on a problem to solve background noise emitted from test objects. > > > > Here's written all on the document, I have > > > > https://www.dropbox.com/sh/38hbzpufqiajm7n/D_vTPEP9mb > > > > What all I am looking to know , > > > > What these mathematical modules could be? > > What these algorithms could serve for? > > Are they contains some logic to remove noise? > > If so, what could be that logic? > > > > Once I heard , it might be Sqrt(), I don't know what it has to do with removal of noise in this structure. > > > > I am sorry if this question is not elaborated at all, I am Informatics student , but somehow I have to solve this problem and build architecture to develop a software in .NET. But before that I have to remove noise as it is an objective of my thesis part. > > > > > > Does these math algorithms needs to be replaces from other algorithms? > > > > > > If some body helps me out, I would be too much obliged for this. > > > > > > Regards > > Muhammad Usman KhalilMates, I am not able to properly elaborate the problem I had. Sorry for this. Being a non DSP student, I am working on noise removal problem due to some reason. It has become my Thesis study of Master's. But I would have to implement a system that would follow the structure ( I mentioned as URL ). All I am working hard to grasp some concepts regarding signals and FFT. I know my data stream would be transformed from time domain to Frequency Domain after FFT. The results of FFT has to be "process" somehow, so that noise can be removed. I studied until now that there's no generic way for removal of noise from contaminated signal. 1 - Either you must have cleaned signal and noise signal ( Apply Wiener Filter or other DSP Filters ). 2 - Or Kalman Filter for taking more better results. This is all what I have researched until now. But the problem is : I have this template from my past workplace side and I am not able to get the answer from those guys somehow, why these Mathematical Modules are there? What kind of modules these are? Once they said these modules would be like "Sqrt". I am sorry , I am not DSP student but Informatics student. So, I am new bie in this domain. But I have got the knowledge about my other modules ( e.g FFT and Block Seperator ). But I have no idea about "Maths" That's why I posted here. Hope you will not angry this time. Regards Usman
Reply by ●July 17, 20132013-07-17
On Wed, 17 Jul 2013 01:45:30 -0700 (PDT), muhammadusman.khalil@gmail.com wrote:>Am Mittwoch, 17. Juli 2013 04:09:30 UTC+2 schrieb muhammadus...@gmail.com: >> Hi mates, >> >> I need to know somehow, the parts of overall structure setup for removing background noise and determining background noise sources. >> >> I am working on a problem to solve background noise emitted from test objects. >> >> Here's written all on the document, I have >> >> https://www.dropbox.com/sh/38hbzpufqiajm7n/D_vTPEP9mb >> >> What all I am looking to know , >> >> What these mathematical modules could be? >> >> What these algorithms could serve for? >> >> Are they contains some logic to remove noise? >> >> If so, what could be that logic? >> >> Once I heard , it might be Sqrt(), I don't know what it has to do with removal of noise in this structure. >> >> I am sorry if this question is not elaborated at all, I am Informatics student , but somehow I have to solve this problem and build architecture to develop a software in .NET. But before that I have to remove noise as it is an objective of my thesis part. >> >> Does these math algorithms needs to be replaces from other algorithms? >> >> If some body helps me out, I would be too much obliged for this. >> >> Regards >> Muhammad Usman Khalil > >Mates, > >I am not able to properly elaborate the problem I had. Sorry for this. > >Being a non DSP student, I am working on noise removal problem due to some reason. It has become my Thesis study of Master's. But I would have to implement a system that would follow the structure ( I mentioned as URL ). All I am working hard to grasp some concepts regarding signals and FFT. > >I know my data stream would be transformed from time domain to Frequency Domain after FFT. The results of FFT has to be "process" somehow, so that noise can be removed. > >I studied until now that there's no generic way for removal of noise from contaminated signal. > >1 - Either you must have cleaned signal and noise signal (Apply Wiener Filter or other DSP Filters ). > >2 - Or Kalman Filter for taking more better results. > >This is all what I have researched until now. >But the problem is : I have this template from my past workplace >side and I am not able to get the answer from those guys somehow, >why these Mathematical Modules are there? What kind of modules >these are? Once they said these modules would be like "Sqrt". > >I am sorry , I am not DSP student but Informatics student. So, >I am newbie in this domain. But I have got the knowledge about > my other modules ( e.g FFT and Block Seperator ). But I have >no idea about "Maths" > >That's why I posted here. Hope you will not angry this time. > >Regards >UsmanHello Usman, I don't know if you volunteered to solve your "noise removal" problem, or if some German professor assigned this problem to you. But in either case, I believe someone has made a BIG mistake. It seems to me that solving some sort of "noise removal" problem, the details of which are unknown at this time, would require a talanted DSP engineer who has years of DSP experience. Even then, success is not guaranteed. I say that because, without knowing any details about your noisy signals and what is meant by "noise removal", it may not be possible achieve acceptable noise removal. So, ... assigning such a difficult problem to someone who new to DSP-that would be you- seems like a bad idea to me. Usman, the reason you have received some silly replies to your original post is because your original post was similar to saying, "My brother is sick, he has a fever. I want to cure my brother but I know nothing about the human body and I know nothing about medicines. What can I do to cure my brother?" You are asking us to help you solve a difficult DSP problem, but we have no information about the nature of your problem. Add to this the fact that you are new to DSP. I suggest you find a simpler DSP problem to solve. Good Luck, [-Rick-]
Reply by ●July 17, 20132013-07-17
Am Mittwoch, 17. Juli 2013 04:09:30 UTC+2 schrieb muhammadus...@gmail.com:> Hi mates, > > > > I need to know somehow, the parts of overall structure setup for removing background noise and determining background noise sources. > > > > I am working on a problem to solve background noise emitted from test objects. > > > > Here's written all on the document, I have > > > > https://www.dropbox.com/sh/38hbzpufqiajm7n/D_vTPEP9mb > > > > What all I am looking to know , > > > > What these mathematical modules could be? > > What these algorithms could serve for? > > Are they contains some logic to remove noise? > > If so, what could be that logic? > > > > Once I heard , it might be Sqrt(), I don't know what it has to do with removal of noise in this structure. > > > > I am sorry if this question is not elaborated at all, I am Informatics student , but somehow I have to solve this problem and build architecture to develop a software in .NET. But before that I have to remove noise as it is an objective of my thesis part. > > > > > > Does these math algorithms needs to be replaces from other algorithms? > > > > > > If some body helps me out, I would be too much obliged for this. > > > > > > Regards > > Muhammad Usman KhalilThanks Rick for your comments. That's why I posted my document , that describes what I am looking for. So, I am summarizing here: Introduction ============ During the measurement of sound emission of test objects the background noise has to be monitored to detect noise that is not emitted by the test object. The different disturbing sources can be detected by several mathematical algorithms. All the algorithms use FFT spectra as input and do some mathematical operations on them. The results are displayed as a waterfall diagram moving over the time axis. Tasks ===== A software component has to be developed that is able to process data in real time from a data source with n channels. The time signals are divided into blocks by a given component and then transformed into the frequency domain by the given FFT component. Then the FFT results are sent to several chains of objects that do mathematical operations on these spectra. The object chains can consist of 1 or more objects and are defined by the user. Here the proposed structure that is figured as : DataStream----->BlockSeperator---->FFT---->(NoiseRemoval) or "Maths"--->WaterFallDisplay. Now , I can copy and past somehow (this is allowed to me by professor )the code for noise removal in any ways. But I NEED to know what could be the logic behind maths ,? So that I can figure out and write on my own. Note: You are right , I am not the guy who has to solve this problem. But I know how to perform Block Separation in signals,FFT, and now how to put the data on WaterFall Spectrum Moreover, I have already seen and worked on "Data Recorder Software" Modules. So I know almost all software related concepts regarding my thesis( i.e How to build the architectures of modules and how to setup services to perform FFT on server or client sides ). But all I am missing Noise removal, and I described above somehow "what kind of noise". Kindly have a look on the document If you feel difficulty still Thanks
Reply by ●July 17, 20132013-07-17
On Tue, 16 Jul 2013 19:09:30 -0700, muhammadusman.khalil wrote:> Hi mates, > > I need to know somehow, the parts of overall structure setup for > removing background noise and determining background noise sources. > > I am working on a problem to solve background noise emitted from test > objects. > > Here's written all on the document, I have > > https://www.dropbox.com/sh/38hbzpufqiajm7n/D_vTPEP9mb > > What all I am looking to know , > > What these mathematical modules could be? > What these algorithms could serve for? > Are they contains some logic to remove noise? > If so, what could be that logic? > > Once I heard , it might be Sqrt(), I don't know what it has to do with > removal of noise in this structure. > > I am sorry if this question is not elaborated at all, I am Informatics > student , but somehow I have to solve this problem and build > architecture to develop a software in .NET. But before that I have to > remove noise as it is an objective of my thesis part. > > > Does these math algorithms needs to be replaces from other algorithms? > > > If some body helps me out, I would be too much obliged for this. > > > Regards Muhammad Usman KhalilWe may all be over-thinking this. I'm not sure just what your prof has in mind (if anything). Frankly, someone on your team (prof included) should be able to answer these questions for the result to do anyone any good other than as an elementary exercise in .NET programming. So, background first, then comments on what might need to go into these mysterious "math" blocks. Background: In order to remove your background noise from the signal, the noise has to be somehow different from the signal. If you know the difference, and if you can figure out how to make an algorithm that sorts out the difference, then you can separate noise from signal. This is what a filter does -- just like a coffee filter separates coffee grounds from what goes into your cup. The coffee filter does it because coffee grounds are a lot bigger than the particles in coffee. Most filters do it because the noise is completely or partially in a different frequency spectrum than the desired signal. Comments: You're already sorting the signal by frequency, so it's hard to see what more might be done with it. Certainly presenting the signal in a manner that makes it easy to see is good. If it hasn't been done, one of the "math" blocks should be something that takes the pairs of complex numbers out of the FFT and finds their magnitude while discarding their phase. (Phase doesn't mean much in your application. Phase can mean a lot in other places, but not here). Another useful "math" block would be to take the logarithm of the amplitude, because people tend to perceive amplitudes logarithmically. Without displaying your graphs logarithmically, you will be suppressing low-amplitude effects, making perfectly audible sounds too small to see on the graph. Beyond that, I don't know what you might do. The block diagram that you have is for something that will be useful to present visual data; if you need to actually filter the signal from the noise, either for listening or for further mechanical processing, then all you'll be doing with the simple useful filtering techniques is zeroing out FFT bins; that'll just show up as a band of "always zero" on the waterfall plot, but it won't tell you anything more about what you want to see. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●July 17, 20132013-07-17
On Wed, 17 Jul 2013 01:45:30 -0700, muhammadusman.khalil wrote:> Am Mittwoch, 17. Juli 2013 04:09:30 UTC+2 schrieb > muhammadus...@gmail.com: >> Hi mates, >> >> >> >> I need to know somehow, the parts of overall structure setup for >> removing background noise and determining background noise sources. >> >> >> >> I am working on a problem to solve background noise emitted from test >> objects. >> >> >> >> Here's written all on the document, I have >> >> >> >> https://www.dropbox.com/sh/38hbzpufqiajm7n/D_vTPEP9mb >> >> >> >> What all I am looking to know , >> >> >> >> What these mathematical modules could be? >> >> What these algorithms could serve for? >> >> Are they contains some logic to remove noise? >> >> If so, what could be that logic? >> >> >> >> Once I heard , it might be Sqrt(), I don't know what it has to do with >> removal of noise in this structure. >> >> >> >> I am sorry if this question is not elaborated at all, I am Informatics >> student , but somehow I have to solve this problem and build >> architecture to develop a software in .NET. But before that I have to >> remove noise as it is an objective of my thesis part. >> >> >> >> >> >> Does these math algorithms needs to be replaces from other algorithms? >> >> >> >> >> >> If some body helps me out, I would be too much obliged for this. >> >> >> >> >> >> Regards >> >> Muhammad Usman Khalil > > Mates, > > I am not able to properly elaborate the problem I had. Sorry for this. > > Being a non DSP student, I am working on noise removal problem due to > some reason. It has become my Thesis study of Master's. But I would have > to implement a system that would follow the structure ( I mentioned as > URL ). All I am working hard to grasp some concepts regarding signals > and FFT. > > I know my data stream would be transformed from time domain to Frequency > Domain after FFT. The results of FFT has to be "process" somehow, so > that noise can be removed. > > I studied until now that there's no generic way for removal of noise > from contaminated signal. > > 1 - Either you must have cleaned signal and noise signal ( Apply Wiener > Filter or other DSP Filters ). > > 2 - Or Kalman Filter for taking more better results. > > This is all what I have researched until now. > But the problem is : I have this template from my past workplace side > and I am not able to get the answer from those guys somehow, why these > Mathematical Modules are there? What kind of modules these are? Once > they said these modules would be like "Sqrt". > > I am sorry , I am not DSP student but Informatics student. So, I am new > bie in this domain. But I have got the knowledge about my other modules > ( e.g FFT and Block Seperator ). But I have no idea about "Maths" > > That's why I posted here. Hope you will not angry this time.We're not angry. We have some members that can be impatient with students, more fools they. RBJ was just borrowing trouble. I fear that you're being assigned a project by people that don't necessarily know what they're asking for. You have my sympathy. Since you mention Kalman filtering I have to comment: a Kalman filter isn't necessarily going to give you better results than a Wiener filter. The filtering technique that you need to use depends entirely on the nature of the signal, of the noise, and what you care about. For most filtering applications where a Wiener filter is helpful, the only thing that a Kalman filter gets you is quicker acquisition. In fact, for those filtering applications where a Wiener filter is indicated, a Kalman filter that has been run for a good long time ends up being a Wiener filter. The "pure" Kalman filter is helpful when you need to have the very best estimate of a system's states (or of signal vs. noise) very early in the process, or when the system is time-varying in some known way -- and that's about it. You can take systems that are nonlinear in a known way, and treat them as if they were linear, time-varying systems, and apply Kalman-filter techniques. These aren't "pure" Kalman filters, though they are based on them (see "extended Kalman" and "unscented Kalman"). Kalman (and Wiener) filters depend on having a near-perfect knowledge of the system to work correctly. There are dodges to making them work well when the system is not known (usually one pretends that there's more noise than there really is, and hopes), but these dodges pretty much immediately take you away from a nice mathematically guaranteed optimal filter and dumps you right into the Land of Seat of the Pants Engineering. You can do the same thing in a more structured way with a so-called H-infinity filter, but these have a much greater computational load if you are trying to estimate the states of a time-varying system. -- Tim Wescott Wescott Design Services http://www.wescottdesign.com
Reply by ●July 17, 20132013-07-17
Rick Lyons <R.Lyons@_bogus_ieee.org> wrote: (big snip)> I don't know if you volunteered to solve your > "noise removal" problem, or if some German professor > assigned this problem to you. But in either case, > I believe someone has made a BIG mistake.> It seems to me that solving some sort of "noise > removal" problem, the details of which are > unknown at this time, would require a talanted > DSP engineer who has years of DSP experience. > Even then, success is not guaranteed. I say that > because, without knowing any details about your > noisy signals and what is meant by "noise removal", > it may not be possible achieve acceptable > noise removal.I suppose this is all true, but sometimes we live with it anyway. As for audio noise reduction, which many of us know better than others, sometimes you just turn down the tone control or treble control. It isn't the best, but sometimes it works well enough. Also, in audio (and likely others) there are two-sided (is that the right term) and single-ended (I believe that is right) noise reduction systems. The more popular dolby and dbx systems are two-sided, encoding the system before noise (vinyl disk or magnetic tape) recording, and decoding it on playback. Single-ended (on playback only) is more difficult, but sometimes must be done. Since audio, and especial spoken word, has little in the high frequencies, but noise is often uniform in frequency space, a low pass filter is sometimes a fine noise reduction system. -- glen
Reply by ●July 17, 20132013-07-17
On Tuesday, July 16, 2013 7:09:30 PM UTC-7, muhammadus...@gmail.com wrote:> Hi mates, > I need to know somehow, the parts of overall structure setup for removing background noise and determining background noise sources. > > I am working on a problem to solve background noise emitted from test objects. > > Here's written all on the document, I have > https://www.dropbox.com/sh/38hbzpufqiajm7n/D_vTPEP9mb > ...The type of structure you show is common to many signal processing applications. A general example is the class of instruments called dynamic spectrum analyzers. Combining that term in a Google with: application notes points to a variety of resources supplied by many venders if instrumentation. ### General Background ### http://www.measurement.net.au/g/5788/application-notes.html The Fundamentals of Signal Analysis Application Note 243 http://literature.agilent.com/litweb/pdf/5952-8898E.pdf ### Example Instruments ### http://www.me.ua.edu/me360/docs/HP-DSA.pdf Effective Machinery Measurements using Dynamic Signal Analyzers Application Note 243-1 The SR785 Data Sheet SR785c.pdf available from site: http://www.thinksrs.com/products/SR785.htm User Manual available through web site: http://www.thinksrs.com/mult/SR785m.htm ### Consider looking at the library at: http://www.bksv.com/library/ for examples of applications, algorithms and instruments for example: http://www.bksv.com/doc/TechnicalReview1996-2a.pdf Browse and learn. Come back with better questions. Dale B. Dalrymple
