Some recent posts have triggered have triggered questions in my mind. I don't think there is a common thread, but I'm putting them in one post as it may highlite a point of confusion that wouldn't otherwise come to mind. 1. For an LTI network in either the analog or discrete domain, for two networks having the same impulse response implies having the same magnitude vs frequency and phase vs frequency response? I've always thought it did and was result of linear superposition holding for an LTI system. 2. This question is prompted in part recent post on using FFT to solve differential equations and the paragraph titled "Partial differential equations" at http://en.wikipedia.org/wiki/Discrete_Fourier_transform . I attempted a BSEE back in the 60's. In a probably a Junior "math" or "signals and system" course we were introduced to a method for solving systems of linear differential equations. "linear" is a vague term as I'm not sure of in what domain things were linear. As I incompletely recall the system mechanically substituted "D" for d/dt {differential with respect to t [total derivative or a partial?]}. That gave a system of algebraic equations which were solved using linear algebra. I haven't the foggiest recollection of how we applied the result to the real world. Does this give a useful hint to anyone of an appropriate Google search term or a specific reference? 3. I have a problem with some FIR filter basics. Just because I have Rick Lyons book in front of me I'll use his section headings and examples. I get lost some where in region of Section 5.2 "Convolution in FIR Filters" (Equation 5-8) and Section 5.3 "Low-pass FIR Filter Design" (Equation 5-8 [First time e to j omega t is used]). I don't have too much problem with complex notation and its advantages per se. It's in the transition from real numbers a weighting factors (i.e. Figure 5-13 "Five-tap low-pass filter implementation ...") to the weightings have a complex value. I've two questions: a. What class of filters can be implemented using ONLY real numbers for the weighting factors? I'm guessing that some sort of band pass filter could be implemented if every n'th factor was alternating between +1 and -1 [with major stability issues probable ;] I can see physically implementing it as a tapped delay line feeding a summing amplifier. b. How could the case of complex coefficients be physically implemented?
Possibly unrelated questions
Started by ●February 15, 2008
Reply by ●February 15, 20082008-02-15
On 15 Feb, 14:27, Richard Owlett <rowl...@atlascomm.net> wrote:> Some recent posts have triggered have triggered questions in my mind. I > don't think there is a common thread, but I'm putting them in one post > as it may highlite a point of confusion that wouldn't otherwise come to > mind. > > 1. For an LTI network in either the analog or discrete domain, > for two networks having the same impulse response implies having the > same magnitude vs frequency and phase vs frequency response? I've always > thought it did and was result of linear superposition holding for an LTI > system.Correct.> 2. This question is prompted in part recent post on using FFT to solve > differential equations and the paragraph titled "Partial differential > equations" athttp://en.wikipedia.org/wiki/Discrete_Fourier_transform.Fourier developed the series now bearing his name as a tool for solving differential equations.> I attempted a BSEE back in the 60's. In a probably a Junior "math" or > "signals and system" course we were introduced to a method for solving > systems of linear differential equations. "linear" is a vague term as > I'm not sure of in what domain things were linear. As I incompletely > recall the system mechanically substituted "D" for d/dt {differential > with respect to t [total derivative or a partial?]}. That gave a system > of algebraic equations which were solved using linear algebra. I haven't > the foggiest recollection of how we applied the result to the real > world. Does this give a useful hint to anyone of an appropriate Google > search term or a specific reference?Kreyzig's "Advanced Engineering Mathemathics" is a standard text which has been available for decades, and shows how to use Fourier methods to solve (P)DEs.> 3. I have a problem with some FIR filter basics. Just because I have > Rick Lyons book in front of me I'll use his section headings and > examples. I get lost some where in region of Section 5.2 "Convolution in > FIR Filters" (Equation 5-8) and Section 5.3 "Low-pass FIR Filter Design" > (Equation 5-8 [First time e to j omega t is used]). I don't have too > much problem with complex notation and its advantages per se. It's in > the transition from real numbers a weighting factors (i.e. Figure 5-13 > "Five-tap low-pass filter implementation ...") to the weightings have a > complex value. > > I've two questions: > > a. What class of filters can be implemented using ONLY real numbers for > the weighting factors? I'm guessing that some sort of band pass filter > could be implemented if every n'th factor was alternating between +1 and > -1 [with major stability issues probable ;] I can see physically > implementing it as a tapped delay line feeding a summing amplifier.If the coefficients of the filter have to be real, the poles and zeros necessarily have to a) come as complex conjugated pairs in s or z domain b) come as one real-valued pole or zero> b. How could the case of complex coefficients be physically implemented?In digital domain one uses complex arithmetic. No problem. In terms of analog cirquits one would need to use some sort of quadrature sampling scheme. Rune
Reply by ●February 15, 20082008-02-15
Hi Richard, I think you are asking some excellent questions. Allow me to attempt to answer. Richard Owlett <rowlett@atlascomm.net> writes:> Some recent posts have triggered have triggered questions in my > mind. I don't think there is a common thread, but I'm putting them in > one post as it may highlite a point of confusion that wouldn't > otherwise come to mind. > > 1. For an LTI network in either the analog or discrete domain, > for two networks having the same impulse response implies having the > same magnitude vs frequency and phase vs frequency response? I've > always thought it did and was result of linear superposition holding > for an LTI system.Yes. This is because both the magnitude and phase response are derived from the transform of the impulse response (the FT for a continuous-time system and the DTFT for a discrete-time system).> 2. This question is prompted in part recent post on using FFT to solve > differential equations and the paragraph titled "Partial differential > equations" at http://en.wikipedia.org/wiki/Discrete_Fourier_transform . > > I attempted a BSEE back in the 60's. In a probably a Junior "math" or > "signals and system" course we were introduced to a method for solving > systems of linear differential equations. "linear" is a vague term as > I'm not sure of in what domain things were linear. As I incompletely > recall the system mechanically substituted "D" for d/dt {differential > with respect to t [total derivative or a partial?]}. That gave a > system of algebraic equations which were solved using linear > algebra. I haven't the foggiest recollection of how we applied the > result to the real world. Does this give a useful hint to anyone of an > appropriate Google search term or a specific reference?You're referring to Laplace transforms. The Laplace transform of a linear differential equation transforms it into an algebraic equaiton.> 3. I have a problem with some FIR filter basics. Just because I have > Rick Lyons book in front of me I'll use his section headings and > examples. I get lost some where in region of Section 5.2 "Convolution > in FIR Filters" (Equation 5-8) and Section 5.3 "Low-pass FIR Filter > Design" (Equation 5-8 [First time e to j omega t is used]). I don't > have too much problem with complex notation and its advantages per > se. It's in the transition from real numbers a weighting factors > (i.e. Figure 5-13 "Five-tap low-pass filter implementation ...") to > the weightings have a complex value. > > I've two questions: > > a. What class of filters can be implemented using ONLY real numbers > for the weighting factors?To answer this question, review the properties of the DTFT of a sequence (think of it as just a sequence, not a filter, for now): namely, that the DTFT of a real sequence is Hermitian symmetric in frequency. If a function f(x) is Hermitian symmetric it means that f(x) = f*(-x), where f*() denotes the complex conjugate of f(). Another way to look at this is that the magnitude of the function is even symmetric (|f(x)| = |f(-x)|) and the phase of the function is odd symmetric (phase(f(x)) = -phase(f(-x))). So the class of filters that can be implemented using ONLY real numbers is preciesly the class of filters that have a Hermitian symmetric frequency response.> I'm guessing that some sort of band pass > filter could be implemented if every n'th factor was alternating > between +1 and -1 [with major stability issues probable ;] I can see > physically implementing it as a tapped delay line feeding a summing > amplifier.You've completely lost me on this. The simple answer is, "Any filter that has a Hermitian symmetric response."> b. How could the case of complex coefficients be physically implemented?Complex numbers can be expressed in either of two forms: polar (r*exp(j*theta)) and rectangular (x + j*y). In a real-world implementation of complex arithmetic operations, you usually use rectangular representation and perform the operations accordingly. This will require more operations than operations with real numbers, a simple example being that a = ax + j*ay added to b = bx + j*by requires ax+bx and ay+by, i.e., two additions instead of just one. -- % Randy Yates % "She tells me that she likes me very much, %% Fuquay-Varina, NC % but when I try to touch, she makes it %%% 919-577-9882 % all too clear." %%%% <yates@ieee.org> % 'Yours Truly, 2095', *Time*, ELO http://www.digitalsignallabs.com
Reply by ●February 15, 20082008-02-15
Randy Yates <yates@ieee.org> writes:> [...] > Richard Owlett <rowlett@atlascomm.net> writes: >> 2. This question is prompted in part recent post on using FFT to solve >> differential equations and the paragraph titled "Partial differential >> equations" at http://en.wikipedia.org/wiki/Discrete_Fourier_transform . >> >> I attempted a BSEE back in the 60's. In a probably a Junior "math" or >> "signals and system" course we were introduced to a method for solving >> systems of linear differential equations. "linear" is a vague term as >> I'm not sure of in what domain things were linear. As I incompletely >> recall the system mechanically substituted "D" for d/dt {differential >> with respect to t [total derivative or a partial?]}. That gave a >> system of algebraic equations which were solved using linear >> algebra. I haven't the foggiest recollection of how we applied the >> result to the real world. Does this give a useful hint to anyone of an >> appropriate Google search term or a specific reference? > > You're referring to Laplace transforms. The Laplace transform of a > linear differential equation transforms it into an algebraic equaiton.PS: An excellent, and very readable, text on this is the old [Spiegel] book. --Randy @BOOK{spiegel, title = "{Applied Differential Equations}", author = "{Murray~R.~Spiegel}", publisher = "Prentice Hall", edition = "third", year = "1981"} -- % Randy Yates % "She tells me that she likes me very much, %% Fuquay-Varina, NC % but when I try to touch, she makes it %%% 919-577-9882 % all too clear." %%%% <yates@ieee.org> % 'Yours Truly, 2095', *Time*, ELO http://www.digitalsignallabs.com
Reply by ●February 15, 20082008-02-15
Rune Allnor wrote:> On 15 Feb, 14:27, Richard Owlett <rowl...@atlascomm.net> wrote:...>> I attempted a BSEE back in the 60's. In a probably a Junior "math" or >> "signals and system" course we were introduced to a method for solving >> systems of linear differential equations. "linear" is a vague term as >> I'm not sure of in what domain things were linear. As I incompletely >> recall the system mechanically substituted "D" for d/dt {differential >> with respect to t [total derivative or a partial?]}. That gave a system >> of algebraic equations which were solved using linear algebra. I haven't >> the foggiest recollection of how we applied the result to the real >> world. Does this give a useful hint to anyone of an appropriate Google >> search term or a specific reference? > > Kreyzig's "Advanced Engineering Mathemathics" is a standard > text which has been available for decades, and shows how > to use Fourier methods to solve (P)DEs.D is a differential operator (specifically, the D operator), invented by Oliver Heavyside, inspired by Boole. Dy == dy/dx, D^2y == d^2y/dx^2, D^3y == d^y/dx^3, etc. It allows algebraic manipulation of homogeneous linear differential equations and is isomorphic with the Laplace transform. Basically, Heavyside invented the use of the operator for solving DEs, but although it worked, the mathematical grounding for it was less than completely rigorous. The engineering approach not being good enough for mathematicians, some of them (lacking Heavyside's creative genius or the ability to supply the missing rigor themselves) searched the literature and rediscovered Laplace. Then they no longer had to hold their noses while using Heavyside transforms; they credited Laplace instead. http://mathworld.wolfram.com/HeavisideCalculus.html ...> In terms of analog cirquits one would need to use some > sort of quadrature sampling scheme.Quadrature, yes. Sampling, no. Jerry -- Engineering is the art of making what you want from things you can get. �����������������������������������������������������������������������
Reply by ●February 15, 20082008-02-15
On Fri, 15 Feb 2008 07:27:14 -0600, Richard Owlett <rowlett@atlascomm.net> wrote:>Some recent posts have triggered have triggered questions in my mind. I >don't think there is a common thread, but I'm putting them in one post >as it may highlite a point of confusion that wouldn't otherwise come to >mind. > >1. For an LTI network in either the analog or discrete domain, >for two networks having the same impulse response implies having the >same magnitude vs frequency and phase vs frequency response? I've always >thought it did and was result of linear superposition holding for an LTI >system. > > >2. This question is prompted in part recent post on using FFT to solve >differential equations and the paragraph titled "Partial differential >equations" at http://en.wikipedia.org/wiki/Discrete_Fourier_transform . > >I attempted a BSEE back in the 60's. In a probably a Junior "math" or >"signals and system" course we were introduced to a method for solving >systems of linear differential equations. "linear" is a vague term as >I'm not sure of in what domain things were linear. As I incompletely >recall the system mechanically substituted "D" for d/dt {differential >with respect to t [total derivative or a partial?]}. That gave a system >of algebraic equations which were solved using linear algebra. I haven't >the foggiest recollection of how we applied the result to the real >world. Does this give a useful hint to anyone of an appropriate Google >search term or a specific reference? > >3. I have a problem with some FIR filter basics. Just because I have >Rick Lyons book in front of me I'll use his section headings and >examples. I get lost some where in region of Section 5.2 "Convolution in >FIR Filters" (Equation 5-8) and Section 5.3 "Low-pass FIR Filter Design" >(Equation 5-8 [First time e to j omega t is used]). I don't have too >much problem with complex notation and its advantages per se. It's in >the transition from real numbers a weighting factors (i.e. Figure 5-13 >"Five-tap low-pass filter implementation ...") to the weightings have a >complex value.The previous items have been well addressed already, I'll add a tiny bit to the stuff below:>I've two questions: > >a. What class of filters can be implemented using ONLY real numbers for >the weighting factors? I'm guessing that some sort of band pass filter >could be implemented if every n'th factor was alternating between +1 and >-1 [with major stability issues probable ;] I can see physically >implementing it as a tapped delay line feeding a summing amplifier.A real-valued set of coefficients will have a response that is symmetric about DC. For many applictations that's fine, since the spectrum of interest is from DC and up or just above DC somewhere. Randy said the same thing, but a little more obliquely.>b. How could the case of complex coefficients be physically implemented?The tap multiplies become complex rather than real. i.e., the same tapped-delay line as usual, only each multiply is complex rather than real. If the input is complex and the tap values are complex, it's straightforward, just, well, complex. Systems that process signals at baseband, i.e., with the spectrum of interest centered at DC rather than above it, do this stuff all the time. Channel equalizers for single-carrier communication systems are often implemented as fully complex FIR filters with adaptive tap coefficients. Eric Jacobsen Minister of Algorithms Abineau Communications http://www.ericjacobsen.org
Reply by ●February 15, 20082008-02-15
Jerry Avins wrote:> Rune Allnor wrote: > >> On 15 Feb, 14:27, Richard Owlett <rowl...@atlascomm.net> wrote: > > > ... > >>> I attempted a BSEE back in the 60's. In a probably a Junior "math" or >>> "signals and system" course we were introduced to a method for solving >>> systems of linear differential equations. "linear" is a vague term as >>> I'm not sure of in what domain things were linear. As I incompletely >>> recall the system mechanically substituted "D" for d/dt {differential >>> with respect to t [total derivative or a partial?]}. That gave a system >>> of algebraic equations which were solved using linear algebra. I haven't >>> the foggiest recollection of how we applied the result to the real >>> world. Does this give a useful hint to anyone of an appropriate Google >>> search term or a specific reference? >> >> >> Kreyzig's "Advanced Engineering Mathemathics" is a standard >> text which has been available for decades, and shows how >> to use Fourier methods to solve (P)DEs. > > > D is a differential operator (specifically, the D operator), invented by > Oliver Heavyside, inspired by Boole. Dy == dy/dx, D^2y == d^2y/dx^2, > D^3y == d^y/dx^3, etc. It allows algebraic manipulation of> homogeneous linear differential equations Oh is that a phrase that brings back memories (nightmares? ;). That's what I was looking for. Now that I know what I'm looking for I can go Forth and Google. > and is isomorphic with the Laplace> transform. Basically, Heavyside invented the use of the operator for > solving DEs, but although it worked, the mathematical grounding for it > was less than completely rigorous. The engineering approach not being > good enough for mathematicians, some of them (lacking Heavyside's > creative genius or the ability to supply the missing rigor themselves) > searched the literature and rediscovered Laplace. Then they no longer > had to hold their noses while using Heavyside transforms; they credited > Laplace instead. http://mathworld.wolfram.com/HeavisideCalculus.html > > ... > >> In terms of analog cirquits one would need to use some >> sort of quadrature sampling scheme. > > > Quadrature, yes. Sampling, no. > > Jerry
Reply by ●February 15, 20082008-02-15
Rune Allnor wrote: My first two questions have been dealt with well enough that I can follow up on my own. Thank you all.> > >>3. I have a problem with some FIR filter basics. Just because I have >>Rick Lyons book in front of me I'll use his section headings and >>examples. I get lost some where in region of Section 5.2 "Convolution in >>FIR Filters" (Equation 5-8) and Section 5.3 "Low-pass FIR Filter Design" >>(Equation 5-8 [First time e to j omega t is used]). I don't have too >>much problem with complex notation and its advantages per se. It's in >>the transition from real numbers a weighting factors (i.e. Figure 5-13 >>"Five-tap low-pass filter implementation ...") to the weightings have a >>complex value. >> >>I've two questions: >> >>a. What class of filters can be implemented using ONLY real numbers for >>the weighting factors? I'm guessing that some sort of band pass filter >>could be implemented if every n'th factor was alternating between +1 and >>-1 [with major stability issues probable ;] I can see physically >>implementing it as a tapped delay line feeding a summing amplifier. > > > If the coefficients of the filter have to be real, the poles > and zeros necessarily have to > > a) come as complex conjugated pairs in s or z domain > b) come as one real-valued pole or zeroSo in the mindset I was in, "If a analog filter can be realized as an RLC network, the discrete domain counter part would be FIR with strictly real coefficients." Correct? And based on Randy's answer, I'll have to read up on "Hermitian" to ask an intelligent version of my follow up question "how to select real coefficients for a desired response?"> > >>b. How could the case of complex coefficients be physically implemented? > > > In digital domain one uses complex arithmetic. No problem. > In terms of analog cirquits one would need to use some > sort of quadrature sampling scheme.Including Jerry's comment, I suspect the the answer for my frame of mind would be "not in a way familiar to an undergrad EE student of 40 years ago." The object of the exercise is to tie some aspects of DSP into things I'm familiar with.
Reply by ●February 15, 20082008-02-15
Hi Richard, One of your comments gave me a little laugh-- "I don't have too much problem with complex notation and its advantages per se." In contrast, I have a terrible problem with complex notation. I'd even go so far at to call it the "scourge of DSP." Let me stand on my soap box a bit. There is no question that complex numbers are elegant and enable some techniques that could not be achieved otherwise. A good example of this is the FFT. They also provide a compact and efficient way of handling the mathematics of DSP. So don't get me wrong; complex notation is a powerful and useful method. However, the vast majority of practical DSP techniques gain no benefit at all from using complex numbers. This includes the big three: Convolution, Spectral Analysis, and Basic Filtering. My mission over the years has been to show that 99% of useful DSP methods can be understood without needing to resort to complex notation. In my mind, complex methods should be viewed as an advanced subject; a second tier of education. For instance, this is how I structured my book. Out of 33 chapters, I don't use complex numbers until the last four. If interested, see www.DSPguide.com. Try starting with Chapter 14. This isn't to downplay other references-- for instance, Rick Lyons book' is really outstanding. I just wanted to give you an alternative approach that may mesh better with your background. Why is this so personal to me? I came through a conventional Ph.D. program in EE with emphasis on DSP. My primary mentor was Tom Stockham, a pioneer in the field and an outstanding instructor. And I did well-- nothing but "A"s. Then I hit industry with quite a shock. I could do integrals like crazy, but couldn't design even the most basic filters. The primary reason I wrote my book was to teach myself useful DSP techniques-- what I should have learned in college, but didn't because they were too busy teaching me complex math. Soap box speech over. Good luck! Regards, Steve
Reply by ●February 15, 20082008-02-15
Hi Richard, One of your comments gave me a little laugh-- "I don't have too much problem with complex notation and its advantages per se." In contrast, I have a terrible problem with complex notation. I'd even go so far at to call it the "scourge of DSP." Let me stand on my soap box a bit. There is no question that complex numbers are elegant and enable some techniques that could not be achieved otherwise. A good example of this is the FFT. They also provide a compact and efficient way of handling the mathematics of DSP. So don't get me wrong; complex notation is a powerful and useful method. However, the vast majority of practical DSP techniques gain no benefit at all from using complex numbers. This includes the big three: Convolution, Spectral Analysis, and Basic Filtering. My mission over the years has been to show that 99% of useful DSP methods can be understood without needing to resort to complex notation. In my mind, complex methods should be viewed as an advanced subject; a second tier of education. For instance, this is how I structured my book. Out of 33 chapters, I don't use complex numbers until the last four. If interested, see www.DSPguide.com. Try starting with Chapter 14. This isn't to downplay other references-- for instance, Rick Lyons book' is really outstanding. I just wanted to give you an alternative approach that may mesh better with your background. Why is this so personal to me? I came through a conventional Ph.D. program in EE with emphasis on DSP. My primary mentor was Tom Stockham, a pioneer in the field and an outstanding instructor. And I did well-- nothing but "A"s. Then I hit industry with quite a shock. I could do integrals like crazy, but couldn't design even the most basic filters. The primary reason I wrote my book was to teach myself useful DSP techniques-- what I should have learned in college, but didn't because they were too busy teaching me complex math. Soap box speech over. Good luck! Regards, Steve
