A New Approach to Linear Filtering and Prediction Problems
In 1960, R.E. Kalman published his famous paper describing a recursive solution to the discrete-data linear filtering problem. Since that time, due in large part to advances in digital computing, the Kalman filter has been the subject of extensive research and application, particularly in the area of autonomous or assisted navigation.
A Quadrature Signals Tutorial: Complex, But Not Complicated
Quadrature signals are based on the notion of complex numbers and perhaps no other topic causes more heartache for newcomers to DSP than these numbers and their strange terminology of j operator, complex, imaginary, real, and orthogonal. If you're a little unsure of the physical meaning of complex numbers and the j = √-1 operator, don't feel bad because you're in good company. Why even Karl Gauss, one the world's greatest mathematicians, called the j operator the "shadow of shadows". Here we'll shine some light on that shadow so you'll never have to call the Quadrature Signal Psychic Hotline for help. Quadrature signal processing is used in many fields of science and engineering, and quadrature signals are necessary to describe the processing and implementation that takes place in modern digital communications systems. In this tutorial we'll review the fundamentals of complex numbers and get comfortable with how they're used to represent quadrature signals. Next we examine the notion of negative frequency as it relates to quadrature signal algebraic notation, and learn to speak the language of quadrature processing. In addition, we'll use three-dimensional time and frequency-domain plots to give some physical meaning to quadrature signals. This tutorial concludes with a brief look at how a quadrature signal can be generated by means of quadrature-sampling.
An Introduction To Compressive Sampling
This article surveys the theory of compressive sensing, also known as compressed sensing or CS, a novel sensing/sampling paradigm that goes against the common wisdom in data acquisition.
Lecture Notes on Elliptic Filter Design
Elliptic filters, also known as Cauer or Zolotarev filters, achieve the smallest filter order for the same specifications, or, the narrowest transition width for the same filter order, as compared to other filter types. On the negative side, they have the most nonlinear phase response over their passband. In these notes, we are primarily concerned with elliptic filters. But we will also discuss briefly the design of Butterworth, Chebyshev-1, and Chebyshev-2 filters and present a unified method of designing all cases. We also discuss the design of digital IIR filters using the bilinear transformation method.
Computing FFT Twiddle Factors
In this document are two algorithms showing how to compute the individual twiddle factors of an N-point decimation-in-frequency (DIF) and an N-point decimation-in-time (DIT) FFT.
Digital Envelope Detection: The Good, the Bad, and the Ugly
Recently I've been thinking about the process of envelope detection. Tutorial information on this topic is readily available but that information is spread out over a number of DSP textbooks and many Internet web sites. The purpose of this blog is to summarize various digital envelope detection methods in one place. Here I focus of envelope detection as it is applied to an amplitude-fluctuating sinusoidal signal where the positive-amplitude fluctuations (the sinusoid's envelope) contain some sort of information. Let's begin by looking at the simplest envelope detection method.
The World's Most Interesting FIR Filter Equation: Why FIR Filters Can Be Linear Phase
This article discusses a little-known filter characteristic that enables real- and complex-coefficient tapped-delay line FIR filters to exhibit linear phase behavior. That is, this article answers the question: What is the constraint on real- and complex-valued FIR filters that guarantee linear phase behavior in the frequency domain?
Introduction to Compressed Sensing
Chapter 1 of the book: "Compressed Sensing: Theory and Applications".
A Wide-Notch Comb Filter
This article describes a linear-phase comb filter having wider stopband notches than a traditional comb filter.
Multirate Systems and Filter Banks
During the last two decades, multirate filter banks have found various applications in many different areas, such as speech coding, scrambling, adaptive signal processing, image compression, signal and image processing applications as well as transmission of several signals through the same channel. The main idea of using multirate filter banks is the ability of the system to separate in the frequency domain the signal under consideration into two or more signals or to compose two or more different signals into a single signal.
A Friendly Introduction to Compressed Sensing
Compared to other signal processing techniques, compressed sensing (or sparse sampling) has caught the interest of many mathematicians, electrical engineers, and computer scientists. The field of compressed sensing is still rapidly evolving. As most papers and textbooks about compressed sensing are at graduate level, the purpose of this paper is to offer a gentler exposure to compressed sensing from a mathematical perspective. By synthesizing my study on compressed sensing as an undergraduate, this thesis covers important concepts in CS such as coherence and restricted isometry property. Several key algorithms in compressed sensing will also be introduced with discussions of their stability, robustness, and performance. In the end, we investigate single-pixel camera as an example of real-world application of compressed sensing.
Savitzky Golay Filter
The Savitzky-Golay filter is a mathematical smoothing filter that is often used in signal processing. It was first described in 1964 by Abraham Savitzky and Marcel Golay. The filter uses a polynomial regression over a series of values to find a smoothed value. One advantage of the Savitzky-Golay filter is that, unlike other smoothing filters, high-frequency components are not simply cut off, but are included in the calculation. As a result, the filter shows excellent properties with regard to the relative maxima, minima and scatter. In this article, the principle of the Savitzky Golay filter is explained and accompanied with MATLAB scripts. Two simple examples will be examined and provided with meaningful representations of the results to help understand these filters. The MATLAB function sgolay and its parameters are also explained and applied so that you can better understand this function and use it for your own applications.
Fractional Delay Farrow Filter
The Fractional Delay Farrow Filter is a digital filter that delays the discrete-time input signal by a fraction of the sample period. There are many applications where such a delay is necessary. As an example one can consider symbol synchronization in digital receivers, conversion between arbitrary sampling frequencies, echo cancellation, speech coding and speech synthesis, modeling of musical instruments, etc.
Decimation and Interpolation with IFIR Filters
In this article, the principle of the IFIR filter is first explained and accompanied by simulation. It also shows how to use the IFIR filters efficiently in the process of decimation and interpolation. Here, too, simulations with Simulink are used to explain the subject in an understandable manner.
Decimation and Interpolation with Polyphase Filters
This article deals with decimation and interpolation in classical and polyphase realizations. The topic is accompanied by many simulations with MATLAB and Simulink, so that the facts can be easily fixed. The simulations can be creatively expanded with your own ideas.
Zero-Order-Hold function as a model for DAConverters
The output of a digital to analog converter, short DAC, is a constant analog signal between two discrete samples. The DAC's output register retains its value from one sample up to the next sample. The network, which converts the binary value of the register into an analog voltage thus supplies a constant voltage and that leads to a stepped output signal. The analog smoothing filter connected at the output together with the frequency response of the DAC, modeled with a Zero-Order-Hold function, results in distortions. These are examined here and solutions to compensate for them are presented.
A Simplified Matlab Function for Power Spectral Density
In an earlier post, I showed how to compute power spectral density (PSD) of a discrete-time signal using the Matlab function pwelch. Pwelch is a useful function because it gives the correct output, and it has the option to average multiple Discrete Fourier Transforms (DFTs). However, a typical function call has five arguments, and it can be hard to remember how to set them all and how they default.
In this post, I create a simplified PSD function by putting a wrapper on pwelch that sets some parameters and converts the output units from W/Hz to dBW/bin. The function is named psd_simple.m, and its code is listed in the appendix.
Fractional Delay FIR Filters
Consider the following Finite Impulse Response (FIR) coefficients:
b = [b0 b1 b2 b1 b0]
These coefficients form a 5-tap symmetrical FIR filter having constant group delay [1,2] over 0 to fs/2 of:
D = (ntaps - 1)/2 = 2 samples
For a symmetrical filter with an odd number of taps, the group delay is always an integer number of samples, while for one with an even number of taps, the group delay is always an integer + 0.5 samples. Can we design a filter with arbitrary delay, say 9.3 samples? The answer is yes -- It is possible to design a non-symmetrical FIR filter with arbitrary group delay which is approximately constant over a wide band, with approximately flat magnitude response [3,4]. Let the desired group delay be:
D = (ntaps - 1)/2 + u
= D0 + u samples, (1)
where we call u the fractional delay and -0.5 <= u <= 0.5. D0 is the fixed portion of the total delay; it is determined by ntaps. The appendix lists a simple Matlab function frac_delay_fir.m to compute FIR coefficients for a given value of u and ntaps. The function provides coefficients with approximately flat delay and frequency responses over a frequency range approaching 0 to fs/2.
In this post, we'll present a couple of examples using the function, then discuss the theory behind it. Finally, we'll look at an example of a fractional delay lowpass FIR filter with arbitrary cut-off frequency.
Digital Filtering in the Frequency Domain
Time domain digital filtering, whether implemented using FIR or IIR techniques, has been very well documented in literature and been thoroughly used in many base band processor designs. However, with the advent of software defined radios as well as CPU support in more recent baseband processors, it has become possible and often desirable to filter signals in software rather than digital hardware. Whereas, time domain digital filtering can certainly be implemented in software as well, it becomes highly inefficient as the number of filter taps grows. Frequency domain filtering, using FFT and IFFT operations, is significantly more efficient and surprisingly easy to understand. This document introduces the reader to frequency domain filtering both in theory and in practice via a MatLab script.
The DFT of Finite-Length Time-Reversed Sequences
Recently I've been reading papers on underwater acoustic communications systems and this caused me to investigate the frequency-domain effects of time-reversal of time-domain sequences. I created this article because there is so little coverage of this topic in the literature of DSP.
A Quadrature Signals Tutorial: Complex, But Not Complicated
Quadrature signals are based on the notion of complex numbers and perhaps no other topic causes more heartache for newcomers to DSP than these numbers and their strange terminology of j operator, complex, imaginary, real, and orthogonal. If you're a little unsure of the physical meaning of complex numbers and the j = √-1 operator, don't feel bad because you're in good company. Why even Karl Gauss, one the world's greatest mathematicians, called the j operator the "shadow of shadows". Here we'll shine some light on that shadow so you'll never have to call the Quadrature Signal Psychic Hotline for help. Quadrature signal processing is used in many fields of science and engineering, and quadrature signals are necessary to describe the processing and implementation that takes place in modern digital communications systems. In this tutorial we'll review the fundamentals of complex numbers and get comfortable with how they're used to represent quadrature signals. Next we examine the notion of negative frequency as it relates to quadrature signal algebraic notation, and learn to speak the language of quadrature processing. In addition, we'll use three-dimensional time and frequency-domain plots to give some physical meaning to quadrature signals. This tutorial concludes with a brief look at how a quadrature signal can be generated by means of quadrature-sampling.
Savitzky Golay Filter
The Savitzky-Golay filter is a mathematical smoothing filter that is often used in signal processing. It was first described in 1964 by Abraham Savitzky and Marcel Golay. The filter uses a polynomial regression over a series of values to find a smoothed value. One advantage of the Savitzky-Golay filter is that, unlike other smoothing filters, high-frequency components are not simply cut off, but are included in the calculation. As a result, the filter shows excellent properties with regard to the relative maxima, minima and scatter. In this article, the principle of the Savitzky Golay filter is explained and accompanied with MATLAB scripts. Two simple examples will be examined and provided with meaningful representations of the results to help understand these filters. The MATLAB function sgolay and its parameters are also explained and applied so that you can better understand this function and use it for your own applications.
Digital Envelope Detection: The Good, the Bad, and the Ugly
Recently I've been thinking about the process of envelope detection. Tutorial information on this topic is readily available but that information is spread out over a number of DSP textbooks and many Internet web sites. The purpose of this blog is to summarize various digital envelope detection methods in one place. Here I focus of envelope detection as it is applied to an amplitude-fluctuating sinusoidal signal where the positive-amplitude fluctuations (the sinusoid's envelope) contain some sort of information. Let's begin by looking at the simplest envelope detection method.
Design IIR Filters Using Cascaded Biquads
This article shows how to implement a Butterworth IIR lowpass filter as a cascade of second-order IIR filters, or biquads. We'll derive how to calculate the coefficients of the biquads and do some examples using a Matlab function biquad_synth provided in the Appendix. Although we'll be designing Butterworth filters, the approach applies to any all-pole lowpass filter (Chebyshev, Bessel, etc). As we'll see, the cascaded-biquad design is less sensitive to coefficient quantization than a single high-order IIR, particularly for lower cut-off frequencies.
A Friendly Introduction to Compressed Sensing
Compared to other signal processing techniques, compressed sensing (or sparse sampling) has caught the interest of many mathematicians, electrical engineers, and computer scientists. The field of compressed sensing is still rapidly evolving. As most papers and textbooks about compressed sensing are at graduate level, the purpose of this paper is to offer a gentler exposure to compressed sensing from a mathematical perspective. By synthesizing my study on compressed sensing as an undergraduate, this thesis covers important concepts in CS such as coherence and restricted isometry property. Several key algorithms in compressed sensing will also be introduced with discussions of their stability, robustness, and performance. In the end, we investigate single-pixel camera as an example of real-world application of compressed sensing.
Decimation and Interpolation with Polyphase Filters
This article deals with decimation and interpolation in classical and polyphase realizations. The topic is accompanied by many simulations with MATLAB and Simulink, so that the facts can be easily fixed. The simulations can be creatively expanded with your own ideas.
A New Approach to Linear Filtering and Prediction Problems
In 1960, R.E. Kalman published his famous paper describing a recursive solution to the discrete-data linear filtering problem. Since that time, due in large part to advances in digital computing, the Kalman filter has been the subject of extensive research and application, particularly in the area of autonomous or assisted navigation.
Lecture Notes on Elliptic Filter Design
Elliptic filters, also known as Cauer or Zolotarev filters, achieve the smallest filter order for the same specifications, or, the narrowest transition width for the same filter order, as compared to other filter types. On the negative side, they have the most nonlinear phase response over their passband. In these notes, we are primarily concerned with elliptic filters. But we will also discuss briefly the design of Butterworth, Chebyshev-1, and Chebyshev-2 filters and present a unified method of designing all cases. We also discuss the design of digital IIR filters using the bilinear transformation method.
Introduction of C Programming for DSP Applications
Appendix C of the book : Real-Time Digital Signal Processing: Implementations, Application and Experiments with the TMS320C55X
Complex Digital Signal Processing in Telecommunications
Digital Signal Processing (DSP) is a vital tool for scientists and engineers, as it is of fundamental importance in many areas of engineering practice and scientific research. The "alphabet" of DSP is mathematics and although most practical DSP problems can be solved by using real number mathematics, there are many others which can only be satisfactorily resolved or adequately described by means of complex numbers. If real number mathematics is the language of real DSP, then complex number mathematics is the language of complex DSP. In the same way that real numbers are a part of complex numbers in mathematics, real DSP can be regarded as a part of complex DSP (Smith, 1999). Complex mathematics manipulates complex numbers - the representation of two variables as a single number - and it may appear that complex DSP has no obvious connection with our everyday experience, especially since many DSP problems are explained mainly by means of real number mathematics. Nonetheless, some DSP techniques are based on complex mathematics, such as Fast Fourier Transform (FFT), z-transform, representation of periodical signals and linear systems, etc. However, the imaginary part of complex transformations is usually ignored or regarded as zero due to the inability to provide a readily comprehensible physical explanation. One well-known practical approach to the representation of an engineering problem by means of complex numbers can be referred to as the assembling approach: the real and imaginary parts of a complex number are real variables and individually can represent two real physical parameters. Complex math techniques are used to process this complex entity once it is assembled. The real and imaginary parts of the resulting complex variable preserve the same real physical parameters. This approach is not universally-applicable and can only be used with problems and applications which conform to the requirements of complex math techniques. Making a complex number entirely mathematically equivalent to a substantial physical problem is the real essence of complex DSP. Like complex Fourier transforms, complex DSP transforms show the fundamental nature of complex DSP and such complex techniques often increase the power of basic DSP methods. The development and application of complex DSP are only just beginning to increase and for this reason some researchers have named it theoretical DSP. It is evident that complex DSP is more complicated than real DSP. Complex DSP transforms are highly theoretical and mathematical; to use them efficiently and professionally requires a large amount of mathematics study and practical experience. Complex math makes the mathematical expressions used in DSP more compact and solves the problems which real math cannot deal with. Complex DSP techniques can complement our understanding of how physical systems perform but to achieve this, we are faced with the necessity of dealing with extensive sophisticated mathematics. For DSP professionals there comes a point at which they have no real choice since the study of complex number mathematics is the foundation of DSP.
