## Maximum Likelihood Estimation

Any observation has some degree of noise content that makes our observations uncertain. When we try to make conclusions based on noisy observations, we have to separate the dynamics of a signal from noise.

## Approximating the area of a chirp by fitting a polynomial

Once in a while we need to estimate the area of a dataset in which we are interested. This area could give us, for example, force (mass vs acceleration) or electric power (electric current vs charge).

## Deconvolution by least squares (Using the power of linear algebra in signal processing).

When we deal with our normal discrete signal processing operations, like FIR/IIR filtering, convolution, filter design, etc. we normally think of the signals as a constant stream of numbers that we put in a sequence

## The Most Interesting FIR Filter Equation in the World: Why FIR Filters Can Be Linear Phase

This blog discusses a little-known filter characteristic that enables real- and complex-coefficient tapped-delay line FIR filters to exhibit linear phase behavior. That is, this blog answers the question:

What is the constraint on real- and complex-valued FIR filters that guarantee linear phase behavior in the frequency domain?I'll declare two things to convince you to continue reading.

Declaration# 1: "That the coefficients must be symmetrical" is not a correct

## Phase and Amplitude Calculation for a Pure Real Tone in a DFT: Method 1

IntroductionThis is an article to hopefully give a better understanding of the Discrete Fourier Transform (DFT) by deriving exact formulas for the phase and amplitude of a non-integer frequency real tone in a DFT. The linearity of the Fourier Transform is exploited to reframe the problem as the equivalent of finding a set of coordinates in a specific vector space. The found coordinates are then used to calculate the phase and amplitude of the pure real tone in the DFT. This article...

## Exact Frequency Formula for a Pure Real Tone in a DFT

IntroductionThis is an article to hopefully give a better understanding of the Discrete Fourier Transform (DFT) by deriving an exact formula for the frequency of a real tone in a DFT. According to current teaching, this is not possible, so this article should be considered a major theoretical advance in the discipline. The formula is presented in a few different formats. Some sample calculations are provided to give a numerical demonstration of the formula in use. This article is...

## DFT Bin Value Formulas for Pure Real Tones

IntroductionThis is an article to hopefully give a better understanding to the Discrete Fourier Transform (DFT) by deriving an analytical formula for the DFT of pure real tones. The formula is used to explain the well known properties of the DFT. A sample program is included, with its output, to numerically demonstrate the veracity of the formula. This article builds on the ideas developed in my previous two blog articles:

## DFT Graphical Interpretation: Centroids of Weighted Roots of Unity

IntroductionThis is an article to hopefully give a better understanding to the Discrete Fourier Transform (DFT) by framing it in a graphical interpretation. The bin calculation formula is shown to be the equivalent of finding the center of mass, or centroid, of a set of points. Various examples are graphed to illustrate the well known properties of DFT bin values. This treatment will only consider real valued signals. Complex valued signals can be analyzed in a similar manner with...

## The Exponential Nature of the Complex Unit Circle

IntroductionThis is an article to hopefully give an understanding to Euler's magnificent equation:

$$ e^{i\theta} = cos( \theta ) + i \cdot sin( \theta ) $$

This equation is usually proved using the Taylor series expansion for the given functions, but this approach fails to give an understanding to the equation and the ramification for the behavior of complex numbers. Instead an intuitive approach is taken that culminates in a graphical understanding of the equation.

Complex...## Sum of Two Equal-Frequency Sinusoids

Some time ago I reviewed the manuscript of a book being considered by the IEEE Press publisher for possible publication. In that manuscript the author presented the following equation:

Being unfamiliar with Eq. (1), and being my paranoid self, I wondered if that equation is indeed correct. Not finding a stock trigonometric identity in my favorite math reference book to verify Eq. (1), I modeled both sides of the equation using software. Sure enough, Eq. (1) is not correct. So then I...

## Filter a Rectangular Pulse with no Ringing

To filter a rectangular pulse without any ringing, there is only one requirement on the filter coefficients: they must all be positive. However, if we want the leading and trailing edge of the pulse to be symmetrical, then the coefficients must be symmetrical. What we are describing is basically a window function.

Consider a rectangular pulse 32 samples long with fs = 1 kHz. Here is the Matlab code to generate the pulse:

N= 64; fs= 1000; % Hz sample...## A Two Bin Solution

IntroductionThis is an article to hopefully give a better understanding of the Discrete Fourier Transform (DFT) by showing an implementation of how the parameters of a real pure tone can be calculated from just two DFT bin values. The equations from previous articles are used in tandem to first calculate the frequency, and then calculate the amplitude and phase of the tone. The approach works best when the tone is between the two DFT bins in terms of frequency.

The Coding...## A Recipe for a Basic Trigonometry Table

IntroductionThis is an article that is give a better understanding to the Discrete Fourier Transform (DFT) by showing how to build a Sine and Cosine table from scratch. Along the way a recursive method is developed as a tone generator for a pure tone complex signal with an amplitude of one. Then a simpler multiplicative one. Each with drift correction factors. By setting the initial values to zero and one degrees and letting it run to build 45 values, the entire set of values needed...

## Amplitude modulation and the sampling theorem

I am working on the 11th and probably final chapter of Think DSP, which follows material my colleague Siddhartan Govindasamy developed for a class at Olin College. He introduces amplitude modulation as a clever way to sneak up on the Nyquist–Shannon sampling theorem.

Most of the code for the chapter is done: you can check it out in this IPython notebook. I haven't written the text yet, but I'll outline it here, and paste in the key figures.

Convolution...

## Two Bin Exact Frequency Formulas for a Pure Real Tone in a DFT

IntroductionThis is an article to hopefully give a better understanding of the Discrete Fourier Transform (DFT) by deriving exact formulas for the frequency of a real tone in a DFT. This time it is a two bin version. The approach taken is a vector based one similar to the approach used in "Three Bin Exact Frequency Formulas for a Pure Complex Tone in a DFT"[1]. The real valued formula presented in this article actually preceded, and was the basis for the complex three bin...

## Analytic Signal

In communication theory and modulation theory we always deal with two phases: In-phase (I) and Quadrature-phase (Q). The question that I will discuss in this blog is that why we use two phases and not more.

## Discrete Wavelet Transform Filter Bank Implementation (part 2)

Following the previous blog entry: http://www.dsprelated.com/showarticle/115.php

Difference between DWT and DWPTBefore getting to the equivalent filter obtention, I first want to talk about the difference between DWT(Discrete Wavelet Transform) and DWPT (Discrete Wavelet Packet Transform). The latter is used mostly for image processing.

While DWT has a single "high-pass" branch that filters the signal with the h1 filter, the DWPT separates branches symmetricaly: this means that one...

## DFT Bin Value Formulas for Pure Complex Tones

IntroductionThis is an article to hopefully give a better understanding to the Discrete Fourier Transform (DFT) by deriving an analytical formula for the DFT of pure complex tones and an alternative variation. It is basically a parallel treatment to the real case given in DFT Bin Value Formulas for Pure Real Tones. In order to understand how a multiple tone signal acts in a DFT it is necessary to first understand how a single pure tone acts. Since a DFT is a linear transform, the...

## A Recipe for a Common Logarithm Table

IntroductionThis is an article that is a digression from trying to give a better understanding to the Discrete Fourier Transform (DFT).

A method for building a table of Base 10 Logarithms, also known as Common Logarithms, is featured using math that can be done with paper and pencil. The reader is assumed to have some familiarity with logarithm functions. This material has no dependency on the material in my previous blog articles.

If you were ever curious about how...

## An Alternative Form of the Pure Real Tone DFT Bin Value Formula

IntroductionThis is an article to hopefully give a better understanding of the Discrete Fourier Transform (DFT) by deriving alternative exact formulas for the bin values of a real tone in a DFT. The derivation of the source equations can be found in my earlier blog article titled "DFT Bin Value Formulas for Pure Real Tones"[1]. The new form is slighty more complicated and calculation intensive, but it is more computationally accurate in the vicinity of near integer frequencies. This...

## Exponential Smoothing with a Wrinkle

IntroductionThis is an article to hopefully give a better understanding to the Discrete Fourier Transform (DFT) by providing a set of preprocessing filters to improve the resolution of the DFT. Because of the exponential nature of sinusoidal functions, they have special mathematical properties when exponential smoothing is applied to them. These properties are derived and explained in this blog article.

Basic Exponential Smoothing

Exponential smoothing is also known as...

## Feedback Controllers - Making Hardware with Firmware. Part 4. Engineering of Evaluation Hardware

Following on from the previous abstract descriptions of an arbitrary circuit emulation application for low-latency feedback controllers, we now come to some aspects in the hardware engineering of an evaluation design from concept to first power-up. In due course a complete specification along with application examples will be maintained on the project website.- Part 1: Introduction
- Part 2:...

## Feedback Controllers - Making Hardware with Firmware. Part 3. Sampled Data Aspects

Some Design and Simulation Considerations for Sampled-Data ControllersThis article will continue to look at some aspects of the controllers and electronics needed to create emulated physical circuits with real-world connectivity and will look at the issues that arise in sampled-data controllers compared to continuous-domain controllers. As such, is not intended as an introduction to sampled-data systems.

- Part 1: Introduction

## Phase and Amplitude Calculation for a Pure Complex Tone in a DFT

IntroductionThis is an article to hopefully give a better understanding of the Discrete Fourier Transform (DFT) by deriving exact formulas to calculate the phase and amplitude of a pure complex tone from a DFT bin value and knowing the frequency. This is a much simpler problem to solve than the corresponding case for a pure real tone which I covered in an earlier blog article[1]. In the noiseless single tone case, these equations will be exact. In the presence of noise or other tones...

## Feedback Controllers - Making Hardware with Firmware. Part 6. Self-Calibration Related.

This article will consider the engineering of a self-calibration & self-test capability to enable the project hardware to be configured and its basic performance evaluated and verified, ready for the development of the low-latency controller DSP firmware and closed-loop applications. Performance specifications will be documented in due course, on the project website here.

- Part 6: Self-Calibration, Measurements and Signalling (this part)
- Part 5:

## Feedback Controllers - Making Hardware with Firmware. Part 8. Control Loop Test-bed

This part in the series will consider the signals, measurements, analyses and configurations for testing high-speed low-latency feedback loops and their controllers. Along with basic test signals, a versatile IFFT signal generation scheme will be discussed and implemented. A simple controller under test will be constructed to demonstrate the analysis principles in preparation for the design and evaluation of specific controllers and closed-loop applications.

Additional design...## A Two Bin Solution

IntroductionThis is an article to hopefully give a better understanding of the Discrete Fourier Transform (DFT) by showing an implementation of how the parameters of a real pure tone can be calculated from just two DFT bin values. The equations from previous articles are used in tandem to first calculate the frequency, and then calculate the amplitude and phase of the tone. The approach works best when the tone is between the two DFT bins in terms of frequency.

The Coding...## Filtering Noise: The Basics (Part 1)

IntroductionFinding signals in the presence of noise is one of the fundamental quests of the discipline of signal processing. Noise is inherently random by nature, so a probability oriented approach is needed to develop a mathematical framework for filtering (i.e. removing/suppressing) noise. This framework or discipline, formally referred to as stochastic signal processing, is often taught in graduate level engineering programs and is covered from different perspectives in excellent...

## Correlation without pre-whitening is often misleading

White LiesCorrelation, as one of the first tools DSP users add to their tool box, can automate locating a known signal within a second (usually larger) signal. The expected result of a correlation is a nice sharp peak at the location of the known signal and few, if any, extraneous peaks.

A little thought will show this to be incorrect: correlating a signal with itself is only guaranteed to give a sharp peak if the signal's samples are uncorrelated --- for example if the signal is composed...

## Least-squares magic bullets? The Moore-Penrose Pseudoinverse

Hello,

the topic of this brief article is a tool that can be applied to a variety of problems: The Moore-Penrose Pseudoinverse.While maybe not exactly a magic bullet, it gives us least-squares optimal solutions, and that is under many circumstances the best we can reasonably expect.

I'll demonstrate its use on a short example. More details can be found for example on Wikipedia, or the Matlab documentation...