It's been some time since I've had anything to do with DSP. With that said, I have a project requiring this skill set here and need help. I kept returning to this website through my search, so I created an account and hopped into the forums. I hope to find someone who can help me understand if I'm on the right path and, if not, possibly point me in the right direction. 

Alright, let's get into the meat of the problem. I am working with an accelerometer that outputs a PDM (pulse density modulation) signal. I must filter and decimate the signal to translate the single-bit stream into a usable acceleration reading. This translation will be done in real-time using an MCU to read the incoming data and perform the necessary math. I was recommended to look into Hogenauer filters, which led me to work on using CIC filters. I have had some success with a rough CIC filter implemented in Python, but this was using discrete data. I am now translating to C and running into issues, mainly due to integrators overflowing (which I understand is by design). I am using a 4th order CIC filter: xn->[i0]->[i1]-[i3]->[i4]->[decimation decision]->[c0]->[c1]-[c3]->[c4]->1/gain->yn

A few additional details:

- Data must be continuously read and filtered for an undetermined amount of time

- Sensor ODR: 781250 Hz (I'd like to be able to decimate by 78 so that I can have ~10K system ODR)

- The PDM was designed to push the noise above 300 Hz. (I may need to add an FIR lowpass to compensate for the CIC non-flat passband.

Now for the questions:

1) Does a CIC filter seem reasonable to implement in C and run continuously on a microprocessor? If not, what type of filter would work well for this?

2) How can I implement a CIC filter in C without worrying about overflow issues? 

Thanks in advance for any help!

UPDATE:

My issue has been resolved. It took me stepping back to compare individual components of the CIC I implemented in python to the one I coded in C, as well as setting my CIC order to 1. My issues for seem to be rooted in not having large enough bit width for the integrators/combs and not using signed values. Once I made a couple of tweaks, I did not see any overflow issues and the results matched what the python code. If anyone has any questions, please feel free to reach out.

Have you confined yourself too much? Try a different accelerometer, at least at first, get that working and then go back. Don't try to do realtime, since that can introduce new anomalies like having to throw-away samples. Some algorithms work well in real-time, others don't. Pipeline the processing which helps to break processing into more manageable (i.e. testable) chunks. Pipelining may simplify making the algorithm multi-processor friendly and/or team development friendly. Create a "typical" data set, work on that until the algorithm works, then go back with newer or live data. Ensure that any dataset contains anticipated anomalies and discontinuies, and what to do when they happen. Sometimes we forget how much more efficient incremental development can be, especially when doing something new. And it may not be entirely new, just, for instance, the output of the accelerometer.

Yes, I am constrained, but it's because I cannot choose the sensor to use. Switching to another accelerometer does not help since it's an issue with interfacing with the specific type of output (PDM), not the accel data itself. Regarding your comment about incremental development, I certainly took this approach. I started with a data set that I collected using a logic analyzer, started playing around with elementary filters (eg, running window), and eventually found my way to CIC filters for decimation. I implemented a crude filter in Python, and the data output looked great. Again, all of this work was not in real-time. When I started getting to real-time calculations, I saw issues with my implementation, so I took a step back, trying to understand the root cause. I could not find the answers, so I came here for assistance. I also do not have anyone in my network with whom I can discuss these matters, so a forum was the perfect place for me to go and ask questions. Anyway, I am absolutely with you on the incremental steps. I am just not sure what I can do at this time to improve my understanding. I'm all ears if you have a clear idea of what process I should take to explore CIC filters or another type of filter you think is reasonable for what I'm trying to accomplish.

If the Python filter looked great, I would continue to refine it to more closely match your real-time implementation, until you see where the issues are encountered.  If you can get a set of the data you're running into your real-time implementation, to run into the Python simulation, it will make that approach even more viable.  

In doing similar exercises, I've developed and used bit-accurate simulations of DSP filters, that make any debugging of processor/real-time issues relatively easy.

Regards,

Robert

Exactly. I was able to break this down a bit more, comparing each individual peice of the CIC in python to that in C to figure out where the problem was coming from. I am now in a position where my C code is giving me the same results as python.

Hello,

As you said PDM is a high frequency sequence (oversampled) and to decode the data you theoretically need a lowpass filter and then downsampling. CIC is an efficient implementation of this process. Inside the CIC there is a section of integrators and my understanding is that you get overflow for this section in your C implementation. A couple of questions and comments: 

1- Do you see the same problem in your python code?

2- Are you using floating point or fixed point? why not increasing your data size?

3- For testing purposes you may scale down your input data and see whether you still get overflow or not. 

4- If you have access to Matlab it will greatly simplify your job:

4.1- Matlab (simulink) has a CIC block, you can test your PDM data with that block and you can use its output as a golden reference (and of course observe any anomaly in your C code)

4.2- You can implement your CIC in the Simulink (or Matlab) with discrete blocks using simple delay elements and check to see whether you still get overflow or not in comparison with your C code.

4.3- If you have Matlab embedded coder you can even generate the C code directly from the CIC block or your discrete implementation and use it in your C project easily. 

4.4 In any case you can compare point by point your Matlab implementation and your C code and see where the problem arises, since there is no multilplication.

With careful design of the data size, scaling and coding you should not observe the overflow. 

 

Post deleted by author

This post has raised a general question regarding single bit versus multibit processing.

I am not familiar with ADC single bit pulse modulation. I know they are based on some sort of pulse changes in tandem with analogue signal, pwm, pdm, pam, pcm ...etc.

We know that processing like filters, fft, etc are done on multi-bit fixed point or floating point. 

So do we need first convert stream to multibit then go from there.

or can we consider single bit stream as ok for processing and so combine filtering + conversion to multi bit in one go.

Additionally: the odd case is pcm as it can be viewed as either pulses or fixed point (just parallelising bits).

Yes exactly, that was a question in my mind for a while. There did not seem to be a lot of resources online around this, but I did find a few posts where people discussed needing to input the individual bits into the filter rather than jumping to convert individual bits into a multi bit value before filtering. In this case the CIC itself does the low pass (at least low pass like) filtering and decimating together. I was able to get this working in both python and C. I am now at the point where I need to prove 1) how efficient are CICs and 2) the accuracy of being able to reproduce the analog signal. At a high level things look good though.

These were great questions and points. I very much appreciate you taking the time to address my post. I was able to take a step back and analyze the python code against the C code, one component of the CIC filter at a time. In doing this, I was able to figure out that I was having issues with overflow due to 1) the size of my integrators/combs and 2) not using signed values. This was fairly easy to see once I set my CIC order to 1 and compared everything.

Unfortunately I do not have access to Matlab at this time. I have a seen quite a number of posts and videos showing how easy it is to develop filters there, so I can see how this will be a valuable tool moving forward.

Thanks for the detailed response!

Reaching back a bit, but if I remember correctly, one of the beauties of CIC filters is that any overflow created in the accumulation (first) stage is exactly undone in the differentiation (second) stage. In hearing aids we would use a single-bit PDM signal and create a 18b or 24b signal, no overflow (however, this included additional decimation filtering).

One of the problems with CIC is that it is not a very sharp filter (i.e. doesn't suppress aliasing very much, if that's all you do). We would do the initial decimation using CIC, then apply IIR filters to give a sharper / stronger aliasing rejection.

In direct answer to your questions:

1) Yes, CIC is quite reasonable in C

2) Overflow - see above.  I _think_ the worst case you have to worry about is gain of 2 for each accumulation stage, and gain of 2 for each differentiation stage.  If you use a 32b integer, it seems like it would be fine.

This is great, so glad to hear that I might be on a decent path. I implemented my filter in C, using 32 bits for each integrator/comb, and still not seeing the results I expect. Would it be ok if I sent you a more thorough breakdown of my filter with a couple questions?

More detail will certainly be helpful.  I would say start with your equations or the source code.

Hi Jon-

I searched on "convert PDM to PCM" and found numerous info and examples, including source code, on various sites including ti.com, dsp.stackexchange.com, reddit, comp.dsp, and here (this long thread https://www.dsprelated.com/thread/6809/mcu-signal-...) Also this informative blog https://abhipray.com/posts/sigproc/pdm_pcm_convers... which points out that brain neurons use PDM 😃😮

My suggestion would be to zero in on accelerometer examples and select a readable, straightforward method of getting to a PCM signal. From there you are back in the "mainstream" and for subsequent signal processing on your MCU you can get abundant help online or from the MCU vendor.

-Jeff

Hey Jeff,

I've also found numerous articles around PDM to PCM conversion. This is probably due to PDM being a common signal with MEMs microphones... Anyways, I was unable to locate anything regarding accelerometers. I can certainly take another look though. I was told by the manufacturer that I should use a low pass filter on the single bit stream before decimating. For some reason I'm under the impression that converting to PCM would bypass filtering first. Am I wrong to think that? I'll look more into this topic. Maybe it's more straight forward than I had thought.

Hi Jon-

If this problem was thrown at me I'd start with this page:

https://www.reddit.com/r/DSP/comments/zzelg5/pdm_t...

which says implement a sharp lowpass filter (5 kHz per your post) and decimate (by 78 per your post; i.e. keep every 78th sample). Then see how that looks in the time domain and FFT it to look at noise level and THD (use MATLAB, Hypersignal, etc). Search brings up references to CIC filters - which should be helpful when you want to optimize (as you don't need to calculate outputs you will discard) -- but you're not to the point of optimizing yet. First just get things working, understand how it works, and make sure your signal quality is acceptable. Then optimize.

-Jeff