Unclipping

I have lately become interested in the processing of audio 
signals recorded at too high a level, and so have clipping.

First, I have a signal that clipped at five samples (but only 
in one channel).  The easy fix is to convert to a mono signal
with the unclipped channel, but I might try to interpolate 
new values for the clipped samples.

But I have another one that has about 17000 clipped points, 
out of about 23 minutes of 44.1kHz sampled stereo audio. 
Specifically, it is a symphony orchestra recording, and the 
clipping is at cymbal crashes, or maybe bass drums. 

The clipping is easily audible, even among the wide spectrum
of a cymbal crash.

But I had the idea that the bad sound from clipping comes
from the unnatural high frequency components, and that while
we like to use 20kHz as the upper limit, very few people can
hear that high, and there is normally not much that high in
a symphony orchestra.  So, an appropriate low-pass filter
should make the clipped points sound better, while not
messing up too much with the rest.

Not knowing so much about actual design of digital filters,
I found an introductory discussion that ends with a fifth
order elliptic filter, with passband to Fs/6, and stop band
starting at Fs/4, 50dB down in the stop band.

http://www.eas.uccs.edu/~mwickert/ece2610/lecture_notes/ece2610_chap8.pdf

Pretty easy to implement a C program to read in the original,
and write out the filtered result, given the IIR filter
coefficients.  And it sounds pretty good!

Then I found another reference, on Butterworth filters,
and this one ends with a second order digital Butterworth
filter at Fs/4. 

So, I implemented that one, and it sounds good, too.

I could implement the complementary high-pass filter, to
hear what I am losing, but haven't done that yet.

The RMS level is about 25dB below the peak, and I thought
it was fine, but didn't expect the peaks from the cymbals.

Is anyone else interested in processing of clipped audio?
I suspect that I am not the first to have this problem.

On 18.07.17 23.29, herrmannsfeldt@gmail.com wrote:
> I have lately become interested in the processing of audio
> signals recorded at too high a level, and so have clipping.
>
> First, I have a signal that clipped at five samples (but only
> in one channel).

5 samples? - no one will ever notice.
Most CD masters are worse.

> The easy fix is to convert to a mono signal
> with the unclipped channel, but I might try to interpolate
> new values for the clipped samples.

Discard high frequencies (e.g. >16kHz) around the clipped samples and 
you get a reasonable interpolation.

> But I have another one that has about 17000 clipped points,
> out of about 23 minutes of 44.1kHz sampled stereo audio.

Now it depends. If they are spread around the entire length no one will 
notice again. If they are stuck together then almost half of a second of 
the sound signal is missing, maybe in several chunks. But in this case 
the information is finally lost.


> Specifically, it is a symphony orchestra recording, and the
> clipping is at cymbal crashes, or maybe bass drums.

In the latter case a local lowpass but with significantly lower 
bandwidth (some 100 Hz) will again do the job.
The wide band signal of the cymbal is more complicated. You might take a 
sample of the same instrument at lower volume from another location and 
overlay it at the point of clipping.


> But I had the idea that the bad sound from clipping comes
> from the unnatural high frequency components, and that while
> we like to use 20kHz as the upper limit, very few people can
> hear that high, and there is normally not much that high in
> a symphony orchestra.

Well, the inaudible frequency components of the clipping will not 
degrade the sound, just because they are inaudible.
Of course, the spectrum is significantly modified by the clipping. But 
this applies to frequencies deep in the audible range at least down to a 
few kHz.

> So, an appropriate low-pass filter
> should make the clipped points sound better, while not
> messing up too much with the rest.

You need to damp several audible frequencies as well. Ideally until the 
local spectrum roughly matches the noise spectrum of the instrument.
But for wide band sounds it is not that easy. We also would detect the 
missing level at the peek in comparison to the neighbor sound. So in 
fact, you need to /amplify/ lower frequencies, that got damped by the 
clipping, while damping the very high ones.


> Not knowing so much about actual design of digital filters,
> I found an introductory discussion that ends with a fifth
> order elliptic filter, with passband to Fs/6, and stop band
> starting at Fs/4, 50dB down in the stop band.

I would use a custom FIR filter kernel, made by inversion of the desired 
frequency response and with zero phase. The complicated task is to apply 
the filter the stronger the more clipping is around.

> Pretty easy to implement a C program to read in the original,
> and write out the filtered result, given the IIR filter
> coefficients.  And it sounds pretty good!

Well, then it's okay.

> I could implement the complementary high-pass filter, to
> hear what I am losing, but haven't done that yet.

This makes no sense at all. The psycho acoustics are highly non linear. 
One of the most important effects is masking, i.e. the existence of 
frequency X reduces the recognition of frequency Y. So if you remove X Y 
would sound entirely different. While when removing Y you might not even 
notice in some cases. (This is the basic idea behind almost any lossy 
audio compression.)

> The RMS level is about 25dB below the peak, and I thought
> it was fine, but didn't expect the peaks from the cymbals.

Well, percussion has very poor crest factors.


> Is anyone else interested in processing of clipped audio?
> I suspect that I am not the first to have this problem.

Pretty sure. ;-)
But most professionals would avoid this by recording at 24 bit with 
let's say 20 dB headroom (to the usual peak level rather than RMS).


Marcel

On Tuesday, July 18, 2017 at 4:11:25 PM UTC-7, Marcel Mueller wrote:

(I wrote)
> > I have lately become interested in the processing of audio
> > signals recorded at too high a level, and so have clipping.

> > First, I have a signal that clipped at five samples (but only
> > in one channel).

> 5 samples? - no one will ever notice.
> Most CD masters are worse.

It isn't all that hard to hear. 

I suspect a bunch of high, but not too high, frequencies that
just sound bad.

That might be a bass drum, which shouldn't have frequencies
anywhere near.

(snip)

> Discard high frequencies (e.g. >16kHz) around the clipped samples and 
> you get a reasonable interpolation.

My thought is to interpolate a parabola to some nearby points.
That is, round off what is now sharp corners.

> > But I have another one that has about 17000 clipped points,
> > out of about 23 minutes of 44.1kHz sampled stereo audio.

> Now it depends. If they are spread around the entire length no one will 
> notice again. If they are stuck together then almost half of a second of 
> the sound signal is missing, maybe in several chunks. But in this case 
> the information is finally lost.

They might be in 1/10th second cymbal crashes.

There is much noise around (that is, anharmonics in the cymbal)
but not enough to cover the really strange sound. 

> > Specifically, it is a symphony orchestra recording, and the
> > clipping is at cymbal crashes, or maybe bass drums.

> In the latter case a local lowpass but with significantly lower 
> bandwidth (some 100 Hz) will again do the job.
> The wide band signal of the cymbal is more complicated. You might take a 
> sample of the same instrument at lower volume from another location and 
> overlay it at the point of clipping.

Local filters are what I haven't figured out yet. 

How to smooth into, and out of, the filtered part.

> > But I had the idea that the bad sound from clipping comes
> > from the unnatural high frequency components, and that while
> > we like to use 20kHz as the upper limit, very few people can
> > hear that high, and there is normally not much that high in
> > a symphony orchestra.

(snip)

> > So, an appropriate low-pass filter
> > should make the clipped points sound better, while not
> > messing up too much with the rest.

> You need to damp several audible frequencies as well. Ideally until the 
> local spectrum roughly matches the noise spectrum of the instrument.
> But for wide band sounds it is not that easy. We also would detect the 
> missing level at the peek in comparison to the neighbor sound. So in 
> fact, you need to /amplify/ lower frequencies, that got damped by the 
> clipping, while damping the very high ones.

> > Not knowing so much about actual design of digital filters,
> > I found an introductory discussion that ends with a fifth
> > order elliptic filter, with passband to Fs/6, and stop band
> > starting at Fs/4, 50dB down in the stop band.

> I would use a custom FIR filter kernel, made by inversion of the desired 
> frequency response and with zero phase. The complicated task is to apply 
> the filter the stronger the more clipping is around.

> > Pretty easy to implement a C program to read in the original,
> > and write out the filtered result, given the IIR filter
> > coefficients.  And it sounds pretty good!

> Well, then it's okay.

> > I could implement the complementary high-pass filter, to
> > hear what I am losing, but haven't done that yet.

> This makes no sense at all. The psycho acoustics are highly non linear. 
> One of the most important effects is masking, i.e. the existence of 
> frequency X reduces the recognition of frequency Y. So if you remove X Y 
> would sound entirely different. While when removing Y you might not even 
> notice in some cases. (This is the basic idea behind almost any lossy 
> audio compression.)

That was just going to be for fun, just to see what is there.

> > The RMS level is about 25dB below the peak, and I thought
> > it was fine, but didn't expect the peaks from the cymbals.

> Well, percussion has very poor crest factors.

> > Is anyone else interested in processing of clipped audio?
> > I suspect that I am not the first to have this problem.

> Pretty sure. ;-)
> But most professionals would avoid this by recording at 24 bit with 
> let's say 20 dB headroom (to the usual peak level rather than RMS).

I have a Tascam DR-1, and record at 24 bits, 44.1kHz.

I try not to get the level too low, as I suspect that there
is noise somewhere in the system.

In the five sample case, that was out of a whole concert, but only
one movement had that peak.  And only one channel.

> 
> Is anyone else interested in processing of clipped audio?
> I suspect that I am not the first to have this problem.

Yes I am interested.
Many people attack this problem from the time domain, I prefer to look at the frequency domain.

For the example you gave, you have 2 distinct situations.

Clipped bass creates harmonics and the remediation is low pass filter. 
But you want the filtering to be dynamic so it doesn't low pass filter everything all the time, just when needed.

Clipped cymbals creates inter-modulation which typically falls into the low and mid range.  In this case you might try high pass filtering, again dynamic.

The fundamental (pun) concept is to pass those frequencies of the desired original and block the harmonics and or intermod created by the clipping

That is easy to say, the hard part is that you want the filtering to be dynamically adaptive.  

The other poster mentioned substitution by editing.  Not really a signal processing method but can be effective and time consuming.   

m

On Tuesday, July 18, 2017 at 7:34:17 PM UTC-5, herrman...@gmail.com wrote:

> My thought is to interpolate a parabola to some nearby points.
> That is, round off what is now sharp corners.

I have done this. It works surprisingly well for clips up to about 10 mSec duration.

If you have a good model for the nonlinearity that produced the
clipping, and a lot of compute time (e.g. non-real-time is okay)
you can try to recreate the original signal (as an oversampled 1-bit 
signal) using the Forney algorithm (MLSE).  

The algorithm for this looks exactly like a Forney equalizer --
in the equalizer algorithm, instead of using the channel impulse
response, use the reconstruction filter (for the 1-bit DAC)
followed by the nonlinearity.

So .. I say do it in the time domain.  :-)


Steve

On Wednesday, July 19, 2017 at 7:10:33 AM UTC-7, Greg Berchin wrote:
> On Tuesday, July 18, 2017 at 7:34:17 PM UTC-5, herrman...@gmail.com wrote:

> > My thought is to interpolate a parabola to some nearby points.
> > That is, round off what is now sharp corners.

> I have done this. It works surprisingly well for clips
> up to about 10 mSec duration.

I hadn't thought about that long, but for a deep bass signal,
such as a big bass drum hit really hard, it would seem that it
could be really close, even over that duration.

Am 18.07.17 um 23:29 schrieb herrmannsfeldt@gmail.com:
> I have lately become interested in the processing of audio
> signals recorded at too high a level, and so have clipping.
> 
> First, I have a signal that clipped at five samples (but only
> in one channel).  The easy fix is to convert to a mono signal
> with the unclipped channel, but I might try to interpolate
> new values for the clipped samples.
> 
> But I have another one that has about 17000 clipped points,
> out of about 23 minutes of 44.1kHz sampled stereo audio.
> Specifically, it is a symphony orchestra recording, and the
> clipping is at cymbal crashes, or maybe bass drums.
> 
> The clipping is easily audible, even among the wide spectrum
> of a cymbal crash.

There is a declipping filter as part of postfish. The source code is here:

	https://svn.xiph.org/trunk/postfish/

Look in declip.c and declip.h. Postfish is/was a project to create a 
high-quality open source filter bank / mixer, which unfortunately, was 
not further developed. It only compiles on quite old Linux boxes. I have 
it running in an Ubuntu 8.04 VM.

Thomas Strohmer also had an impressive demonstration based upon the 
irregular sampling theory; unfortunately, no code is available. This is 
the page:
	https://www.math.ucdavis.edu/~strohmer/research/audio/audio.html

	Christian