AUDIO CLIPPING ALGORITHM

Does anyone knows a good algorithm that takes as input a WAV file and
tells me about problems of saturation? I have to implement it in matlab
and then C++.

Thank you.
---------------------------------------
Posted through http://www.DSPRelated.com

Am 29.02.16 um 16:15 schrieb rickowens:
> Does anyone knows a good algorithm that takes as input a WAV file and
> tells me about problems of saturation? I have to implement it in matlab
> and then C++.

Analog or digital clipping? Digital clippin should be easy to detect, 
because the signal then does not exceed a certain threshold. Analog 
clipping will be a more challenging task.

If it is of any help, the postfish (a discontinued audio filter bank) 
contains a reconstruction filter which can declip audio. Works quite 
well, and I think this filter is relatively unique:

https://svn.xiph.org/trunk/postfish/

There used to be a research project on declipping using the non-uniform 
sampling theorem.

https://www.math.ucdavis.edu/~strohmer/research/audio/audio.html

I once tried to contact this guy, but got no answer.

	Christian

On 29.02.16 16.15, rickowens wrote:
> Does anyone knows a good algorithm that takes as input a WAV file and
> tells me about problems of saturation? I have to implement it in matlab
> and then C++.

Calculate the sample value distribution. It will show many bad things 
that have been done with the audio data.

If the distribution ends rather abruptly at some value, this is an 
indication of clipping or improper use of compression. Normally the 
distribution goes asymptotically to zero.

Furthermore if the distribution is not smooth but more like a comb, it 
is likely that gain changes without dithering have been applied to the data.


Marcel

On 04.03.16 15.46, rickowens wrote:
> if I understood correctly.. you mean something like that?
>
> m=max(f);
> k=min(f);
> for i=1:N
>      v(i)=0;
>      z(i)=0;
>      if (f(i)==m)
>          v(i)=1;
>      end;
>      if (f(i)==k)
>          z(i)=1;
>      end;
> end;

No, the sample value distribution is no scalar but a vector.

f := array of N elements
for i = 0 to N
   h(f(i)) = h(f(i)) + 1

Now h(x) is the sample value distribution.

Have a look at them. You will see whether the samples are probably clipped.
Because the values around x = 0 are usually large you may plot log(h(x)) 
instead.

To get a scalar, calculate the crest factor of your samples, i.e.
   max(abs(f(i))) / RMS(f(i))
where RMS is the root mean square, i.e. square root of sum over i of f(i)^2.
The lower the crest factor the higher is the chance of highly compressed 
audio data.

However, the crest factor is very sensitive to single bad samples 
because of the max. So be careful.

Another measure might use the median of absolute the sample value as 
reference instead of max. This is very stable. But you need to


> in this way I have a kind of clipping index, right?
>
> My goal is to quantify "how much clipped" is my tune.. I don't know if
> this is enough!

If you only count the samples at min and max, you might get wrong 
results, because if further processing has been done after the clipping 
then your algorithm will not trigger, since the clipping is no longer 
restricted to one sample value. So the clipping border becomes soft.
Soft clipping is common practice to reduce high frequency harmonics that 
can destroy speakers.


Marcel

>Am 29.02.16 um 16:15 schrieb rickowens:
>> Does anyone knows a good algorithm that takes as input a WAV file and
>> tells me about problems of saturation? I have to implement it in
matlab
>> and then C++.
>
>Analog or digital clipping? Digital clippin should be easy to detect, 
>because the signal then does not exceed a certain threshold. Analog 
>clipping will be a more challenging task.
>
>If it is of any help, the postfish (a discontinued audio filter bank) 
>contains a reconstruction filter which can declip audio. Works quite 
>well, and I think this filter is relatively unique:
>
>https://svn.xiph.org/trunk/postfish/
>
>There used to be a research project on declipping using the non-uniform 
>sampling theorem.
>
>https://www.math.ucdavis.edu/~strohmer/research/audio/audio.html
>
>I once tried to contact this guy, but got no answer.
>
>	Christian

I have to estimate both! The last one is good, however, I'm more focused
on estimating the "amount" of clipping
---------------------------------------
Posted through http://www.DSPRelated.com

>On 04.03.16 15.46, rickowens wrote:
>> if I understood correctly.. you mean something like that?
>>
>> m=max(f);
>> k=min(f);
>> for i=1:N
>>      v(i)=0;
>>      z(i)=0;
>>      if (f(i)==m)
>>          v(i)=1;
>>      end;
>>      if (f(i)==k)
>>          z(i)=1;
>>      end;
>> end;
>
>No, the sample value distribution is no scalar but a vector.
>
>f := array of N elements
>for i = 0 to N
>   h(f(i)) = h(f(i)) + 1
>
>Now h(x) is the sample value distribution.
>
>Have a look at them. You will see whether the samples are probably
clipped.
>Because the values around x = 0 are usually large you may plot log(h(x))

>instead.
>
>To get a scalar, calculate the crest factor of your samples, i.e.
>   max(abs(f(i))) / RMS(f(i))
>where RMS is the root mean square, i.e. square root of sum over i of
>f(i)^2.
>The lower the crest factor the higher is the chance of highly compressed

>audio data.
>
>However, the crest factor is very sensitive to single bad samples 
>because of the max. So be careful.
>
>Another measure might use the median of absolute the sample value as 
>reference instead of max. This is very stable. But you need to
>
>
>> in this way I have a kind of clipping index, right?
>>
>> My goal is to quantify "how much clipped" is my tune.. I don't know if
>> this is enough!
>
>If you only count the samples at min and max, you might get wrong 
>results, because if further processing has been done after the clipping 
>then your algorithm will not trigger, since the clipping is no longer 
>restricted to one sample value. So the clipping border becomes soft.
>Soft clipping is common practice to reduce high frequency harmonics that

>can destroy speakers.
>
>
>Marcel

Sorry but I can't understand what h(x) does exactly :(
---------------------------------------
Posted through http://www.DSPRelated.com

On Monday, February 29, 2016 at 7:15:24 AM UTC-8, rickowens wrote:
> Does anyone knows a good algorithm that takes as input a WAV file and
> tells me about problems of saturation? 

Some time ago, I wrote a C program that takes a WAV file and calculates
the minimum, maximum, and RMS values.  If the minimum or maximum hit the 
largest or smallest (it works for at least 16 and 24 bit WAV), it counts how many
such clip points there are.

Assuming 16 bit WAV, find (or count) the points at 32767 and -32768.

I have some WAV files with just a few such points, but it is enough to hear.

On Wed, 13 Apr 2016 13:42:43 -0700 (PDT), herrmannsfeldt@gmail.com
wrote:

>On Monday, February 29, 2016 at 7:15:24 AM UTC-8, rickowens wrote:
>> Does anyone knows a good algorithm that takes as input a WAV file and
>> tells me about problems of saturation? 
>
>Some time ago, I wrote a C program that takes a WAV file and calculates
>the minimum, maximum, and RMS values.  If the minimum or maximum hit the 
>largest or smallest (it works for at least 16 and 24 bit WAV), it counts how many
>such clip points there are.
>
>Assuming 16 bit WAV, find (or count) the points at 32767 and -32768.
>
>I have some WAV files with just a few such points, but it is enough to hear.

This will work if the clipping is digital, such as exceeding the range
of an A/D converter.  But what about analog clipping?  That can be
somewhat "soft" clipping, so that all the affected points may not be
represented by the extreme digital values.  Perhaps something could be
done with histograms, counting all the values within a certain
percentage of the extreme values as "saturated".

On Monday, April 18, 2016 at 12:32:27 PM UTC-7, Robert Scott wrote:

(snip, I wrote)
> >Some time ago, I wrote a C program that takes a WAV file and calculates
> >the minimum, maximum, and RMS values.  If the minimum or maximum hit the 
> >largest or smallest (it works for at least 16 and 24 bit WAV), it counts how many
> >such clip points there are.

> >Assuming 16 bit WAV, find (or count) the points at 32767 and -32768.

> >I have some WAV files with just a few such points, but it is enough to hear.
 
> This will work if the clipping is digital, such as exceeding the range
> of an A/D converter.  But what about analog clipping?  That can be
> somewhat "soft" clipping, so that all the affected points may not be
> represented by the extreme digital values.  Perhaps something could be
> done with histograms, counting all the values within a certain
> percentage of the extreme values as "saturated".

If you know the actual, or likely, distribution of peaks, I suppose you
could do that.  

I believe that there are some digital recording systems with electronic
soft clipping.  There is also automatic level, which seems to do bad things
to the sound overall.   In the case of electronic soft clipping, you might be
able to find the actual transform and undo it. 

I try to turn off anything that isn't needed between the source and the ADC.
No filters, no clip reduction.  I have recordings with three clipped samples.
Interesting, for that one, all are in one channel, so by using the other channel,
converting to a monophonic signal, I can remove the clip.

In the case of soft clipping, it could be interesting to look at the two
stereo channels and compare them. That might supply enough information
to extract the soft clip transformation and invert it.  

Reminds me, I was looking not so long about  the blind signal
separation problem:

https://en.wikipedia.org/wiki/Blind_signal_separation

Given two stereo channels, doing appropriate correlations, such
as through principle component analysis, you might accurately
reconstruct clipped values, even with soft clipping.

But I was mostly interested in making a single clipped peak sound better.
It doesn't have to be all that close.

On Mon, 18 Apr 2016 15:32:34 -0700 (PDT), herrmannsfeldt@gmail.com
wrote:

>> This will work if the clipping is digital, such as exceeding the range
>> of an A/D converter.  But what about analog clipping?  That can be
>> somewhat "soft" clipping, so that all the affected points may not be
>> represented by the extreme digital values.  Perhaps something could be
>> done with histograms, counting all the values within a certain
>> percentage of the extreme values as "saturated".
>
>If you know the actual, or likely, distribution of peaks, I suppose you
>could do that.  
>
>I believe that there are some digital recording systems with electronic
>soft clipping.  There is also automatic level, which seems to do bad things
>to the sound overall.   In the case of electronic soft clipping, you might be
>able to find the actual transform and undo it. 
>
>I try to turn off anything that isn't needed between the source and the ADC.
>No filters, no clip reduction.  I have recordings with three clipped samples.
>Interesting, for that one, all are in one channel, so by using the other channel,
>converting to a monophonic signal, I can remove the clip.
>
>In the case of soft clipping, it could be interesting to look at the two
>stereo channels and compare them. That might supply enough information
>to extract the soft clip transformation and invert it.  

Theoretically you could invert the transform, provided that transform
did not discard information.  But practically, any transform that
severely compresses the range (such as soft limiting) does discard
some resolution, so that when the range is re-expanded, the result
will have more quantization error than it did before compression.