# reciprocals for quantization

Started by October 6, 2006
```Hi,
I'm building a quantization table to use with the IMG_quantize function
on a TI DSP to quantize the DCT of an image.
The function basically does this:

short quantValue (short v)
{
int q, round;

round = 1 << (shift -  1)
q = (v * recip + round) >> shift

return (short) q;
}

I used the algorithm to find the reciprocal on Jones page:
http://www.cs.uiowa.edu/~jones/bcd/divide.html

The first problem I found is that using the "optimal" reciprocal leads
to a "negative" short, which would give -v/d instead of v/d. I arranged
to have a reciprocal with a leading 0 to avoid that.

My problem is: I need the rounded down division, not the rounded one.
To do so, I'm doing:
if (v > d/2) v -= d/2;
else if (v < -d/2) v += d/2;

but that's very unefficient.

I think I could find a reciprocal that automatically rounds down, but I
can't think of a (simple) way to do that...
Any help/pointer?

Thanks,
M

```
```>>>>> "metiu" == metiu uitem <metiu.uitem@gmail.com> writes:

metiu> Hi,
metiu> I'm building a quantization table to use with the IMG_quantize function
metiu> on a TI DSP to quantize the DCT of an image.
metiu> The function basically does this:

metiu> short quantValue (short v)
metiu> {
metiu>     int q, round;

metiu>     round = 1 << (shift -  1)
metiu>     q = (v * recip + round) >> shift

metiu>     return (short) q;
metiu> }

metiu> I used the algorithm to find the reciprocal on Jones page:
metiu> http://www.cs.uiowa.edu/~jones/bcd/divide.html

metiu> The first problem I found is that using the "optimal" reciprocal leads
metiu> to a "negative" short, which would give -v/d instead of v/d. I arranged
metiu> to have a reciprocal with a leading 0 to avoid that.

What is the reciprocal you are trying to find?

Ray
```