Forums

reciprocals for quantization

Started by metiu uitem October 6, 2006
Hi,
I'm building a quantization table to use with the IMG_quantize function
on a TI DSP to quantize the DCT of an image.
The function basically does this:

short quantValue (short v)
{
    int q, round;

    round = 1 << (shift -  1)
    q = (v * recip + round) >> shift

    return (short) q;
}

I used the algorithm to find the reciprocal on Jones page:
http://www.cs.uiowa.edu/~jones/bcd/divide.html

The first problem I found is that using the "optimal" reciprocal leads
to a "negative" short, which would give -v/d instead of v/d. I arranged
to have a reciprocal with a leading 0 to avoid that.

My problem is: I need the rounded down division, not the rounded one.
To do so, I'm doing:
if (v > d/2) v -= d/2;
else if (v < -d/2) v += d/2;

but that's very unefficient.

I think I could find a reciprocal that automatically rounds down, but I
can't think of a (simple) way to do that...
Any help/pointer?

Thanks,
M

>>>>> "metiu" == metiu uitem <metiu.uitem@gmail.com> writes:
metiu> Hi, metiu> I'm building a quantization table to use with the IMG_quantize function metiu> on a TI DSP to quantize the DCT of an image. metiu> The function basically does this: metiu> short quantValue (short v) metiu> { metiu> int q, round; metiu> round = 1 << (shift - 1) metiu> q = (v * recip + round) >> shift metiu> return (short) q; metiu> } metiu> I used the algorithm to find the reciprocal on Jones page: metiu> http://www.cs.uiowa.edu/~jones/bcd/divide.html metiu> The first problem I found is that using the "optimal" reciprocal leads metiu> to a "negative" short, which would give -v/d instead of v/d. I arranged metiu> to have a reciprocal with a leading 0 to avoid that. What is the reciprocal you are trying to find? Ray