Hi, I'm building a quantization table to use with the IMG_quantize function on a TI DSP to quantize the DCT of an image. The function basically does this: short quantValue (short v) { int q, round; round = 1 << (shift - 1) q = (v * recip + round) >> shift return (short) q; } I used the algorithm to find the reciprocal on Jones page: http://www.cs.uiowa.edu/~jones/bcd/divide.html The first problem I found is that using the "optimal" reciprocal leads to a "negative" short, which would give -v/d instead of v/d. I arranged to have a reciprocal with a leading 0 to avoid that. My problem is: I need the rounded down division, not the rounded one. To do so, I'm doing: if (v > d/2) v -= d/2; else if (v < -d/2) v += d/2; but that's very unefficient. I think I could find a reciprocal that automatically rounds down, but I can't think of a (simple) way to do that... Any help/pointer? Thanks, M
reciprocals for quantization
Started by ●October 6, 2006
Reply by ●October 6, 20062006-10-06
>>>>> "metiu" == metiu uitem <metiu.uitem@gmail.com> writes:metiu> Hi, metiu> I'm building a quantization table to use with the IMG_quantize function metiu> on a TI DSP to quantize the DCT of an image. metiu> The function basically does this: metiu> short quantValue (short v) metiu> { metiu> int q, round; metiu> round = 1 << (shift - 1) metiu> q = (v * recip + round) >> shift metiu> return (short) q; metiu> } metiu> I used the algorithm to find the reciprocal on Jones page: metiu> http://www.cs.uiowa.edu/~jones/bcd/divide.html metiu> The first problem I found is that using the "optimal" reciprocal leads metiu> to a "negative" short, which would give -v/d instead of v/d. I arranged metiu> to have a reciprocal with a leading 0 to avoid that. What is the reciprocal you are trying to find? Ray