Forums

basic questions on jpeg compression

Started by manishp April 12, 2012
Folks,

I have few basic questions on related to compression used in JPEG.

 1. Is the DCT applied over Y, CB and CR separately?
 2. Is there a specific reason why DCT is chosen over DFT?
 3. Can quantization step be roughly seen as a filter since it basically
tones down high frequency components?
    As an observation on point 3 above, we know that filter aims to achieve
high attenuation in stop band while quantization does not totally
attenauate HF components but only reduces their overall contribution
 4. Another related question is, in an earlier question I had asked if
 filters really need conversion from time to frequency domain before they
can operate. The response was that filters are designed to operate directly
on time domain inputs and no conversion is normally needed.
But in the above case, clearly conversion is done to frequency domain
(though DCT) before further steps can be applied.

Comments would be highly appreciated?

manishp <manishp.p18@n_o_s_p_a_m.gmail.com> wrote:

> I have few basic questions on related to compression used in JPEG.
> 2. Is there a specific reason why DCT is chosen over DFT?
It has better boundary conditions, and so doesn't cause as much visible artifacts. The DFT periodic boundary conditions will tend to cause the block boundaries to be equal, and be visible. -- glen
On Thu, 12 Apr 2012 04:43:21 -0500, manishp wrote:

> Folks, > > I have few basic questions on related to compression used in JPEG. > > 1. Is the DCT applied over Y, CB and CR separately? > 2. Is there a > specific reason why DCT is chosen over DFT? > 3. Can quantization step be > roughly seen as a filter since it basically > tones down high frequency components?
If you broaden your definition of "filter" enough, anything is a filter. But quantization isn't a filter in most people's dictionary. The idea of the filter-quantize is that you identify the features that 'matter' more (low spatial frequency signal) and quantize that with more bits, and you identify the features that 'matter' less (high spatial frequency) and quantize more. The idea is that you're _going_ to lose information, so you try to pick and choose what information you care least about to discard.
> As an observation on point 3 above, we know that filter aims to > achieve > high attenuation in stop band while quantization does not totally > attenauate HF components but only reduces their overall contribution
I think you misunderstand what quantization does. Quantization distorts. When you try to interpret its effects in the frequency domain you might talk about it as you are -- but here in the time domain, where we live, quantization distorts.
> 4. Another related question is, in an earlier question I had asked if > filters really need conversion from time to frequency domain before > they > can operate. The response was that filters are designed to operate > directly on time domain inputs and no conversion is normally needed. But > in the above case, clearly conversion is done to frequency domain > (though DCT) before further steps can be applied.
All operations are performed in the time domain. We live in the time domain, to us the frequency domain is a mathematical abstraction. Everything we build operates in the time domain. When something that we have built performs a DCT, it does operations in the time domain on data that was measured in the time (and perhaps time-spatial) domain. The output of the DCT is an approximation of variables that may exist in _a_ frequency domain, but the numbers that are spat out of the hardware only have meaning in the time domain. Do this simple experiment: Go harvest a picture of something of aesthetic value off of the Internet (I'm trying to get my mind out of the gutter, so I won't suggest porn -- directly). Take it's 2D Fourier transform. Display the Fourier transform of the picture next to the original. Is the Fourier transform still interesting? Is it good looking? Does it make you drool (oh wait -- I said I wouldn't suggest that...)? So when you look at a picture in what you are calling the "frequency domain" (which is really a spatial domain rendering of an approximation of one particular frequency domain) -- it does nothing for you. But the original picture has meaning that your nervous system can perceive. Thus, time domain vs. frequency domain. -- My liberal friends think I'm a conservative kook. My conservative friends think I'm a liberal kook. Why am I not happy that they have found common ground? Tim Wescott, Communications, Control, Circuits & Software http://www.wescottdesign.com
Tim Wescott <tim@seemywebsite.com> writes:
> [...] > All operations are performed in the time domain. We live in the time > domain, to us the frequency domain is a mathematical abstraction. > Everything we build operates in the time domain.
At some basic level that is true enough. But you know there are "frequency domain filters" and there is a damned good reason that's what they're called. -- Randy Yates DSP/Firmware Engineer 919-577-9882 (H) 919-720-2916 (C)
On 12/04/2012 18:24, Randy Yates wrote:
> Tim Wescott<tim@seemywebsite.com> writes: >> [...] >> All operations are performed in the time domain. We live in the time >> domain, to us the frequency domain is a mathematical abstraction. >> Everything we build operates in the time domain. > > At some basic level that is true enough. But you know there are > "frequency domain filters" and there is a damned good reason that's what > they're called.
I am sure there are quite a few physicists who will tell you the time domain is a mathematical abstraction too. And the frequency domain is by no means entirely abstract, it is in the vernacular too. People will generally refer to a pitch going up or down; rarely indeed will they describe it as getting shorter or longer. Richard Dobson
Tim Wescott <tim@seemywebsite.com> wrote:
 
(snip, someone wrote)

>> 3. Can quantization step be roughly seen as a filter since >> it basically tones down high frequency components?
> If you broaden your definition of "filter" enough, anything is a filter. > But quantization isn't a filter in most people's dictionary.
Sampling is often described as multiplying by a train of delta functions, such that the result is still described in continuous time, though it is zero except at specific discrete times. If you do that, it seems that one could also describe quantization as a filter, with the result still described in continuous time. It seems to me that you could also quantize a continuous time signal, though the result wouldn't be contiguous (in the mathematical sense).
> The idea of > the filter-quantize is that you identify the features that 'matter' more > (low spatial frequency signal) and quantize that with more bits, and you > identify the features that 'matter' less (high spatial frequency) and > quantize more. The idea is that you're _going_ to lose information, so > you try to pick and choose what information you care least about to > discard.
(snip)
> I think you misunderstand what quantization does. Quantization > distorts. When you try to interpret its effects in the frequency domain > you might talk about it as you are -- but here in the time domain, where > we live, quantization distorts.
>> 4. Another related question is, in an earlier question I had asked if >> filters really need conversion from time to frequency domain before >> they >> can operate. The response was that filters are designed to operate >> directly on time domain inputs and no conversion is normally needed. But >> in the above case, clearly conversion is done to frequency domain >> (though DCT) before further steps can be applied.
> All operations are performed in the time domain. We live in the time > domain, to us the frequency domain is a mathematical abstraction.
I suppose that it mostly true in audio, much less obvious in optics, and probably not in anything higher than optical frequencies. The ear uses mechanical filters to convert frequencies into the spatial domain, and then detect them. As far as what we see, it is much easier to describe the eye as a frequency domain sensor than a time domain sensor. There are systems that do optical frequency counting after downconversion to something actually countable. That is, countable differences between two optical frequencies. (I have seen a picture of two lasers shining on a microwave mixer diode, then connected to a high speed frequency counter.) You can almost describe optics as the time domain interaction with electrons in atoms. At X-ray and gamma frequencies, though, even that analogy breaks down.
> Everything we build operates in the time domain. When something that we > have built performs a DCT, it does operations in the time domain on data > that was measured in the time (and perhaps time-spatial) domain. The > output of the DCT is an approximation of variables that may exist in _a_ > frequency domain, but the numbers that are spat out of the hardware only > have meaning in the time domain.
There is a favorite optical trick of converting spatial frequency to the FT domain, though the use of a lens. (One can argue about it being continuous or discrete, but usually not cosine transform. The phase information is still there.)
> Do this simple experiment:
(snip)
> Is the Fourier transform still interesting? Is it good looking? Does it > make you drool (oh wait -- I said I wouldn't suggest that...)?
There is, in one chapter of "Feynman Lectures on Physics" a discussion of a rainbow. He describes one in the frequency-spatial space, that is, optical frequency spectrum and view angle, on a graph, and then discusses the beauty of the graph compared to the view of an actual rainbow. Maybe a better example of time domain, though, would be the DFT of a song. (That is, time domain to temporal frequency instead of space to spatial frequency.) Reminds me, though, of Kepler's music of the spheres. -- glen
On 4/12/12 1:10 PM, Tim Wescott wrote:
 > On Thu, 12 Apr 2012 04:43:21 -0500, manishp wrote:
..
 >>      As an observation on point 3 above, we know that filter aims to
 >>      achieve
 >> high attenuation in stop band while quantization does not totally
 >> attenauate HF components but only reduces their overall contribution
 >
 > I think you misunderstand what quantization does.  Quantization
 > distorts.  When you try to interpret its effects in the frequency
 > domain you might talk about it as you are -- but here in the time
 > domain, where we live, quantization distorts.

and i think the main point for the OP to this is that there isn't a nice 
Fourier transform of what this time-domain operation does.  it's not 
like an integrator or some filter that *is* fully describable in the 
frequency domain.  but ...

...

On 4/12/12 1:42 PM, Richard Dobson wrote:
> On 12/04/2012 18:24, Randy Yates wrote: >> Tim Wescott<tim@seemywebsite.com> writes: >>> [...] >>> All operations are performed in the time domain. We live in the time >>> domain, to us the frequency domain is a mathematical abstraction. >>> Everything we build operates in the time domain. >> >> At some basic level that is true enough. But you know there are >> "frequency domain filters" and there is a damned good reason that's what >> they're called. > > I am sure there are quite a few physicists who will tell you the time > domain is a mathematical abstraction too. > > And the frequency domain is by no means entirely abstract, it is in the > vernacular too. People will generally refer to a pitch going up or down; > rarely indeed will they describe it as getting shorter or longer.
pedestrians have an idea of frequency response, at least when something "sounds tinny", there is a quantitative vista that is in the frequency domain every much as what we experience as moment-to-moment reality is. and sometimes we can come up with models in the frequency domain, even if there are limitations and inaccuracies (a.k.a. approximations) in them models. e.g. the quantization operation referred above. if the quantization level is sufficiently smaller than the signal level (and we can argue about how low that is), we model it as additive white noise with a rectangular p.d.f., unless we do something other than simple quantization. sometimes we do things like add triangular p.d.f. dither and noise shape to a known power spectrum. *then*, it's very legitimate to view this solely in the frequency domain. it is true that the moments above the 2nd, regarding the additive error, is coupled to the signal, but the 1st and 2nd moments (the DC level and AC power) are fully decoupled from the signal getting quantized. i doubt you could sense any difference between the quantized signal and the unquantized signal with this uncorrelated additive noise added. i know that this is true with audio, and i sorta think it is with 2-dim images or other kind of data. i am not saying that it would be wise to quantize that way for any particular signal, it might be worse to do this for bitmap images, i dunno. i'm just saying that this kind of quantization would look just like the unquantized image with some kinda "snow" added. -- r b-j rbj@audioimagination.com "Imagination is more important than knowledge."
On 4/12/12 3:39 PM, glen herrmannsfeldt wrote:
> >> All operations are performed in the time domain. We live in the time >> domain, to us the frequency domain is a mathematical abstraction. > > I suppose that it mostly true in audio,
it surely doesn't have to be. i was trying to make the case (dithered and noise-shaped quantization) that often in audio we don't need the time domain for a good model of quantization.
> much less obvious in optics, > and probably not in anything higher than optical frequencies.
i dunno, i'll bet that effects from non-linear operations can be a problem to model in any territory of the spectrum.
> The ear uses mechanical filters to convert frequencies into the > spatial domain, and then detect them. As far as what we see, it is > much easier to describe the eye as a frequency domain sensor than a > time domain sensor.
hmmm.
> > There are systems that do optical frequency counting after > downconversion to something actually countable. That is, countable > differences between two optical frequencies. (I have seen a picture > of two lasers shining on a microwave mixer diode, then connected to > a high speed frequency counter.) > > You can almost describe optics as the time domain interaction with > electrons in atoms. At X-ray and gamma frequencies, though, even > that analogy breaks down. > >> Everything we build operates in the time domain. When something that we >> have built performs a DCT, it does operations in the time domain on data >> that was measured in the time (and perhaps time-spatial) domain. The >> output of the DCT is an approximation of variables that may exist in _a_ >> frequency domain, but the numbers that are spat out of the hardware only >> have meaning in the time domain. > > There is a favorite optical trick of converting spatial frequency > to the FT domain, though the use of a lens. (One can argue about > it being continuous or discrete, but usually not cosine transform. > The phase information is still there.) > >> Do this simple experiment: > > (snip) > >> Is the Fourier transform still interesting? Is it good looking? Does it >> make you drool (oh wait -- I said I wouldn't suggest that...)? > > There is, in one chapter of "Feynman Lectures on Physics" a discussion > of a rainbow. He describes one in the frequency-spatial space, that is, > optical frequency spectrum and view angle, on a graph, and then > discusses the beauty of the graph compared to the view of an actual > rainbow. > > Maybe a better example of time domain, though, would be the DFT of > a song.
hmmmmmmmmm.
> (That is, time domain to temporal frequency instead of space > to spatial frequency.) Reminds me, though, of Kepler's music > of the spheres.
<...stroking chin...> -- r b-j rbj@audioimagination.com "Imagination is more important than knowledge."
On 4/12/12 3:39 PM, glen herrmannsfeldt wrote:
> > Sampling is often described as multiplying by a train of delta > functions, such that the result is still described in continuous > time, though it is zero except at specific discrete times. > > If you do that, it seems that one could also describe quantization > as a filter, with the result still described in continuous time. >
well, and think Robert Wannamaker and Stan Lipshitz have published about this. i think this is how they derived their main result that rectangular-shaped additive dither (of width equal to the quantization step) will fully decouple the 1st moment (the mean) of the quantization error from the signal and that adding triangular pdf dither (the sum of two independent rectuangular pdf dithers) decoupled both the 1st and 2nd moment of the quantization error from the signal getting quantized.
> It seems to me that you could also quantize a continuous time signal, > though the result wouldn't be contiguous (in the mathematical sense).
hmmm. what'sa mathematically contiguous result? another hmmm i don't understand yet. -- r b-j rbj@audioimagination.com "Imagination is more important than knowledge."
robert bristow-johnson <rbj@audioimagination.com> wrote:

(snip, I wrote)
>> much less obvious in optics, >> and probably not in anything higher than optical frequencies.
> i dunno, i'll bet that effects from non-linear operations > can be a problem to model in any territory of the spectrum.
A gamma ray might be connected to the time domain motion of quarks in the nucleus, but quantum mechanics doesn't really allow one to look at them that way. In any case, we don't have any tools that will do it. And electron-photon scattering at gamma ray energies also doesn't have a good time domain represention.
>> The ear uses mechanical filters to convert frequencies into the >> spatial domain, and then detect them.
Consider a wave traveling in a system where velocity decreases as it travels. (That is, non-linear.) Then look at the amplitude as a function of position for different frequencies.
>> As far as what we see, it is much easier to describe the eye >> as a frequency domain sensor than a time domain sensor.
> hmmm.
(big snip)
>> Maybe a better example of time domain, though, would be the DFT of >> a song.
> hmmmmmmmmm.
It seems that it wouldn't take so long to FFT a whole CD, though I suppose track by track makes more sense.
>> (That is, time domain to temporal frequency instead of space >> to spatial frequency.) Reminds me, though, of Kepler's music >> of the spheres.
> <...stroking chin...>
-- glen