basic questions on jpeg compression

Folks,

I have few basic questions on related to compression used in JPEG.

 1. Is the DCT applied over Y, CB and CR separately?
 2. Is there a specific reason why DCT is chosen over DFT?
 3. Can quantization step be roughly seen as a filter since it basically
tones down high frequency components?
    As an observation on point 3 above, we know that filter aims to achieve
high attenuation in stop band while quantization does not totally
attenauate HF components but only reduces their overall contribution
 4. Another related question is, in an earlier question I had asked if
 filters really need conversion from time to frequency domain before they
can operate. The response was that filters are designed to operate directly
on time domain inputs and no conversion is normally needed.
But in the above case, clearly conversion is done to frequency domain
(though DCT) before further steps can be applied.

Comments would be highly appreciated?

manishp <manishp.p18@n_o_s_p_a_m.gmail.com> wrote:

> I have few basic questions on related to compression used in JPEG.

> 2. Is there a specific reason why DCT is chosen over DFT?

It has better boundary conditions, and so doesn't cause as much
visible artifacts.

The DFT periodic boundary conditions will tend to cause the block
boundaries to be equal, and be visible.

-- glen

On Thu, 12 Apr 2012 04:43:21 -0500, manishp wrote:

> Folks,
> 
> I have few basic questions on related to compression used in JPEG.
> 
>  1. Is the DCT applied over Y, CB and CR separately? 
>  2. Is there a
>  specific reason why DCT is chosen over DFT? 
>  3. Can quantization step be
>  roughly seen as a filter since it basically
>   tones down high frequency components?

If you broaden your definition of "filter" enough, anything is a filter.  
But quantization isn't a filter in most people's dictionary.  The idea of 
the filter-quantize is that you identify the features that 'matter' more 
(low spatial frequency signal) and quantize that with more bits, and you 
identify the features that 'matter' less (high spatial frequency) and 
quantize more.  The idea is that you're _going_ to lose information, so 
you try to pick and choose what information you care least about to 
discard.

>     As an observation on point 3 above, we know that filter aims to
>     achieve
> high attenuation in stop band while quantization does not totally
> attenauate HF components but only reduces their overall contribution

I think you misunderstand what quantization does.  Quantization 
distorts.  When you try to interpret its effects in the frequency domain 
you might talk about it as you are -- but here in the time domain, where 
we live, quantization distorts.

>  4. Another related question is, in an earlier question I had asked if
>  filters really need conversion from time to frequency domain before
>  they
> can operate. The response was that filters are designed to operate
> directly on time domain inputs and no conversion is normally needed. But
> in the above case, clearly conversion is done to frequency domain
> (though DCT) before further steps can be applied.

All operations are performed in the time domain.  We live in the time 
domain, to us the frequency domain is a mathematical abstraction.  
Everything we build operates in the time domain.  When something that we 
have built performs a DCT, it does operations in the time domain on data 
that was measured in the time (and perhaps time-spatial) domain.  The 
output of the DCT is an approximation of variables that may exist in _a_ 
frequency domain, but the numbers that are spat out of the hardware only 
have meaning in the time domain.

Do this simple experiment:

Go harvest a picture of something of aesthetic value off of the Internet 
(I'm trying to get my mind out of the gutter, so I won't suggest porn -- 
directly).

Take it's 2D Fourier transform.

Display the Fourier transform of the picture next to the original.

Is the Fourier transform still interesting?  Is it good looking?  Does it 
make you drool (oh wait -- I said I wouldn't suggest that...)?

So when you look at a picture in what you are calling the "frequency 
domain" (which is really a spatial domain rendering of an approximation 
of one particular frequency domain) -- it does nothing for you.

But the original picture has meaning that your nervous system can 
perceive.

Thus, time domain vs. frequency domain.

-- 
My liberal friends think I'm a conservative kook.
My conservative friends think I'm a liberal kook.
Why am I not happy that they have found common ground?

Tim Wescott, Communications, Control, Circuits & Software
http://www.wescottdesign.com

Tim Wescott <tim@seemywebsite.com> writes:
> [...]
> All operations are performed in the time domain.  We live in the time 
> domain, to us the frequency domain is a mathematical abstraction.  
> Everything we build operates in the time domain.  

At some basic level that is true enough. But you know there are
"frequency domain filters" and there is a damned good reason that's what
they're called.
-- 
Randy Yates
DSP/Firmware Engineer
919-577-9882 (H)
919-720-2916 (C)

On 12/04/2012 18:24, Randy Yates wrote:
> Tim Wescott<tim@seemywebsite.com>  writes:
>> [...]
>> All operations are performed in the time domain.  We live in the time
>> domain, to us the frequency domain is a mathematical abstraction.
>> Everything we build operates in the time domain.
>
> At some basic level that is true enough. But you know there are
> "frequency domain filters" and there is a damned good reason that's what
> they're called.

I am sure there are quite a few physicists who will tell you the time 
domain is a mathematical abstraction too.

And the frequency domain is by no means entirely abstract, it is in the 
vernacular too. People will generally refer to a pitch going up or down; 
rarely indeed will they describe it as getting shorter or longer.

Richard Dobson

Tim Wescott <tim@seemywebsite.com> wrote:
 
(snip, someone wrote)

>>  3. Can quantization step be  roughly seen as a filter since 
>>     it basically tones down high frequency components?

> If you broaden your definition of "filter" enough, anything is a filter.  
> But quantization isn't a filter in most people's dictionary.  

Sampling is often described as multiplying by a train of delta
functions, such that the result is still described in continuous
time, though it is zero except at specific discrete times.

If you do that, it seems that one could also describe quantization
as a filter, with the result still described in continuous time.

It seems to me that you could also quantize a continuous time signal,
though the result wouldn't be contiguous (in the mathematical sense).

> The idea of 
> the filter-quantize is that you identify the features that 'matter' more 
> (low spatial frequency signal) and quantize that with more bits, and you 
> identify the features that 'matter' less (high spatial frequency) and 
> quantize more.  The idea is that you're _going_ to lose information, so 
> you try to pick and choose what information you care least about to 
> discard.

(snip)

> I think you misunderstand what quantization does.  Quantization 
> distorts.  When you try to interpret its effects in the frequency domain 
> you might talk about it as you are -- but here in the time domain, where 
> we live, quantization distorts.

>>  4. Another related question is, in an earlier question I had asked if
>>  filters really need conversion from time to frequency domain before
>>  they
>> can operate. The response was that filters are designed to operate
>> directly on time domain inputs and no conversion is normally needed. But
>> in the above case, clearly conversion is done to frequency domain
>> (though DCT) before further steps can be applied.

> All operations are performed in the time domain.  We live in the time 
> domain, to us the frequency domain is a mathematical abstraction.  

I suppose that it mostly true in audio, much less obvious in optics,
and probably not in anything higher than optical frequencies.

The ear uses mechanical filters to convert frequencies into the
spatial domain, and then detect them. As far as what we see, it is
much easier to describe the eye as a frequency domain sensor than a
time domain sensor. 

There are systems that do optical frequency counting after
downconversion to something actually countable. That is, countable
differences between two optical frequencies. (I have seen a picture
of two lasers shining on a microwave mixer diode, then connected to
a high speed frequency counter.)

You can almost describe optics as the time domain interaction with
electrons in atoms. At X-ray and gamma frequencies, though, even
that analogy breaks down. 

> Everything we build operates in the time domain.  When something that we 
> have built performs a DCT, it does operations in the time domain on data 
> that was measured in the time (and perhaps time-spatial) domain.  The 
> output of the DCT is an approximation of variables that may exist in _a_ 
> frequency domain, but the numbers that are spat out of the hardware only 
> have meaning in the time domain.

There is a favorite optical trick of converting spatial frequency
to the FT domain, though the use of a lens. (One can argue about
it being continuous or discrete, but usually not cosine transform.
The phase information is still there.)

> Do this simple experiment:

(snip)

> Is the Fourier transform still interesting?  Is it good looking?  Does it 
> make you drool (oh wait -- I said I wouldn't suggest that...)?

There is, in one chapter of "Feynman Lectures on Physics" a discussion
of a rainbow. He describes one in the frequency-spatial space, that is,
optical frequency spectrum and view angle, on a graph, and then
discusses the beauty of the graph compared to the view of an actual
rainbow.

Maybe a better example of time domain, though, would be the DFT of
a song. (That is, time domain to temporal frequency instead of space
to spatial frequency.) Reminds me, though, of Kepler's music
of the spheres.

-- glen

On 4/12/12 1:10 PM, Tim Wescott wrote:
 > On Thu, 12 Apr 2012 04:43:21 -0500, manishp wrote:
..
 >>      As an observation on point 3 above, we know that filter aims to
 >>      achieve
 >> high attenuation in stop band while quantization does not totally
 >> attenauate HF components but only reduces their overall contribution
 >
 > I think you misunderstand what quantization does.  Quantization
 > distorts.  When you try to interpret its effects in the frequency
 > domain you might talk about it as you are -- but here in the time
 > domain, where we live, quantization distorts.

and i think the main point for the OP to this is that there isn't a nice 
Fourier transform of what this time-domain operation does.  it's not 
like an integrator or some filter that *is* fully describable in the 
frequency domain.  but ...

...

On 4/12/12 1:42 PM, Richard Dobson wrote:
> On 12/04/2012 18:24, Randy Yates wrote:
>> Tim Wescott<tim@seemywebsite.com> writes:
>>> [...]
>>> All operations are performed in the time domain. We live in the time
>>> domain, to us the frequency domain is a mathematical abstraction.
>>> Everything we build operates in the time domain.
>>
>> At some basic level that is true enough. But you know there are
>> "frequency domain filters" and there is a damned good reason that's what
>> they're called.
>
> I am sure there are quite a few physicists who will tell you the time
> domain is a mathematical abstraction too.
>
> And the frequency domain is by no means entirely abstract, it is in the
> vernacular too. People will generally refer to a pitch going up or down;
> rarely indeed will they describe it as getting shorter or longer.


pedestrians have an idea of frequency response, at least when something 
"sounds tinny", there is a quantitative vista that is in the frequency 
domain every much as what we experience as moment-to-moment reality is.

and sometimes we can come up with models in the frequency domain, even 
if there are limitations and inaccuracies (a.k.a. approximations) in 
them models.

e.g. the quantization operation referred above.

if the quantization level is sufficiently smaller than the signal level 
(and we can argue about how low that is), we model it as additive white 
noise with a rectangular p.d.f., unless we do something other than 
simple quantization.  sometimes we do things like add triangular p.d.f. 
dither and noise shape to a known power spectrum.

*then*, it's very legitimate to view this solely in the frequency 
domain.  it is true that the moments above the 2nd, regarding the 
additive error, is coupled to the signal, but the 1st and 2nd moments 
(the DC level and AC power) are fully decoupled from the signal getting 
quantized.  i doubt you could sense any difference between the quantized 
signal and the unquantized signal with this uncorrelated additive noise 
added.

i know that this is true with audio, and i sorta think it is with 2-dim 
images or other kind of data.  i am not saying that it would be wise to 
quantize that way for any particular signal, it might be worse to do 
this for bitmap images, i dunno.  i'm just saying that this kind of 
quantization would look just like the unquantized image with some kinda 
"snow" added.

-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

On 4/12/12 3:39 PM, glen herrmannsfeldt wrote:
>
>> All operations are performed in the time domain.  We live in the time
>> domain, to us the frequency domain is a mathematical abstraction.
>
> I suppose that it mostly true in audio,

it surely doesn't have to be.  i was trying to make the case (dithered 
and noise-shaped quantization) that often in audio we don't need the 
time domain for a good model of quantization.

> much less obvious in optics,
> and probably not in anything higher than optical frequencies.

i dunno, i'll bet that effects from non-linear operations can be a 
problem to model in any territory of the spectrum.

> The ear uses mechanical filters to convert frequencies into the
> spatial domain, and then detect them. As far as what we see, it is
> much easier to describe the eye as a frequency domain sensor than a
> time domain sensor.

hmmm.

>
> There are systems that do optical frequency counting after
> downconversion to something actually countable. That is, countable
> differences between two optical frequencies. (I have seen a picture
> of two lasers shining on a microwave mixer diode, then connected to
> a high speed frequency counter.)
>
> You can almost describe optics as the time domain interaction with
> electrons in atoms. At X-ray and gamma frequencies, though, even
> that analogy breaks down.
>
>> Everything we build operates in the time domain.  When something that we
>> have built performs a DCT, it does operations in the time domain on data
>> that was measured in the time (and perhaps time-spatial) domain.  The
>> output of the DCT is an approximation of variables that may exist in _a_
>> frequency domain, but the numbers that are spat out of the hardware only
>> have meaning in the time domain.
>
> There is a favorite optical trick of converting spatial frequency
> to the FT domain, though the use of a lens. (One can argue about
> it being continuous or discrete, but usually not cosine transform.
> The phase information is still there.)
>
>> Do this simple experiment:
>
> (snip)
>
>> Is the Fourier transform still interesting?  Is it good looking?  Does it
>> make you drool (oh wait -- I said I wouldn't suggest that...)?
>
> There is, in one chapter of "Feynman Lectures on Physics" a discussion
> of a rainbow. He describes one in the frequency-spatial space, that is,
> optical frequency spectrum and view angle, on a graph, and then
> discusses the beauty of the graph compared to the view of an actual
> rainbow.
>
> Maybe a better example of time domain, though, would be the DFT of
> a song.

hmmmmmmmmm.

> (That is, time domain to temporal frequency instead of space
> to spatial frequency.) Reminds me, though, of Kepler's music
> of the spheres.


<...stroking chin...>



-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

On 4/12/12 3:39 PM, glen herrmannsfeldt wrote:
>
> Sampling is often described as multiplying by a train of delta
> functions, such that the result is still described in continuous
> time, though it is zero except at specific discrete times.
>
> If you do that, it seems that one could also describe quantization
> as a filter, with the result still described in continuous time.
>

well, and think Robert Wannamaker and Stan Lipshitz have published about 
this.  i think this is how they derived their main result that 
rectangular-shaped additive dither (of width equal to the quantization 
step) will fully decouple the 1st moment (the mean) of the quantization 
error from the signal and that adding triangular pdf dither (the sum of 
two independent rectuangular pdf dithers) decoupled both the 1st and 2nd 
moment of the quantization error from the signal getting quantized.

> It seems to me that you could also quantize a continuous time signal,
> though the result wouldn't be contiguous (in the mathematical sense).

hmmm.  what'sa mathematically contiguous result?  another hmmm i don't 
understand yet.


-- 

r b-j                  rbj@audioimagination.com

"Imagination is more important than knowledge."

robert bristow-johnson <rbj@audioimagination.com> wrote:

(snip, I wrote)
>> much less obvious in optics,
>> and probably not in anything higher than optical frequencies.

> i dunno, i'll bet that effects from non-linear operations
> can be a problem to model in any territory of the spectrum.

A gamma ray might be connected to the time domain motion of
quarks in the nucleus, but quantum mechanics doesn't really allow
one to look at them that way. In any case, we don't have any tools
that will do it. And electron-photon scattering at gamma ray
energies also doesn't have a good time domain represention.

>> The ear uses mechanical filters to convert frequencies into the
>> spatial domain, and then detect them. 

Consider a wave traveling in a system where velocity decreases
as it travels. (That is, non-linear.) Then look at the amplitude
as a function of position for different frequencies. 

>> As far as what we see, it is much easier to describe the eye 
>> as a frequency domain sensor than a time domain sensor.

> hmmm.

(big snip)
>> Maybe a better example of time domain, though, would be the DFT of
>> a song.

> hmmmmmmmmm.

It seems that it wouldn't take so long to FFT a whole CD, though
I suppose track by track makes more sense.

>> (That is, time domain to temporal frequency instead of space
>> to spatial frequency.) Reminds me, though, of Kepler's music
>> of the spheres.

> <...stroking chin...>

-- glen