Forums

Chroma Decimation in Digital Video Signals

Started by Randy Yates October 14, 2005
Hi,

I've seen several cases where the chroma decimation, e.g.,
when going from YCrCb 4:4:4 to 4:2:2, is done by simply
throwing away every other sample. I have two questions
regarding this operation:

  1. If there were a 1-D signal, simply throwing away
  every other sample without first filtering would cause (potentially)
  aliasing. Why doesn't this happen in a chroma signal, or
  why doesn't it matter if it does happen?

  2. Since this is a 2-D signal, why isn't a 2-D filter
  required?

--Randy

Hello Randy,

This may be only a little related to your question but here goes. In still 
photography the common ways to represent images are in RGB and CMYK color 
spaces where the color and luminance info are entangled.  However from what 
I understand YCrCb is a lot like L*A*B in that the color info and the 
brightness info are separated from each other. And my experience with 
processing images in L*A*B colorspace shows most of the high spatial 
frequency info is in the L channel (Luminance) and the two color channels A 
and B have very little high spatial frequency content. Since in this 
viewpoint, the color channels are sort of already lowpassed in the sense 
that they don't contain much high frequency info so aliasing is not much of 
an issue. This is not to say the data has been filtered, but rather this 
lack of high color spatial frequency content seems to be a common property 
of many images. There are cases where the color channels can have high 
frequency data and aliasing may prove to be a problem here. But it may not 
be strongly perceived.

Clay



"Randy Yates" <yates@ieee.org> wrote in message 
news:1129298509.776847.49730@z14g2000cwz.googlegroups.com...
> Hi, > > I've seen several cases where the chroma decimation, e.g., > when going from YCrCb 4:4:4 to 4:2:2, is done by simply > throwing away every other sample. I have two questions > regarding this operation: > > 1. If there were a 1-D signal, simply throwing away > every other sample without first filtering would cause (potentially) > aliasing. Why doesn't this happen in a chroma signal, or > why doesn't it matter if it does happen? > > 2. Since this is a 2-D signal, why isn't a 2-D filter > required? > > --Randy >
Clay S. Turner wrote:
> Hello Randy, > > This may be only a little related to your question but here goes. In still > photography the common ways to represent images are in RGB and CMYK color > spaces where the color and luminance info are entangled. However from what > I understand YCrCb is a lot like L*A*B in that the color info and the > brightness info are separated from each other. And my experience with > processing images in L*A*B colorspace shows most of the high spatial > frequency info is in the L channel (Luminance) and the two color channels A > and B have very little high spatial frequency content. Since in this > viewpoint, the color channels are sort of already lowpassed in the sense > that they don't contain much high frequency info so aliasing is not much of > an issue. This is not to say the data has been filtered, but rather this > lack of high color spatial frequency content seems to be a common property > of many images. There are cases where the color channels can have high > frequency data and aliasing may prove to be a problem here. But it may not > be strongly perceived.
I was under the impression that much of the low-frequency nature of the color signal arose in the eye, rather than the image. I can imagine that that doesn't matter. Jerry -- Engineering is the art of making what you want from things you can get. &#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;
Randy Yates wrote:
> Hi, > > I've seen several cases where the chroma decimation, e.g., > when going from YCrCb 4:4:4 to 4:2:2, is done by simply > throwing away every other sample. I have two questions > regarding this operation: > > 1. If there were a 1-D signal, simply throwing away > every other sample without first filtering would cause (potentially) > aliasing. Why doesn't this happen in a chroma signal, or > why doesn't it matter if it does happen? > > 2. Since this is a 2-D signal, why isn't a 2-D filter > required? > > --Randy
Aliasing could occur, yes, but see the other response. As for #2, in 4:2:2 the chroma is subsampled only in the horizontal dimension, so a 1-D filter along that horizontal would be all that is required to prevent aliasing. In 4:1:1, you would need a 2-D filter since chroma is subsampled horizontally and vertically. Cheers! --M
"Jerry Avins" <jya@ieee.org> wrote in message 
news:jqmdndlO187BedLeRVn-sw@rcn.net...
> > I was under the impression that much of the low-frequency nature of the > color signal arose in the eye, rather than the image. I can imagine that > that doesn't matter. > > Jerry > --
Hello Jerry, I'm just speaking from my experience in working in the L*A*B colorspace. Many graphic artists subscribe to the maxim of blur early and sharpen late. Often the color channels can take a little bit of gausian blur without affecting (much) the detail in the image. This follows from there not being a lot of high frequency color info. This blurring really helps reduce color noise due to over processing an image. The L channel on the other hand gets sharpened at the end of the processing. And sharpening this way avoids colored halos. Certainly the human visual system has its own lowpass filtering as I'm sure you know how this was implemented in NTSC. Maybe this natural adaptation is in response to most scenes lacking a lot of high color frequency detail. Clay
On 14 Oct 2005 07:01:49 -0700, "Randy Yates" <yates@ieee.org> wrote:

>I've seen several cases where the chroma decimation, e.g., >when going from YCrCb 4:4:4 to 4:2:2, is done by simply >throwing away every other sample. I have two questions >regarding this operation: > > 1. If there were a 1-D signal, simply throwing away > every other sample without first filtering would cause (potentially) > aliasing. Why doesn't this happen in a chroma signal, or > why doesn't it matter if it does happen? > > 2. Since this is a 2-D signal, why isn't a 2-D filter > required? > >--Randy
Aliasing will only happen if there's energy above fs/4, which Clay suggests there likely won't be in the color info. So throwing away every other sample in that case won't affect much other than sample spacing. The perception issue is key. A while back when somebody was asking about maintaining registration between the compressed audio and compressed video during packet dropouts or rate mismatches, I learned from our internal people working on that that a common way to handle it is either drop or repeat video frames. Dropping or repeating a video frame is apparently (and I can easily see this) much more tolerable to a user than dropping or repeating an audio packet. The impression I get from that is that such things that we'd suspect to be travesties from a signal processing perspective aren't really so bad to the eye. Eric Jacobsen Minister of Algorithms, Intel Corp. My opinions may not be Intel's opinions. http://www.ericjacobsen.org
Clay S. Turner wrote:
> "Jerry Avins" <jya@ieee.org> wrote in message > news:jqmdndlO187BedLeRVn-sw@rcn.net... > >>I was under the impression that much of the low-frequency nature of the >>color signal arose in the eye, rather than the image. I can imagine that >>that doesn't matter. >> >>Jerry >>-- > > > > Hello Jerry, > > I'm just speaking from my experience in working in the L*A*B colorspace. > Many graphic artists subscribe to the maxim of blur early and sharpen late. > Often the color channels can take a little bit of gausian blur without > affecting (much) the detail in the image. This follows from there not being > a lot of high frequency color info. This blurring really helps reduce > color noise due to over processing an image. The L channel on the other hand > gets sharpened at the end of the processing. And sharpening this way avoids > colored halos.
I think that's because of the way that the visual system processes images. Most sharp color transitions coincide with luminance transitions and can be inferred from them. High color frequencies without corresponding luminance transitions would seem to be rare.
> Certainly the human visual system has its own lowpass filtering as I'm sure > you know how this was implemented in NTSC. Maybe this natural adaptation is > in response to most scenes lacking a lot of high color frequency detail.
If the high-frequency color information can be recreated from intensity, maybe it's a form of data compression. Jerry -- Engineering is the art of making what you want from things you can get. &#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;&#2013266095;
Randy Yates wrote:

> I've seen several cases where the chroma decimation, e.g., > when going from YCrCb 4:4:4 to 4:2:2, is done by simply > throwing away every other sample. I have two questions
Which is not really a good thing... Throwing away samples, I mean, not the two questions :-)
> regarding this operation: > > 1. If there were a 1-D signal, simply throwing away > every other sample without first filtering would cause (potentially) > aliasing. Why doesn't this happen in a chroma signal, or > why doesn't it matter if it does happen?
Yes, it could cause aliasing, but it depends also on the source. For example color information in PAL/NTSC modulated composite signals is quite low passed, so subsampling can be done without further filtering. Different story if the UV signals come from direct RGB444 conversion. In this case low pass filtering is required, also because the 422 result could be, in case 420 is the final target, co-sited or interspersed (whatever is written), then filtering is a must.
> 2. Since this is a 2-D signal, why isn't a 2-D filter > required?
422 sumsamples the UV signals only in the horizontal direction, so 1-D filter is enough. In case of 420, 2-D filtering is required, of course. bye, -- piergiorgio
Throwing away every other sample is often done, but is not the best
idea. The media processors I know best (Philips TriMedia/Nexperia)
apply a 5-tap filter when changing from eg YUV444 to YUV422 (and from
YUV422 to YUV420 etc). The shape of filter is important, too, and may
be changed for different types of image. In some cases this os done by
digitizing or reconstruction hardware, in some by dedicated
co-processors.

Chroma decimation by throwing away is often acceptable, but it depends
what you mean by 'acceptable' - it does not give you such a good
picture. In fact, the new Nexperia processors enhance the chroma by
some clever processing that tries to put back what was left out, and
this is beginning to become perceived by viewers. I thik the reason for
the less-demanding processing being so common is, that viewers have
been trained to accept fairly poor picture quality.

Horizontal filtering is really a quite simple 1-D case, albeit one that
requires polyphase filter, and so can be done 'on the fly' as a signal
is captured or rconstructed - to do a 2D filter requires line or frame
buffering, but this is also often done.

One problem in this area is, lots of people 'know' how to process video
- but fewer seem able to produce pictures that look good - that
requires a lot of domain knowledge. When I was involved with
demonstrating video to prospective customners (ie TV manufacturers) I
noticed that engineers would often be proud of their video processing
but the intended customers would write it off very quickly because he
picture quality was not good enough. To be able to look for video
quality is not all that easy, as we get distracted by the content and
by being so used to watching awful quality.

Chris
===================
Chris Bore
BORES Signal Processing
chris@bores.com

The chroma aliasing certainly occurs and can be seen for example as
false color fringes on high frequency details - eg closely spaced white
fence posts.

Chris
=========================
Chris Bore
BORES Signal Processing
www.bores.com