"Encoding Efficiency" Definition

Hi Folks,

I scoured Google and believe it or not couldn't come up with
a definition of "encoding efficiency" (or coding efficiency)
in the context of video and audio codecs (MPEG4-AAC/LC, e.g.).

It seems to me the obvious definition is

 k = 1 - (output bitrate) / (input bitrate)

Is this reasonable? Does anyone know of anything
better, or can you suggest something better?

--Randy

> It seems to me the obvious definition is
>
>  k = 1 - (output bitrate) / (input bitrate)
>
> Is this reasonable? Does anyone know of anything
> better, or can you suggest something better?

Hi Randy,

are you talking about source coding or channel coding? Since you
mentioned MPEG I presume you are talking about source coding, in which
case I cant help you. If you are asking about channel coding, then the
code efficiency is more usually described as the bandwidth efficiency
of the code, which is given in information bits, per second, per Hertz.
To illustrate by example, an uncoded 16-QAM signal has a bandwidth
efficiency of 4 b/s/Hz while using conventional trellis coded
modulation with a 64-point QAM constellation gives bandwidth efficiency
of 5 b/s/HZ, since there are 5 information bits, one of which is fed to
rate 1/2 convolutional encoder to give a total of 6 bits which are
mapped to the constellation point. 

Porterboy

Randy Yates wrote:
> Hi Folks,
>
> I scoured Google and believe it or not couldn't come up with
> a definition of "encoding efficiency" (or coding efficiency)
> in the context of video and audio codecs (MPEG4-AAC/LC, e.g.).
>
> It seems to me the obvious definition is
>
>  k = 1 - (output bitrate) / (input bitrate)
>
> Is this reasonable? Does anyone know of anything
> better, or can you suggest something better?

Hi Randy,

I don't think that's a reasonable F.O.M. for a lossy codec like AAC, as
the user will specify the input and output bit rates to the coder.  I
can get whatever 'k' I want just by specifying different command line
arguments to FAAC (etc.).
I.e. the 'efficiency' as you define it is not a characteristic of the
encoding algorithm.

What's more important is the subjective audio quality as a function of
the output bitrate.  Unfortunately this is a little harder to express
with a single number.

Regards,
Allan

Hi Porterboy,

I'm talking about source coding. 

--Randy

Hi Allan,

Thank you for your response. What does F.O.M. mean?

I think I see your point. I agree that output bitrate alone is pretty
meaningless - it must be coupled with signal quality.
However, I don't see how you can specify the input
bitrate - that is determined by the signal you're encoding.

I was going to include signal quality (just a rough, subjective
rating) as part of my testing, so I think I've got this covered.

Along the same lines, you could also say encoding efficiency
depends on the input signal complexity. For example,
Stravinski's Firebird Suite is going to be harder to
encode at given output bitrate and signal quality than
Aerosmith. Thus the reported encoding efficient is going
to be a function of the input signal complexity in my testing.

I just needed a simple encoding efficiency parameter. Perhaps
it would be better to call it "compression ratio?"

--Randy

Randy Yates wrote:
> Hi Folks,
> 
> I scoured Google and believe it or not couldn't come up with
> a definition of "encoding efficiency" (or coding efficiency)
> in the context of video and audio codecs (MPEG4-AAC/LC, e.g.).
> 
> It seems to me the obvious definition is
> 
>  k = 1 - (output bitrate) / (input bitrate)
> 
> Is this reasonable? Does anyone know of anything
> better, or can you suggest something better?

If you think that is reasonable, it seems to follow that you must think 
it reasonable that selecting the poorest quality option for the codec is 
more efficient than selecting the highest quality option. Most opinions 
of codec efficiency weigh heavily on the quality of the results, and 
that can be very subjective. Codec performance measurement is good a 
subject for starting a fight. :-)

Regards,
Steve

Randy Yates wrote:
> Hi Allan,
>
> Thank you for your response. What does F.O.M. mean?

Figure Of Merit.
( http://www.acronymfinder.com/ )

> I think I see your point. I agree that output bitrate alone is pretty
> meaningless - it must be coupled with signal quality.
> However, I don't see how you can specify the input
> bitrate - that is determined by the signal you're encoding.
>
> I was going to include signal quality (just a rough, subjective
> rating) as part of my testing, so I think I've got this covered.
>
> Along the same lines, you could also say encoding efficiency
> depends on the input signal complexity. For example,
> Stravinski's Firebird Suite is going to be harder to
> encode at given output bitrate and signal quality than
> Aerosmith. Thus the reported encoding efficient is going
> to be a function of the input signal complexity in my testing.
>
> I just needed a simple encoding efficiency parameter. Perhaps
> it would be better to call it "compression ratio?"

Consider this:
I can't tell the difference between 16 bit and 24 bit audio samples.
If I AAC both of them (to, say, 128kbps), I still won't be able to tell
the difference, but the compression ratio (bitrate out / bitrate in)
will be very different.
This is why I think compression ratio (or encoding efficiency,
whatever) is pretty useless for this sort of work.

The quality of the output *at a particular output bit rate* is what
matters.


I'm curious about why you are intested in metrics like compression
ratio.

Regards,
Allan

> Randy Yates wrote:
> > Hi Allan,
>
> > Thank you for your response. What does F.O.M. mean?
>
> Figure Of Merit.
> ( http://www.acronymfinder.com/ )
>
> - Hide quoted text -
> - Show quoted text -
> > I think I see your point. I agree that output bitrate alone is pretty
> > meaningless - it must be coupled with signal quality.
> > However, I don't see how you can specify the input
> > bitrate - that is determined by the signal you're encoding.
>
> > I was going to include signal quality (just a rough, subjective
> > rating) as part of my testing, so I think I've got this covered.
>
> > Along the same lines, you could also say encoding efficiency
> > depends on the input signal complexity. For example,
> > Stravinski's Firebird Suite is going to be harder to
> > encode at given output bitrate and signal quality than
> > Aerosmith. Thus the reported encoding efficient is going
> > to be a function of the input signal complexity in my testing.
>
> > I just needed a simple encoding efficiency parameter. Perhaps
> > it would be better to call it "compression ratio?"
>
> Consider this:
> I can't tell the difference between 16 bit and 24 bit audio samples.
> If I AAC both of them (to, say, 128kbps), I still won't be able to tell
> the difference, but the compression ratio (bitrate out / bitrate in)
> will be very different.
> This is why I think compression ratio (or encoding efficiency,
> whatever) is pretty useless for this sort of work.
>
> The quality of the output *at a particular output bit rate* is what
> matters.

Hmmm. You make good sense, Allan. I'll think this through a little
more.

> I'm curious about why you are intested in metrics like compression
> ratio.

I'm currently contracting for a company that has licensed some video
and audio codecs and they're interesting in characterizing their
performance. 

--Randy

Randy Yates wrote:

> I'm currently contracting for a company that has licensed some video
> and audio codecs and they're interesting in characterizing their
> performance. 

Unfortunately there's no such thing as a black box where you can put
encoders in at one side and get ratings out on the other side.

For the audio codecs, a subjective quality evaluation at a set bitrate
will get you good info, but such a test is quite time consuming and
difficult to set up (if you want to do it correctly, at least).

If the codecs are of significantly differing performance, there are some
objective quality measuring methods, e.g. ITU BS.1387. But these rely on
the quality measuring tool to be smarter than the codec itself - for
state of the art stuff, that is not going to be valid and you can get
wrong results.

So basically, no easy solution to this one.

-- 
GCP

Randy,
With the number of different types of codes, (compression codes,
translation codes, etc.), there really isn't a good consistent measure
of effeciency.  For example, compression codes are usually "graded" by
their compression ratio.  But this does nothing for a measure of
readability after decoding.  For channel codes, sometimes the rate is
used.

It is preferred to "grade" codes by their performance.  This recognizes
that there is a difference between coding effeciency and quality of
service (QoS) of a code.  For Subjective performance, the typical has
been SNR, SEGSNR, and other forms of the signal and noise ratios.
There has been a lot of activity lately in the area of objective as
well as subjective measures.  Study Group 15 (now that I think about
it, it might be Study Group 11.  I belonged to both and get them mixed
sometimes) created several subjective measures for coders.  They have
tried to apply some of these as a measure for signal improvement, with
which I strongly disagreed.  I would suggest looking at the
contributions on this subject by Study Groups 15 and 11.  I know Study
Group 15 was looking at this with regard to signal enhancement.  If I
remember correctly, Study Group 11 was looking at coding performqnce
measures.  When I left the ITU, both groups were beginning to examine
an algorithm to give an objective measure.

Good luck,

Maurice Givens, P.E.
Givens Burnet, Inc.
Chicago, IL