Forums

MELP Test Vectors

Started by Unknown January 27, 2005
Hi,

I am looking for MELP test vectors to verify my implementation. Can
someone point me in the right direction?

I expected they would be easy to find - considering the DDVPC have 3
publically available MELP implementations - but I can not find test
vectors anywhere.

Regards,
Nick

Probably the best reference for MELP and MELPe is Compandent Inc., which 
is the only company that participated in the MELPe's R&D and is still 
actively working on it, supporting it, and its implementation.
See:
http://www.MELPe.com

VNG


npelly@gmail.com wrote:
> Hi, > > I am looking for MELP test vectors to verify my implementation. Can > someone point me in the right direction? > > I expected they would be easy to find - considering the DDVPC have 3 > publically available MELP implementations - but I can not find test > vectors anywhere. > > Regards, > Nick >
Other companies working on MELPe include:

  -Harris
  -Thales Group
  -Signalogic

Work by these companies includes algorithm level.

Another source for MELP/MELPe test vectors:

  http://www.signalogic.com/codec_samples.htm

Jeff Brower
Signalogic

PS.  Although he did not say, VNG is Oded Gottesman, alias Compandent,
Inc.

The crystal clear fact is that Compandent is the only company that 
worked on MELPe, contributed to its standard in terms of source code & 
Test Vectors, and still provides professional and reliable MELPe related 
solutions.  Compandent are the same speech coding experts that 
contributed to the actual MELPe standard.  Compandent in fact originated 
the reference "certified" Test Vectors, and therefore out of the other 
options mentioned, it is probably the best source for the requested 
"certified" Test Vectors.

See for yourself the very helpful MELPe / MELP FAQ:
http://www.compandent.com/melpe_faq.htm

The other companies may have merely done some "uncertified" internal 
programming work related to MELPe, but have NOT contributed to its 
standard reference or to its "certified" Test Vectors.  Their work 
probably relied upon Compandent's work, and is likely to even include 
some of it.   But they may have deviated from it, and who knows by how 
much...  Some of those companies out there who have done some internal 
programming work don't even have any real speech coding expert...  It is 
somewhat similar to the difference between having an opinion of a 
beginner nurse as compared to experienced doctor's, a doctor who even 
participated in a scientific research of the medical condition...
[I apologize for the very late posting.  I have been lurking for years, but
when I read the above it stuck in my craw badly enough that I simply had to
respond.  The misinformation circulating with regard to MELP is sometimes
beyond belief.]

Actually, the best place to obtain the test vectors is not Compandent, it
is the NATO STANAG 4591 standards site which has the 'golden' reference
code and over a hundred megabytes of test vectors.  If you don't produce
the same results as the golden reference, you're not MELPe [1].
Compandent's "MELPe++" is named that for a reason - because it's not bit
exact to the official MELPe standard.

NATO will transfer or mail you copies of the standards documents, vectors,
source code, and other material free of charge, so long as:

1) You're based in a NATO country, and
2) You have a valid reason to want to use MELPe.

Compandent has been very aggressive in posting about MELPe IP, which is
curious as it appears to base its entire claim on the copyright of a single
source file (melp_syn.c) in the published NATO standard.  The only other
Compandent copyrights related to MELP that I could find are for an old TI
C5000 series assembly language port of the code.

It is my understanding that the US government holds dual copyright and
patent rights on this standard, as it was developed with US government
funds.  As a dual rights holder, the government can give licenses to
whoever it wants, and it has given those rights to anyone using the
standard in a governmental application.  I believe for commercial
applications, you must license the appropriate IP.  IANAL - make sure you
do your own research.


[1] - There is an alternate process to matching the golden reference: 
hundreds of hours of listening tests with many different listeners to
verify that your implementation has better quality than the golden
reference.  This is beyond the budgets and time constraints of all but the
largest companies.


This really should be addressed, even if it is on an old forum, as it
contains a direct falsehood.


> [Compandent] is probably the best source for the requested >"certified" Test Vectors.
This is false. The official NATO STANAG 4591 site is the best place to get the official MELPe test vectors. They are available free of charge, as is the source code and design documentation, to all entities in the NATO organization that can demonstrate a valid need. The "valid need" test is a very low bar. Student researchers at a university can easily obtain a complete copy of all available material.
>The other companies may have merely done some "uncertified" internal >programming work related to MELPe, but have NOT contributed to its >standard reference or to its "certified" Test Vectors.
This talk of certification is misleading. NATO is the only certification body that matters here, and they have a simple certification policy: If it's bit exact to the NATO test vectors, it's NATO certified. [1] It doesn't matter who worked on it.
>Their work >probably relied upon Compandent's work, and is likely to even include >some of it.
I have a copy of the MELPe source tree available to me. It contains no Compandent copyright notices. A quick copyright search on the web shows that Compandent has registered US copyrights on precisely one [2] of the two score source files that make up the 'golden' reference. I have been able to find no patent references. TI on the other hand, holds copyrights on nearly half of the code base and makes numerous patent claims. Thales and Microsoft also figure prominently. AT&T has copyrighted and patented the NPP, a huge and non-trivial piece of work compared to the derivative modifications that are Compandent's registration. There is something seriously goofy going on with all these Compandent posts. Buyer beware. [1] - NATO certification can also be done by a lengthy and expensive set of listening tests, which verify that the implementation under test has better voice quality than the golden reference. No individual company would bother to take this route, as even a cleanroom implementation doesn't get you around the patent restrictions. If you license the patents, you usually get a license for the source code too; the same people own it. [2] - It should be noted that the NATO version of the file Compandent claims (melp_syn.c) contains copyright headers from both TI and Microsoft, but not from Compandent. As this file was in existence prior to the 2001 copyright registration, and is largely the same as earlier versions, Compandent's contributions would make this at best a derivative work. It may even fall into the category of being an uncopyrightable 'trivial modification'.
Argh, I apologize for the dual posting.  I was certain that my first
message had been eaten and destroyed.  The content is roughly the same,
though I was angry at having lost my work when posting the second message.
The NATO 'request for access' site is here:

https://s5066.nc3a.nato.int/authrequest.php

Note that you should request 4591 access.  5066 is a whole different
beast.
YOU ARE VERY VERY WRONG!!!

NATO provided the original speech files, and Compandent's code generated 
the reference Test Vector!

The Compandent MELPe++ fully complies with the reference Test Vectors!

Many bugs and problems that exist in the old, buggy, and premature code 
that you referred to, were not revealed in the standard Test Vectors, 
but were revealed in extended data bases and field experiments.

Compandent contributed to many MELPe files - including C files and 
Assembly files, some of which were registered in the US Copyright 
Office.  Included are Copyrigh numbers: TX0006833875 , TX0006845690, 
TX0006845325, TX0006844314, TX0006845691, TX0006845330, TX0006845769, 
TX0006842148, and TX0006874773.

Any derivative of Compandent code such as porting C code to c64xx or 
porting c549 code to c54xx code or to c55xx code etc., require licensing 
Compandent's MELPe IPR!

YOU ARE VERY VERY WRONG!!!

NATO provided the original speech files, and Compandent's code generated 
the reference Test Vector!

The Compandent MELPe++ fully complies with the reference Test Vectors!

Many bugs and problems that exist in the old, buggy, and premature code 
that you referred to, were not revealed in the standard Test Vectors, 
but were revealed in extended data bases and field experiments.

Compandent contributed to many MELPe files - including C files and 
Assembly files, some of which were registered in the US Copyright 
Office.  Included are Copyright numbers: TX0006833875 , TX0006845690, 
TX0006845325, TX0006844314, TX0006845691, TX0006845330, TX0006845769, 
TX0006842148, and TX0006874773.

Any derivative of Compandent code such as porting C code to c64xx or 
porting c549 code to c54xx code or to c55xx code etc., require licensing 
Compandent's MELPe IPR!