Quality of academic papers

Hi all,

While I'm bitching about the price of papers downloaded from IEEE 
Xplore I thought I'd raise another issue.

How many people here have spent considerable time working through 
an academic paper only to find that the paper contained glossed 
over details making the paper indecipherable, contained gross 
errors, or claimed results that simply weren't reproduceable?

I've actually come up with a solution to this problem at least
for the field of DSP;  require that all papers include code
demonstrating that the technique does in fact work. The code
could be released in source code form, in languages like
Matlab, Mathematica, Python, C, C++ or Fortran and should be 
released under a licence which allows the code to reused; say
a BSD, MIT or GPL style license.

I know this is a pretty radical idea, but I've wasted too much
time in my career being bamboozled by papers which were written
to pad the authors publication list rather than with the aim of
spreading knowledge.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo
+-----------------------------------------------------------+

   GPLG
  GPLGPLGP
 GPLGPLGPLGP
GPLGP
GPL                 MICROSOFT
GPLGP
 GPLGPLGPLGP
  GPLGPLGPL
    GPLGPL

I have a really big axe to grind about this one!

I find a lot of papers using purely mathematics to explain something,
without much descriptive text to accompany the equations, personally an
equation on a piece of paper is never ever enough and it always needs
explaining to some level.  If you are new to a subject, you're not going
to understand why a person has used the letter 'Y' instead the letter 'X',
sometimes you don't want to have to research this to accomplish something.

I come from a computer science background, some of the papers I look up,
I'm sure are pretty simplistic in nature, but the subject is clouded to
such a point the only way to understand a paper, is to do exactly the same
research as the author, just to understand it! (which is completely beyond
the point in the first place)

I sometimes think people just publish papers, not to inform others, but
just to say "look, look what I've done!"

I would like to see more examples accompanying papers in C++, but I just
don't see that happening, most papers I've seen however, if you do a
google on the subject, will probably result in finding some sort of matlab
code, someone else external to the paper has gone off and done all the hard
work.

Actually, when browsing forums, I get quite offended when people ask for
code for a hideously complicated task, and obviously haven't thought about
it at all, and just went straight to the "asking for help stage", that
really annoys me after all the hard work I put into decoding.

-Moose.

>Hi all,
>
>While I'm bitching about the price of papers downloaded from IEEE 
>Xplore I thought I'd raise another issue.
>
>How many people here have spent considerable time working through 
>an academic paper only to find that the paper contained glossed 
>over details making the paper indecipherable, contained gross 
>errors, or claimed results that simply weren't reproduceable?
>
>I've actually come up with a solution to this problem at least
>for the field of DSP;  require that all papers include code
>demonstrating that the technique does in fact work. The code
>could be released in source code form, in languages like
>Matlab, Mathematica, Python, C, C++ or Fortran and should be 
>released under a licence which allows the code to reused; say
>a BSD, MIT or GPL style license.
>
>I know this is a pretty radical idea, but I've wasted too much
>time in my career being bamboozled by papers which were written
>to pad the authors publication list rather than with the aim of
>spreading knowledge.
>
>Erik
>-- 
>+-----------------------------------------------------------+
>  Erik de Castro Lopo
>+-----------------------------------------------------------+
>
>   GPLG
>  GPLGPLGP
> GPLGPLGPLGP
>GPLGP
>GPL                 MICROSOFT
>GPLGP
> GPLGPLGPLGP
>  GPLGPLGPL
>    GPLGPL
>

moosedude wrote:

> I would like to see more examples accompanying papers in C++,

Not trying to be a programming language bigot here, but C++
is probably the worst possible language for providing proof
of concept implementations, mainly because of all the boiler
plate code required to do even simple linear algebra in C++.

Languages like Matlab and Mathematica are far better fit. Even 
Python with its extremely high level of readability is a better 
fit.

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo
+-----------------------------------------------------------+
"These days, she [Chahdortt Djavann] is a member of a government advisory 
council. There is also a French mullah on this council. "Why aren't boys
veiled?", she asked with a smile. The mullah cleared his throat and 
answered that the truth of the Koran states that because girls are unclean 
from birth it is only natural that they be veiled. Against this level of 
stupidity, blasphemy is a duty."
-- http://gatesofvienna.blogspot.com/2006/12/west-must-stop-tormenting-itself.html

"moosedude" <phill_holland@hotmail.com> wrote in
news:m_-dnYWogM0DFD7YnZ2dnUVZ_rqhnZ2d@giganews.com: 

> Actually, when browsing forums, I get quite offended when people ask
> for code for a hideously complicated task, and obviously haven't
> thought about it at all, and just went straight to the "asking for
> help stage", that really annoys me after all the hard work I put into
> decoding. 
> 
> 

But these folk shouldn't be getting code directly from published material 
either.  There are some real intellectual property issues that preclude the 
well-meaning ideas expressed in this thread.

The real answer is to make sure the referees do their jobs somehow. Pay 
them if necessary.

-- 
Scott
Reverse name to reply

Erik de Castro Lopo wrote:
> moosedude wrote:
>
> > I would like to see more examples accompanying papers in C++,
>
> Not trying to be a programming language bigot here, but C++
> is probably the worst possible language for providing proof
> of concept implementations, mainly because of all the boiler
> plate code required to do even simple linear algebra in C++.
>
> Languages like Matlab and Mathematica are far better fit. Even
> Python with its extremely high level of readability is a better
> fit.

Fortran 90 and later versions feature operations on whole arrays and
array sections, as in Matlab. Two free Fortran compilers exist,
gfortran and g95, whereas Matlab and Mathematica are proprietary. Much
Matlab code may run on Octave and Scilab, with only slight
modification, but I don't know of a free substitute for Mathematica
that accepts the same language.

Other things being equal (they may not be), it is better to use a
standardized programming languages that respect backwards
compatibility, such as C, C++, or Fortran to present algorithms. Python
3 will NOT be backwards compatible with the current Python 2.x. To
given an example of the pitfalls this creates, in Python, 1/2 equals 0
at present (integer division is used), but its designer has decided
that 1/2 to equal 0.5 (convert 1 and 2 to floats, then divide) in
future versions.

Scott Seidman wrote:
> "moosedude" <phill_holland@hotmail.com> wrote in
> news:m_-dnYWogM0DFD7YnZ2dnUVZ_rqhnZ2d@giganews.com:
>
> > Actually, when browsing forums, I get quite offended when people ask
> > for code for a hideously complicated task, and obviously haven't
> > thought about it at all, and just went straight to the "asking for
> > help stage", that really annoys me after all the hard work I put into
> > decoding.
> >
> >
>
> But these folk shouldn't be getting code directly from published material
> either.  There are some real intellectual property issues that preclude the
> well-meaning ideas expressed in this thread.

Much research appearing in academic journals is funded by the
government, for example by the National Science Foundation in the
United States. If the general public is paying for it, the public
should have access to the code. I would go further and say that the
"deliverables" for government-funded scientific research should include
not just published papers but well-documented software, when
programming is an important part of the research.

"Beliavsky" <beliavsky@aol.com> wrote in news:1168436258.869518.257740
@i39g2000hsf.googlegroups.com:

> If the general public is paying for it, the public
> should have access to the code. 


"Should" and "the law" unfortunately, are two very different things.  The 
public does have access to the publications these days, but not the code, 
the reagents, the royalties generated from IP, etc.


-- 
Scott
Reverse name to reply

Scott Seidman wrote:
> "Beliavsky" <beliavsky@aol.com> wrote in news:1168436258.869518.257740
> @i39g2000hsf.googlegroups.com:
>
> > If the general public is paying for it, the public
> > should have access to the code.
>
>
> "Should" and "the law" unfortunately, are two very different things.  The
> public does have access to the publications these days, but not the code,
> the reagents, the royalties generated from IP, etc.

I think future NSF grants contracts should be written so that
everything is put in the public domain.

"Beliavsky" <beliavsky@aol.com> wrote in
news:1168441817.484123.266440@k58g2000hse.googlegroups.com: 

> 
> Scott Seidman wrote:
>> "Beliavsky" <beliavsky@aol.com> wrote in
>> news:1168436258.869518.257740 @i39g2000hsf.googlegroups.com:
>>
>> > If the general public is paying for it, the public
>> > should have access to the code.
>>
>>
>> "Should" and "the law" unfortunately, are two very different things. 
>> The public does have access to the publications these days, but not
>> the code, the reagents, the royalties generated from IP, etc.
> 
> I think future NSF grants contracts should be written so that
> everything is put in the public domain.
> 
> 

A lovely idea, but few universities and investigators would sign on the 
bottom line.


-- 
Scott
Reverse name to reply

moosedude skrev:
> I have a really big axe to grind about this one!

Get in line...

> I sometimes think people just publish papers, not to inform others, but
> just to say "look, look what I've done!"

No, not even that.

The purpoose of academic publishing these days, is not to
communicate new insight, not to promote own activity, but to
go through certain ceremonial motions in order to obtain an
academic degree or tenure in a university.

The main point is that the grad students should have a certain
number of articles published inside a certain time frame, so their
universities can graduate them and claim reimbursement for
the commercial product the degree is.

That's all there is to it:

- The student needs to publish in order to get the degree.
- The university needs the student to graduat in order to
  claim financial reimbursements.
- No one care about any readers of the journals where these
  papers are published. The papers are not intended to be
  useful, they are intended as items in the authors'
  publication lists. The more items on that list, the better.

Rune