memset or not ?

Jake wrote:
>> It is one of these kernel programming functions, like bcopy(),
>> that nobody simulating a filter has any business using.
>>
>> It fills a given range of memory with a constant.
> 
> Ok, but _why_ ? I am just asking because I want to learn so I can explain
> to the next "lamer" why he/she shouldn't use it. I mean what's the point of
> just accepting postulates without understanding the reasons behind.

The reason why I don't like memset/memcpy in DSP code is that it breaks
type safety. I often have various different fract types (1.15, 4.28,
etc.). I can easily catch accidental mistakes (using a debugging version
with each type defined as an individual struct) when the code reads
       for (i = 0; i < N; ++i)
          dest[i] = src[i];
but not when the code just stomps around the raw memory.

Other than that, memset is surely fine to initialize big buffers upon
system start. And it's definititely better than code using unreadable,
inefficient countdown loops.


  Stefan

On Sep 15, 4:23 pm, glen herrmannsfeldt <g...@ugcs.caltech.edu> wrote:
> Tim Wescott <t...@seemywebsite.com> wrote:
>
> (snip)
>
> > '83?!?  Wow -- I'm moving up!  Of course, that's when it found it's way
> > into a book.
> > I remember the first time I thought of some cool and 'unique' innovation
> > in sampled data systems, only to find it in the literature -- for
> > implementation on vacuum tubes.
>
> Better than Cooley and Tukey who found out, years later, that their
> algorithm had been published by Gauss in 1805.  (Without looking
> it up, I believe 1805 is before vacuum tubes.)

I will never forget the look on my PhD advisor's
(Oscar Buneman) face in 1965 when he learned that
Cooley and Tukey had published Gauss' Algorithm.
Oscar had been using the algorithm since the 1930s.

Quite a bit of his voluminous body of work involved
fast methods of computer simulation via particle-field
interactions (microwave electronics, plasma physics,
gravitation, weather, fluids, etc). The FFT was used
to convert space-time to wavenumber-time. The fast
computation of the field at time t was then used to
calculate particle positions at time t+dt, etc.

Some of the readers may be more familiar with his
later work on the Fast Hartley Transform.

Searching on Google Web

Buneman Hartley  3,440 hits

Greg

On 16/09/2010 17:54, Stefan Reuther wrote:
> Jake wrote:
>>> It is one of these kernel programming functions, like bcopy(),
>>> that nobody simulating a filter has any business using.
>>>
>>> It fills a given range of memory with a constant.
>>
>> Ok, but _why_ ? I am just asking because I want to learn so I can explain
>> to the next "lamer" why he/she shouldn't use it. I mean what's the point of
>> just accepting postulates without understanding the reasons behind.
>
> The reason why I don't like memset/memcpy in DSP code is that it breaks
> type safety.


Type safety is a general concern to programmers in any context - it is 
not something exclusive or even particularly special to dsp. The void 
pointer is the beast that compromises type safety - not any one library 
function in particular.


Richard Dobson

Richard Dobson  <richarddobson@blueyonder.co.uk> wrote:

>On 16/09/2010 17:54, Stefan Reuther wrote:

>> The reason why I don't like memset/memcpy in DSP code is that it breaks
>> type safety.

>Type safety is a general concern to programmers in any context - it is 
>not something exclusive or even particularly special to dsp. 


Sure, but memset() harks back to a non-type-safe style of kernel
programming.   So it's reasonally correct to say DSP programmers
should not be using it, in the modern world.

I would go further and say DSP programmers should be using C++
containers and not plain C arrays anymore, but there is probably
less consensus on that.

Steve

"Jake" <jake@nospam.invalid.com> wrote in message 
news:4c90d33f$0$50442$14726298@news.sunsite.dk...
>I have a workbuffer with values that needs to be re-arranged, 
>so...initially...I did it like this:
>
> for (i = (N - 1); i >= 0; i--)
> {
>   workbuffer[Q * i] = workbuffer[i];
>   memset(&workbuffer[(Q * i) + 1], 0, (Q-1) * sizeof(int16));
> }
>
> but I was told not to use memset. I don't know exactly why I am not 
> allowed to use memset in a loop.

Perhaps they meant it's not to be used in that particular algorithm?
Anyway, I've come across a couple of instants where a loop is faster than 
the library function.
I don't know about modern compilers, as this was a few years ago.
But if you have 128 bit registers, or any special media registers like that, 
then maybe creating your own memset could be quicker. As you can blat as 
much memory as possible with each write. Although you'll need to make sure 
you align your allocated memory to 16 bytes.

Cheers,
Dave.

VelociChicken <turn@invalid.com> wrote:
(snip)

> Anyway, I've come across a couple of instants where a loop is 
> faster than the library function.

> I don't know about modern compilers, as this was a few years ago.
> But if you have 128 bit registers, or any special media registers 
> like that, then maybe creating your own memset could be quicker. 
> As you can blat as much memory as possible with each write. 
> Although you'll need to make sure you align your allocated 
> memory to 16 bytes.

Well, in that case memset() should store enough to get to the
next 16 byte boundary, loop for as many such blocks as possible,
then store bytes to the end.

I believe that S/370 MVCL is supposed to do that internally
(maybe in microcode).   The IA32 manuals specify that REP STOS
is the fastest way to set large blocks of memory, even though
it seems to do it a byte at a time.  It might be that there is
internal optimization for REP STOS and aligned blocks.

As I said, memset() is intended to be efficient for large blocks, 
but maybe less efficient for smaller blocks.

-- glen

Richard Dobson wrote:
> On 16/09/2010 17:54, Stefan Reuther wrote:
>> Jake wrote:
>>> Ok, but _why_ ? I am just asking because I want to learn so I can
>>> explain to the next "lamer" why he/she shouldn't use it. I mean
>>> what's the point of just accepting postulates without
>>> understanding the reasons behind.
>>
>> The reason why I don't like memset/memcpy in DSP code is that it breaks
>> type safety.
> 
> Type safety is a general concern to programmers in any context - it is
> not something exclusive or even particularly special to dsp.

True, of course. But it happens that DSP is the field where I most often
face the problem of zeroing or copying large arrays :-) (And where I'm
needing the raw iron performance, and cannot hide everything behind nice
C++ classes that do everything for me.)


  Stefan

robert bristow-johnson wrote:


> 
> Vlad, am i a lamer because i didn't know about memset()?

Sure. As well as anyone who got involved in this stupid thread.

VLV

Stefan Reuther  <stefan.news@arcor.de> wrote:

>Richard Dobson wrote:

>> Type safety is a general concern to programmers in any context - it is
>> not something exclusive or even particularly special to dsp.

>True, of course. But it happens that DSP is the field where I most often
>face the problem of zeroing or copying large arrays :-) (And where I'm
>needing the raw iron performance, and cannot hide everything behind nice
>C++ classes that do everything for me.)

True, but the incidence of not practicing type-safety when one
should is far higher than the incidence of not being able to do
so for performance reasons.

Steve

In article <4c90d33f$0$50442$14726298@news.sunsite.dk>,
Jake <jake@nospam.invalid.com> wrote:
>I have a workbuffer with values that needs to be re-arranged,
>so...initially...I did it like this:
>
>for (i = (N - 1); i >= 0; i--)
>{
>   workbuffer[Q * i] = workbuffer[i];
>   memset(&workbuffer[(Q * i) + 1], 0, (Q-1) * sizeof(int16));
>}

Serves you right for using workbuffer in two different ways.

/* The safety provided by comparing two sizes here is unsurpassed */
workbuffer = malloc(  Q*N*sizeof(int16) );
memset(workbuffer, 0, Q*N*sizeof(int16));

/* Here you need to be careful */
for (i = 0; i<N; i++ )
{
   workbuffer[Q * i] = workbuffer_input[i];
}

It all depends on trade offs, copying words one by one may be
both safe and fast and have a small enough foot print.

>Comments about not using memset in a loop are also welcomed.

Some will argue to not use memset at all. I wouldn't care a bit.
In the end you may look at the assembler output.
I would care about what *that* has to say.

>
>Thank you.
>

Others argue to use C++:

Any safety provided by the type system of C++ is offset
by the unreliability of implementing an inherently
ill-defined language and the problems to cope with its
complexity.

(Hell, I should save that for my thesis.)

Groetjes Albert

--
-- 
Albert van der Horst, UTRECHT,THE NETHERLANDS
Economic growth -- being exponential -- ultimately falters.
albert@spe&ar&c.xs4all.nl &=n http://home.hccnet.nl/a.w.m.van.der.horst