Python for DSP

On 03/24/2015 12:45 AM, Rob Gaddi wrote:
[...]
> Everything but math.  I used to use Octave for most of my numerical
> analysis stuff, but god is all the stuff around the numerical analysis a
> kick in the teeth.  Simple things like reading all the .csv files in a
> directory, inferring the sample rate from the file name, and graphing the
> resultant waveform force you into areas like file and string handling
> that Octave (and Matlab with it) are completely crippled in.
>

Sure, Matlab (and probably Octave) is mainly a tool for numeric / DSP
computations and simulations. But to me file and string handling are quite
OK. We even use it for a complete framework of C++ code generation, i.e.
Matlab code is prducing C++ code. Of course one could do that in Perl or
python as well. But overall we see it as advantage to have one tool 
everybody is familiar with for use as shell, for visualization, 
simulations, algorithm development (mainly Simulink),... Expensive? Yes, 
no doubt. But i see no real alternative overall, just for single 
applications which would result in a more fragmented development 
environment. May be OK, if you work for yourself but not in environments 
with double digit numbers of developers.
And of course Matlab does not run on any target hardware, it is used to 
develop the target hardware or programs running there!

Servus
Wolfgang

On Thu, 26 Mar 2015 16:14:32 -0400, Max <Max@sorrynope.com> wrote:

>On Mon, 23 Mar 2015 18:33:27 -0700 (PDT), Mark DeArman
><s.d.m@ieee.org> wrote:
>
>>On Sunday, March 22, 2015 at 7:09:00 AM UTC-7, Max wrote:
>>> Seems that Python is catching on for some DSP work. The recent
>>> Coursera course co-sponsored by Julius Smith III used Python rather
>>> than Matlab.
>>> 
>>> I've searched for DSP libraries but there's kind of a confusing
>>> jumble. Some are just using SciPy.  Does anyone know of specific
>>> Python libs that would extend SciPy's scope?
>>
>>I'm in the process of changing our "DSP" Post-Processing application over to a python/NumPy based scripting language interface from hard coded menu items. It was pretty easy to integrate Python 2.7.3 and a custom NumPy with  the Windows WPF front end interface and Intel C fast closed source DSP backend stuff.  I also put together a IPython test system to start experimenting with.  Right now I still use MATLAB for all prototyping, but I think the NumPy tools have come a long way, and you can't beat the licensing.
>
>Mark, you mentioned using an Intel C library in conjunction with the
>standard Python libs.  Is it DSP-oriented?  What does it do that the
>standard Python libs don't do?
> 

The Intel Math Kernel Library and the Intel compiler just offer CPU
targeting for optimization.  So you're going to see a significant
speed improvement for some types of DSP applications on certain CPUs.
Normally NumPy is built with MKL also.  It does two things when used
in combination with Python and NumPy.  For speed critical algorithms,
you can build them in C with MKL and then just return the result as a
NumPy array for further interactive processing in Python.  Second, it
can protect the source code for commercial algorithms.  The end-user
can use your algorithm interactively with in Python to perform some
processing task.

Mark

On Thu, 26 Mar 2015 21:06:46 -0700, Mac Decman
<dearman.mark@gmail.com> wrote:

>On Thu, 26 Mar 2015 16:14:32 -0400, Max <Max@sorrynope.com> wrote:

>>Mark, you mentioned using an Intel C library in conjunction with the
>>standard Python libs.  Is it DSP-oriented?  What does it do that the
>>standard Python libs don't do?

>The Intel Math Kernel Library and the Intel compiler just offer CPU
>targeting for optimization.  So you're going to see a significant
>speed improvement for some types of DSP applications on certain CPUs.
>Normally NumPy is built with MKL also.  It does two things when used
>in combination with Python and NumPy.  For speed critical algorithms,
>you can build them in C with MKL and then just return the result as a
>NumPy array for further interactive processing in Python.  Second, it
>can protect the source code for commercial algorithms.  The end-user
>can use your algorithm interactively with in Python to perform some
>processing task.
>
>Mark

Thanks for the info, Mark.  I'll look into that. Thanks.

On Thu, 26 Mar 2015 16:24:56 -0400, Max wrote:

> On Mon, 23 Mar 2015 23:45:34 +0000 (UTC), Rob Gaddi
> <rgaddi@technologyhighland.invalid> wrote:
> 
>>On Mon, 23 Mar 2015 16:46:19 -0400, Randy Yates wrote:
>  ...
>>> Or that GNU Octave doesn't?
> 
>>Everything but math.  I used to use Octave for most of my numerical
>>analysis stuff, but god is all the stuff around the numerical analysis a
>>kick in the teeth.
> 
> Rob, are you saying that Python's math libs (NumPy, SciPy) fall short in
> that respect?
> 

No no, that Octave's tools for things outside of numerical analysis fall 
short.  I find Python's math libs handle all of the math side of things 
as well as Octave ever did, and as fast as Octave ever did, while letting 
me take advantage of the fact that Python's language and standard library 
are infinitely superior to Octave's for all the ancillary stuff.

>>Simple things like reading all the .csv files in a directory, inferring
>>the sample rate from the file name, and graphing the resultant waveform
>>force you into areas like file and string handling that Octave (and
>>Matlab with it) are completely crippled in.
> 
> Yeah, I'm reminded of everything that good old Visual Basic was
> -supposed- to be back in the day (I really hated that language), but
> Python has structure, more elegant object-orientation, and of course the
> language and libs have a very pragmatic focus.  Reading CSV files is a
> good example.
> 

>>Python is NOT the be all and end all of languages.  The interpreter is
>>slow, the runtime enormous, and when people talk about using it in an
>>embedded environment my eyes roll hard enough to hurt.
> 
> I had just mentioned in another post that it runs on a lot of micros,
> like Raspberry Pi and Arduino. But maybe you're referring to
> DSP-oriented processors?
> 

The RPi is a bit of a different story; it's got the aggregate horsepower 
of every PC I owned for the first 2/3 of my life.  But yes, I'm talking 
Blackfins, or Cortex-Ms, or cute little power-sipping MSP430s.  The sorts 
of things that madmen keep trying to run Linux on due to abject terror 
that they might be forced to write a state machine.

>>And please don't get me started on the disaster that actually
>>distributing applications written in Python has turned out to be.
> 
> I've been wondering about that.  Again, I'm new to Python, but I
> understand that the compilers do a decent job.  Is that not your own
> experience?  Or maybe there's a significant runtime lib that needs to
> ship with them?

The compilers don't compile, the compilers bundle; you basically get a 
ball that's the interpreter and the runtime and (hopefully) all the 
libraries and either the source or the byte-code.  It's fine when all you 
need is things in the standard library, but bigger libraries (matplotlib 
notoriously) start having exciting corner cases.

I'm coming at this specifically from a Linux (Ubuntu) standpoint, where 
we wasted the better part of a month trying to get cx_Freeze to actually 
lock an entire package down, finding one place after another where the 
system needed to be massaged into submission until we finally realized 
that our result was more bailing wire and duct tape than steel, and was 
predicated on so many assumptions that there wasn't any point anymore and 
scrapped the entire idea.

There is now a dedicated Python Packaging Authority (https://www.pypa.io), 
because things have gotten so bad with various approaches spawning and 
forking and dying and being abandoned.  Hopefully by mid-2016 or so the 
situation will be better.  But right now it's a pretty glaring hole in my 
ability to recommend people to "just use Python".

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.