Forums

Python for DSP

Started by Max March 22, 2015
On 03/24/2015 12:45 AM, Rob Gaddi wrote:
[...]
> Everything but math. I used to use Octave for most of my numerical > analysis stuff, but god is all the stuff around the numerical analysis a > kick in the teeth. Simple things like reading all the .csv files in a > directory, inferring the sample rate from the file name, and graphing the > resultant waveform force you into areas like file and string handling > that Octave (and Matlab with it) are completely crippled in. >
Sure, Matlab (and probably Octave) is mainly a tool for numeric / DSP computations and simulations. But to me file and string handling are quite OK. We even use it for a complete framework of C++ code generation, i.e. Matlab code is prducing C++ code. Of course one could do that in Perl or python as well. But overall we see it as advantage to have one tool everybody is familiar with for use as shell, for visualization, simulations, algorithm development (mainly Simulink),... Expensive? Yes, no doubt. But i see no real alternative overall, just for single applications which would result in a more fragmented development environment. May be OK, if you work for yourself but not in environments with double digit numbers of developers. And of course Matlab does not run on any target hardware, it is used to develop the target hardware or programs running there! Servus Wolfgang
On Thu, 26 Mar 2015 16:14:32 -0400, Max <Max@sorrynope.com> wrote:

>On Mon, 23 Mar 2015 18:33:27 -0700 (PDT), Mark DeArman ><s.d.m@ieee.org> wrote: > >>On Sunday, March 22, 2015 at 7:09:00 AM UTC-7, Max wrote: >>> Seems that Python is catching on for some DSP work. The recent >>> Coursera course co-sponsored by Julius Smith III used Python rather >>> than Matlab. >>> >>> I've searched for DSP libraries but there's kind of a confusing >>> jumble. Some are just using SciPy. Does anyone know of specific >>> Python libs that would extend SciPy's scope? >> >>I'm in the process of changing our "DSP" Post-Processing application over to a python/NumPy based scripting language interface from hard coded menu items. It was pretty easy to integrate Python 2.7.3 and a custom NumPy with the Windows WPF front end interface and Intel C fast closed source DSP backend stuff. I also put together a IPython test system to start experimenting with. Right now I still use MATLAB for all prototyping, but I think the NumPy tools have come a long way, and you can't beat the licensing. > >Mark, you mentioned using an Intel C library in conjunction with the >standard Python libs. Is it DSP-oriented? What does it do that the >standard Python libs don't do? >
The Intel Math Kernel Library and the Intel compiler just offer CPU targeting for optimization. So you're going to see a significant speed improvement for some types of DSP applications on certain CPUs. Normally NumPy is built with MKL also. It does two things when used in combination with Python and NumPy. For speed critical algorithms, you can build them in C with MKL and then just return the result as a NumPy array for further interactive processing in Python. Second, it can protect the source code for commercial algorithms. The end-user can use your algorithm interactively with in Python to perform some processing task. Mark
On Thu, 26 Mar 2015 21:06:46 -0700, Mac Decman
<dearman.mark@gmail.com> wrote:

>On Thu, 26 Mar 2015 16:14:32 -0400, Max <Max@sorrynope.com> wrote:
>>Mark, you mentioned using an Intel C library in conjunction with the >>standard Python libs. Is it DSP-oriented? What does it do that the >>standard Python libs don't do?
>The Intel Math Kernel Library and the Intel compiler just offer CPU >targeting for optimization. So you're going to see a significant >speed improvement for some types of DSP applications on certain CPUs. >Normally NumPy is built with MKL also. It does two things when used >in combination with Python and NumPy. For speed critical algorithms, >you can build them in C with MKL and then just return the result as a >NumPy array for further interactive processing in Python. Second, it >can protect the source code for commercial algorithms. The end-user >can use your algorithm interactively with in Python to perform some >processing task. > >Mark
Thanks for the info, Mark. I'll look into that. Thanks.
On Thu, 26 Mar 2015 16:24:56 -0400, Max wrote:

> On Mon, 23 Mar 2015 23:45:34 +0000 (UTC), Rob Gaddi > <rgaddi@technologyhighland.invalid> wrote: > >>On Mon, 23 Mar 2015 16:46:19 -0400, Randy Yates wrote: > ... >>> Or that GNU Octave doesn't? > >>Everything but math. I used to use Octave for most of my numerical >>analysis stuff, but god is all the stuff around the numerical analysis a >>kick in the teeth. > > Rob, are you saying that Python's math libs (NumPy, SciPy) fall short in > that respect? >
No no, that Octave's tools for things outside of numerical analysis fall short. I find Python's math libs handle all of the math side of things as well as Octave ever did, and as fast as Octave ever did, while letting me take advantage of the fact that Python's language and standard library are infinitely superior to Octave's for all the ancillary stuff.
>>Simple things like reading all the .csv files in a directory, inferring >>the sample rate from the file name, and graphing the resultant waveform >>force you into areas like file and string handling that Octave (and >>Matlab with it) are completely crippled in. > > Yeah, I'm reminded of everything that good old Visual Basic was > -supposed- to be back in the day (I really hated that language), but > Python has structure, more elegant object-orientation, and of course the > language and libs have a very pragmatic focus. Reading CSV files is a > good example. >
>>Python is NOT the be all and end all of languages. The interpreter is >>slow, the runtime enormous, and when people talk about using it in an >>embedded environment my eyes roll hard enough to hurt. > > I had just mentioned in another post that it runs on a lot of micros, > like Raspberry Pi and Arduino. But maybe you're referring to > DSP-oriented processors? >
The RPi is a bit of a different story; it's got the aggregate horsepower of every PC I owned for the first 2/3 of my life. But yes, I'm talking Blackfins, or Cortex-Ms, or cute little power-sipping MSP430s. The sorts of things that madmen keep trying to run Linux on due to abject terror that they might be forced to write a state machine.
>>And please don't get me started on the disaster that actually >>distributing applications written in Python has turned out to be. > > I've been wondering about that. Again, I'm new to Python, but I > understand that the compilers do a decent job. Is that not your own > experience? Or maybe there's a significant runtime lib that needs to > ship with them?
The compilers don't compile, the compilers bundle; you basically get a ball that's the interpreter and the runtime and (hopefully) all the libraries and either the source or the byte-code. It's fine when all you need is things in the standard library, but bigger libraries (matplotlib notoriously) start having exciting corner cases. I'm coming at this specifically from a Linux (Ubuntu) standpoint, where we wasted the better part of a month trying to get cx_Freeze to actually lock an entire package down, finding one place after another where the system needed to be massaged into submission until we finally realized that our result was more bailing wire and duct tape than steel, and was predicated on so many assumptions that there wasn't any point anymore and scrapped the entire idea. There is now a dedicated Python Packaging Authority (https://www.pypa.io), because things have gotten so bad with various approaches spawning and forking and dying and being abandoned. Hopefully by mid-2016 or so the situation will be better. But right now it's a pretty glaring hole in my ability to recommend people to "just use Python". -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix.