Forums

Is Raspberry PI 2 B suitable for a real-time audio effects application?

Started by Mauritz Jameson January 4, 2016
On 05/01/16 05:39, Les Cargill wrote:
> Half that for 7ish msec latency on a modest dual-core i5: > > http://us.focusrite.com/usb-audio-interfaces/scarlett-18i20 > > I doubt you'll use that as-is on a Pi.
If you want many cores (i.e. >10!) /known/ latency and /zero/ jitter, use the XMOS processors and boards. http://www.xmos.com/ http://www.digikey.co.uk/product-search/en?keywords=xmos
On 05/01/16 04:06, Mauritz Jameson wrote:

> I haven't made up my mind about that. I haven't heard about DietPI, so I'll look into that. Have you developed algorithms for real-time data processing for Raspberry PI? I am very interested in hearing about other people's experience with this board (in the context of how well it performs when it comes to processing acquired data in real time).
See http://www.xmos.com/applications/hi-res-audio
On Tuesday, January 5, 2016 at 12:33:31 AM UTC-5, Les Cargill wrote:
> angrydude wrote: > > On Monday, January 4, 2016 at 9:25:18 PM UTC-5, Mauritz Jameson > > wrote: > >> I am about to start working on a project which involves real-time > >> audio processing for an audio effects application. The application > >> is supposed to acquire and process audio from 8 individual audio > >> input channels at a sample rate of 48kHz. The audio processing > >> algorithm generates audio for 8 individual audio output channels. > >> The audio sample format is 16-bit signed LPCM. Since it's for a > >> real-time audio effects application, latency is also important. If > >> audio is looped from input to output without any processing, the > >> round-trip latency should not exceed 12ms. > >> > >> The application should receive 256*8 input samples every 5.33ms > >> (via an audio callback), process those samples and transmit 256*8 > >> output samples (also via callback). The 256*8 samples are > >> received/sent in some kind of "interleaved" format. > >> > >> I am currently researching different hardware platforms and I was > >> wondering if the Raspberry PI 2 B could be a suitable choice. The > >> board itself seems to have a lot of horse power (900MHz Quad Core > >> ARM CPU), but it only has audio input. So my questions are: > >> > >> 1. Is the Raspberry PI 2 B suitable for real-time audio > >> processing? 2. If the answer to [1] is yes, then I would like to > >> know if there are any add-on audio boards for the Raspberry PI 2 B > >> on the market with 8 input channels and 8 output channels? The > >> board must support 48kHz sample rate. 3. If the answer to [1] or > >> [2] is no, which HW platform would you recommend? > >> > >> The reason why I'm looking at the Raspberry PI 2 B is the > >> attractive price, but there might be other types of boards on the > >> market which are more suitable for this kind of application. > >> > >> > >> Block Diagram: http://postimg.org/image/f9hfj6kf5/ > > > > On the second question the answer is definitely no: there are no > > add-on boards on the market with 8 ins and 8 outs - you'll have to > > build your own board > > > > The most popular Wolfson audio card for Pi has a combination of > > stereo inputs > > > > With that many ins-outs you are looking at professional (PC or Mac) > > audio interfaces at $1000 or more > > > > Half that for 7ish msec latency on a modest dual-core i5: > > http://us.focusrite.com/usb-audio-interfaces/scarlett-18i20 > > I doubt you'll use that as-is on a Pi. > > Les Cargill
That thing requires Mac or Windows PC. Raspberry Pi 2 can run a version of Windows but I doubt they'll work together. And the form factor is not suitable for small device: it's only suitable for e.g. writing VST audio effects plug-ins for full-featured DAW software like Cubase running on e.g. a Windows PC. OP might be better off with OEM USB data acquisition cards if he can get them for cheap - they are all too expensive. Other than that - build your own.
On 05.01.16 03.25, Mauritz Jameson wrote:
> I am about to start working on a project which involves > real-time audio processing for an audio effects application. > The application is supposed to acquire and process audio from 8 individual audio > input channels at a sample rate of 48kHz. The audio > processing algorithm generates audio for 8 individual audio output > channels.
The important question is: is a Raspi capable of 16 Audio streams in parallel? Which I/O hardware do you intend to use? And what is the acceptable latency.
> If audio is looped from input to output without any processing, the > round-trip latency should not exceed 12ms.
Forget it. The only option for this is SPI, but the data rate is probably too high.
> 1. Is the Raspberry PI 2 B suitable for real-time audio processing?
If you are smart enough to use the GPU even a RPi 1 can deal with audio processing. I made some tests with 2*64k FIR Filter kernels. No Problem so far.
> 2. If the answer to [1] is yes, then I would like to know if there are > any add-on audio boards for the Raspberry PI 2 B on the market with 8 input channels > and 8 output channels?
Forget it. I was not able to handle 2*24/196 full duplex over SPI with Wolfson Audio.
> 3. If the answer to [1] or [2] is no, which HW platform would you recommend?
Sorry, I never used anything but ordinary PCs for such purposes. The limiting factor of the RPi is the I/O subsystem and the latency. With appropriate buffers the operation is no big deal (assuming you don't do too extravagant things with the audio). But the real time response (without sample drops) is a problem. Marcel
angrydude wrote:
> On Tuesday, January 5, 2016 at 12:33:31 AM UTC-5, Les Cargill wrote: >> angrydude wrote: >>> On Monday, January 4, 2016 at 9:25:18 PM UTC-5, Mauritz Jameson >>> wrote: >>>> I am about to start working on a project which involves real-time >>>> audio processing for an audio effects application. The application >>>> is supposed to acquire and process audio from 8 individual audio >>>> input channels at a sample rate of 48kHz. The audio processing >>>> algorithm generates audio for 8 individual audio output channels. >>>> The audio sample format is 16-bit signed LPCM. Since it's for a >>>> real-time audio effects application, latency is also important. If >>>> audio is looped from input to output without any processing, the >>>> round-trip latency should not exceed 12ms. >>>> >>>> The application should receive 256*8 input samples every 5.33ms >>>> (via an audio callback), process those samples and transmit 256*8 >>>> output samples (also via callback). The 256*8 samples are >>>> received/sent in some kind of "interleaved" format. >>>> >>>> I am currently researching different hardware platforms and I was >>>> wondering if the Raspberry PI 2 B could be a suitable choice. The >>>> board itself seems to have a lot of horse power (900MHz Quad Core >>>> ARM CPU), but it only has audio input. So my questions are: >>>> >>>> 1. Is the Raspberry PI 2 B suitable for real-time audio >>>> processing? 2. If the answer to [1] is yes, then I would like to >>>> know if there are any add-on audio boards for the Raspberry PI 2 B >>>> on the market with 8 input channels and 8 output channels? The >>>> board must support 48kHz sample rate. 3. If the answer to [1] or >>>> [2] is no, which HW platform would you recommend? >>>> >>>> The reason why I'm looking at the Raspberry PI 2 B is the >>>> attractive price, but there might be other types of boards on the >>>> market which are more suitable for this kind of application. >>>> >>>> >>>> Block Diagram: http://postimg.org/image/f9hfj6kf5/ >>> >>> On the second question the answer is definitely no: there are no >>> add-on boards on the market with 8 ins and 8 outs - you'll have to >>> build your own board >>> >>> The most popular Wolfson audio card for Pi has a combination of >>> stereo inputs >>> >>> With that many ins-outs you are looking at professional (PC or Mac) >>> audio interfaces at $1000 or more >>> >> >> Half that for 7ish msec latency on a modest dual-core i5: >> >> http://us.focusrite.com/usb-audio-interfaces/scarlett-18i20 >> >> I doubt you'll use that as-is on a Pi. >> >> Les Cargill > > That thing requires Mac or Windows PC.
There exist Linux drivers for it. No clue if the drivers can run on a RasPi. The driver stack looks like it's ALSA so there may be hope. https://community.ardour.org/node/5751 ( that's for the 18i6; hopefully, the 18i20 is the same only moreso ) Caveat caveat caveat of course....
> Raspberry Pi 2 can run a version of Windows but I doubt they'll work together. > And the form factor is not suitable for small device: it's only suitable for e.g. writing VST audio effects plug-ins for full-featured DAW software like Cubase running on e.g. a Windows PC. > > OP might be better off with OEM USB data acquisition cards if he can get them for cheap - they are all too expensive. > Other than that - build your own. >
Could be, although those are usually lower sample rate. -- Les Cargill
Tom Gardner wrote:
> On 05/01/16 05:39, Les Cargill wrote: >> Half that for 7ish msec latency on a modest dual-core i5: >> >> http://us.focusrite.com/usb-audio-interfaces/scarlett-18i20 >> >> I doubt you'll use that as-is on a Pi. > > If you want many cores (i.e. >10!) /known/ latency > and /zero/ jitter, use the XMOS processors and boards. > > http://www.xmos.com/ > http://www.digikey.co.uk/product-search/en?keywords=xmos
Way cool, Tom. Thanks for the link. -- Les Cargill
On 05/01/16 23:46, Les Cargill wrote:
> Tom Gardner wrote: >> On 05/01/16 05:39, Les Cargill wrote: >>> Half that for 7ish msec latency on a modest dual-core i5: >>> >>> http://us.focusrite.com/usb-audio-interfaces/scarlett-18i20 >>> >>> I doubt you'll use that as-is on a Pi. >> >> If you want many cores (i.e. >10!) /known/ latency >> and /zero/ jitter, use the XMOS processors and boards. >> >> http://www.xmos.com/ >> http://www.digikey.co.uk/product-search/en?keywords=xmos > > > Way cool, Tom. Thanks for the link.
It is, isn't it. It may or may not surprise you that many of the key technologies in the XMOS processors were created by Prof David May - who was the lead architect of the Transputer and Occam.
Tom Gardner wrote:
> On 05/01/16 23:46, Les Cargill wrote: >> Tom Gardner wrote: >>> On 05/01/16 05:39, Les Cargill wrote: >>>> Half that for 7ish msec latency on a modest dual-core i5: >>>> >>>> http://us.focusrite.com/usb-audio-interfaces/scarlett-18i20 >>>> >>>> I doubt you'll use that as-is on a Pi. >>> >>> If you want many cores (i.e. >10!) /known/ latency >>> and /zero/ jitter, use the XMOS processors and boards. >>> >>> http://www.xmos.com/ >>> http://www.digikey.co.uk/product-search/en?keywords=xmos >> >> >> Way cool, Tom. Thanks for the link. > > It is, isn't it. > > It may or may not surprise you that many of the key > technologies in the XMOS processors were created by > Prof David May - who was the lead architect of the > Transputer and Occam. >
Ah, interesting. What that needs is about 4 Lightpipe interfaces and some sort of simple control protocol to connect over TCP/IP to a control surface/knob farm, laptop or tablet. Throw in a software system to reuse VST plugins at the source code level and you might have something. Of course the big plugin guys would probably never bother. Maybe they would; Waves might sell a lot of plugins to bar bands for something like that. I'm not that fond of the Presonus offerings; something like this with a semi-open-source software basis might take over the MI world. -- Les Cargill
On Tuesday, January 5, 2016 at 8:09:43 PM UTC-5, Les Cargill wrote:
> Tom Gardner wrote: > > On 05/01/16 23:46, Les Cargill wrote: > >> Tom Gardner wrote: > >>> On 05/01/16 05:39, Les Cargill wrote: > >>>> Half that for 7ish msec latency on a modest dual-core i5: > >>>> > >>>> http://us.focusrite.com/usb-audio-interfaces/scarlett-18i20 > >>>> > >>>> I doubt you'll use that as-is on a Pi. > >>> > >>> If you want many cores (i.e. >10!) /known/ latency > >>> and /zero/ jitter, use the XMOS processors and boards. > >>> > >>> http://www.xmos.com/ > >>> http://www.digikey.co.uk/product-search/en?keywords=xmos > >> > >> > >> Way cool, Tom. Thanks for the link. > > > > It is, isn't it. > > > > It may or may not surprise you that many of the key > > technologies in the XMOS processors were created by > > Prof David May - who was the lead architect of the > > Transputer and Occam. > > > > Ah, interesting. > > What that needs is about 4 Lightpipe interfaces and some sort of > simple control protocol to connect over TCP/IP to a control > surface/knob farm, laptop or tablet. > > Throw in a software system to reuse VST plugins at the source code level > and you might have something. Of course the big plugin guys would > probably never bother. Maybe they would; Waves might sell a lot > of plugins to bar bands for something like that. > > I'm not that fond of the Presonus offerings; something like this with a > semi-open-source software basis might take over the MI world. > > -- > Les Cargill
*Throw in a software system to reuse VST plugins at the source code level* Don't those poor developers of the most successful VST plugins try hard to hide and obfuscate source code to make a buck ? Doesn't help much of course - as soon as someone makes a few six figures on some plugin it gets cracked and voila.. Then they try to use USB dongles but those things suck big time
angrydude wrote:
> On Tuesday, January 5, 2016 at 8:09:43 PM UTC-5, Les Cargill wrote: >> Tom Gardner wrote: >>> On 05/01/16 23:46, Les Cargill wrote: >>>> Tom Gardner wrote: >>>>> On 05/01/16 05:39, Les Cargill wrote: >>>>>> Half that for 7ish msec latency on a modest dual-core i5: >>>>>> >>>>>> http://us.focusrite.com/usb-audio-interfaces/scarlett-18i20 >>>>>> >>>>>> I doubt you'll use that as-is on a Pi. >>>>> >>>>> If you want many cores (i.e. >10!) /known/ latency >>>>> and /zero/ jitter, use the XMOS processors and boards. >>>>> >>>>> http://www.xmos.com/ >>>>> http://www.digikey.co.uk/product-search/en?keywords=xmos >>>> >>>> >>>> Way cool, Tom. Thanks for the link. >>> >>> It is, isn't it. >>> >>> It may or may not surprise you that many of the key >>> technologies in the XMOS processors were created by >>> Prof David May - who was the lead architect of the >>> Transputer and Occam. >>> >> >> Ah, interesting. >> >> What that needs is about 4 Lightpipe interfaces and some sort of >> simple control protocol to connect over TCP/IP to a control >> surface/knob farm, laptop or tablet. >> >> Throw in a software system to reuse VST plugins at the source code level >> and you might have something. Of course the big plugin guys would >> probably never bother. Maybe they would; Waves might sell a lot >> of plugins to bar bands for something like that. >> >> I'm not that fond of the Presonus offerings; something like this with a >> semi-open-source software basis might take over the MI world. >> >> -- >> Les Cargill > > *Throw in a software system to reuse VST plugins at the source code level* > > Don't those poor developers of the most successful VST plugins try hard to hide and obfuscate source code to make a buck ?
Beats me. I don't see the source of any plugins. The idea is to provide a "VST host" interface for the new thing that VST plugin makers should be able support to by just recompiling and shipping binaries.
> Doesn't help much of course - as soon as someone makes a few six figures on some plugin it gets cracked and voila.. > Then they try to use USB dongles but those things suck big time >
-- Les Cargill