The Living Thing / Notebooks :

Music software frameworks

and programming languages, for music

Usefulness: 🔧 🔧 🔧
Novelty: 💡
Uncertainty: 🤪
Incompleteness: 🚧

Kaspa König:

Disco ears by Kaspar König
Disco ears by Kaspar König

Nothing sounds quite right? Roll your own audio! Emphasis here is on quickly prototyping things; What’s the fastest way to try a new algorithm, etc. Good enough for installations, not necessarily for shipping to punters. (Some here are potentially production-grade, though, e.g. Juce.)

General

The Audio Programmer is a blog and podcast about general audio programming stuff.

Whole languages

Audio-specific programming languages. Why do people invent new DSLs for audio? This is a terrible idea. Anyway, there are some.

tl;dr If you must use a musical DSL, Csound might not be hip, but it’s the one that avoids the more horrible pathologies of its bitrotten competitors. You’ll miss some sequencing capabilities of supercollider, but not the debugging experience, or the inconvenient GPL licensing. Possibly overtone or Tidalcycles could get you there, if you want a more maintained language? Licensing is still tricky though.

For a satisying rant/overview see Michael Gogins list which includes a few I missed

Supercollider

Supercollider, dealt with elsewhere is an audio application and programming language. It comprises two parts, a backend synth scsynth, which is really good,and a front-end sequencer sclang, which looks good but upon closer inspection is a mess.

Fortunately, because of its modular structure you can use the synthesis part of supercollider, scsynth, which is pretty good, with some other languages than the rickety front-end, sclang.

Note that all of these share a common defect, which is that because it is purely GPL’d, it it legally difficult to release real software using it or put it on iphones etc.

Sonic Pi

Supercollider+ruby=sonicpi

Sonic pi is a flagship livecoding music and general coding education language. Vexingly, because it’s so damned friendly it’s hard to find any concrete specs of API docs. How do I even know it’s ruby? I inspect the code from the examples. Then I confirmed by deducing from the manual developer instructions. But the word “ruby” is not mentioned anywhere. There are some powerful ideas in here, like the live sequencing loops, but it’s hard to excavate the advanced instructions you need for advanced usages, and the FX presets will annoy the crap out of you real fast.

overtone

Supercollider+clojure=overtone

The archetypal hipster FRP language, clojure, has a functional composition library to work around supercollider’s sclang pathologies, called overtone.

Tidalcycles

Supercollider+haskell=Tidalcycles.

Thanks be the excellent Alex McClean for Tidal, which still, IMO, does not get sequencing right, but does good at getting a subset of sequencing sensible. See Tidal.

foxdot

Supercollider+python=foxdot

Foxdot is a great hack, even though it’s not my own preferred style. Discussed under art python.

supercolliderjs

supercollider+javascript=supercolliderjs

supercolliderjs is a pragmatic workaround for sclang oddities from one of the supercollider core developers. The alternative means of javascript audio. This is very attractive if you want a client-server infrastructure and JS guis. However, the community is small and the hacks are fiddly so I spent most of my time with this thing just being confused.

You need to link up a lot of messy bits to make it go.

Csound

Csound is another supercollider competitor with more open licensing, and more sustained development. It’s useful enough that it has its own page. The especially good features are that it’s small, liberally licensed, and designed for clean foreign function interfaces, and thus it works seamlessly with other languages, unlike supercollider which eschewed that by design.

Chuck

chuck is a recent competitor to supercollider; still not that actively developed, surely in part because their source-management is AFAICT innocent of all innovations in collaborative source code since the 90s, so contributing is too tedious for anyone to bother. Get it to together and get a public github URL, Princeton hermits.

DSLs

SOUL

SOUL (source) is a mini-language and API for local compilation of DSP algorithms, including to browsers etc. BAcked by susbstantial industry interests

Faust

Faust is a functional language (in fact a DSL) thoughtful enough to address a lot of the qualms I have around niche languages, i.e. it is a DSL which compiles DSP code to other mainstream languages, such as C++ and javascript; as it generates normal code in real programming languages I listed it here so as to avoid slandering it by association. It has its own page now because it is complex and useful enough.

Reajs

ReaJS is a plugin library from Reaper DAW that allows custom VST design using their weird custom language. cul de sac.

Audio libraries for mainstream programming languages

Wait what? You can just use a normal programming language with a lovely IDE and a sensible debugger, and still make music?

JuliaAudio

JuliaAudio does audio processing in Julia

STK

STK is a C++ library implementing various synthesis algorithms in a bareback style, with a special emphasis on physical modelling algorithms for virtual instruments.

Macimillian

maximilian is Mick Grierson’s C++ library for doing audio graphs, and seems to be a pragmatic choice for many artists. (Direct source link)

Juce

Juce seems to be the cross-platform Audio-GUI of choice and can host faust code; so that’s nice. I wonder if it can plug into maximilian?

Raffael Seyfried wrote the best tutorial for this that I have seen, while developing his marvelous plugin GRNLR

Beads

Beads brings realtime audio to java. Here’s an intro. It’s an excellent no-fuss way of creating musical audio rapidly. Recommended if you handle java. Disclaimer: I work with the creator of this library author, Ollie Bown, and am therefore biased.

OpenAL

Light spatialization is the specialty of the gaming-oriented open-source OpenAL

Misc python

Python options listed under python audio

PRotoplug

protoplug looks cool;

But few venture into this ghetto.

Gstreamer

gstreamer is a generic multimedia pipline library that seems to pop up in lots of neat projects.

Patchers

Graphical dataflow programming languages. A specific kind of DSL, with a visual function graph interface.

For the record, I hate programming using visual dataflow, although I do like visualizing my programs this way. The patchers that offer only a GUI workflow are arse-backwards for my work process, forcing you to click a million times to do every simple thing. But they are famous, and seem simple to the sort of people who prefer clicking on 100 buttons instead of tapping 100 keys. These are the same kind of people who will drive their cars until the rims are on the road rather than take half an hour to pump up the tyres.

Some of the patchers avoid this – faust, and puredata (the latter is is pure patcher per default, but you can persuade it). Also supercollider and chuck are usually classed as patchers, but AFAICT patching style visualisation is broken for both these, so I don’t know what’s going on in this world anymore. Aside: this is being mainstreamed slightly outside of audio by projects like Luna, which approach pure functional languages in this way. Well, Haskell at least.

There is a giant hub of patcher enthusiasts at patchstorage.

MaxMSP

MaxMSP, the expensive, bloated, pretty, popular one, with crashy-but-convenient Ableton Live integration. It is now called Max, which makes it impossible to google.

Puredata

Puredata, the slim, clever open source alternative to MaxMSP, with the remarkable feature of running on smartphones. It’s neat, but there is much too much clicking involved in it in the default setup, and awful abstractions. Puredata by itself is too barebones to do anything interesting with; It’s more of a platform than an app. Get some higher-level abstractions e.g.

The entire raison d’être of PD got extra raisins with these next generation, free, open-source, smartphone-friendly apps: MobMuPlat and MobMuPlatEditor. (Pity about the name.)

MobMuPlat is a standalone iOS+Android app which hosts and loads from a list of available works. Creating your own work consists of two parts. First, creating a graphical user interface (GUI) with the MobMuPlat Editor (macOS and Java versions available); second, creating the audio engine using the graphical programming language Pure Data (PD). With both of these applications open, and data sent between the two, you can simulate the app behavior on your laptop/desktop. Once development is complete, just drag the two saved files into the “Documents” folder of iTunes, and they are uploaded to your iOS device and can be opened in the MobMuPlat app. (For android, get the files onto your device storage folder anyway you like, then open them in the MobMuPlat app).

MobMuPlat can do synthesis, sampling, MIDI, OSC, networking via local wifi, query and set hardware characteristics like tilt/compass/camera flash, display images and vector graphics, be used within AudioBus (iOS only), take joystick/gamepad input (Android only) and much more.

However, Csound also has this facility, so YMMV.

ecocSYSTEM

Monologx’s ecoSYSTEM is… am not sure.

Reaktor

Executable sound

Left-field entrant, the incredible open-source executable sound, which has an elegant approach – it pumps out demo-scene-style DSP executables.

There are lots more, some of ’em even more expensive than the usurious Reaktor. Everyone likes this way of doing things, and the punters seem happy to cough up cash for it too.

See also Faust, above, which technically fits on this category but Does It Right, in terms of having a real language as well as a dataflow visualisation, and oh p.s. being free, and incredible. However, it doesn’t have good instrument support, (voice allocation, polyphony, VSTi compilation targets) so I don’t actually use it; I merely admire it from afar.

Other

MrsWatson is useful: given a VST, a VST preset and a MIDI and/or audio file, it will render the output of said VST. Command-line. Open-source.

Refs

Lazzarini, Victor, Steven Yi, Joachim Heintz, Øyvind Brandtsegg, and Iain McCurdy. 2016. Csound: A Sound and Music Computing System. Springer.

McCartney, James. 2002. “Rethinking the Computer Music Language: SuperCollider.” Computer Music Journal 26 (4): 61–68. https://doi.org/10.1162/014892602320991383.

Orlarey, Yann, Dominique Fober, and Stephane Letz. 2009. “FAUST: An Efficient Functional Approach to DSP Programming.” https://hal.archives-ouvertes.fr/hal-02159014/.

Puckette, Miller. 1996. “Pure Data: Another Integrated Computer Music Environment.” Proceedings of the Second Intercollege Computer Music Concerts, 37–41.

———. 2007. The Theory and Technique of Electronic Music. Hackensack, NJ: World Scientific Publishing Co.

Puckette, Miller S. 1997. “Pure Data.” In ICMC.

Vercoe, Barry. 1986. “Csound.” The CSound Manual Version 3.

Vercoe, Barry, and Dan Ellis. 1990. “Real-Time CSound: Software Synthesis with Sensing and Control.” In ICMC, 90:209–11.

Wang, Ge, Perry R. Cook, and Spencer Salazar. 2015. “ChucK: A Strongly Timed Computer Music Language.” Computer Music Journal 39 (4): 10–29. https://doi.org/10.1162/COMJ_a_00324.

Wilson, Scott, David Cottle, and Nick Collins. 2011. The SuperCollider Book. The MIT Press.