The Living Thing / Notebooks :

Audio synthesis in python

sometimes you just want to use the wrong tool for the job.

Python’s audio analysis toolkit is impressive; see machine listening. However, its synthesis is mediocre. Which is not to say terrible. Although I might, later. But in a pinch, you can get audible feedback from that audio analysis.

DIY, bareback style.

Tedious, but do-able. The best example I know is paulstretch, which creates a lovely phase vocoder from raw FFT, reasonably compactly. For real-time things this is not easy, nor are many basic audio techniques easy even offline.


Not quite trivial.

Here is a comparison of options. Summary:

Reading files

If you want to read MP3, audioread is simple and easy, but it breaks in opaque ways when you use it concurrently and has crappy error handling (everything is NoBackendError). This is the librosa system.

If you don’t care about MP3 then SoundFile does the job, but it is hard to compile.

if you want to load everything and do it really fast, but have a tricky time with trapping the cause of errors, you can invoke ffmpeg from python, which is very fast and does various FX processing for free. This is what I now do.

Offline editing/FX

Notable non-abandoned projects include pydub, amen and pippi.


pippi is a little idiosyncratic but seems to do synthesis quick-n-easy with reasonable optimisation using cython. The documentation and packaging are a mess, though. I think it might even do realtime? Heavily developed.

brew install libsndfile
pip install pippi


Amen has strong machine-listening tools, integrating librosa, but its effects are weak sauce - just cutting and pasting audio around in time with nice crossfades. It does cute re-edits, but nothing else. The idea of keeping the analysis metadata attached to the samples is nice though.

brew install libsndfile1  libav-tools
pip install amen


pydub has a lots of audio DSP, effects and editing procedures, but only basic audio analysis. Also, weirdly, it aims to be pure python (i.e. no numpy) which makes some things embarrassingly slow, and means there is a lot of re-implementing numpy. So it runs everywhere, but not great anywhere. (maybe this would be fast with a jit-compiled python?)

PyWorld Vocoder

More specialised: pyworld is a python wrapper for a speech-specific anlaysis-resynthesis method WORLD, by Masanori Morise. This will really only work on things that are very much like solo human voice.

Fancy realtime audio libraries


pyo (github) is a python audio processing framework by Olivier Bélanger. Supports python 2.7 and 3.5-3.6. It wants to run a wxPython gui, which is its own kind of inconvenience in turn, as it conscripts you into a toolkit war. Nonetheless it can do some neat stuff, and wxPython GUI is pretty good, so if you don’t mind a mildly opinionated library, this is a nice thing to work with. It’s a one-man shop, indicates very impressive productivity on the part of its creator. This guy has more or less reimplemented the supercollider scsynth infrastructure.

It claims to be

a Python module written in C to help DSP script creation. Pyo contains classes for a wide variety of audio signal processing. With pyo, the user will be able to include signal processing chains directly in Python scripts or projects, and to manipulate them in real time through the interpreter. Tools in the pyo module offer primitives, like mathematical operations on audio signals, basic signal processing (filters, delays, synthesis generators, etc.), but also complex algorithms to create sound granulation and other creative audio manipulations. pyo supports the OSC protocol (Open Sound Control) to ease communications between softwares, and the MIDI protocol for generating sound events and controlling process parameters. pyo allows the creation of sophisticated signal processing chains with all the benefits of a mature, and widely used, general programming language.:

Here is an example

>>> s = Server().boot()
>>> s.start()
>>> wav = SquareTable()
>>> env = CosTable([(0,0), (100,1), (500,.3), (8191,0)])
>>> met = Metro(.125, 12).play()
>>> amp = TrigEnv(met, table=env, dur=1, mul=.1)
>>> pit = TrigXnoiseMidi(
>>> out = Osc(table=wav, freq=pit, mul=amp).out()

See also cecilia, a gui for pyo.

If you don’t want to use the default distribution by weird OS-specific installer packages which want to invade your system python installation, that is optional:

brew install liblo libsndfile portaudio portmidi --universal
git clone
cd pyo
python install --use-coreaudio --use-double

Note that you might still have to deal with some wxPython weirdness on OSX.


csound supports python – specifically embedding of and within python.


FoxDot runs actual supercollider scripts from python. (As opposed to pyo, which implements a synthesis server that Looks like supercollider. It comes with an IDE, which is a waste of time IMO, but many other worthwhile tricks, including a nice scheduler (see the docs) Rather than claiming to be a universal solution to audio, it’s a righteous hack that does some startling things very well and some other things not at all. A good start, creatively speaking.


( is a generic multimedia pipline library that seems to pop up in lots of neat projects. It happens to have [extensive python support() See Brett Virren’s tutorial.


audiolazy, by Danilo de Jesus da Silva Bellini, looks great for technical audio analysis and synthesis, although a bit clunky for, you know, synths. Intermittently updated.

Prioritizing code expressiveness, clarity and simplicity, without precluding the lazy evaluation, and aiming to be used together with Numpy, Scipy and Matplotlib as well as default Python structures like lists and generators, AudioLazy is a package written in pure Python proposing digital audio signal processing (DSP), featuring:


Surprisingly, GNURadio supports extensive optimized DSP for python using a high-performance compiled real time dataflow graph.


Real time

Real time audio

PyAudio provides Python bindings for PortAudio, the cross-platform audio I/O library. With PyAudio, you can easily use Python to play and record audio on a variety of platforms. PyAudio is inspired by:

pyPortAudio/fastaudio: Python bindings for PortAudio v18 API.

tkSnack: cross-platform sound toolkit for Tcl/Tk and Python.

Real time MIDI

Mido, iirc. TBC.