For a few years now I’ve been recording the PSNS’s Curious Minds lecture series with the resultant videos making their way to Youtube.
A couple of years ago I got into audio and hifi a bit more – mostly for home-cinema purposes, upgrading the telly from a 28″ monitor to a more respectable unit and swapping the soundbar (ewww, hahaha) for a proper amp and speakers.
Of course, merely passively absorbing cinema wasn’t enough, and I already have a regular supply of audio for the PSNS videos needing editing, so I went one bigger and learned my way around Ardour, a free & open-source DAW. Non-destructive editing certainly has its moments.
I even upgraded my audio recording gear (oh wait, yes, I got into sound recording for a while… 🤪), buying an ambisonic microphone (ie one that records the entire sphere of audio at all times, allowing me to synthesize the microphone direction and pickup pattern in processing after the event – ideal for moving a lecturer from beside the podium to centre-mono for the duration of their talk and then simulating a stereo split facing the audience for the subsequent Q&A session).
Podcasts
For the past couple of years, podcasts have become a staple listening experience whilst out on long road-trips (which tend to happen most weekends). One of my favourites is the Nature podcast – keeping up with science news and stuff. They’re quite engaging – worth listening to with as much non-driving concentration as possible. After a while, one of the things I noticed is that all episodes finish with credits “produced by, …sound designed by” – at which point there would be a gentle crescendo and the ambient background noise would become somewhat more apparent… like, hang around, there was ambient background? How long has that been going on subconsciously?
Generating “Music”
I’m thinking the PSNS lectures would benefit considerably from better designed audio – nothing too dramatic or imposing, but a subtle subconscious ambient bed of sounds. One thing I don’t have time – or much talent 😉 – for is sitting down composing 1.5hr of music only for it to be relegated to the background channel at -30dB. So, we need a way of automating the accompaniment. There are at least 3 ways of doing this:
- recording long background noises – leaving a few microphones around the back garden, some woods, near a waterfall, that kind of thing – but it imposes too much of a context on the audio that might not be to all listeners’ tastes;
- using granular synthesis (sampling the audio waveform into hundreds of tiny bits and rearranging them at random);
- somehow sampling and generating actual “music” in a way compatible with the source audio, as long as it chooses to go on
Aeolian Understory
So I asked a few LLMs and came up with a strategy. If we analyze the spectrum of the incoming audio, we can choose one or two prevailing frequencies; coupled with chord patterns suitable for ambient music (lots of suspensions and diminished chords), we can generate as much music as we like by saying the pattern is chosen randomly, moving slowly over time, and the detected frequencies must be present in it somewhere, with such things as inversions and choice of octave being less important as long as the resultant notes don’t jump around too much.
So I set to and had my LLM of choice vibe-code me a VST3 plugin. Nothing too complex. A couple of days’ work, on and off.
And here we are: a new github repository for Aeolian Understory, my first attempt writing any kind of audio / DSP plugin.
It comes in two halves:
- on the left, a regular granular synthesizer – you get to choose how many grains to use, their size and density and so on, and how much to smear them by. It mostly transforms regular speech into belching frogs…
- on the right half, the harmony analyzer and generator – it samples prominent frequencies (within a permitted range) and maps them to chord patterns as above (again, within a permitted range), with a choice of outputs: built-in sine, square or sawtooth wave (good for testing), or “midi” which enables an entire output MIDI controller channel.
Both halves have a mixer, for controlling the blend from dry to wet (ie how much the filter retains the original signal).
Usage
The best way I’ve found to use this is to retain the granular synth at low mixture to bulk-out the audio – it’s almost like a form of distortion – and then using Ardour or one’s DAW of choice to route the MIDI output to a separate MIDI track with a full synthesizer on board. By default, using Generic MIDI Synthesizer, the plugin targets violins, cellos and double-bass, giving the effect of a string ensemble, but there’s nothing stopping you using SurgeXT and warm synth pads or any number of other virtual instruments, to taste.
For the subtle bed effect, fade the MIDI track/buses much quieter – maybe -30dB or so – and implement auto-ducking against the primary audio channel so the speech “wins”. It might also help to add an EQ strongly favouring the bass <150Hz and a little bit of treble >6kHz but avoiding the midtones between.
Grab it from github and enjoy! 😊