"It's possible that our grandchildren will look at us and say 'You mean people used to listen to the same thing over and over again?'" - Brian Eno
This is a fairly simple start to a larger idea.
I attended a lecture in 2014 by Dr. Andy Farnell on Procedural Audio who spoke, in part, about the distinction between fixed and performance mediums (ie film vs stage, album vs concert). Making note of the fact that while a theatre performance has a fixed structure and the story envokes a mood, it also adapts itself to the space and time of the specific performance.
Though, undoutably, much of the draw of performance art is owed to community and social connection, I think there's a case to be made that some of the power of perfomance is in its subtle unpredictability.
While the world of popular cinematic storytelling is, at least in part, beginning to push itself out of a fixed format (Black Mirror: Bandersnatch / Neflix's growing library of interactive content) and the world of video gaming, which has long-touted interactive storytelling, is approaching cinematic realism, popular recorded music is still very much fixed.
Procedural music itself is not a new thing, the video game and contemporary composition communities have been exploring it for a long while (Steve Reich's It's Gonna Rain was recorded in 1965). But, as of yet, nothing seems to exist in the realm of song.
The, possibly obvious, solution that I would like to explore would be to record a song in such a way that you have some number (say 10) of each individual part (ie, 10 takes of the vocal, 10 of the drums, 10 of the guitar, etc). Then on play back, you choose a random selection of each part. On a simple song with 5 parts (Guitar, Vocals, Drums, Bass, Synth) this would yield 100,000 unique combinations.
Widespread internet accessibility and the popularity of streaming music could make this potentially very achievable.
So I started simply and created a test using SoX and PHP to combine the parts of a short piece into a new random composite file. And this is what can be found at the top of this page.
<?php
// Choose one of each of the other tracks
$rhythm = $rhythmArray[array_rand($rhythmArray)];
$piano = $pianoArray[array_rand($pianoArray)];
$synth = $synthArray[array_rand($synthArray)];
$clarinet = $clarinetArray[array_rand($clarinetArray)];
// Full filename
$newfile = create_unique_filename();
// Use SoX CLI to create the new mix
exec("sox -m $rhythm $piano $synth $clarinet Audio/$newfile");
?>
<audio src="Audio/<?php echo $newfile; ?>" preload="auto" />
The idea, however, isn’t satisfied with the proof of concept largely for its lack of portability. So I started looking into the Matroska container format. Since the Matroska format can contain any number of streamable tracks, it would be possible to package up the parts of the song into a distinct file.
So in early 2021, I started to work on expanding the idea with two Flutter-based desktop applications (here & here) to read and write Matroska files using FFmpeg. This second proof of concept worked as well but had two main issues:
- Visually it was hard to break out of the Google/Flutter style.
- As soon as the complexity of a song file moved beyond a very simple test, playback of the audio became very slow.
So a little while later, in mid-2022, I decided to address the first issue by replacing the flutter applications with ElectronJS applications in order to make use of the flexibility of CSS styling and, at the same time, decided to name the project after the Greek sea-god Proteus who represents mutability and is the root of the adjective 'protean'.
After making a first test of the authoring app in Electron, I found that, though the styling issues were much improved, the performance issues were not and, furthermore, the bundled application created by Electron was enormous.
So, after a little more research, I found Tauri which offers nearly everything that I was looking for with electron but with significantly improved performance. Since the Tauri backend is written in Rust, I began working on writing Rust-based tools to compile and playback my Matroska files / Proteus Audio (.prot) files as well as the Authoring and Playback applications.
There's still much to do with the project so, if you would like to follow along, you can keep tabs on this page or watch the github repos. If you'd like talk about the idea, feel free to give me a shout!