brain of mat kelcey...

intro

video synthesis and manipulation looks super fun but video synthx hardware is very expensive :(

so i was wondering, could we convert video to audio and then use eurorack for glitching things instead?

yes! but with one big caveat; bandwidth! audio bandwidth is measured in kHz whereas video is more commonly Mhz.

so if we want to serialise video to audio and back we have to put up with a pretty low resolution and framerate

data flow

let's consider the following data flow

• capture "video" on a raspberry pi at 320x240 at 5FPS
• resize to 128x75 and send via SPI to tiliqua
• encode frames as audio on tiliqua and send out to eurorack at 192 kHz
• read back audio from eurorack and decode back to frames
• send frames out to HDMI resized to 1024x600 at 25.175 Mhz

demo and code

this all works surprisingly well!

code on github