HoloDecks is an umbrella project for a number of studies which focus on transforming sound through different mediums. This particular part of the HoloDecks project uses software to transform sound into physical 3D printed objects, sound sculptures.
The base shape chosen for the visualisation is a disc which has always been a ubiquitous shape associated with music storage formats such as compact discs and vinyl records. The disc also acts as a convenient base layer that 3D elements can be printed on. Using a generative process, a 3D mesh form is created on the disc as a representation of the audio from a selected song.
A custom application made using openFrameworks loads in a sound file and analyses it to attain the shape of the audio from a song. This is simply like a audio equaliser, which breaks down the sound into a number of bars that bounce up and down to the music. ofxFFT is an addon developed for audio analysis and it allows me to control the sound values that are coming in, smoothing or thresholding etc. The application uses the audio values to generate a 3D mesh in real-time. openFrameworks has good mesh support, namely the ofMesh class has all the functions you need for generating openGL meshes and it can export meshes in the .ply format. Also worth mentioning is the easy to use ofxObjLoader addon which can export meshes in .obj format, ready for 3D printing.
The initial prototyping is done on the MakerBot Replicator 2 which we use all the time at the studio. The MakerBot has been great for rapid prototyping with surprisingly good quality and has allowed me to quickly print and test my 3D experiments. The layered printing process of the MakerBot does pose some constraints on the 3D design where overhangs in the 3D models become tricky to print without supports. So I’ve simplified the sound sculpture formula so the objects always grow upwards and taper off towards the top, which makes it is easy to print on the MakerBot. Once I’m happy with the MakerBot print, I upload it to Shapeways for professional printing using the SLS (Selective Laser Sintering) process. I find Shapeways prints to be of higher precision and quality but also a different aesthetic. There is something endearing about the MakerBots visible layers and I can see that in 10 years when 3D printing has transcended to a whole new level of supreme quality, people will be returning to this aesthetic in the same way that old school pixel graphics never get old.
Augmented Sound Sculpture.
In this part of the project I was interested in exploring how AR (augmented reality) can interact with 3D printed objects. Since proper 3D object tracking on mobile devices is still poor / non-existent, I used the HoloDecks label as a 2D flat marker which was glued to the bottom of the sound sculpture. A custom AR mobile app built using openFrameworks and ofxQCAR was then used to track the marker and map a virtual 3D model over the sculpture. This technique allowed to make it appear as if the physical sculpture was audio reactive, each triangle changing size and color in response to the music.