Numbers, a record label originating from Glasgow, had the opportunity to showcase artists at this year’s Sonar Festival in Barcelona. The artists Redinho, Spencer, Deadboy, Jackmaster, and Lory D each had a spot in the set. Adam Rodgers, co-founder of Numbers and Remote-Location approached Thomas Traum, Adam Finlay and myself to create live visuals for the show. We compared a number of technologies and frameworks– OpenFrameworks, Cinder, Unity3D, Cocos2d, WebGL, Processing, and good ol’ Flash. Doing something in 3D was a priority, and since we only had a 2-3 week timeline, the decision really came down to what would be the fastest to develop.
The app runs as a standalone Mac / PC executable file. Audio comes in via OSC from VDMX which handles spectrum bracketing, gain, and smoothing. Have a look at the tutorial if you’d like more info about this setup. We met only once to discuss scene ideas, the rest sort of spilled out of “it would be cool if…” comments. Some scenes, like the gravity balls, were pretty much done in a few hours. Others like the ragdoll falling through space, which had 9 separate asset modes, took over a week to get right. The scene with falling extruded text was created literally the day of the show in the hotel room. In the end, I probably wrote less lines of code than in a standard Flash microsite.
In the end there were a dozen scenes, each with at least 3 cameras, along with various modes to toggle. The keyboard was literally packed with toggles and options. All this in about 2 weeks of work, not bad at all considering the product that came out of it. One additional benefit of using Unity is that now we have the opportunity to easily port it to iPhone or iPad with very little difficulty.
A Brief Tutorial on Creating a Sound Visualizer with Unity3D
Note: These directions are directed toward Mac users.
Part I. Preparing the Audio for OSC
This reroutes audio inputs and outputs. Perfect for what we’re doing.
2) For now we’ll use the computer’s output as our input.
- Go to Sound in your System Preferences
- In the Output tab, select Soundflower (2ch) (see picture)
3) Download and install VDMX demo: http://www.vidvox.net/
The demo won’t let you save files, but that won’t be necessary for our project.
4) Download the example project files from github: https://github.com/miketucker/UnityAudioVisualizerTutorial
Direct link to zip file: https://github.com/miketucker/UnityAudioVisualizerTutorial/zipball/master
5) Open the .vdmx5 file with VDMX. Only use the other vdmx file if you happen to be running an old version…
6) Try playing music from any app (Spotify, iTunes, etc) and you should see the wave forms in the Audio Analysis 1 window.
7) Go into preferences, and ensure there is a OSC Output port like the one pictured below.
Congrats, we’re done setting up the audio. There’s a lot to explore in VDMX– but this is enough to get some interesting visuals.
Part II. The App
1) If you don’t already have it, go download Unity Indie (it’s free): http://unity3d.com/
There’s a few things in Pro version that we used, such as Depth of Field, Bloom effects, etc.
But none if that is really necessary for what we’re doing in this project.
2) Open the same folder we used earlier ( https://github.com/miketucker/UnityAudioVisualizerTutorial )
3) Open the file ExampleOscApp/Assets/Scene1.unity, this should start up Unity3D with the first scene opened
4) In the Hierarchy window you’ll see a list of names, these are the objects inside the scene.
Each object can have scripts attached to them. I’ll briefly run through each object and the scripts attached.
Camera – It’s from this view that we’ll see the running app.
GUILayer, Flare Layer, Audio Listener: These are by default added to every camera. Don’t worry about them right now.
Script – Rotate Around: I wrote this, It makes the camera spin around the centre, it also goes up and down with a sine wave.
Capsule, Cube, Cylinder, and Sphere – These are 3D objects in the scene. We’re going to have them scale to the music.
Script – ScaleObject: This is where we get to have fun. Only 2 lines of code to get it scaling to our OSC music! The public variable oscChannel is a number 0-3 based on our original 4 bars from VDMX. 0 is bass, 1 and 2 are midtones, and 3 is treble.
Point light – just a light to give the 3d objects some depth. OSCStuff – this connects OSC from VDMX to our Unity app.
Notice how the IP and ports match the ones we set in VDMX.
The code was found on this forum: http://forum.unity3d.com/threads/16882-MIDI-or-OSC-for-Unity-Indie-users/page2
Part III. Onwards and Upwards
1) Once you wrap your head around how Unity deals with project structures it’s time to move onto Scene 2.
2) This scene is the exact scene from our Sonar Festival project. Very simple to make, with some beautiful outcomes.
3) There’s some new things going on here:
• Keyboard Manager: I used this class to map keyboard keys to different functions
• Multiple cameras: These can be switched in playback by KeyboardManager
• Duplicating objects: the Dupe object is duplicated using the Icosahedron vertices
• Physics: Each sphere has a rigidbody which means it will bump into other spheres
• Changing materials in playback: If you hit the J, K, L, and semicolon keys in playback, the spheres will change colors. This is triggered by KeyboardManager and can be found in the AttractionObject class
5) Thanks and hope you enjoyed the introduction!
— Here are some more images from the Sonar Festival: