Mac, Members, Tutorials
comments 51

Visuals for Sonar Festival + VDMX to Unity3D [Tutorials]

sonar-15

Numbers, a record label originating from Glasgow, had the opportunity to showcase artists at this year’s Sonar Festival in Barcelona. The artists Redinho, Spencer, Deadboy, Jackmaster, and Lory D each had a spot in the set. Adam Rodgers, co-founder of Numbers and Remote-Location approached Thomas Traum, Adam Finlay and myself to create live visuals for the show. We compared a number of technologies and frameworks– OpenFrameworks, Cinder, Unity3D, Cocos2d, WebGL, Processing, and good ol’ Flash. Doing something in 3D was a priority, and since we only had a 2-3 week timeline, the decision really came down to what would be the fastest to develop.

Despite my own lack of experience with the platform, Unity3D stood out as the clear choice. Unityscript, essentially their modified Javascript, ended up being quick to learn (particularly if you’ve done JS or Flash in the past.) Alternatively, Unity supports C# and Boo. It was only a matter of days before I was churning out decent prototypes for possible scenes. In addition, Tommi and both Adams could use Cinema4D as a quick way to contribute assets to the project. Unity’s native inclusion of 3D rigidbody support ended up saving us days of work. Rather than having to rely on an external library, out of the box 3D objects can simply have the attribute toggled on. Mass, friction, directional and angular velocity vectors– it’s all there. Many of the visuals were driven simply by sending the volume straight into the velocity values.

The app runs as a standalone Mac / PC executable file. Audio comes in via OSC from VDMX which handles spectrum bracketing, gain, and smoothing. Have a look at the tutorial if you’d like more info about this setup. We met only once to discuss scene ideas, the rest sort of spilled out of “it would be cool if…” comments. Some scenes, like the gravity balls, were pretty much done in a few hours. Others like the ragdoll falling through space, which had 9 separate asset modes, took over a week to get right. The scene with falling extruded text was created literally the day of the show in the hotel room. In the end, I probably wrote less lines of code than in a standard Flash microsite.

In the end there were a dozen scenes, each with at least 3 cameras, along with various modes to toggle. The keyboard was literally packed with toggles and options. All this in about 2 weeks of work, not bad at all considering the product that came out of it. One additional benefit of using Unity is that now we have the opportunity to easily port it to iPhone or iPad with very little difficulty.

Credits: NumbersRemote-LocationThomas TraumAdam Finlay and Mike Tucker

 

 

A Brief Tutorial on Creating a Sound Visualizer with Unity3D

Note: These directions are directed toward Mac users.

Part I. Preparing the Audio for OSC

We’re going to use OSC to convert music into variables that we’ll use as our input for the visualizer. 1) Download and install Soundflower: http://cycling74.com/products/soundflower/

This reroutes audio inputs and outputs. Perfect for what we’re doing.

2) For now we’ll use the computer’s output as our input.

  •  Go to Sound in your System Preferences
  •  In the Output tab, select Soundflower (2ch) (see picture)

3) Download and install VDMX demo: http://www.vidvox.net/

The demo won’t let you save files, but that won’t be necessary for our project.

4) Download the example project files from github: https://github.com/miketucker/UnityAudioVisualizerTutorial

Direct link to zip file: https://github.com/miketucker/UnityAudioVisualizerTutorial/zipball/master

5) Open the .vdmx5 file with VDMX. Only use the other vdmx file if you happen to be running an old version…

6) Try playing music from any app (Spotify, iTunes, etc) and you should see the wave forms in the Audio Analysis 1 window.

7) Go into preferences, and ensure there is a OSC Output port like the one pictured below.

 

Congrats, we’re done setting up the audio. There’s a lot to explore in VDMX– but this is enough to get some interesting visuals.

 

Part II. The App

1) If you don’t already have it, go download Unity Indie (it’s free): http://unity3d.com/

There’s a few things in Pro version that we used, such as Depth of Field, Bloom effects, etc.

But none if that is really necessary for what we’re doing in this project.

2) Open the same folder we used earlier  ( https://github.com/miketucker/UnityAudioVisualizerTutorial )

3) Open the file ExampleOscApp/Assets/Scene1.unity, this should start up Unity3D with the first scene opened

4) In the Hierarchy window you’ll see a list of names, these are the objects inside the scene.

Each object can have scripts attached to them. I’ll briefly run through each object and the scripts attached.

Camera – It’s from this view that we’ll see the running app.

GUILayer, Flare Layer, Audio Listener: These are by default added to every camera. Don’t worry about them right now.

Script – Rotate Around: I wrote this, It makes the camera spin around the centre, it also goes up and down with a sine wave.

Capsule, Cube, Cylinder, and Sphere – These are 3D objects in the scene. We’re going to have them scale to the music.

Script – ScaleObject: This is where we get to have fun. Only 2 lines of code to get it scaling to our OSC music!  The public variable oscChannel is a number 0-3 based on our original 4 bars from VDMX. 0 is bass, 1 and 2 are midtones, and 3 is treble.

Point light – just a light to give the 3d objects some depth. OSCStuff – this connects OSC from VDMX to our Unity app.

Notice how the IP and ports match the ones we set in VDMX.

The code was found on this forum: http://forum.unity3d.com/threads/16882-MIDI-or-OSC-for-Unity-Indie-users/page2

5) Try running the scene by pressing the Play button. The shapes should bounce to the music!

Part III. Onwards and Upwards

1) Once you wrap your head around how Unity deals with project structures it’s time to move onto Scene 2.

2) This scene is the exact scene from our Sonar Festival project. Very simple to make, with some beautiful outcomes.

3) There’s some new things going on here:

• Keyboard Manager: I used this class to map keyboard keys to different functions

• Multiple cameras: These can be switched in playback by KeyboardManager

• Duplicating objects: the Dupe object is duplicated using the Icosahedron vertices

• Physics: Each sphere has a rigidbody which means it will bump into other spheres

• Changing materials in playback: If you hit the J, K, L, and semicolon keys in playback, the spheres will change colors. This is triggered by KeyboardManager and can be found in the AttractionObject class

4) During playback, jump back to VDMX and play around with those values, it’s a great way to control the visuals.

5) Thanks and hope you enjoyed the introduction!

— Here are some more images from the Sonar Festival:

  • Billy Corbin
    You need to login to view this content.