All of the visuals in the above video were created using NyArtoolkit for Processing. NyARToolkit is an augmented reality toolkit built with 100% pure Java. It is derived from ARToolkit-2.72.1. Like Processing itself it’s open source and free! In this tutorial you will learn how to use it to place computer generated imagery correctly onto real world footage. To do this in real-time NyArtoolkit uses markers – black and white images – to determine the three-dimensional position and orientation in the real world. Most likely you will have seen something like this before, but now you will learn how to do it yourself using freely available software.

All right so let’s start with the general setup. What do you need to get started?

Continue reading.... Augmented Reality with Processing [Tutorial, Processing]

The AntiMap by Trent Brooks is an Open Source creative toolset for recording and visualising your own data. The project currently consists of a phone utility application (AntiMap Log – iOS/Android) for data capture, and a couple of web/desktop applications (AntiMap Simple (Javascript) and AntiMap Video (oF)) for post analysis and data visualisation.

The mobile application uses the GPS and Compass sensors on your phone to extract and store the following data: latitude, longitude, compass direction, speed (kph), distance (kms), time (milliseconds), and finally an input field for optional points of interest. Recorded data is saved to a CSV file at 30 FPS with filenames matching the date & time they were created. Eg: ‘150811_1230_08.csv’, was created on the 15th August 2011, at 12:30pm and 8 seconds. For Android users, files can be retrieved from the ‘AntiMap’ folder on the SD card. For iPhone users, files can be retrieved from your device in iTunes/Apps/AntiMap Log Documents (just under the ‘File Sharing’ heading). The mobile applications are free and Open Source, with Processing (Android) and Openframeworks (iPhone). Source code is provided in hope that users can learn from and find interesting ways to visualise their data.

Continue reading.... AntiMap [oF, Android, iPhone, Javascript, Processing]

Latest by Joshua Nimoy is this synaesthetic, interactive musical experience provides six original modes in which the player may produce music.

Advanced configuration is possible by editing the contents of the bundled settings.yml. By changing the values of that file, you can control the window properties, startup behaviour, user interface detail, and OSC network parameters. When set up on multiple machines, Bandwidth’s ‘grid’ mode will broadcast OSC messages and most other modes will receive.

Continue reading.... Bandwidth [openFrameworks, Mac, Windows, Sound]

Using code that Zach Lieberman posted on the oF forums which allows you to show the pixels directly below the OF app window in a texture, Jesus Gollonet created this little experiment which converts all content below the window into dithered image. This allows you to move the window around the screen exploring the dithered space in color image/video/etc.

Continue reading.... Realtime Dithered Screen Capture [openFrameworks]

Last week The Mill unrevealed their rear projected, 5’x 3’ interactive touch screen which is an interactive visualization of their portfolio (spanning 20 years) developed in Cinder. What makes this screen unique is extensive effort behind design and software development by the NY Digital team comprising Andrew Bell and an incredible group designers, flash developers and digital artists. For those that may be unaware, Andrew Bell is the main person behind Cinder, powerful, intuitive toolbox for creative programming, previously an in-house tool at the Barbarian Group where Andrew worked, now open sourced and developed by Andrew as part of the Mill’s R+D. We got a chance to to ask Andrew about the project including some making-of details and  features we will see in future releases of Cinder, including the forthcoming Cinder Timeline API – tweening library.

Continue reading.... The Making of the Mill Touch [Cinder]

Latest by Justin Windle is this ‘Super Recursion Toy’ created using Canvas. He writes:

Revisiting one of my favorite topics – recursion. This canvas based tool allows you to explore branching algorithms & rendering techniques, as well as save a snapshot of your creations. It’s also open source, which is always nice…

Continue reading.... Super Recursion Toy [Javascript]

Plask is a programming environment for multimedia and computational design. Created by Dean McNamee, Plask was built objC, V8 Javascript Engine, Skia rasterization library, NodeJS, Cocoa and OpenGL, FreeImage and Syphon.

Currently available for Mac OSX only. The download includes the “PlaskLauncher.app” and a number of basic examples to get you started.

Released under the BSD license, source is available on github.com/deanm/plask
Credits: code and ideas from: Nicolas Garcia Belmonte, Thatcher Ulrich, Ryan Alexander and Marcin Ignac

-

Projects below: Roskilde Festival Activity by Marcin Ignac and Native Instruments Razor by Dean McNamee.

Muze is the latest creation of Teaugue Labs, a collective operating alongside Teague, a long established industrial design agency and a name behind products for companies such as samsung and microsoft. The device itself is a musical instrument that ‘plays with you’ and aims to provoke a two way dialog between musicians.

The device reads a palette of notes that it can in-turn interpret and compose into various rhythms and phrases that are strung together to form something musical. The user can then influence these strings of notes and rhythms to create entirely new compositions using a simple collection of 8 triggers/knobs which are manually inserted. No single knob controls a single function, but rather a blend of functions derived from it’s rotation.

The team explains the thinking:

A couple of us started talking about the state of musical instruments, digital music creation, and how so much of it buckles under the weight of heavy user interfaces and the desire for more knobs, buttons and faders. What if we were to create a device that sings to you and has its own musical inclinations, yet can also engage in a two way dialog with another musician? Not something that can be controlled as much as be guided and influenced – and as a result guides and influences the user.

But Muze also has its own desire to explore and will continually improvise on the melodies it creates with you. It is out of this ability for it to self-create that Muze becomes a partner and not just an instrument. For instance, we have played with it and then left it to play over lunch. When we return it has come-up with something completely new, yet derivative. Sometimes what Muze creates is enjoyable, sometimes not. At which point you give Muze a little nudge and it creates something new.

All of the code and circuits are open source. You can check out the Arduino code and Eagle circuit schematic on the site. The team is planning to make it more musical, robust, and simple and would love to hear your thoughts, suggestions.

Project Page

Designed and developed by Ivan Safrin, Polycode is a new free, open-source, cross-platform framework for creative code. You can use it as a C++ API or as a standalone scripting language to get easy and simple access to accelerated 2D and 3D graphics, hardware shaders, sound and network programming, physics engines and more.

On top of the core C++ API, Polycode offers a Lua-based scripting system with its own set of compilation tools. The Lua API mirrors the C++ API and can be used to easily create prototypes and even publish complete applications to multiple platforms without compiling C++.

Polycode is made possible by Lua, Freetype, libpng, PhysicsFS, JSONCpp,tinyxml, Ogg Vorbis, Box2D, Bullet Physics and many people and projects which inspired the effort.

More information at polycode.org and github. For full list of features see here.

For more tools, see our new tools page.

–

The color of space by Ivan Safrin using Polycode.

One idea is to use color as a simple, abstract representation of the personal perception of geographical location. My original plan was to map New York City in such a manner, but collecting this kind of information locally seemed like an insurmountable task, so the data was collected via the internet for the entire country using a google form posted to reddit.com, with zip codes serving as geographical anchors to place data into context.

“MNDR – Cut Me Out” Music video.

Official music for MNDR’s Cut me out directed by Timothy Saccenti. Polycode and the Polycode Kinect module were used to capture and render live 3D data from a Microsoft Kinect device for this music video.

Would you like to create what you see in those videos? Well, read on! Because in this article I will show you how you can do just that using Processing and Toxiclibs. As Processing’s biggest open source collection of libraries, Toxiclibs can assist you in areas like geometry, physics, math and color. With so much code candy for the taking, the libs can still be a bit daunting for many people, especially Processing beginners. That’s why – in addition to great functionality and documentation – clear and inspiring examples on how to use the library are so important. Fortunately the collection of code examples bundled with the libs is growing steadily. I hope my examples can add to that and be helpful to those learning how to use this wonderful collection of code which is shared and continuously developed by Karsten Schmidt.

I’ve already shared the two code examples from the first video on my blog. As the video shows these concern creating, picking and dragging polygon shapes (example 1) and the destruction of voronoi tesselated circles (example 2). The full source code and a more detailed explanation of those two examples can be found HERE. In this follow-up I will share the source code for the two brand new examples you see in the second video. This time I’m venturing a little deeper into the physics capabilities of Toxiclibs, more specifically the VerletSpring2D class (example 3) and we will explore a whole new area of the libs, namely the color library (example 4). All of the examples are commented as much as possible. So by running them and looking through the code, you should be able to understand what’s going on. The rest of this blog post can be considered additional background information ranging from general description to specific pointers. Note (06/05/2011): Karsten came up with some useful suggestions to further improve the code, which of course I was happy to apply. This explains why the visuals when running the code may differ ever so slightly from what you see in the movie.

Example 3: The Infinite Rope

What’s better than a rope? Exactly. An INFINITE rope! This example demonstrates both the creation and efficient removal of particles, springs and behaviors. The three pillars of the physics system. Specifically it uses the VerletSpring2D class, which can connect two VerletParticles in space. For simplicity it’s kept 2D, but everything with regard to physics in Toxiclibs also has a 3D equivalent. Let me describe the way it works and some of the specific choices and solutions. When the mouse is dragged a new particle is created and connected to the last one, effectively making a digital rope. Release the mouse to start a new rope. Push behavior is added to each particle to make it look a little more realistic. For aesthetic reasons, the color of every segment is determined by the direction of each spring.

The most important part of the sketch however, is the code that removes off-screen objects. This is absolutely imperative to keep things running smoothly. Let me elaborate on two specific aspects with regard to the removeOffscreen() function. First, it’s running backwards through the for loop! This is because we are removing things from the list. Meaning the list is getting shorter while you are going through it. Therefore you need to go backwards to prevent problems and to make sure you cover every item in the list. Second, notice that I remove behavior(i+1) for particle(i). The reason is that the first behavior on the list is the gravity we added in setup(). Therefore the behavior of particle 1 can be found in position 2 of the behavior list and so on.

//  Toxiclibs Code Example: The Infinite Rope
//  by Amnon Owed (05/05/2011)
//  minor refactorings by Karsten Schmidt (06/05/2011)

import processing.opengl.*;

import toxi.physics2d.behaviors.*;
import toxi.physics2d.*;
import toxi.geom.*;
import toxi.color.*;

VerletPhysics2D physics;
VerletParticle2D prev;

int continuous,current; // variables to create a new continuous line on each mouse drag
 
void setup() {
  size(1280,720,OPENGL);
  physics = new VerletPhysics2D();
  // add gravity in positive Y direction
  physics.addBehavior(new GravityBehavior(new Vec2D(0,0.1)));
  // set the stroke weight of the line
  strokeWeight(2);
}
 
void draw() {
  background(255);
  // update all the physics stuff (particles, springs, gravity)
  physics.update();
 
  // draw a line segment for each spring and change the color of it based on the x position
  for(VerletSpring2D s : physics.springs) {
    // map the direction of each spring to a hue
    float currHue=map(s.b.sub(s.a).heading(),-PI,PI,0,1);
    // define a color in HSV and convert into ARGB format (32bit packed integer)
    stroke(TColor.newHSV(currHue,1,1).toARGB());
    line(s.a.x,s.a.y,s.b.x,s.b.y);
  }
 
  // remove stuff that is off the screen to keep things running smoothly ;-)
  removeOffscreen();
}
 
void removeOffscreen() {
  // remove off-screen springs
  for (Iterator<VerletSpring2D> i=physics.springs.iterator(); i.hasNext();) {
    VerletSpring2D s=i.next();
    if (s.a.y > height+100 || s.b.y > height+100) {
      i.remove();
    }
  }
 
  // remove off-screen particles & behaviors
  for (int i=physics.particles.size()-1; i>=0; i--) {
    VerletParticle2D p = physics.particles.get(i);
    if (p.y > height+200) {
      physics.removeParticle(p);
      ParticleBehavior2D b = physics.behaviors.get(i+1);
      physics.removeBehavior(b);
    }
  }
}
 
void mouseDragged() {
  // create a locked (unmovable) particle at the mouse position
  VerletParticle2D p = new VerletParticle2D(mouseX,mouseY);
  p.lock();
  // if there is at least one particle and this is the current continuous line
  if (physics.particles.size() > 0 && continuous == current) {
    // get the previous particle (aka the last in the list)
    VerletParticle2D prev = physics.particles.get(physics.particles.size()-1);
    // create a spring between the previous and the current particle of length 10 and strength 1
    VerletSpring2D s = new VerletSpring2D(p,prev,10,1);
    // add the spring to the physics system
    physics.addSpring(s);
  } else {
    current = continuous;
  }
  // unlock previous particle
  if (prev!=null) {
    prev.unlock();
  }
  // add the particle to the physics system
  physics.addParticle(p);
  // create a forcefield around this particle with radius 20 and force -1.5 (aka push)
  ParticleBehavior2D b = new AttractionBehavior(p,20,-1.5);
  // add the behavior to the physics system (will be applied to all particles)
  physics.addBehavior(b);
  // make current particle the previous one...
  prev=p;
}
 
void mouseReleased() {
  if (prev!=null) {
    prev.unlock();
  }
  continuous++;
}

Example 4: NamedColors

Aesthetically somewhat similar, but technically completely different, this example is meant to demonstrate how to use TColors in general and NamedColors in particular. If Vec2D/Vec3D is the heart of the geometry lib, then you could say TColor is the heart of the color lib. It’s the basis for much greater possibilities such as color ranges, themes and gradients. But to grasp this part of Toxiclibs, I think it’s best to start with a basic example. For this I chose the NamedColors since they are less abstract than working with numbers alone. In the color portion of Toxiclibs there is a list of 143 NamedColors that you can use. They have names like azure, darkturquoise, lavender, orange and last but not least peachpuff. When working with TColors it’s important to remember that you need to convert them into something that can be used in a Processing fill() or stroke() function. In this example you can see that every time the color is actually used, it’s converted into a packed ARGB int using the toARGB() function.

So let me walk through the sketch real quick. In setup() all the names are loaded into an ArrayList of strings for the purpose of sorting them alphabetically. Running the sketch presents you with the full color palette. I’ve applied some ZoomLensInterpolation to bring out the selected color (mouseX) and made it move up and down with the user (mouseY). There are some checks to make sure both the name and it’s background are kept within the boundaries of the screen. The right mouse button changes the background color, while the left mouse button creates a colorWorm at the mouse position. Press any key to toggle the palette on/off, the mouse functionality will keep working. The colorWorm is basically a list of up to 25 points, a direction and a certain color. It starts at the mouse position and then moves randomly, adding new points along the way. Since the directional change is limited to 30 degrees, it will generally keep going into a certain direction instead of wriggling around the same spot. To make it a little smoother all the points are loaded into a Spline2D which is then subdivided. From the vertices that come out, the line is drawn.

//  Toxiclibs Code Example: NamedColors
//  by Amnon Owed (05/05/2011)
//  minor refactorings by Karsten Schmidt (06/05/2011)

import processing.opengl.*;
import toxi.geom.*;
import toxi.color.*;
import toxi.math.*;

ArrayList <String> names = new ArrayList <String> ();
ArrayList <ColorWorm> worms = new ArrayList <ColorWorm> ();

ZoomLensInterpolation zoomLens = new ZoomLensInterpolation();

boolean showColorPalette = true;
int selectedColorID;

// screen center
Vec2D center;

// background color (readonly colors can't be modified)
ReadonlyTColor bg;

void setup() {
  size(1280, 720, OPENGL);
  center = new Vec2D(width/2, height/2);
  // create a list of all the Toxiclibs NamedColors
  names = NamedColor.getNames();
  // sort it alphabetically
  Collections.sort(names);
  textFont(createFont("SansSerif", 28));
  // set zoom lens to a dilating characteristic
  // setting the first parameter to a negative value will create a bundling effect
  zoomLens.setLensStrength(0.45, 1);
  // set the background color to deepskyblue
  bg = NamedColor.getForName("deepskyblue");
}

void draw() {
  // convert the bg color into ARGB color format (32bit packed integer)
  background(bg.toARGB());

  // run through all the worms (backwards cause we are also removing some from the list)
  for (Iterator<ColorWorm> i=worms.iterator(); i.hasNext();) {
    ColorWorm w = i.next();
    // if the worm's last point is 'off the screen' remove the worm
    // distanceToSquared() is faster than distanceTo() since it avoids
    // the square root calculation and we don't need precise values here...
    if (w.points.get(0).distanceToSquared(center) > 640000) {
      i.remove();
    } 
    else {
      // otherwise update and display the worm
      w.update();
      w.display();
    }
  }

  // set the zoom location based on the normalized mouseX (0.0 .. 1.0 interval)
  float normX=(float)mouseX / width;
  // interpolate focal point to new mouse position (15% step per frame)
  zoomLens.setLensPos(normX, 0.15);
  // determine the selected color based on mouseX
  // by finding which color area contains mouseX
  float focalX=zoomLens.interpolate(0, width, normX);
  for (int i=0, num=names.size()-1; i<=num; i++) {
    float x=zoomLens.interpolate(0, width, (float)i/num);
    float x2=zoomLens.interpolate(0, width, (float)(i+1)/num);
    // select color if focalX is between x and x2
    if (focalX >= x && focalX < x2) {
      selectedColorID=i;
      break;
    }
  }

  // toggle the color palette
  if (showColorPalette) {
    drawColorPalette();
  }

  if (mousePressed) {
    // Create worms with the LEFT mouse button
    if (mouseButton == LEFT) {
      Vec2D mouse = new Vec2D(mouseX, mouseY);
      ReadonlyTColor c = NamedColor.getForName(names.get(selectedColorID));
      worms.add(new ColorWorm(mouse, c));
      // Change the background color with the RIGHT or MIDDLE mouse button
    } 
    else {
      bg = NamedColor.getForName(names.get(selectedColorID));
    }
  }
}

// Press ANY key to toggle the color palette
void keyPressed() {
  showColorPalette = !showColorPalette;
}

class ColorWorm {
  List <Vec2D> points = new ArrayList <Vec2D> ();
  Vec2D direction;
  TColor c;

  ColorWorm(Vec2D origin, ReadonlyTColor c) {
    // at the origin point (mouseX,mouseY)
    points.add(origin);
    // create a copy of the readonly color for later manipulation
    this.c = c.copy();
    // create a random direction
    direction = Vec2D.randomVector();
  }

  void update() {
    // every second frame (not too fast, not too slow)
    if (frameCount % 2 == 0) {
      // create a new point given the last point's coordinates
      Vec2D p = points.get(points.size()-1).copy();
      // rotate the direction randomly somewhere between -30 and 30 degrees
      direction.rotate(radians(random(-30, 30)));
      // create a movement vector in that direction with a random magnitude between 15 and 30
      Vec2D move = direction.getNormalizedTo(random(15, 30));
      // move the point in that direction and with that distance
      p.addSelf(move);
      // add the new point to the list
      points.add(p);
    }

    // truncate at 25 points (remove the oldest point)
    while (points.size () > 25) {
      points.remove(0);
    }
  }

  void display() {
    // need at least 3 points to construct a spline
    if (points.size()>2) {
      // create Spline2D from the points
      Spline2D s = new Spline2D(points);
      // subdivide it by 8 into a list of vertices
      List <Vec2D> vertices = s.computeVertices(8);
      noFill();
      strokeWeight(2);
      // draw a line through all the vertices
      beginShape();
      for (int i=0,num=vertices.size()-1; i<=num; i++) {
        Vec2D v = vertices.get(i);
        // the position in the list determines the transparency of the segment
        c.setAlpha(map(i, 0, num, 0, 1));
        // convert the color into ARGB color format (32bit packed integer)
        stroke(c.toARGB());
        vertex(v.x, v.y);
      }
      endShape();
    }
  }
}

void drawColorPalette() {
  noStroke();

  // display all the colors over the width of the screen
  for (int i=0,num=names.size()-1; i<=num; i++) {
    // determine the color swatch's position & width based on
    // it's relative position and the zoom location (mouseX)
    float x = zoomLens.interpolate(0, width, (float)i / num);
    float x2 = zoomLens.interpolate(0, width, (float)(i+1) / num);
    // convert the color into ARGB color format (32bit packed integer)
    fill(NamedColor.getForName(names.get(i)).toARGB());
    // move the colors vertically with mouseY
    rect(x, mouseY-15, x2-x, 30);
  }

  // get the name of the selectedColor
  String name = names.get(selectedColorID);
  float ascent = textAscent();
  float textwidth = textWidth(name);
  // keep the text and it's background fill within screen boundaries
  float x = min(mouseX, width-textwidth-8);
  float y = min(mouseY + 52, height-4);
  // draw a white text background
  fill(255);
  rect(x, y-ascent-4, textwidth+8, ascent+8);
  // draw a black text
  fill(0);
  text(name, x+4, y);
}

That concludes this round of code sharing. For all things Toxiclibs go to toxiclibs.org. A description of how to install contributed libraries for Processing can be found here. If you would like to know if and how Toxiclibs can help you with your project, but are unsure of where to start, I suggest asking for help in the Processing forum. There are quite a few people over there (including Karsten himself sometimes) who can help you out with advice and maybe even code examples. So good luck and get creative! ;-)