Cinder, Featured, Tutorials
comment 1

Creating Instagram Kaleidoscope with Cinder – Tutorial

header copy

header

Cinder has the reputation of having a steep learning curve that’s difficult to overcome for programmers not familiar with C++. I’m not going to lie and tell you that there’s a shortcut that doesn’t require learning some C++, as well as some banging of heads against a wall, but I also assure you that curve isn’t as steep as it may seem. And, your head doesn’t have to have too many bruises to start creating some great visual effects.

I’d like to share a project that I created –- not just the final product, but how I was able to accomplish it. It’s an application written in Cinder that grabs photos from Instagram to create real-time animated kaleidoscopes. We’ll look at the technical details of the (simplified) version that is now included as a sample with Cinder (0.8.5+). If you want to follow along with the sample code, you can check it out on github.

The Kaleidoscope Math

Part of the appeal of a kaleidoscope is that it messes with your mind enough to be mesmerizing, but is actually quite simple. It’s really just a grid of equilateral triangles positioned so that each adjacent triangle is a mirror image of the one next to it. I started by drawing some sketches by hand to begin working out my technique. I began with one triangle, then drew reflected triangles around it, and subsequently built out the rest of the grid, quickly realizing that six triangles make a lovely, easy to manipulate hexagon.

sketch

To construct one hexagon, you draw six triangles from the same “origin”, rotating each one by 60º around the same pivot point. Every other triangle is flipped along its x axis to create a mirror image of the triangle adjacent to  it. Combined this makes a full 360º set of triangles.

hexagon2

This modular hexagon shape is easy to throw into a loop and repeat. The only thing you have to do to tile a set of hexagons is offset a column’s x position by 150% of one triangle’s width (a hexagon’s width equals the width of 2 triangles) and offset the y position of each hexagon by half of the hexagon’s height (1 triangle’s height).

hexagon-grid2

Here’s some code to explain the loops that create this grid of hexagons. TrianglePiece  is a custom class that draws each triangle.

// loop through the amount of columns you want
for( int i = 0; i < amtX; i++ ) {
    float startX = tri_width * 1.5f * i;

    // loop through the amount of rows you want
    for( int j = 0; j < amtY; j++ ) {
        // alternate the y position
        float startY = ( i % 2 == 0 ) ? ( tri_height * 2 * j - tri_height ) : ( tri_height * 2 * j );
        loop through the 6 triangles that make the hexagon
        for( int k = 0; k < 6; k++ ) {
            // because every piece is a mirror of the one next to it, every other has to be reversed on the x scale
            int scaleX = ( k % 2 == 0 ) ? 1 : -1;
            Vec2f scale( scaleX * tri_scale, tri_scale );

            // create the triangle piece and add it to a vector that holds references to them all
            // TrianglePiece constructor arguments:
            //  start: center point of the hexagon that this triangle piece is part of
            //  pt1, pt2, pt3: equilateral triangle points to construct the triangle
            //  60 * k: the rotation of the triangle in the context of the hexagon
            //  scale: the scale of the triangle 
            Vec2f start( startX, startY );
            TrianglePiece *tri = new TrianglePiece( start, pt1, pt2, pt3, 60 * k, scale );
            mTriPieces.push_back( tri );
        }
    }
}

We now have our vector of TrianglePieces, but how do we animate the kaleidoscope? It may not be obvious, but as it it turns out these triangles never actually move. The effect comes from “sliding” the triangle of texture coordinates we sample from.

This video illustrates how a triangular sample from the original image (on the right) animates to create the kaleidoscope effect (on the left).

If you take a look at the updateMirrors function, you’ll see an affine transformation matrix(MatrixAffine2f mtrx = MatrixAffine2f::identity();) which has some translating, scaling, and rotation being applied to it. The matrix then translates the initial triangle point positions into to the area that is to be reflected in all of the kaleidoscope triangle pieces.

// updateMirrors function
// define the initial equilateral triangle point positions
Vec2f mSamplePt1( -0.5, -( sin( M_PI / 3 ) / 3 ) );
Vec2f mSamplePt2( mSamplePt1.x + 1, mSamplePt1.y);
Vec2f mSamplePt3( mSamplePt1.x + ( cos( M_PI / 3 ) ), mSamplePt1.y + ( sin( M_PI / 3 ) ) );

// translate the points via an affine matrix
MatrixAffine2f mtrx = MatrixAffine2f::identity();
mtrx.translate( mSamplePt.value() );
mtrx.scale( mSampleSize );
mtrx.rotate( ( getElapsedFrames() * 4 ) / 360.0 ); // degrees, not radians

// convert the matrix positions into cartesion coordinates
mSamplePt1 = mtrx.transformPoint( mSamplePt1 );
mSamplePt2 = mtrx.transformPoint( mSamplePt2 );
mSamplePt3 = mtrx.transformPoint( mSamplePt3 );

// normalize the coordinates so that they are in the 0.0 - 1.0 range
mSamplePt1 /= mMirrorTexture->getSize();
mSamplePt2 /= mMirrorTexture->getSize();
mSamplePt3 /= mMirrorTexture->getSize();

// loop through all the pieces and pass along the current texture and it's coordinates
...
for( int i = 0; i < vec->size(); i++ ) {
    (*vec)[i].update( mMirrorTexture, mSamplePt1, mSamplePt2, mSamplePt3 );
    ...
}

Within TrianglePiece, you can see how the those passed texture coordinates are used to draw your triangle

// TrianglePiece::draw function
// mVertices = equilateral triangle points
// mTexVertices = passed texture coordinates
// draw the texture to the triangle
mDrawTex->enableAndBind();
gl::drawSolidTriangle( mVertices, mTexVertices );
mDrawTex->unbind();
glColor4f( 1.0, 1.0, 1.0, 1.0 );
glPopMatrix();

The Timeline API

The ci::Timeline is Cinder’s class for managing property animations or “tweens.” If you happen to have a background in Flash development (like me) the “tween” concept should be familiar. In this sample, I use the Timeline to animate everything that moves on screen. The Timeline is also used to set up the event sequencing for switching between photos.

To illustrate the Timeline’s tweening functionality, let’s use the TextRibbon class as an example. This class renders the ribbon containing the name of the Instagram user for our current photo. We animate its alpha value as it appears and disappears.

The basic syntax is as follows:

app::timeline().apply( &mCurAlpha, 1.0f, 0.4f, EaseOutQuint() ).delay( 4 );

Translated to English, this code basically says, “Hey Timeline! Animate the value of the mCurAlpha variable to 1.0 in 0.4 seconds, using an EaseOutQuint easing function, but don’t start for another 4 seconds”. The member variable mCurAlpha is defined in the header as a special cinder type ci::Anim<float> which treats the value as a float that a timeline is able animate and apply a tween to.

Let’s take a look at another example of where the Timeline is used, this time in TrianglePiece.cpp. Here we animate another alpha variable, and we make use of the optional startFn callback. This feature of the Timeline lets us add a function to “fire” when a tween is initiated. In this case, we’re asking it to call setVisible with the parameter true when the tween begins.

app::timeline().apply( &mAlpha, 0.0f, randFloat(0.2f, 0.7f), EaseInQuint() ).delay(0.5)
    .startFn( [&] { setVisible( true ); } );

Again, in English this says “Animate the mAlpha variable to a value of 0.0, over a random period of between 0.2 and 0.7 seconds, using the EaseInQuint easing function, but don’t start for 0.5 seconds. Also right before you start animating this I want you to call thiss setVisible() function, and pass it a parameter of true.”

You might notice that we wrote the body of the code in-line, using a C++11 lambda function. Lambdas are a powerful technique that some other languages have had for some time, but are relatively new in C++. In essence, lambdas let you write a nameless function to stand in for a std::function. Cinder allows you to use them a number of different places in its APIs. You can read more about lambdas in this article.

There’s actually quite a bit of sophisticated functionality available with properties like the startFn, which can be set per-animation. There are similar callbacks for events like when the tween updates or when it completes, among others.

To explain how the Timeline can be used to sequence events, let’s look at the sample’s image transitions. For every image that gets “kaleidoscoped”, there is a phase where you see the image in its original unaltered format. After a few seconds, it transitions back into a kaleidoscope of the next image in sequence and the process repeats. All of the timing and transitions in this app are managed by using the Timeline API.

Here’s how we set that up:

app::timeline().add( [&] { changePhase( 0 ); }, timeline().getCurrentTime() + delayOffset );
app::timeline().add( [&] { changePhase( 1 ); }, timeline().getCurrentTime() + delayOffset + MIRROR_DUR);

This would read as “Call changePhase and pass a value of 0, but wait delayOffset seconds to do so.” The second line does the same thing, but passes a value of 1 instead and with a longer delay, MIRROR_DUR, which signifies the amount of time the previous phase should be active for. With these 2 lines, the app is now set to go through each animation phase one time. After changePhase is called with 1 as the parameter, the app loads a new image to display, and the sequence repeats itself.

ConcurrentCircularBuffer

Another Cinder feature this sample uses heavily is ci::ConcurrentCircularBuffer. Essentially, a concurrent circular buffer is a container that makes it easy to manage an array of data in a thread-safe manner. It’s perfect for applications (like this one) that load data on a background thread, and then make that data available to the main thread. I used this in the InstagramStream class to store downloaded instagram data and images.

While threading is a separate topic, it’s an important concept for understanding the advantages of using the ConcurrentCircularBuffer. But put simply, threading can be described as the programming technique that enables multiple streams of code to execute simultaneously. This allows us (in our particular case) to download new images from Instagram in a background thread while we’re simultaneously drawing our kaleidoscope of the current image. You can see how easy it is to create the thread that handles loading the instagram data in the startThread function.

void InstagramStream::startThread(string url){
    mThread = make_shared<thread>( bind( &InstagramStream::serviceGrams, this, url ) );
}

Using make_shared<thread> does what it sounds like, it actually creates a shared pointer that references the new thread that handles the instagram API calls. The concept of shared pointers is another topic that deserves its own post, but you can read more about it here)

The main reason multithreaded programming is difficult is that exchanging and sharing data between threads requires some specialized techniques to ensure that the threads don’t step on each other’s toes. Fortunately, Cinder makes a common case, where one thread produces data and one thread consumes data, quite simple, with a built-in class called ci::ConcurrentCircularBuffer. It mimics the behavior of a normal queue (you push things on the front and pop things off the back), but behind the scenes it’s doing all of the nasty threaded synchronization for you.

In this example, the items in our buffer are instances of a custom Instagram class that holds data pertaining to each Instagram image.

To add that data to the buffer, we simply pushFront() it from the background thread:

mBuffer.pushFront( Instagram( userName, imageUrl, image ) );

If there’s no room in the buffer (ours has room for 10 images), then the pushFront() call automatically puts the background thread to sleep until there’s room again in the buffer. To pull data out of that buffer we call popBack().

Instagram result;
mBuffer.popBack( &result );

We call this from the app’s main thread when we want the next Instagram item in the buffer. That’s all there is to setting up a background thread to safely load data in Cinder.

JSON API

The built-in JsonTree class in Cinder handles parsing Instagram’s REST API.

Cinder makes it simple to load data from a url directly into a JsonTree object:

JsonTree( loadUrl( url ) );

The raw Instagram JSON data looks roughly like this:

"data": [
{
    "images": {
        ...
        "standard_resolution": {
            "url": "http://distilleryimage5.s3.amazonaws.com/1dd174cca14611e1af7612313813f8e8_7.jpg",
            "width": 612,
            "height": 612
        }
    },
    ...
    "user": {
        "username": "gregkepler",
        "full_name": "Greg Kepler"
    }
    ...
}

And here’s how the the JSON API extracts the username and image url:

string userName = (*resultIt)["user"]["username"].getValue();

// get the URL and load this instagram image
string imageUrl = (*resultIt)["images"]["standard_resolution"]["url"].getValue();

resultIt is the iterator for walking the JsonTree object. To get the userName string, this reads as “In the JsonTree object (resultIt), find the the node whose name is ‘user’. Once you have that, find the child node named ‘username’. Whatever the value of that node is, assign it to the userName variable.” Repeat for all of the JSON data you need and pass those values in when creating your new Instagram object.

The Instagram API

Here at The Barbarian Group, we’re big fans of Instagram as you might have seen with Screenstragram and GE’s Tumblr. Instagram is ideal to work with since the images tend to be pretty to look at, are consistent in resolution, and are plentiful in interesting content.

Their API allows for a variety of different queries from which you can grab images. Included in this sample is functionality to load popular images, images within a distance of a specific geographic location, and images containing a particular hashtag.

There are many more calls you can make, some of which require authentication. Like many APIs, Instagram uses OAuth to authenticate sessions, which requires a user to log in with their username and password before accessing data. That’s great for apps such as Screenstagram, which makes a screensaver from your actual Instagram photos. But we wanted to keep things simple here and decided to use API calls that require only a client id. The client id is the identification number that’s created for each registered Instagram app. You can sign up for one on Instagram’s developer site, which you should definitely sift through if you plan on using the API. It’ll uncover lots of gems including their API console. When playing with this sample, be sure to replace the sample’s CLIENT_ID with the one that you get when you register.

Once you’ve got your client id, you’re on your way. Take a look at the InstagramStream.h file and you’ll see the following options to start up the InstagramStream object:

// popular images
InstagramStream( const std::string &clientId );

// images with a specific tag
InstagramStream( const std::string &searchPhrase, const std::string &clientId );
InstagramStream( const std::string &searchPhrase, const int &minId, const int &maxId, const     std::string &clientId );

// Search for media in a given area.
InstagramStream( ci::Vec2f loc, float dist, int minTs, int maxTs, std::string clientId );
InstagramStream( ci::Vec2f loc, float dist, std::string clientId );
InstagramStream( ci::Vec2f loc, std::string clientId );

As an example, to get the stream of instagram images that are tagged with “pizza”, you would define your stream (mInstastream) by saying:

mInstaStream = make_shared<InstagramStream>( "pizza", CLIENT_ID );

To get the popular images stream, it’s set up so that all you have to pass is your client id. Just be prepared to see a good amount of One Direction pics (or whatever boy band is in the charts these days).

mInstaStream = make_shared<InstagramStream>( CLIENT_ID );

Now that you have easy access to all those awesome instagram photos at your disposal, it’s your turn to do something amazing with them.

That should cover everything you’ll need to know about the Cinder kaleidoscope sample. You can check out the source on github. Play with it. Make it better. And as always, there’s a community of crazy smart and eager developers on the Cinder forum to help you out with any questions or if you just want to show off your latest work.

  • Raphaël de Courville

    Nice tutorial. Encoding alert: the & (ampersand) appears as “& amp;” in the code samples. Similar problem for the characters and maybe others.