Intro to Computational Media: In This Together

Tonight came the dreadful news of Steve Jobs’ death.  The man was brilliant and dragged us all forward into a better world.  Unfortunately he was also very secretive and he did not allow us to embrace him and thank him and love him while we all knew he was dying.  For all he had given us, he could have let us given back.  But maybe that was his way.  He didn’t need us.  But I think we probably needed him.

Apparently Earthlink/Mindspring decided tonight to hold a night of internet silence in his honor, because my internet’s been down all evening.  Now I’m tethered to my Android phone, blogging on my MacBook Air.

Our assignment for ICM this week was to come up with a chance composition, similar to Iannis Xenakis or John Cage and their reactions against the chaos of the World Wars.  I feel like I’d come up with a chance composition with the “Call Your Mum” piece but wanted to do something similar to the Sol LeWitt drawing 92.  At the same time, I happened somehow across the OpenCV library, that also directly supports Processing. After a simple install, you can import the library into a Processing sketch and pull in a live video feed from your webcam, and then manipulate it.

After that, I quickly knew I wanted to add a pixelated random RGB color filter to grit it up, which lent itself to a low-fi pirate shortwave kind of look.  Then I was thinking about Desmond from LOST and his attempts to contact Penny.  Then I thought about the movies Children of Men and Minority Report and V for Vendetta and other dystopian future movies which I love.  But since once of my goals in the NYU-ITP program was to come back to positive themes, I made the user’s chat with the love interest, “cipher”, open to improving the situation through human connection, even if just through a shell prompt.  Once contact is established, the palette turns yellowish, with hopeful photos being shown subliminally.  When cipher wants to hear “it” one last time, almost anyone should understand that cipher wants to hear “I love you” one last time before it’s all over.  The user either figures it out, causing the tint to turn pink, for love, or the signal begins to get jammed, resulting in an inevitable disconnection without resolution of feelings.  The code is called “In This Together”, a reminder of the Nine Inch Nails song.

Click here to view the embedded video.

One thing I had some trouble understanding in the code was how strings are treated.  Since I’m used to PHP, I’m used to just declaring strings like any other variable, but Java treats arrays more strictly as something relating to single character arrays and pointers (at least, that’s what it seems like to me), so I’m not too sure I could safely navigate more string-based projects without running into huge errors.

To sum up, I don’t think I really did any functions within functions (sup dawg) to produce fractals or a random stochastic process composition like Xenakis did.  However, I did create some external functions and used the randomized RGB codes to create a pixelated video filter.  I did take in user input and use an algorithm within a function.

Demo movie:

Click here to view the embedded video.

Screenshots:

Code, download links, etc. below the jump:

For download:  applet. (probably doesn’t work since it requires some Java/extra libs)

Here is my code:

// Ben Turner
// NYU-ITP ICM project

// "chance composition", "In This Together"

// x position for upwards black interlacing rect
int i=300;

// timer for overall animation
int counter=0;
boolean counterConsole = false;

// loads OpenCV
import hypermedia.video.*;
import java.awt.*;
OpenCV opencv;
int threshold = 80;

// needed to build the right-side terminal shell
int consoleLine = 1;
int consoleLineCount = 15;

// holds console message buffer
String[] messages2 = new String[50];
StringBuffer command = new StringBuffer();
String s;

// holds cipher's pre-arranged messages
String[] messages = new String[50];

// changes tenor of piece depending on user input & time
String globalTint;

// counts chars on console line in order to move cursor
int keyCount = 0;

// did user respond to cipher yet?
boolean pingBack = false;

void setup() {
  size(800, 350);
  background(0);

  globalTint = "green";

  // need a console-like fixed-width font
  PFont font;
  font = loadFont("LucidaConsole-48.vlw");
  textFont(font, 40);

  // init empty arrays to avoid errors
  for (int j = 0; j < 50; j++) {
     messages[j] = "";
     messages2[j] = "";
  }

  smooth();
  frameRate(15);

  // OpenCV needs this
  opencv = new OpenCV(this);
  opencv.capture(450, 350);
}

void draw() {
  background(0);

  // loading random subliminal images
  PImage b0, b1, b2, c0, c1, c2, d0, d2;
  b0 = loadImage("martiallaw.jpg");
  b1 = loadImage("riot.jpg");
  b2 = loadImage("com.jpg");
  c0 = loadImage("moon.jpg");
  c1 = loadImage("ww2kiss.jpg");
  c2 = loadImage("mlk.jpg");
  d0 = loadImage("orpheus.jpg");
  d2 = loadImage("kiss.jpg");

  // inputs camera footage for OpenCV
  opencv.read();
  image(opencv.image(), 0, 0);

  // puts a color tint over captured video image
  if (globalTint == "yellow") {
    fill(255, 255, 0, 100);
  }
  else if (globalTint == "red") {
    fill(255, 0, 0, 100);
  }
  else if (globalTint == "pink") {
    fill(200, 0, 200, 100);
  }
  else {
    fill(0, 255, 0, 100);
  }

  rect(0, 0, 450, 350);

  // adds subliminal images
  if (counter%50 == 0) {
    if (globalTint == "green" || globalTint == "red") {
      tint(0, 200, 0);
      image(b0, 0, 0);
    }
    else if (globalTint == "yellow") {
      tint(200, 200, 0);
      image(c0, 0, 0);
    }
    else if (globalTint == "pink") {
      tint(255, 0, 255);
      image(d0, 0, 0);
    }
    tint(255);
  }
  if (counter%30 == 0) {
    if (globalTint == "green" || globalTint == "red") {
      tint(0, 200, 0);
      image(b1, 0, 0);
    }
    else if (globalTint == "yellow") {
      tint(200, 200, 0);
      image(c1, 0, 0);
    }
    tint(255);
  }
  if (counter%80 == 0) {
    if (globalTint == "green" || globalTint == "red") {
      tint(0, 200, 0);
      image(b2, 0, 0);
    }
    else if (globalTint == "yellow") {
      tint(200, 200, 0);
      image(c2, 0, 0);
    }
    else if (globalTint == "pink") {
      tint(255, 0, 255);
      image(d2, 0, 0);
    }
    tint(255);
  }

  // some black matting for video
  fill(0, 100);
  rect(0, 0, 450, 350);

  if (counter >= 600) {
    globalTint = "red";
    fill(255);
    text("SIGNAL ACTIVELY JAMMED", 50, 300);
  }

  if (counter >= 1000) {
    fill(255);
    textSize(30);
    text("CONNECTION TERMINATED", 250, 175);
  }

  fill(255, 100);
  textSize(20);
  text("LIVE", 380, 30);

  // black bg for console text
  fill(0);

  textSize(12);
  fill(161, 211, 147);

  messages[0] = "cipher> honey, are you there? it's urgent.";
  messages[1] = "cipher> my life is in danger.";
  messages[2] = "cipher> pls respond, give proof you're there.";
  messages[3] = "cipher> i don't have much time...";
  messages[4] = "cipher> let me just hear you say it";

  if (counter%400==0 && counter > 0 && counter < 850) {
    addConsoleLine(messages[4]);
  }
  if (counter == 300) {
    addConsoleLine(messages[3]);
  }
  else if (counter == 200 && pingBack == false) {
    addConsoleLine(messages[2]);
  }
  else if (counter == 50) {
    addConsoleLine(messages[1]);
  }
  else if (counter == 5) {
    addConsoleLine(messages[0]);
  }

  displayConsole();

  s = command.toString();
  fill(255);
  text(s, 520, 335);
  fill(161, 211, 147);

  text("root:~$ ", 465, 335);

  drawCursor();

  videoFeed();

  i-=20;

  fill(0, 100);

  // interlacing rect
  if (i > 20000) {
    counter = 10000;
  }
  counter++;
}

void addConsoleLine(String newMessage) {
  for (int f=10;f>0;f--) {
    messages2[0] = newMessage;
    messages2[f] = messages2[f-1];
  }
}

void displayConsole() {
  for (int f=10;f>0;f--) {
    text(messages2[f], 465, 320-f*consoleLineCount);
  }
}

void keyTyped() {
  if (key == RETURN || key == ENTER) {
    s = command.toString();
    addConsoleLine(s);
    if (pingBack == false) {
      pingBack = true;
      addConsoleLine("cipher> oh thank god you're there.");
      addConsoleLine("cipher> is it you? what school did we meet at?");
    }
    if (s.matches("(?i).*tisch.*") || s.matches("(?i).*itp.*") || s.matches("(?i).*nyu.*")) {
      addConsoleLine("cipher> so it IS you...!");
      globalTint = "yellow";
    }
    if (s.matches("(?i).*i love you.*")) {
      addConsoleLine("cipher> i love you too");
      addConsoleLine("cipher> stay with me here, till they find me.");
      globalTint = "pink";
    }
    if (s.matches("(?i).*help.*")) {
      addConsoleLine("cipher> can't help, too risky. don't come find me.");
    }
    if (s.matches("(?i).*counter.*")) {
      addConsoleLine("counter is currently: " + counter);
    }
    s = "";
    command.setLength(0);
    keyCount = 0;
  }
  else {
    command.append(key);
    fill(255);
    text(key, 520+keyCount*7, 335);
    keyCount++;
  }
}

// creates random colored matrix look
void videoFeed() {
  int j = 0;
  int k = 0;
  while (k = 900) {
      j = 0;
      k += 20;
    }
    if (globalTint == "yellow") {
      fill(random(200, 255), random(200, 250), 0, 50);
    }
    else if (globalTint == "red") {
      fill(random(0, 255), 0, 0, 100);
    }
    else if (globalTint == "pink") {
      fill(random(0, 255), 0, random(0, 255), 30);
    }
    else {
      fill(0, random(0, 100), 0, 50);
    }
    rect(j, k, 20, 20);
    j += 20;
  }
}

// draws and moves console cursor
void drawCursor() {
  if (counter%15 == 0) {
    fill(0);
  }
  else {
    fill(161, 211, 147);
  }
  rect(520+keyCount*7, 325, 7, 12);
}

Professor Dewey-Hagborg also requested that we find an example of something that is modular in design, in preparation for us learning more about building in modularity and object oriented stuff into our code.  The first example I had in mind was IKEA.  Their parts are modular, their shelves are mostly modular, and the whole design is usually reliant on the same Phillips head screwdrivers and Allen wrenches.

Here’s an example of the IKEA instructions, for you fortunate souls who have never had to assemble IKEA shit before.

Here’s a better example of how IKEA projects turn out: