Creative Machines

Three methods for creative collaboration with machines — the seed of my creative technology practice

Creative Technologist Self-directed 2008

Summary

  • Built three tools for a 2008 BFA thesis at RISD.
  • IDA: an articulated ink-brush arm driven by four Arduino-wired PlayStation 2 joysticks.
  • IRAH: infrared-sensor gloves map hand position to a projected brush.
  • BARB: a wheeled robot that trails a gravity-fed ink brush and steers around obstacles people place in its path.
  • Across all three, the drawing itself becomes a record of the negotiation between person and machine.
  • The seed of a my creative technology practice: building tools that put people in collaboration with machines they can't fully control.
The Interactive Drawing Arm mid-drawing, ink brush sweeping across a paper canvas inside a wood frame.

For my BFA thesis at the Rhode Island School of Design, I built a set of three tools for making drawings. Each one encouraged the user to embrace what they couldn't fully control. The goal was to get people out of their own heads: to set out drawing a bird on a branch, end up with something else entirely, and treat that drift as the point rather than a mistake.

Looking back, this is where the thread that runs through all of my creative-technology work started. I'm still building tools that put people in collaboration with machines and asking them to embrace what they can't fully control.

Interactive Drawing Arm (IDA)

IDA is an articulated ink-brush arm mounted inside an empty picture frame. Three joints and a pan motor each wired to a repurposed PlayStation 2 joystick through an Arduino microcontroller.

Because each joystick controls exactly one motor, the arm is hard to drive alone and much more interesting with friends. Up to four people can crowd around the frame, each in charge of one degree of freedom, trying to coordinate a drawing in real time. The intent was to make collaboration unavoidable, and to make the inevitable miscommunications part of the result.

IDA in use: three players, four joysticks, one ink brush.
The IDA prototype: a salvaged picture frame with the articulated brush arm and electronics rigged to one side.
The prototype rig — a salvaged picture frame, four motors, the joints of the arm, and the bottle of ink.

The drawings that come out of IDA all share a family resemblance: jittery, overlapping marks that nobody could draw on purpose. They're a record of four people trying to agree.

An ink drawing made with IDA: dense overlapping strokes forming an organic mass. An ink drawing made with IDA: more open and gestural, with arcing strokes. An ink drawing made with IDA: faint sweeping arcs and scattered drips. An ink drawing made with IDA: bold curved strokes layered over each other.
A selection of drawings made by groups of four players collaborating on IDA.

InfraRed to Audio Hands (IRAH)

IRAH is the more digital sibling of IDA. The user wears a pair of gloves fitted with infrared distance sensors and stands in front of a rear-projected canvas. As they move their hands across the canvas — left and right, up and down, closer and farther — the projected brush follows in x, y, and z, with depth controlling brush size.

At the same time, the system maps hand position to pitch, so the user hears the drawing as they make it. The setup is designed to be playable with your eyes closed: you can stop looking at the canvas and try to draw by sound alone, listening for the shape you're making.

IRAH translates hand position into brush position, brush size, and pitch.
The IRAH installation: a laptop on a pedestal with a translucent canvas hanging from the ceiling, rear-projected with a soft pixelated drawing.
IRAH installed — laptop driving the projection, canvas hanging from the ceiling.

Barrier Avoiding Robotic Brush (BARB)

BARB is a small wheeled robot, housed in an inverted Tupperware container. A gravity-fed ink brush hangs off the back, low enough to trail a line along whatever surface the bot is driving over. Two infrared sensors on the front detect obstacles in its path and steer it around them.

People play with BARB by placing objects in its path, lifting it, or redirecting it. The bot reacts to its environment, the environment reacts to the people, and the drawing falls out as a record of all of that negotiation.

BARB held in one hand: a white inverted Tupperware container with foam wheels, an obstacle-detection sensor on the front, and a brush mount hanging off the back. BARB from another angle, showing the rear-mounted brush trailing low to the ground.
BARB — an upside-down Tupperware container, foam wheels, two IR sensors on the front, and a gravity-fed brush trailing the rear.

One thread

All three of these are the same project at different scales: build a tool that has its own opinions, hand it to a person, and see what comes out of the collaboration. That instinct — building tools that put people in productive conversation with machines — is still what I'm doing now, just with bigger teams and more precise tools.