partyBot

Code available on Github

partyBot is my first attempt at combining embedded electronics and software to create something interactive. By moving the partyBot plushie, a drawing of partyBot on a program built with openFrameworks also moves around. As partyBot moves around, it collides with surrounding balls which emit a certain tone when hit.

Overview Video


Demo with a Projector


An Anecdote!

Before I go into details, I thought I’d share a quick story explaining how this project came to be.

Last year for Halloween I made myself a robot costume (also called partyBot) that flashed LEDs and played songs with an Arduino. It was pretty awesome. After walking around downtown in this costume, I guess most people started associating me with robots.

SWAG

I say pB, what shall we do?

This is where the major headaches came in. When I first started, I was pretty new to Object Oriented Programming so I spent a significant amount of time learning up on the OOP aspects of C++. I didn’t end up using polymorphism, but I sure sharpened up on my knowledge on pointers!

1. Class pb: This is where pB is drawn onto the screen, takes in accelerometer values and moves him accordingly.
2. Class ofBall: I’ve built upon the moving balls tutorial on the oF website. I’ve also added collision functionality as well that is based off of the Bouncy Bubbles example in the Processing Website.
3. Class midi: This class handles the communication of MIDI commands to the Music Instrument Shield.

[partyBot] I’ve attached an Arduino Lilypad onto pB with the Firmata standard example loaded onto it. That’s it. The Firmata protocol sends the accelerometer data to the computer where it is processed.

[MIDI Relay] I would have connected the Music Instrument Shield directly to my computer using an FTDI board to convert from USB to serial but unfortunately, ofSerial does not support the 31250 baud rate which is the standard for MIDI. I decided that the easiest way to get around this problem was to use an Arduino Uno as a relay to convert from 9600 baud (which is used to communicate to my computer) to 31250.

Construction

Wiring up partyBot was alot of fun. I never thought my electronics training would have me pull out a needle and some conductive thread but here I was sewing like a BAWS.

Awwwwww yiss…

All wired up and ready to go!

LEDs ALWAYS make a project better.

Other than the large amount of time taken to learn Object Oriented Programming and writing up the classes, there really was only one major debugging pain. Serial Communications.

First, I used an Arduino Mini Pro to act as the MIDI Relay. This meant that the relay would be connected to my computer using the FTDI breakout board. Initially, the program ran well until after about two minutes in. At that point, things would fall out of sync. Tones would not play after colliding with pB until about two seconds after. About three minutes in, no tones would be played and my computer would go into kernel panic.

This happened every time, so I decided to take a look at the crash report and found the FTDI Driver lurking around. Finding the FTDI Driver in the report didn’t really surprise me since the crash log indicates all the processes that were running at the time of crash (or, kernel panic) and the FTDI Driver definitely would have been running. Still, I thought to myself “what if the driver is the cultprit?”

I decided to use an Arduino Uno as the relay instead. The Uno doesn’t rely on FTDI drivers for communication so I shouldn’t see such a horrific crash this time. I was right. No more kernel panics, BUT tones still fell out of sync after about five minutes and the program began to lag big time.

At this point I was pretty frustrated so I decided to mention this to a programming guru at work. He suggested that it was probably the serial communications that was mucking everything up and told me to try having the relay send an acknowledge message to the computer before the next MIDI command was sent. So, I went home and wrote these lines of code:

[c] Serial.println(input); // Resend to computer MIDI command
serialOut.print(input); // Relay to Music Instrument Shield the MIDI command
inputString = "";
stringComplete = false;

Serial.print("e"); // Acknowledge message to send. Computer sends next commands upon reception
[/c]
Guess what?? This DIDN’T work! Aaaarrrrggghhhhh! At least there weren’t any kernel panics happening. Because the program was lagging so much, I had a hunch that maybe the computer was having so much trouble keeping up with the bytes of data coming into the buffer. After all, data WAS being sent to the computer. So, I got rid of the acknowledge portions of code in the relay and I noticed that I was sending data to my laptop before all this acknowledge stuff (Serial.println(input);). I commented that out as well.

That worked. I patted myself on the back and went to sleep.

Ending Notes

This was my first formal foray into openFrameworks and Object Oriented Programming and I don’t think I did too badly. I would love to do more projects that link embedded electronics and software more closely in the future. I would also love to work with other software libraries like openCV or openCL. There’s a whole lotta stuff to learn out there. I would like to use a software synth (or even delve into Apple Audio Units) next time so that I won’t need to use a hardware synth. If anyone has suggestions, definitely let me know!

Thanks for reading!

I want to thank two people who have been awesome in helping me complete this project.
First, to Rob my programming guru who never fails to pull me out of coding trouble.
Second, to Crystal for making me mini-partyBot. I probably would have made something pretty lame as my first oF project if it wasn’t for you and your mad fabric skills.