Code available on Github
partyBot is my first attempt at combining embedded electronics and software to create something interactive. By moving the partyBot plushie, a drawing of partyBot on a program built with openFrameworks also moves around. As partyBot moves around, it collides with surrounding balls which emit a certain tone when hit.
Demo with a Projector
Before I go into details, I thought I’d share a quick story explaining how this project came to be.
Last year for Halloween I made myself a robot costume (also called partyBot) that flashed LEDs and played songs with an Arduino. It was pretty awesome. After walking around downtown in this costume, I guess most people started associating me with robots.
One of those people was a friend of mine and she decided to make me my very own mini partyBot plushie. This naturally melted my heart and Dan Mangan apparated in my apartment and sang a song about how robots needed love or something.
At the time, I had just learned about e-textiles and I was itching to do apply that to partyBot (pB) but I didn’t know what to do. Sure, I could make it flash a bunch of LEDs and stuff but that would be boring.
Enter openFrameworks. I’ve always wanted to make something interactive, something that combined embedded electronics with really cool things on the software side. pB was the perfect opportunity to make such a thing.
On pB, there is a Lilypad Arduino which is hooked up to an ADXL335 Accelerometer breakout board (Lilypad style) as well as four LEDs (also Lilypad style). pB connects to my laptop using the FTDI breakout board.
To play tones, I decided to make use of the SFE Music Instrument Shield (yet again) connected to an Arduino UNO. The setup is connected to my laptop and receives commands to play certain tones.
This is where the major headaches came in. When I first started, I was pretty new to Object Oriented Programming so I spent a significant amount of time learning up on the OOP aspects of C++. I didn’t end up using polymorphism, but I sure sharpened up on my knowledge on pointers!
The program is built on top of openFrameworks. I’ve created three main classes (not including “testApp”):
- Class pb: This is where pB is drawn onto the screen, takes in accelerometer values and moves him accordingly.
- Class ofBall: I’ve built upon the moving balls tutorial on the oF website. I’ve also added collision functionality as well that is based off of the Bouncy Bubbles example in the Processing Website.
- Class midi: This class handles the communication of MIDI commands to the Music Instrument Shield.
I’ve attached an Arduino Lilypad onto pB with the Firmata standard example loaded onto it. That’s it. The Firmata protocol sends the accelerometer data to the computer where it is processed.
I would have connected the Music Instrument Shield directly to my computer using an FTDI board to convert from USB to serial but unfortunately, ofSerial does not support the 31250 baud rate which is the standard for MIDI. I decided that the easiest way to get around this problem was to use an Arduino Uno as a relay to convert from 9600 baud (which is used to communicate to my computer) to 31250.
Wiring up partyBot was alot of fun. I never thought my electronics training would have me pull out a needle and some conductive thread but here I was sewing like a BAWS.
My conductive thread of choice can be purchased at Sparkfun. It comes in a bobbin and contains 30 feet of thread. I’ve tried using a different kind of thread. It comes in 50 feet but I found this thread to be far less flexible than the former and more difficult for hand sewing – but that’s my opinion.
Other than the large amount of time taken to learn Object Oriented Programming and writing up the classes, there really was only one major debugging pain. Serial Communications.
First, I used an Arduino Mini Pro to act as the MIDI Relay. This meant that the relay would be connected to my computer using the FTDI breakout board. Initially, the program ran well until after about two minutes in. At that point, things would fall out of sync. Tones would not play after colliding with pB until about two seconds after. About three minutes in, no tones would be played and my computer would go into kernel panic.
This happened every time, so I decided to take a look at the crash report and found the FTDI Driver lurking around. Finding the FTDI Driver in the report didn’t really surprise me since the crash log indicates all the processes that were running at the time of crash (or, kernel panic) and the FTDI Driver definitely would have been running. Still, I thought to myself “what if the driver is the cultprit?”
I decided to use an Arduino Uno as the relay instead. The Uno doesn’t rely on FTDI drivers for communication so I shouldn’t see such a horrific crash this time. I was right. No more kernel panics, BUT tones still fell out of sync after about five minutes and the program began to lag big time.
At this point I was pretty frustrated so I decided to mention this to a programming guru at work. He suggested that it was probably the serial communications that was mucking everything up and told me to try having the relay send an acknowledge message to the computer before the next MIDI command was sent. So, I went home and wrote these lines of code:
Serial.println(input); // Resend to computer MIDI command serialOut.print(input); // Relay to Music Instrument Shield the MIDI command inputString = ""; stringComplete = false; Serial.print("e"); // Acknowledge message to send. Computer sends next commands upon reception
Guess what?? This DIDN’T work! Aaaarrrrggghhhhh! At least there weren’t any kernel panics happening. Because the program was lagging so much, I had a hunch that maybe the computer was having so much trouble keeping up with the bytes of data coming into the buffer. After all, data WAS being sent to the computer. So, I got rid of the acknowledge portions of code in the relay and I noticed that I was sending data to my laptop before all this acknowledge stuff (Serial.println(input);). I commented that out as well.
That worked. I patted myself on the back and went to sleep.
This was my first formal foray into openFrameworks and Object Oriented Programming and I don’t think I did too badly. I would love to do more projects that link embedded electronics and software more closely in the future. I would also love to work with other software libraries like openCV or openCL. There’s a whole lotta stuff to learn out there. I would like to use a software synth (or even delve into Apple Audio Units) next time so that I won’t need to use a hardware synth. If anyone has suggestions, definitely let me know!
Thanks for reading!
I want to thank two people who have been awesome in helping me complete this project.
First, to Rob my programming guru who never fails to pull me out of coding trouble.
Second, to Crystal for making me mini-partyBot. I probably would have made something pretty lame as my first oF project if it wasn’t for you and your mad fabric skills.