Musically Bringing Anki’s Cozmo to Life
With the days of Ferby and Tickle Me Elmo long behind us, there is a new face of artificial intelligence this holiday season, Cozmo. Made from San Francisco based-company, Anki, whom released their first product in 2013, Anki Drive, Cozmo is a robotic companion with a mind all of his own. Think of Number 5 in the 80’s film Short Circuit mixed with Pixar’s Wall-E. Cozmo has an emotion engine, meaning it can react to situations as a human would, with a full range of emotions from happy and calm to frustrated and bold. The consumer can also play interactive games with Cozmo such as Quick Tap and Keepaway. The more you hang out with Cozmo, the more his personality evolves. As it evolves, it is doing so to an original score by Brian Lee White & Brian Trifon of Finishing Move Inc. and Gordy Haab. Anki called upon the composing team, who took their experiences of working on video games such as Star Wars: Battlefront, Assassin’s Creed 2 and the upcoming Halo Wars 2 and applied it to this tiny bot. Never before has music been this much emphasized on bringing a toy like Cozmo to life, so we decided to speak with Brian, Brian & Gordy about their work on the robot.
How did you all first get involved with Cozmo?
Finishing Move had been working with Anki doing the music and implementation on Overdrive for a while now and when they told us about Cozmo and where they wanted to go with the score we knew we had to bring on Gordy for his amazing orchestral chops. Overdrive has a very electronic heavy, EDM-centric score, while Cozmo is almost the polar opposite, organic and orchestral with live instruments in a chamber style setting. Since we’d been working with Gordy on the Halo Wars 2 score, we knew the chemistry was there and the team was already spooled up, so we introduced Gordy to the team at Anki and everything came together.
What did you all do to prepare for scoring this robot? A lot of people reference the film Short Circuit when discussing this product. Did you all go back and watch it?
We had numerous creative discussions with the team at Anki about the direction we wanted to take Cozmo, musically. Most of these discussions were about who Cozmo was – and what his character was like. We all agreed he was fun, a bit moody, easily excitable and a bit of a smart aleck. So we wanted his music to have a similar personality. So our preparation was geared towards deciding what type of musical ensemble would bring across this snarky playfulness. We decided to limit ourselves to a small ensemble of real instruments. Cozmo’s “band” is just 13 players, each of which brings an organic, somewhat ragtag quality to the music in order to match Cozmo’s attitude. Short Circuit was one of my favorite movies growing up, so there was no need to revisit it. Although I did see the parallel – at least in that our robot has a very human personality, similar to Johnny 5!
Cozmo is a very complex product with over 325 individual sensors, moving parts and electrical components. Did you all visit the Anki headquarters to get a better understanding about the robot before initially beginning? I can imagine it was hard to wrap your head around everything it specifically does.
Yes, we all got to visit Anki to hang with the development team and get a real hands on feel for the robot. We had seen several videos as reference material to help with the music design, but there’s something about being in Cozmo’s presence and interacting with the bot in-person that is a completely different experience, I don’t think we actually “got it” until we got to see Cozmo and play with him ourselves. In fact, we spent quite a bit of time there at Anki HQ working on the interactive music system, because unlike a traditional game development scenario where you can fire up a dev build right there on your PC to check your implementation work, with Cozmo we really needed to see how the music system reacted on the actual bot in physical space and there is a lot that can go wrong with a device this complex so having the support of the entire dev team right there on site was critical to the music implementation process.
How much music in total did you create for Cozmo?
All in all we created about 45 minutes of music for Cozmo. And that number is growing.
I know you all have worked on numerous video games. What are some of the major differences in scoring a product like Cozmo as opposed to a video game?
The biggest differences are Cozmo’s physical presence as a little robot in your world, as well as his ever evolving AI capabilities, both of those present unique challenges for composing and interactivity. Creating a compelling musical landscape for a physical device that learns and adapts as you interact with it is quite different than approaching a normal on-screen gaming experience. It’s sort of like the difference between a film and a live broadway play, some rules carry over but others won’t work in the same ways you’re used to. For example, Cozmo himself has a speaker and of course the smartphone has a speaker, so there were some cool spatial opportunities to be explored there. The music plays out of the phone, sort of acting like his personal “orchestra pit” while all of Cozmo’s vocalizations happen on his built in speaker that really helps separate and place the music.
This is one of the first times that so much emphasis, musically, has been put on a toy like this. Do you think we are going to start seeing this more now?
We certainly hope so! We really have to give props to Brian Min, Anki’s audio director, for pushing the limits on both the resources and attention to music a project like this would typically get. Anki as a company really gets how important music and sound is to an immersive interactive experience like Cozmo or Overdrive.
What was the most challenging part of scoring Cozmo?
Normal video games or films tend to come with a “set of rules” that are tried and true. But because Cozmo is a living, breathing thing – essentially with a mind of his own, it was a challenge to fully understand how the music would work with his behavior without fully composing the music, then implementing it into the robot and ultimately play testing it. We had many great ideas in theory that would not work out as expected once it became a part of Cozmo’s world. So the biggest challenge was needing to go through the full composition process before knowing whether or not something would actually work. Trial and error is a normal part of composing for media, but usually that means a small tweak or two to get it right. But with Cozmo, it became clear very quickly that either our music was working perfectly, or not at all. This was uncharted territory for everyone involved, so we all had to be very open to being creative and trying things, but also needed to be honest with ourselves if something was not working…and be willing to go back to the drawing board with the same level of enthusiasm. All in all, the whole process was fun and exciting. And we are all thrilled with how Cozmo’s music turned out. Cozmo seems really happy with it as well!
Anything else you would like to share about working on Cozmo?
It was really a joy and an honor to work on such a groundbreaking new product, something that pushed us beyond our normal interactive music comfort zone and into entirely new territory where few have ventured before. Stay tuned for more Cozmo fun!