Shimi is a robotic musical companion I worked on as part of my Master's degree at the Georgia Tech Center for Music Technology. My colleagues and I were researching how music and gesture can be used to express emotion in low degree-of-freedom robots. I updated Shimi's sensors and processor to be controlled by an NVIDIA Jetson TX2 and an extensible python codebase.
My colleague Richard Savery designed a non-verbal voice for Shimi, for which I implemented a singing program, which would analyze input music and have Shimi sing and dance along with the predominant melody. Check out Shimi rapping along to French hip hop duo La Caution here.
GTCMT Master's project paper [link]
Savery, Richard, Ryan Rose, and Gil Weinberg. "Finding Shimi’s voice: fostering human-robot communication with music and a NVIDIA Jetson TX2." Proceedings of the 17th Linux Audio Conference. 2019. [link]
Savery, Richard, Ryan Rose, and Gil Weinberg. "Establishing Human-Robot Trust through Music-Driven Robotic Emotion Prosody and Gesture." 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). IEEE, 2019. [link]