A Retro Smart Assistant

This project is an open-ended assignment, which was part of the Creative Coding course at Pitt. As a part of the Studio Arts Department, the course explores “the fundamentals of coding and digital circuits with a focus placed specifically on creative application (visual arts, sound/music, theater).”

The assignment was an exploration of the utilization of hardware components controlled by the programming concepts taught during the course. As an open assignment we worked directly with the professor, Jeremy Boyle, to create an idea for the studio work.

The Idea

The premise was to create a radio styled speaker inspired by the space age design of the mid 1900s. It would incorporate an embedded voice assistant which, when activated, would trigger an animatronic character to “come to life”. In essence, the project was a reimagined Google Home or Amazon Echo.

Experimentation

Initial research and prototyping of the animatronics soon led to the realization a 3D printer would help manufacture some of the parts. Soon a prototype of the animatronic eye mechanism was created. The “eye” interacted with servos to “look” up, down, left, and right. Then work began to incorporate a mechanical iris/diaphragm to the “eye” for additional expression. It was a lot of trial, error, fabrication, and maybe a bandage or two.

Hardware Design

As the first full-scale project, I knew of the importance to fully plan and map out all of the physical aspects of the design. I researched radios and other household items from the 1950’s and 1960’s, noting the materials and other elements, to choose an overall aesthetic and style -- in-short, retro-futurism. I made some iterative sketches before finally settling on a design.

While the sketches progressed, I also delved into Fusion 360 tutorials. Although initially a novice with CAD software, it quickly became a vital part of my tool belt. I soon translated components of my final sketches into 3D mock-ups.

Paper templates of the structure’s components were also printed out to scale. I recently inherited a scroll-saw, and one viewing of a vintage, VHS-tape introductory video later, got to work cutting out the face plate from plywood. The rest of the body structure was constructed from MDF. The MDF is inexpensive, and would later be covered by sheets of zinc, cut to size using metal shears and given detailed cutouts with a rotary tool.

The cutouts on the face-plate for the eye and dial on the face plate were fitted with 3D-printed rings, coated with metallic paint. Initially I worked on a speaker “cover” which would act as an eyelid mechanism to open and close like a hand-held fan. In the end, I decided to expose the eye, covering it with a machinery-like cage instead.

"Hey Google"

The voice assistant part of the project required a lot more careful planning. I started out just trying to control music with voice commands, with the intention to build out more interactivity over time. I prototyped Python code that would run javascript and interact with the Spotify API. I couldn't find a seamless way to have music playback, and since that was essential to this being a radio, I decided to pivot to a more all-in-one approach using a fully-fledged, smart assistant.

Various solutions that would be compatible and seamless with a fully featured voice assistant were researched and tested without actually purchasing a smart speaker outright. I was able to find only a few options at the time in Spring of 2020. For instance, I looked at the SnowBoy community project for training a hot-word detection model, and using that to in turn trigger a voice assistant. Despite Amazon Development’s support of the project, it wasn’t consistent enough at even detecting the prompt “Alexa”.

Next was the Google Assistant SDK, which I could easily load and run on a Raspberry Pi. There was a catch, however -- It no longer supported hot-word detection. The older version was supported by the Google Assistant Library, but that code was deprecated two months before I started this project. What timing! I managed to still grab a copy of this deprecated Library, install it, and it worked! Ignoring the fact service for it could stop working any day, for the scope of the project it would do. I modified the example script to control the GPIO pins of the Raspberry Pi, which were fed as triggers to an Arduino development board that controlled the servos. The separate processor was the simplest way to both have the assistant and servos operating simultaneously.