Strings On-Line

An interactive art installation presented during the International Conference on New Interfaces for Musical Expression (NIME) 2020.

guitars and robots

The installation is based on a collection of self-playing guitars and interactive robots.

Conceptual information

This physical-virtual installation explores the concept of entrainment, the process in which two interacting systems assume the same period. The online audience can interact with six self-playing guitars through distance-controlling three Dr Squiggles robots. The installation is physically located in the fourMs Lab at the University of Oslo, Norway. Users watch an online video stream of the installation with binaural audio and interact with the robots (and indirectly with the guitars) through a web interface.

The guitars "listen" to their surroundings through built-in microphones and respond sonically to the sounds produced by the Dr Squiggles robots. The guitars hang in thin lines from the ceiling and are therefore free to (slowly) spin around. The sonic result is a complex interplay between rhythm and chaos, structure and variation.

Guitars hanging from ceiling
Six self-playing guitars hanging in the ceiling, with three Dr Squiggles robots on the floor.

People are invited to interact with three Dr Squiggles robots sitting next to the guitars. The interaction is happening through a simple web interface. With the push up of a button, the robots will play on metal rods, and the guitars will pick up these sounds. As such, the installation blurs between the physical and the virtual, local and global, and it plays with conceptions of differences between acoustic and digital instruments.

Robot
Dr Squiggles is an interactive musical robot. It taps on metal rods with its eight "feet".

Behind the Scenes

Technical information

Each of the guitars is equipped with a Bela micro-computer, which produces electronic sound through an actuator placed on the back of the guitars. There are no external speakers; all the sound generation is coming from the vibration of the acoustic guitar. The sound is created through a physical model (the Karplus Strong algorithm). The guitars listen to their environment through a microphone and use a Firefly-inspired algorithm to entrain each other. The pitch of the tones and colour of the light is controlled through an infrared distance sensor.

Microcontroller
A Bela micro-controller with a custom-built board with additional sensors (accelerometer and microphone) and light control.

Each of the Dr Squiggles robots has eight legs, with a motor attached to them, through which they can tap on objects to create rhythmic patterns. They have a powerful onboard computer (Raspberry Pi) and are therefore fully autonomous, enabling explorations of swarm-like activities. For the installation, each of the robots can be controlled remotely through a web interface.

Screen with video
The installation is streamed online 24/7 for the duration of the conference. Users can interact with the robots by pushing buttons in an online form (right on screen).

The streaming is done through a Zoom webinar. We tried different types of streaming solutions, including on the main video platforms. However, none of them provided a low enough latency. So we decided to go for a Zoom solution, which is a good compromise between quality and latency. Unfortunately, the browser version of Zoom is not as optimized as the standalone version. So, even though it is not ideal for the user to have two windows open simultaneously, we decided to go for the standalone Zoom version and a separate control window.

Organizer

RITMO and fourMs Lab
Published June 21, 2020 9:04 PM - Last modified Sep. 18, 2024 2:29 PM