Heartbeat — Waveplant

HEARTBEAT: DEEPENED connection through interactive sounD

A deep-dive into the considerations presented during the design of a reactive audio system
By Tyler Felson, published Jan. 24, 2024

How can interactive sound design most effectively mediate a uniquely personal connection between brand and participant? Waveplant recently designed, programmed, and composed an interactive soundscape for Heartbeat, an installation manufactured by Humancraft in which the real-time heartbeat detection of up to six participants was translated into a collective musical experience.

Interactive sound design must provide a discernible, clear correlation between a participant’s action and what is generated — someone should wonder, and then be able to answer, “What sound am I?”. This correlation is optimized through the synthesis of multiple real-time factors, demonstrated in Heartbeat through the following:

Individual Pulses: sounds were triggered only upon a detected heartbeat, driving the rhythm and ensuring a direct connection to the participant’s actions.
Heart Rate: the average beats-per-minute of a participant’s pulse determined the texture of their individual sound, shifting from soft and smooth at lower rates to bright and short at higher rates. This indirect reaction serves as a consistent, perhaps sub-conscious reference point for the participant.
Spatial Position: each participant had a designated pitch, produced solely from the speaker at their station. The fixed quality of pitch and proximity avoided confusion and provided definite ownership of an individual contribution.
Engagement State: in stretches of inactivity, a station would revert to a background bed of pads. This distinction in texture, rhythm, and tone offered a powerful initial point of engagement, enabling participants to immediately understand the impact of their interaction.

These first two factors illuminate the delicate balance between direct and indirect data required within an interactive system. Relying solely on direct, unfiltered data will result in inconsistent quality, despite a very apparent correlation. Too big an emphasis on indirect, passive data prohibits the participant from actively engaging with the experience. A successful interactive sound design must feel alive and dynamic while safeguards work behind-the-scenes to ensure it sounds good in a given moment. An essential safeguard of the Heartbeat system was the assignment (quantization) of each pulse detection to the nearest 16th-note. The notes therefore remained within a recognizable groove — without this automatic process the soundscape would have consistently sounded disjunct and sporadic. Instead, channels of direct and indirect interaction existed in symbiosis.

Variation is essential in any interactive environment to sustain engagement over time; a participant will tire quickly if a system lacks the promise of ongoing discovery. Gradual global development within a soundscape fosters extended curiosity by introducing an additional dimension to the experience. The consideration then arises: how and when do you introduce variation?

Heartbeat addressed this problem by modulating in 15-minute cycles between three distinct sonic languages: synthetic, mallet percussion, and orchestral. These instrumentations, tied together by their emotional quality, occupy different textural spaces. Striking the right balance is once again crucial: too close a relationship between sonic languages (ex: violin, cello, double bass) prevents continued discovery, while too much distance (saxophone, church organ, metal guitar) weakens the cohesion and impact of an experience.

The implementation of an interactive audio system demands a complex grasp of software and hardware — a unique expertise that aligns seamlessly with Waveplant’s technical capabilities. While developing three custom software solutions to receive, adjust, and sonify the hardware data, Waveplant prototyped a system that mimicked the intended capabilities of the installation. This enabled an essential stage of efficient creative development before interactive functionality had been fully achieved.

In harnessing the transformative potential of interactive audio, Waveplant stands ready with expertise and enthusiasm to elevate your experiential production. We look forward to collaborating with you!

Listen to and explore Heartbeat:

ARTICLES