Multi-Media Creative Projects

I began maturing an interest in multi-media art projects during my interdisciplinary masters program. Here I designed, created, and installed projects that often have an interactive component. My goal with these projects is to engage participants in play, have them consider their interactions with others, and more deeply consider the consequences of their actions. In these projects, I imbue my own personality and experiences.

Celestial Sequencer

This project was a comment on the consequences of creation. A participant is given a celestial world to experience either on PC or VR. They are able to create celestial bodies that swirl around them. These celestial objects carry sounds, which are ambisonically encoded, for spatialized audio. On feature was intentionally designed to be absent from this experience: the ability to remove objects once they have been created. A participant may get carried away with the sounds and sights that they are able to create. Then they must face their creations. Greedy designers are left to face rather loud and dissonant soundscapes.

Audiovisual sequencer, where a participant must face their creations.

Contrastingly, another version of this experience was created for users to create playful soundscapes. Users were given an additional mechanic of removing their creations, ore options in the sounds which they can create. More massive celestial objects are given deeper tones. This was intended to be used a creative outlet. Additive synthesis for Tibetan sound-bowl physical modeling was used to create a meditative soundscape.

Audiovisual sequencer, where users can creative relaxing soundscapes

Synchronizing Fireflies

In this project, I wanted to capture and represent a memory from my childhood, while providing a visually interesting audio visualizer. A much less frequently visited national park, in Congaree SC is home to one of the only known species of fireflies, which synchronize their flashes. Unfortunately, due to human environmental impact in this area, the population of these remarkable fireflies is diminishing. I hoped to capture the astonishment and awe that I felt when I first experienced these fireflies as a kid.

Audio visualizer demo interaction

I’ve continued to use this visualizer, repurposing it for presentations on neural synchrony in human-derived organoid model systems. I use this to demonstrate a property which both synchronizing fireflies and our brains have in common: paracrine signaling. Additionally, I’ve used this visualizer to highlight an application of piezoelectric reporters for neural sub-type identification in dense electrode array recordings of neural activity.

Human-derived organoid model, demonstrating neural synchrony. Visualized via microscopy and calcium signal indicator
Demo from presentation, of paracrine signaling and use-case for piezo cell-type reporting. When paracrine signaling rule is programmatically introduced to this system of randomly flashing fireflies, they spontaneously synchronize their flashes.

Close Enough

This multimedia art installation came about during the pandemic. Here I explored programmatically designing an interactive art piece, that runs on a computationally constrained linux embedded system. The system uses an object detection neural net, real-time tracking, sound synthesis, multi-sensor integration, and ambisonic sound source spatialization. Details of the system build can be found here. The experience is one that seems as if sound is emanating from individuals interacting in the space. As they approach one another, more dissonant voices are added at a higher amplitude. The effect it creates is one that where you may experience discomfort in getting close to others. However, using a galvanic skin response sensor, when you are close enough to touch, the piece may suggest something else.

Early demo of build during the pandemic

I’ve continued with this project, and have introduced new mechanics. I have received permission to install a professional build at the GlassBox Gallery on UCSB campus, in December 2023.