By stepping into the installation space, the user controls how they experience the stories of loss, told through two voices transmitting via a headset — using TouchDesigner motion tracking and immersive, old-media-styled music and SFX.
Category:
Installation
Year:
2024
Roles:
Researcher, Graphic Designer, Photographer, UX/UI Designer, Sound Enginer, Composer.
Inspired by the overlaying techniques of Robert Rauschenberg and the interactive art of Emmanuel Van der Auwera, we were interested in creating a multifaceted experience, utilizing the full effects of sound, photography, and motion tracking.
We began by researching Japanese photographer Ken Kitano and planning the initial photography trials for the mock lineup.
Six separate shoots were executed, varying in style, color, grain variation, background, pose, and distance. This was done to determine the strongest final composition in the following steps.
Through several tests, we found that the eyes of each model became a distraction and gave an uneasy feeling. While this was partially expected, the results appeared more unsettling and distracting than anticipated. Minimizing this element led to a stronger and more thoughtful outcome.
We then moved into assembling the photographic piece. By sliding the clear film photographs between A4-sized glass panels, we trialed several arrangements to test the composition of the full piece. After comparing 20 experimental arrangements, we identified the most visually captivating layout.
Overlaying a variety of distances, poses, and backgrounds in one specific arrangement proved to be the strongest. We separated each photo with a pane of glass, which further enhanced the piece by creating a subtle, unexpected lenticular effect when viewed from different angles.
We built on this by pressing the final two photographs within the same pane of glass. This added more variety and allowed the two images to blend more than the others, creating additional depth and a more abstracted focus on the deeper layers of the piece.
We then constructed a functional two-sided frame to hold the glass panels together. As a deviation from the traditional one-sided frame, the structure was initially weaker — especially under the weight of the glass.
To alleviate this issue and keep the piece balanced, a wooden stand was assembled to support the frame.
We composed a simple guitar-based soundscape to emphasize the melancholic tone of the project. Inspired by soft acoustic guitar instrumentals and ambient soundscapes, the audio helped shape the overall experience and added emotional depth to the spoken voice sections.
The audio was then mixed and processed to mimic the sound of an old tape recorder, enhancing immersion and making it feel like a vintage phone receiver message. This effect was thoroughly tested to ensure it felt authentic and achieved the intended emotional impact.
The voices were a primary aspect of this experience and needed to carry significant emotional weight. The stories told in each recording had to feel both powerful and authentic. Numerous iterations were conducted to achieve the final result.
The audio proved most effective when using two participants, recorded through an iPhone speaker and then heavily altered to mimic the sound of an old answering machine.
This blended seamlessly with the music composition. By drawing on the emotionally charged subject of losing a loved one, and the feelings expressed by both speakers, the final audio conveyed its message in a raw, emotionally authentic, and visceral form.
The next stage was to piece everything together using motion tracking. This was fully realized through TouchDesigner’s blob detection, which was linked directly to the audio. The user could control which voice they heard by moving to one of the two sides of the glass piece (over the tape marking), creating the illusion of hearing the voice through the prominent face they could clearly see.
The music was panned and contrasted against the voice lines and accompanying sound effects. Additional sound effects were included to enhance user control and immersion — such as a tape recorder rewind at the end of a voice line and an answering machine beep to indicate when the user had switched sides.
Several small tests were conducted in the installation space after the final assembly. These helped balance the audio levels for the environment. Minor adjustments were made, such as using noise-cancelling headphones to achieve the full effect, and making amendments to the motion tracking and lighting setup.
To add final details, a logo was created using the same materials as the glass piece. Additionally, a simple diagram was produced to demonstrate how to use the installation, displayed on the wall and on a computer screen behind the setup.
The final piece was a successful dive into an emotional experience, with two voices tracked based on the user’s position.
The strong soundscape created an immersive and emotional backdrop, allowing the user to fully engage with the stories shared through the audio. All of this was brought together by the central glass photography piece placed at the heart of the installation.