Beginning this project I was inspired by the overlaying techniques of Robert Rauschenberg and the interactive digital art of Emmanuel Van der Auwera. I was interested in creating a multifaceted experience, utilising the full effects of sound, photography, and motion tracking.

I began by researching these two artist as well as Japanese photographer Ken Kitano and planning the initial photography trials for the mock lineup of how this might work. The concept at this stage would primiliar utilise the photography installation and sound to create a sort of art gallery-like experience, where a tour guide would speak with the visual accompaniment of the art. However, blending this together for a more immersive experience, using ambient music and human interaction to decide how you want to experience the audio.

Six separate shoots were executed, varying in style, colour, grain variation, background, pose, and distance. I did this to determine the strongest possible final composition in the following steps of overlaying the portraits.

I repeated this process a few times, with different compositions, and this proved more difficult than originally conceived. I found that the eyes of each model became a distraction and gave an uneasy feeling when overlayed with one another haphazardly, so special attention to how each photograph was taken and how they were overlayed were equally as important. From this I also understood that colour only muddied the final composition, distance of shot would play a major role is creating visually pleasing final outcome.

I then moved into assembling the photography piece. I printed the photographs onto translucent clear film and placed them between repurposed A4-sized glass panels. I again at this stage trailed different arrangements to test the composition of the full piece. After the glass created a distance, this altered the piece yet again. After comparing 20 experimental arrangements, I identified the best layout for what the piece needed to be.

The glass enhanced the piece by creating a subtle, unexpected lenticular effect when viewed from different angles. Building on this by pressing the final two photographs within the same pane of glass, this added more variety and allowed the two images to blend more than the two other photographs, separated individually by a glass panel, creating additional depth and a more abstracted focus on the deeper layers of the piece.

I then constructed a functional two-sided frame to hold the glass panels together. As this isn’t typical of regular one-sided frames, the structure was initially weaker, and suffered from wearing glue that would over time separate with the weight of the glass panels.

To alleviate this issue and keep the piece balanced, a wooden stand was assembled to support the frame and also provide stronger focus for the piece.

I composed a simple guitar-based soundscape to emphasise the melancholic tone of the project. Inspired by soft acoustic guitar instrumentals and ambient soundscapes, the audio helped shape the overall experience and added emotional depth to the personal accounts of grief.

The audio was then mixed and processed to mimic the feeling of an old tape recorder, enhancing the immersion and making it sound like a old, vintage answering phone message. This made each personal accounts feel more authentic, and as if you were listening in on an emotional message left by someone meant for a loved one after a missed call.

The personal accounts were a key focal point of this experience and needed to carry the right emotional weight. The stories told in each recording had to feel both powerful and authentic when listened to. Because of this, several interviews were conducted before this finally came through.

I soon discovered the audio was most effective when using two participants, recorded through a simple microphone and then heavily altered in recording software to achieve a similar effect and feeling of the ambient soundscape (an old answering machine message).

Eventually this blended seamlessly with the music composition. The emotionally charged subject matter, and the unique and human feelings expressed by both speakers, lead to a final audio that conveyed its intended effect in an authentic, and visceral form.

The next stage was to piece everything together with the motion tracking concept. I achieved through TouchDesigner, specifically with blob detection, which was linked directly to the audio.

The concept was for the user could control which voice they heard by moving to one of the two sides of the glass piece (over the tape marking on the floor), creating the illusion of hearing the voice through the prominent face they could clearly see on that side. To bring more prominence to the voices the soundscape was panned to the left and contrasted against the voice lines that were panned to the right and accompanying sound effects which stayed centre.

Additional sound effects were included to enhance the sense of control and immersion. These were: A tape recorder rewinding at the end of a voice line as if playing on repeat with physical media, and an answering machine beep to indicate when the user had switched sides, as if you were pressing play on your answering machine.

Coming to the end of the process, small tests were conducted in the installation space after the final assembly. These helped balance the audio levels for the environment. Minor adjustments were made, such as using noise-cancelling headphones to achieve the immersive feeling, and making amendments to the motion tracking and lighting setup.

To add final details, a logo was created using the same materials as the glass installation piece and a simple diagram was created to demonstrate how to use the installation.

The final installation was a successful dive into an emotional experience, with two voices tracked based on the user’s position.

The strong soundscape created an immersive and emotional backdrop, allowing you to fully engage with the stories shared through the experiences of grief. All of this was brought together by the central glass photography piece placed at the heart of the installation.