Sinestesia
Sinestesia started as a series of artistic experiments where my goal was to visualise the movement of pure frequencies. All these experiments started in 2018, and since then I have been working both conceptually and technically on it.
The title
I would like to start by dissecting the way I baptised this project, Sinestesia, as it is a good first step to understand how it has evolved since 2018. My interpretation of this word appears to conflict with its English translation. Sinestesia, in Portuguese, denotes a stylistic device often used in Portuguese literature and poetry. Like a metaphor, this figure of speech is a way of relating entities from distinct universes. It illustrates an intersection of the senses, where the traits of one sense, such as sight, are attributed to another, like sound. Examples such as ‘the dress was of a screaming colour’ or ‘the sound of the piano was blue’ demonstrate how a writer can translate qualities from the sense of sight to the sense of sound, or vice-versa.
So, to me, this experimentation has consistently served as a stylistic tool, bringing tactile and visual qualities to something typically imperceptible by the human eye: sound waves. One of my first mistakes when developing this project in London was deciding to translate the title of this project to English, Synesthesia, as it led to people questioning me about the neurological condition, estimated to affect between 3 and 5 percent of the population. As I didn’t want to make a statement about this condition, or had the credibility to do so, and because it actually changed the way the audience would understand my project, I decided to embrace the original title in Portuguese. It took me some time to understand this conflict that was derived from the semantics of my title, but it was the first learning step for me to further question myself about my personal curiosities in this project, and to investigate the concept that I was looking for. It was also a learning lesson that showed me how influential the title of an art project can be on the audience.
The technicities
The first setup for this project was quite experimental, as can be seen in the picture below, consisting of a speaker inside a metal pan covered in a latex material. This material is stretched in order to vibrate with the frequencies being played on the speaker. A mirror is glued on this surface and a laser is reflected through it, so the vibrations can be seen on the wall.
In the following video, it is possible to understand how the experiment operates. I play around with blending multiple pure frequencies spanning from 70Hz to 140Hz. The more frequencies are being played at the same time, the more intricate the projected object becomes, as the sound waves produce more complex diffraction patterns.
This remains the foundation for the current iteration of the project. However, thus far, I’ve experimented with various iterations to understand what works and what doesn’t. In the following sections, I will navigate through each of these stages, culminating in the version currently under development.
First stage: long-exposure photography
[2018]
In this initial iteration of Sinestesia, I wanted to toy with long-exposure photography to explore the way the object moves in the space over time. Every image I propose portrays small segments of music from a variety of genres, rock, pop, jazz and classical music. By keeping this identifying feature hidden, I explore how fragments of a composition can become visual lilliputian identity cards for the composition as a whole.
At this stage, I was still trying to figure out what to do with this analog system I had come across with. I quickly realized that using music didn’t yield as intriguing reflections as pure frequencies did. Therefore, the first significant change from this point onward was to restrict the sound being played to a range of frequencies.
Second stage: installation
[2019-2020]
I see this stage as the pivotal moment that has influenced my decisions up to the present day. I created a simple installation where curated frequencies played in a loop on the structure displayed above. The focal point of this experiment was the three-dimensional aspect resulting from the blending of these frequencies. The combination of the auditory and visual elements creates a synesthetic experience where the boundaries between the senses fade away. In the installation space, the audience meet a threshold of perception, where the incorporeal sound waves become almost tangible to the participants.
This was the first moment where I understood the importance of space for my concept. After establishing this foundation as a WIP, I began exploring various variations of this installation.
Third stage: interactivity
[2022]
In this third stage, I began experimenting with interactivity. I designed a physical computing structure that incorporates an array of sensors strategically placed within the exhibition space to detect the presence and movement of the audience. These sensors capture data in real time that is then transformed into auditory signals. The movement induced by the sound waves creates intricate objects that morph in sync with the audience’s movements. By merging sound and sight, the audience experiences an instance of chaos where sound waves becomes visually perceivable, fostering a collaborative and immersive experience that is both personal and shared.
After investigating the role of interactivity in this experience, I concluded it was not the right direction to pursue. It made the experience flat as it became playful and chaotic, and led people to focus too much on the direct translation of their movement to the sound. This disturbed the experience of blending the senses together, resulting in a gamefied version of my project. I also realised people where more curious about the object that was generating the sound rather than the object detecting their movement. Finally, I understood that there is an implied interaction of the body with the sound, a physical one, where the sound waves are affected by the physical space and the bodies, reflecting and refracting accordingly. Therefore, forcing an interaction of the audience with the sound device was not only redundant but also trivial.
I decided to change directions.
Fourth stage: sculpture
[2023]
Until now, I was too focused on the interaction between the audience and the sound. At this stage of the project, I chose to take a step back and analyse its core features and reflect on their contribution to my overarching concept. My aim was to delve deeper into the essence of this project, dedicating more time to its conceptual exploration. I realised the significance of space as the medium through which sound waves propagate, even though they remain invisible to our eyes. This is when the idea of presence started to take place. I began by translating my thought into a few questions, and now I’m beginning to connect them with the technical and physical aspects of the project to seek answers.
What is this connection of human and light? And what is the relationship between light, vision and sound? How does the individual or the audience experience the sound? And light? What responses do the pure frequencies have on the human body? How do these vibrations affect the audience? And how does the presence of the audience affect the vibrations?
These were extremely important in prompting me to question my approach. Upon reflection, I shifted my focus to the object responsible for translating frequencies into movement– my structure. Initially, I concealed the analog structure responsible for producing sound and the projection. However, it became evident that this should be the focal point of my project, rather than the laser gimmick.
During this same time period, I started to find an interest in sculpture, as I just had developed my project Evanescence. Viewing this structure as a sculpture led me to redirect my attention away from the laser and towards the creation a centre piece, all while examining the conceptual implications of my decisions. I developed a WIP version of two sculptural objects that had a table-top height.
Fortunately, I had the opportunity to exhibit this work in progress and assess how the audience interpreted it. While certain aspects, like the resemblance to a DJ set, didn’t work as effectively, I ultimately recognised that I was finally progressing in a direction that satisfied me.
Fifth stage: materiality
[2024]
Currently, my focus lies in further experimenting with the sculptural aspect of Sinestesia, exploring different forms and shapes. Through this exploration, I’ve come to realise my intention to incorporate a third sense that was previously overlooked: touch. My investigation now centres on vibrations, and not so much the ability to see the sound. However, as I delve deeper into the conceptualisation of Sinestesia, I’ve encountered some obstacles, leading me to seek solutions through practical experimentation. I believe research and practice should go hand in hand.
I realised that the essence of interaction lies within the material itself, and that there’s no need to impose a specific mode of interaction. Instead, I can maintain openness for the audience to engage with it freely. By delving into the haptics of these experiments, I hope to create opportunities for the public to interact with Sinestesia in diverse ways.
With this in mind, a new concept began to emerge: the notion of composition, encompassing not only sound composition but also the physical arrangement of the space. I found that projecting the laser onto a wall diverted people’s attention away from the sculpture I was creating. My focus has shifted towards enhancing the sensory experience of the audience and fostering their connection with the vibrations being produced. Currently, I’m exploring the materiality of my sculptures and examining how different materials influence the audience’s experience.
My main references at the moment are Lachlan Turczan and Harzoon Mirza. Mirza was already a reference for me before, but now more than ever as I intend to delve deeper into the notion of composition. Mirza works with the space and the materiality of his installations (first row of images below), while Turczan (second row and video) has been an inspiration particularly in encouraging me to take a leap I had been avoiding: incorporating water into my sculptures.
Liquid Loops, from The Joshua Light Show (Video ), has been also an influence in this stage of the project. I’m exploring how oil can impede the movement of water. See below both my experiments and my reference. I hope to continue to explore this by introducing oils with different densities, as well as glycerin.
I am currently designing multiple cube-like structures, each one exploring a different matterial. I am starting to learn about metal-work, as metal is known for its excellent sound reflection properties. You can see my plan below.
My last experimentation was with silicone, as it is such a sensitive a flexible material. My goal is to create a cube that can showcase the patterns created by the vibration of the sound waves. I started by creating smaller prototypes, as can be seen below. After scaling the cube to a bigger dimension, I found some obstacles, mainly the difficulty to actually see the patterns. During the next month I will further experiment with this material in order to achieve what is seen in the first video.
My next step will be developing my metalworking skills in order to craft various boxes capable of housing speakers internally. If done right, they will vibrate with the speaker and reflect any light that is directed towards it, generating interesting patterns. Jonathan Prince’s metal sculptures serve as a significant source of inspiration for these upcoming endeavors.
