The light that falls onto your eyes is meaningless. How the brain translates this ambiguous information into useful behaviour is not known. Here we address this question by enabling humans to experience the same visual world that our bumblebees experienced in the Bee Matrix, but with their ears. By hearing colour and associating those contextual sounds a meaning to each sound in context, we hope to explain the principles of perception generally. Our ‘Synesthetic’ software programme translates light into sound.
The programme outputs the converted sound on up to 81 separate tracks, which can be played on the Sound Wall system. One can specify the spatial location of any instrument (violin, piano, voice, etc.), as well as its synesthetic relationship to light in space and tone (e.g., red = C#, etc.). The input to the programme can be imported images, quicktime/AVI movies and/or video on-line via a fire-wire camera. This system is then used to construct sound and music in 2D to explore the science, design and art of perception.