Trans-sensory mappings are interesting for different reasons.
First, it has clearly an entertaining and artistic dimension, as attested by various attempts to do so by musicians, painters, and filmmakers. The space between different expressions of similar things is worth to explore. Isn’t poetry a way to explore how the musicality of a language can interact with the sense conveyed by the words?
Second, the exploration could lead to new ways to express what we feel when we listen to a piece of music, when we look at a set of images. Appropriate words are difficult to find to describe a musical experience. Imagine we can give a visual interpretation of a set of Mozart variations with a carefully chosen set of photos. This would very likely broaden our vocabulary to speak about Mozart.
Thirdly and perhaps mainly, this could also tell us something about beauty, harmony and aesthetic experiences in general. Indeed, to link a piece of music with a strip of images, will oblige us to reproduce or to translate in a visual dialect a set of emotions conveyed by a piece of music, a musical dialect. Is there a common structure shared by the two? Can we go beyond syntactical or superficial aspects of melodies and images to link them and reach some kind of deeper structure directly associated to the notion of harmony, beauty?
Finally, this will tackle indirectly a topic, which is seldom addressed in current research to our knowledge. What makes the beauty, the harmony, not of a single image, but of a set of images? How could the choice of music impact the choice of photos or the other way round? How can we elaborate a sequence of photos, which as a whole is harmonious? Could we introduce like in music, variations on a theme, repetitions, and changes of tone? What makes the beauty, not of a single image but of a sequence of photos is possibly partly hidden along the road explored in this site.
What associations are favored by people?
Mathematical approaches make it possible – from an a priori definition of what a good association looks like – to find optimal mappings. But how human beings when faced with the task of mapping photos on music or images on savours do operate is another issue. I have realised some experiements to compare the performances of persons with those of the mapping devices I have designed. The results are summarized in a section of this site.