Experimental conditions
Two simple songs – Happy Birthday to You and Jeux Interdits – and three sets of photos – photos of flowers, of a little girl and of snowy landscapes – were presented to ten subjects. Their task was, for each pair of song and set of photos, to create a musical strip.
This task was also carried out with the mapping device and gave for Jeux Interdits the three following musical strips
Main results
The exercise proved to be difficult as evidenced by the comments made by the interviewees, the time taken to construct the strips and the hesitations observed. This can be explained by the cognitive load of the task; numerous photos to combine with several musical segments.
- The subjects make choices very different from those proposed in the optimal adjustments (partly) presented above.
- When musical segments are close, subjects tend to associate more similar images. This is mostly observed for Jeux Interdits but must be confirmed because the trend is not very marked.
- The subjects would have preferred to associate one or several images with the music as a whole, without being obliged to follow the progression of the music through the segments. This is in fact what they have implicitly done as one finds similar photos in all the stripes but for different musical segments.
- The songs are related to events with strong associations in the minds of the subject. For instance, Happy Birthday to You evokes a happy moment. And this seems to have refrained the subjects to choose cactuses that suggest pain. The mapping device does not work that way and in fact chose some cactuses.
- There is probably a seed effect. The choices of the first and second pictures are partially arbitrary (choice of origin and unity), a bit like choosing to interpret an air a given tone or with a specific instrument. This is corroborated by the diversity of photographs chosen to represent the first two musical segments, especially for snowy landscapes.
- The comments made by the subjects at the end of the exercise, however, confirm some basic assumptions of my exploration. Similar moods, states of mind, feelings are caused by sounds and images and can be used to link the visual and acoustic worlds. When asked about what motivated their choices, people referred to emotions and moods: “Happy birthday means joy like the red color”, “Faded flowers are sad as the end of Jeux Interdits”, ” The brightest pictures are associated with the strongest segments “,” Music as a facial expression can be pensive” , ” Progressions in the smiles fit musical crescendo “,” Softness of the music does not go with the pungent cactuses “.