The different experiments
Four different experiments have been performed to compare the way the subjects associate images, flavors and music with the results of the mapping device.
In a first exploration, 10 subjects were asked to build musical strips on 2 different airs. More specifically, they had to carry out 6 tasks: to map 4 photos taken from three different sets of pictures (flowers (9), photos of a little girl (9) and landscapes under snow (22)) on two airs: Happy Birthday to You and Jeux Interdits.
A second experiment made it possible to study the way 10 subjects associate images and zakouskis. Again, the subjects were asked to draw from a set of 10 images – which represented “architectural elements” – 4 photos likely to marry 4 zakouskis served as an aperitif.
A third experiment carried out with 13 subjects analysed the possible matches between flavors and images, with 4 beers and 10 photos of dolls.
Finally, in a last experiment, I proposed to 36 subjects a choice between two musical strips and this for 4 musical tunes.
General results
The results show that
1. Some songs and music are more difficult to translate into images. To use an analogy, it is harder to dance on some tunes than on others. The algorithms do not have this kind of difficulty.
2. When flavors are associated with images, subjects are influenced by visual elements of food: the shape of a zakouski, the color of a beer.
3. When the tunes are strongly connoted, the connotations interfere with the choices which are made. Happy Birthday to You evokes a happy moment, that is to say smiles, flowers … surely not cactuses as proposed by the mathematical algorithm.
4. The subjects use very different strategies; some truly seek to reproduce the temporal structure of music – repetitions, variations – others propose more global associations – often based on emotions aroused by the piece of music; a last category finally builds narratives that underlie both music and images to uncover a common thread.
5. The subjects’ choices correspond statistically with the choices made by the mapping device in the last experiment, but this result can be explained in many different ways.