Three musical strips on the song Happy Birthday to you
Where do we find this type of association?
This kind of association can be found in
- linguistic expressions which qualify a musical interpretation as intense or brilliant, a timbre as clear;
- musical compositions inspired by visual sensations or paintings linked in one way or another to music (see for instance Unravelling Bolero by Anne Adams – Unravelling bolero);
- visual music (see for instance Mary Ellen Bute http://www.centerforvisualmusic.org/Bute.htm);
- visual animations like those produced with Media Player;
- some neurological disorders (like synesthesia) where input to different senses are mixed.
Mathematical techniques can be used to map images on music. This requires to cut a piece of music in different segments, to quantify those segments, and to represent them in an appropriate space. Images must be processed in a similar way: digitalization and representation in appropriate spaces. Musical strips can then be creaetd by associating some images to musical segments with an appropriate mapping device.
A device which maps structures
More particularly, we consider here, that in each domain the objects (musical segments, images) are represented by points in multidimensional spaces. A piece of music – being a sequence of segments – is then represented by a broken line; to map images on the piece of music, a similar path has to be found in the space of images (which could have a different number of dimensions). What is meant here by “similar” requires clarification. A mapping is regarded as good when the distances between the images mapped onto the musical segments are close to the distances of the corresponding segments. This requires of course some normalization process, as the units on the different axes are sometimes arbitrary and so are the distances.
Some results obtained with the mapping device
Little girl on “Happy birthday to you”
Snow on “Jeux interdits”
Wire chairs on “Mozart variations” (K 265)