Home | Site Map | Watch | FAQ | History | Store | Contact

Conversation between Alex and Stephen (prev top next)


Stephen, you wrote

I'm not sure I'm understanding what you think the relationships are among meaning, narrative, emotion, and formal elements (form, structure, process, etc.). For example, there are (what I think of as) emotional effects associated with different types of modulation, cadence, etc., and when I look at a score, I see those forms, and I know what the emotion associated with them might be. So I don't see how emotion could be eliminated.

I find the requirement to verbally interpret my visual imagery for the purpose of our discussions quite challenging. I am applying to give a paper (detailing my work) at a conference about art and education in Heidelberg later this year (inSea 2007). This is certainly helping me elucidate my thoughts.

Only the other day I stumbled across your manifesto. In particular the section "my choices"

the objective versus subjective content of music you can have a passage that makes you feel a certain way but what is it in it that makes you feel that way? can a visual analogue also make you feel that way? if the visual analogue represents the feeling rather than the cause is it being too pushy?


In the process of generating a visual analogy of music you can initially  focus on the formal elements that underpin musical structure, intervals, rhythm, chords, keys, etc. Ideally these would  be informed by  established culturally defined systems of notation to make them more accessible (and this style of visual representation is what I refer to whenever I use the term music visualisation loosely).  This is fine for offering a dry anatomical analogy of  musical structure, however, the real challenge of music visualisation is to find ways of embedding emotional analogies into the visual interpretation without being prescriptive, or pushy, or in other words, forcing an emotional narrative onto the audience. This is, of course, based on the assumption that emotional expression can be choreographed into otherwise abstract images, which there is already plenty of precedence for in animated Hollywood films.

Minsky writes: "a thing has meaning only after we have learned some ways to represent and process what it means, or to understand its parts and how they are put together."

I agree with this. We already have a well developed musical lexicon which composers use, for example the chord progression V-I for finality, or I-IV for expectation of further movement, minor modalities for sad or subdued emotions and major modalities for happier more excited emotions. Indeed, Huron investigates and attempts to build a relationship between some of these formal elements and their emotional impact. These are your elemental components which compound into larger emotional tapestries. Visually notated, however, these are essentially meaningless elements. They are  elements which, aurally, hold emotive content, but not visually. An example would be, reading the score to  the adagio in Mahler's 5th symphony. A moving piece of music to listen to, but to uncover that emotion in the notation requires a mental translation into the aural realm.  Within our current visual lexicon, it is emotionally barren. A particular musical phrase may invoke the emotion of transcendent beauty, but translated into its visual format, the formal structural detail which leads the performance of this phrase to its emotional experience, is meaningless. On one level I see this  as a data resolution issue.

I don't think of expressive nuance as unquantifiable -- just "unquantified to date."

In our modern world of computer based data analysis we have  a whole series of tools available to embed a great deal of resolution into  acoustic data to assist in visually representing phrasing and articulation and instrumentation. This allows an unprecedented capacity to represent subtle nuances and these nuances are essential in shaping the emotional content of a musical phrase. However, as you identified,

it's essential that people recognize that there's a correspondence, but once that's a given, it's no longer something you have to consider, and other things can be seen as "important" -- and that's the direction we need to strive.

So despite  high resolution visualisation, representation of acoustic nuance etc, we still need to address the question of emotional content. Emotion is an emergent phenomenon from musical structure. To represent this faithfully requires the establishment of an analagous aesthetic in the visual realm, ie, one that emerges out of the visual structure within the music. This is a fundamental problem I struggle with. There is already an established animation industry very adept at giving inanimate objects personality and emotion. It is very possible to choreograph a particular phrase-entity in accordance with established notions of postural and visual declamation, this, however, requires me to declare what that emotion is going to be. This is prescriptive narrative. This is what I want to avoid.

This is why I am investigating these mathematical relationships:

I know what you referring to when you say "mathematical" and I do believe there are visual strategies based on this ... but I wouldn't call it "mathematical" ... the relationships can be described mathematically, but the aspect of them that's most important in music is only related to those relationships by happenstance.  For example, you and I would agree that an octave is consonant, relative to a major seventh or a minor ninth.  What accounts for this?  You might be tempted to say "it's the 2-to-1 ratio of the frequencies."  But we don't actually have a way of sensing that ratio directly.

I acknowledge that this may not be the whole story behind our appreciation of certain intervals or harmonic progressions, however these relationships offer a simple foundation upon which to build a larger visual aesthetic. And in reference to Minsky, its value lies in that it offers a basic explanation, or meaning, to the parts and why they are put together. The simple  visual ratio of 2:1 versus a ratio of 16:15 can be easily exploited as a basis for a contrasting visual aesthetic. The powerful relationship between the tonic and the dominant can be  represented through the simple ratio of 3:2. Simplicity is paramount when dealing with an  uneducated audience. Simplicity is also important in avoiding the distraction of complex conscious interpretive demands.    My exploration into visualisation is not meant to be a treatise on the psychoacoustic processes behind music appreciation. Neither is my focus on these relationships without substantiation as these relationships do exist.

Ultimately, whether this is effective as a system for introducing a visual aesthetic to our notions of harmony, consonance and dissonance, tension and release, to the point of invoking some emotional response, remains to be seen. I am  working on it ...

I will detail over the coming months the methods (with examples) of how i propose to do this. However I am curious as to how you yourself approach the issue of emotional analogy.


(prev top next)