Surprise plays an important role in music, but this wasn't something I paid much attention to until I read David Huron's excellent Sweet Anticipation. Before then, my videos were little more than a scrolling version of a completely static bar-graph score (the only difference being that the notes brightened when they were sounding) so, visually speaking, there weren't any surprises in them (except perhaps if something unexpected scrolled on from the right).

My scores got more "animated" in 2010 when I started to show melodic motion by having shapes glide between notes as they were being played ...

     Debussy, First Arabesque

... but, if anything, this resulted in less surprise, since it gave the viewer a better idea of when the next note was about to sound.

In 2011, I started experimenting with other ways of showing motion from one note to the next. In the first versions of these ...

     Bach, Preludio, Partita in E Major
     Debussy, Syrinx

... I still indicated note positions in the future, but in 2013, I experimented with not showing the future ...

     Mozart, Queen of the Night

This was effective in some pieces, but it still didn't get at what I considered essential about surprise, namely, the violation of expectation. In the Queen of the Night aria, you don't know exactly what's coming, visually, but unless it's the first time you've heard the piece, you have an idea of what's coming musically, and you can pretty much imagine what the visualization is going to look like.

Fortunately, in 2017, the pieces I was working on gave me some ideas. Chopin's third scherzo has a figuration that is striking, beautiful, and unexpected; hiding these notes until they played was the obvious thing to do:

     Chopin, Scherzo No. 3 in C-sharp minor, opus 39

Musicians use the phrase "deceptive cadence" to refer to a sequence of chords which sounds like it's going to end a certain way but then goes in a different direction. Bach uses both a deceptive chord sequence and decepting writing ...

     Bach, A Musical Offering, Trio, 4th mvt

... by pretending to be about to end the piece, but then going on. In this case, I showed what was apparently the end of the piece, but then revealed the (hidden) future when the music got to that point and kept going.

With the next example, I'm not sure it's exactly surprise, though it does involve not completely showing the notes in the future; in it ...

     Bach, A Musical Offering, Canon perpetuus

... the violin (red) plays the same music as the flute (green), but inverted (mirrored), and delayed in time; here, I don't show the notes the violin plays until the flute plays them.

This piece ...

     Mussorgsky/Rimsky-Korsakov, Night on Bald Mountain

... gave me the opportunity to do something that's even closer to what I was thinking of in the wake of Huron's book: to depict what you're expecting will happen, and then have it change when it turns out not to happen.

Beethoven's Große Fuge presents challenges to performers, listeners, and to music visualizers. The most daunting of these is probably the sheer complexity: there's so much going on in some passages, so many fragments of the subject happening at once, that's it's hard to know "where to look." In my first version of the animation ...

     Beethoven, Große Fuge

... these passages were just a mess, visually. I decided that in these places, it wasn't so important to see the notes in the past and future—there was plenty going on in the present to pay attention to! So, in this version ...

     Beethoven, Große Fuge

... the notes of the main subject only become visible right before they're about to sound, and fade away shortly after they've sounded.

The day I posted this video, I was thinking about it while the dental hygienist was working on my teeth, and I saw that I'd stumbled onto an important principle: when something appears unexpectedly, your eyes are drawn to it, and by having the subject fragments appear right before they sounded, I was directing the viewer's attention to what was important. Composers do a similar thing when writing fugues: shortly before the subject happens, the voice about to play it will drop out, to make a "hole" in the musical texture, so that the statement of the subject is not a continuation of what was happening before but an unexpected entrance. I need to think about how this idea might be used in other ways ...

... but meanwhile, any videos I make that feature surprise in a significant way will be added to the Surprise Playlist.

Stephen Malinowski, September 5, 2017




Sometimes, when accents happen in unexpected places, we refer to it as syncopation. Is syncopation a form of surprise that could be visualized? That's something I want to explore. In this passage (about twenty seconds, starting here) ...

     Chopin, Fantaisie in F minor, opus 49

... accented chords appear in unexpected places, and I've depicted this by showing them in the expected places until they are expected to sound, and in the unexpected places when they actually sound. This is effective, I think. I look forward to seeing whether the same idea can be applied to faster and more regular types of syncopation.

Stephen Malinowski, April 11, 2018



Syncopation, continued ...

I'm not sure whether the tied notes in the opening of the 3rd movement of Bach's 6th Brandenburg concerto ...

... count as syncopated, technically, but they're close enough I thought I should experiment with them.

In the first version of the animation, I just had the tied notes explode. This gave them a nice visual accent, but there were problems: the explosion effect begins at the onset of the note, so it extends into the time when the following strong beat is happening, which has the effect of making the timing of the syncopated note less clear, and hiding the highlighting of the strong beats.

In the second version, I tried making the notes on the strong beats (immediately after the syncopated notes) appear out of nothing when they sounded (as opposed to merely being highlighted). This helped, but the timing of the syncopation was still unclear.

In the third version, I tried reversing the effects: having the syncopated note appear out of nothing and the strong beats assemble out of nothing. This was not an improvement, because now the syncopated notes were appearing during the time the strong beat notes were assembling; very confusing.

In the fourth version, I gave up on using two contrasting effects, and had both sets of notes (syncopated and strong beats) assemble. By having the assembling happen quickly, I was able to prevent the times from overlapping, so the effect was relatively clear, but I think it would be better to use contrasting effects somehow.

Stephen Malinowski, October 9, 2018