Home | Site Map | Watch | FAQ | History | Store | Contact

Music Animation Machine Frequently-Asked Questions (FAQ)

Introduction

  • What is the Music Animation Machine?
  • How does the MAM "notation" work?

    MAM Videos

  • How do you make your videos?
  • What is on the videos?
  • Can you made a MAM video from my .mp3 (.wav, .m4v, ...) file?
  • Could you please do a MAM video of ____________?
  • How was the video of the Toccata and Fugue in D Minor produced?
  • Can you make me a custom video from my MIDI files?
  • Do you sell the MAM video in PAL format?
  • Do you sell any other videos besides the First Demonstration Reel?
  • I have trouble seeing the darker notes; what can I do?
  • How are the MAM performances on the videotapes created?

    MAM Software

  • Is there a version of the MAM for { Windows, NT, MacOS, Linux, ... }?
  • I don't hear anything when I run Tantrum; why not?
  • Could the MAM software be used in live performance?
  • How are the MAM movies made?
  • Can I make a movie of my own music from the MAM Player display?

    The Future of the MAM (design/improvements)

  • Why don't you show dynamics by changing the bar size?
  • Could the MAM's bar-graph display be combined with standard notation?
  • Could the MAM display be generated from live audio, or from a CD?
  • What other things might be in the MAM's future?

    MAM and teaching

  • Can the MAM help me learn to read music?
  • Could the MAM be used as a notation for singers?
  • What is the MAM for?

    The Conductor Program and Tapper

  • Why did you write the conductor program article?

    Other

  • What are you working on now?
  • Didn't I see the MAM on TV?
  • How did you pick the colors used in Harmonic Coloring?
  • What about using synaesthetic colors for the MAM?
  • Is there some way to turn a drawing back into music?
  • Do you sell posters of the MAM?
  • Is the MAM useful for (or interesting to) the deaf?
  • What happened to the page on your website that I can't find anymore?
  • How does the MAM relate to numerology, metaphysics, parapsychology, aliens, etc.?
  • Why are you doing this?


    How do you make your videos?

    The first step is composition; some pieces, I write myself, some I arrange, and some (the best) are by other people.  The next step is to get a rendition of the piece; if somebody else has performed the piece and saved the result as a MIDI file, I can use that as is, but usually I need to practice it myself and then do a performance.  This could be a live performance (like with this Fantasy by William Byrd), or it could be done with the conductor program (compare the "without expression" and "with expression" videos in the Malinowski section of that article), or it could be entered into a sequencer (like the original Music Animation Machine editor) one part at a time.  Most pieces require some editing, too.  The audio for this is also created at this point.  Then, the MIDI data is rendered into individual video frames; this is done with software which is pretty much the same as the MAM MIDI Player, but with file output; there are various versions of this software that do different kinds of displays.  Finally, the audio and frames are assembled into a movie, and titles are added; I do this in Adobe Premiere, and then for the versions I post online, I create the iPod-playable and Google-compatible versions with Apple's QuickTime player (the Pro version that can export files).


    What is on the video?

    To get an idea of what's on the videos, read the Viewer's Guides


    Can you make a MAM video from my .mp3 (.wav, .m4v, ...) file?

    In general, no.

    The visual part of a MAM video shows the pitch and the beginning and end time of notes.  A Standard MIDI file contains this information in a clear and unambiguous format, and can therefore be used to generate a MAM display.  Most music notation software (e.g. Sibelius, Finale, etc.) can convert their internal data representations into the standard MIDI file format.

    By contrast, an audio file (.wav, .mp3, etc.) or a multimedia file containing audio (.mov, .m4v, etc.) consists of a sequence of numbers which encode the pressure of the air at a microphone.  The problem of automatically extracting note times and pitches from audio data is one that's currently being worked on by people in the field of auditory scene analysis.  There are products on the market (WIDI, Melodyne, Solo Explorer, etc) that attempt to translate audio into MIDI, but so far none are reliable enough for use in extracting MIDI data for the use in MAM scores.


    What is on the video?

    To get an idea of what's on the videos, read the Viewer's Guides


    Could you please do a MAM video of ____________?

    Here's what I need (or want) in order to do a MAM movie of a piece of music:

  • Love of the music.
  • MIDI data (what the graphical score is generated from).
  • Audio data (what forms the sound track).
  • Permission (or, for the piece to be in the public domain).

    In some cases, audio of adequate quality can be generated from the MIDI data, but in the case of music for strings, orchestra, voice, large ensembles, or most popular music, it's usually not. I'm currently developing a tool to align MIDI and audio data, but until that's complete I need audio and MIDI that were generated from the same performance(s).

    So, if a piece of music is in the public domain (composer died before 1920) or you composed it yourself, or you can get the composer's permission, and there's a MIDI file of a performance, then the answer may well be "yes" (assuming I like the music enough to spend the time working on it).  Unfortunately, this puts a lot of popular music (and jazz, which you may or may not consider "popular") out of reach.


    How was the video of Toccata and Fugue in D Minor produced?

    Since this piece used almost all the techniques I used on any video, it serves as a good example of what's involved; I've posted some production notes for it here.


    Can you make me a custom video from my MIDI files?

    Several years ago, I thought that people might want this, so I geared up for it: got the necessary equipment, added an offer to the MAM Viewer's Guide that came with the video, and told people about it. After a year or two of waiting, I decided nobody was interested, and dismantled the equipment and moved on to other things.

    It's possible that in the future, I will write a MAM-style viewer for MIDI files.


    Do you sell the MAM video in PAL format?

    The 1990 and 1996 videotapes are available in PAL; the DVD video is not. The price (USD 25) includes shipping anywhere in the world.


    Do you sell any other videos besides the First Demonstration Reel?

    Yes; in 1996 I did a video as a commission for a banking consortium called Gulfnet. I originally called this "The Gulfnet Program" but now that I'm selling copies of it, I'm calling it "Second Demonstration Reel." The program contents and ordering information for this videotape can be found here .


    I have trouble seeing the darker notes; what can I do?

    The way to adjust your TV for viewing the MAM at its best is:

  • Go to a section of the tape that has music playing,
  • Turn the TV's "brightness" control all the way up, and then down until the background is completely black (but no further),
  • Turn the TV's "color" control all the way down, and then back up until just before the point where the bars of color smear or become unclear.
  • If your TV has a Dolby switch, set it to off .


    How are the MAM performances on the videotapes created?

    First, the music is entered into the MAM software (which is a MIDI recorder or "sequencer"), using one of these methods:

  • Play the piece into the MAM at tempo. The William Byrd Voluntarie was done this way.
  • Play the piece into the MAM at a tempo slower than full performance tempo, then speed it up. The Beethoven Bagatelle was done this way.
  • Play individual parts into the MAM separately. The parts can be different hands (for keyboard music) or different instruments for ensemble music). The Bach Brandenburg concerto movement was done this way.

    For some pieces, the music is then "performed" using the conductor program to give it more expression.

    Finally, the piece is edited, using the MAM performance editor. In this step, small changes in timing and dynamics are made, and the colors of notes are chosen.


    Why don't you show dynamics by changing the bar size?

    Good question. There are a lot of reasons the display is as basic as it is. The original reason is that the Music Animation Machine software was designed to run on 1984-vintage IBM PC's, which didn't have enough power for special effects.

    But there are other considerations. Short notes are already smaller than longer notes; if you also make loud notes big and quiet notes small, then it gets really hard to see short, quiet notes.

    Also, notes often get louder or softer while they're sounding; this would suggest that the notes change shape. That's fine, but then the question becomes: how do you do the highlighting? If you light up the whole note at once, then you might have this big bright part suddenly appearing, even though the start of the note is quiet. If you only light up the "now" moment, then you end up with a very narrow stripe of highlighting, which is hard to see.

    I'm not saying that nothing like this is possible; it's just not straightforward, and the things I've tried so far didn't look good enough to keep.


    Could the MAM display be generated from live audio, or from a CD?

    At the moment, no, this is not possible. Extracting "notes" from an audio signal is a difficult task, one which people have been working on for a long time. For a single instrument (or voice), the problem is relatively easy, but as the number of simultaneous notes increases, reliable decisions about which parts of the signal belong to which instruments become more and more difficult. As analytical tools improve and computers get faster, a satisfactory solution gets closer. One of the people working on this is Lloyd Watts .


    Is there a version of the MAM for { Windows, NT, MacOS, Linux, ... }?

    As of 2006jun11, there is a Music Animation Machine MIDI File Player for Microsoft Windows . You can download it for free here.


    I don't hear anything when I run Tantrum; why not?

    Because there's nothing to hear; it's a visual-only program.


    Could the MAM software be used in live performance?

    Sure. Of course, since it can't know what's coming, you won't see the future (which is the best part)!


    How are the MAM movies made?

    There are several steps to the process. First, I create the rendition. For pieces that I can perform adequately myself, I'll use a MIDI recording of my performance as a starting point. For pieces that are well-suited to the conductor program, I'll perform them using that tool and save the result. Some pieces have enough instruments that it works better to perform the part for each instrument separately, and layer them (sound on sound). Sometimes I use a combination of techniques.

    Once I have a rough draft of the rendition, I edit it. I'll correct wrong notes, and change the timings and dynamic levels of notes.

    In the original version of the Music Animation Machine, the next step was to play back the rendition and record the computer's video output and the synthesizer's audio output.

    Nowadays, the audio and video are processed separately. The edited performance is played back through a synthesizer (or sampler) and its output is recorded. The edited performance is also processed to produce individual video frames. The audio and video are then imported into video editing software and exported as a movie.


    Can I make a movie of my own music from the MAM Player display?

    I haven't tried doing this myself, but at least one user has reported good results by using the Hyperionics HyperCam software.


    Can the MAM help me learn to read music?

    First, let’s consider how people learn to speak, read, and write their native language. By the time a person begins to learn to read, they’ve usually amassed a vocabulary of thousands of words, mastered the structure of their language (syntax, grammar, case, tense, etc.), and spent several years interpreting the utterances of other people and improvising utterances of their own. When they learn to read and write, they are merely learning the system of notation for a language in which they are already quite fluent.

    On the other hand, most people who want to learn to read music are not fluent interpreters and improvisers of the kind of music they’d like to be able to read. So, we have to ask: what would they have to learn for us to be able to say that they could “read music?” Here are some things that are involved in being a fluent reader of music, in approximate order of difficulty.

  • To understand the meanings of the symbols used in conventional music notation: note names, rhythmic symbols, marks for dynamics and articulation, etc.
  • To understand the fundamentals of music theory: scales, chords, etc.
  • To perform pitches at sight.
  • To perform rhythms at sight.
  • To perform chords at sight.
  • To imagine (“hear in one’s inner ear”) what a single line of music sounds like by looking at it.
  • To play a melody expressively at sight (by hearing in one’s inner ear what it should sound like, and then playing it appropriately to its character - that is to say, to interpret it).
  • To imagine what a chord progression sounds like by looking at it.
  • To play a chord progression expressively at sight.
  • To imagine what a complex score sounds like by looking at it.
  • To play a complex score expressively at sight.

    In what way might the MAM help with these?

    Since the MAM doesn’t use the symbols of conventional music notation, it wouldn’t help much with notes, rhythms, harmonies, etc.

    Where it might help is in dealing with the complexity of a full score. Reading music is demanding, even with a single line; when reading a full score, it’s hard to broaden one’s focus and keep track of many things happening at once. Watching MAM scores might help prepare a person for this.

    And in a general way, watching MAM scores could prepare a young musician for what to expect in music notation generally. MAM scores can be understood by young children; it’s possible that this would influence how they conceived of (and listened to) music, and that this would help them to learn to read music. (This is speculation, of course.)


    Could the MAM be used as a notation for singers?

    Yes, certainly. In fact, I worked briefly on such a project with a Korean businessman who wanted to release a MAM sing-along video in his country. (Unfortunately, the Korean copyright system is set up so that it's very expensive to make a small number of copies of a video with lots of pieces on it, and he decided that he couldn't risk the investment.)


    What is the MAM for?

    When I first started working on the MAM, I did it because I thought it would be fun to see the result. Once I'd seen it myself, I kept working on it to share it with other people. Since then, people have used it for various things. For example, some music teachers use it in their music history and music theory classes. But I still think of it as mostly for fun.



    Why did you write the conductor program article?

    I first wrote Tapper because I wanted to use it; I modified it to make it easier for my friends to use (especially Christopher Strangio and the Dalbys), and I rewrote it for Windows so that it could run on a more modern machine (so that my friend Mervin Lane could use it).  At that point, since I'd done most of the work necessary for it to run on MacOSX, I wrote a version for that, too, with the idea of making both versions available as freeware.   Since most people don't know quite what the conductor program is or why anybody might want to use it, I decided to accompany the release of the freeware with an article describing it.  I didn't want people to think Tapper was representative of the full potential of the conductor program, so I put in a history section to put it in context, to show that it was an early and very minimal implementation.

    Over the years, I've participated in hundreds of discussions about the conductor program, and while I'm very interested in the idea, I'd grown a little weary of going over the same ground, so I took the article as an opportunity to make all the points about it that I and other people had made in those discussions, with the hope that it would save me some repetition.

    Finally: I think the conductor program is a wonderful thing, and I wanted to advertise my interest in it, to increase the likelihood that I'd get to know other people who are working on it.

    Who is the intended reader for the article?  Anybody who is interested, or who becomes interested.  It's not especially intended as a "popular" article; I expect that people who are puzzled, curious or confused will search the web for explanations, write me, or remain puzzled and confused.

     

    What are you working on now?

    Besides this page, you mean?   ;-)

    I'm starting to work on the tools I'll need to work on DVD #2. The first of these is a tool to align MIDI and audio data. Once I've completed DVD #2, things I'd like to spend time on include:

  • A video of Chopin's opus 10 etudes.
  • New animations for the Well-Tempered Clavier (this is a long-back-burnered collaboration with Peter Taussig). 
  • A documentary of my recent experiments (perhaps in collaboration with Stefan Sargent).
  • Porting the rest of my previous visualization tools to Windows and Mac OS X.
  • Learning how to package MAM movies as stand-alone graphics applications with embedded (high-quality) audio.
  • Developing a video game that teaches music sight-reading.


    Didn't I see the MAM on TV?

    Yes! Starting in November 2000, Classic Arts Showcase has been broadcasting selections from the two Music Animation Machine videotapes. CAS's satellite broadcasts can be received in all of North and South America, and their programming is rebroadcast in many cities, typically by community and non-profit stations; to find their station in your area, click here .


    What about using synaesthetic colors for the MAM?

    Some synaesthetes associate colors with musical pitches. For people without perfect pitch, colors are more memorable than pitches. Wouldn't it be great if we could somehow combine synaesthesia and the Music Animation Machine? Unfortunately, not everybody has synaesthesia, and among those who do, there is no agreement about which colors go with which pitches. For example, Elizabeth A. Pector, M.D. has one set of associations, while Cassidy Curtis has another (and Tony De Caprio yet another, but this link is temporarily broken). So, while it would be possible to make a MAM display that was satisfying for one particular synaesthete, another would think it was all wrong.

    But more to the point: synaesthetes with color/pitch synaesthesia already have perfect pitch, so they don't need help recognizing pitches. For the rest of us, I believe that any given random coloring makes as much sense as another (though I know in my heart that G is brown); therefore, I've focused on colorings which are governed by some recognizable rule, such as in my Harmonic Coloring scheme.


    How did you pick the colors used in Harmonic Coloring?

    The main idea of the coloring is: apply twelve colors from the color wheel to the twelve pitches around the circle of fifths. Once you've decided to do that, there are four more choices:

  • Which color/pitch to start on. I've chosen blue as the "home pitch," since blue suggests repose and resolution (as opposed to, say, red, which suggests activity).
  • Which direction to use in mapping the pitches and the colors. I've chosen to associate the I-to-V direction with the blue-to-red direction (since motion toward V is more often associated with more activity). This means that the I-to-IV direction is toward green.
  • Which direction to use when presenting the colors and pitches on a circle. I've chosen to make the I-to-V direction clockwise.
  • Which exact colors to use. I tried to choose colors so as to maximized the minimum perceptible difference between any two colors. With the variability in computer monitors and TV screens, this turned out to be nearly impossible.

    These choices are all fairly arbitrary. For example, if a piece modulates into a different key, the (local) I is no longer blue, and so the association of blue=home, red=dominant no longer holds.


    Is there some way to turn a drawing back into music?

    Yes. If you're interested in exploring this, you might want to check out:

  • "Musical Sketchpad" in Morton Subotnik's Creating Music software.
  • The UPIC System.
  • The MetaSynth.
  • Iannis Xenakis's Mycenae Alpha (YouTube video).
  • Face as sound at 3:10 (YouTube video).

  • Do you sell posters of the MAM?

    No, but you're welcome to. :-)


    Is the MAM useful for (or interesting to) the deaf?

    I don't know -- but the question is certainly interesting to me. I have not gotten much feedback about the MAM from deaf people, so I don't know the answer. I've sent tapes to Gallaudet University and to various deaf people I've known of, but I've received only one or two responses; from those, it appears that the MAM display is no more interesting to deaf people than it is to hearing people when they watch it with the sound turned off, which is to say: not very. In a way, this result, though disappointing, makes a certain amount of sense: why, after all, should it be any different?

    The fact that the MAM is not very interesting when unaccompanied by sound suggests that it is deficient: that there are aspects of musical sound which are not well represented in the MAM display. This is something I'm working on: to figure out what aspects of our perception of sound are not present in the MAM, and try to invent ways of showing them.

    On a related subject: Will Pitkin (link to follow) is working on a project the goal of which is to help deaf infants exercise the auditory cortex of their brain by presenting them with visualizations of the sounds around them during their "babbling" period. For this to be maximally useful, the visualizations will have to be comprehensible to the eye in much the same way that sound is comprehensible to the ear -- which is to say: he's going to have to solve many of the same problems that I have to.



    What happened to the page on your website that I can't find anymore?

    When I redesign the web site, I don't throw out old pages, but I do sometimes retire a page from active duty. Here is a list of the retired pages:

  • The original history page.
  • The original home page.
  • What you won't find on this site.
  • An earlier version of the "recent experiments" page.


    How does the MAM relate to numerology, metaphysics, parapsychology, aliens, etc.?

    I don't know.  These people might have some ideas, though:

  • Spectrum Muse: the explosive new knowledge and technology of color, music, numbers and waves with applications in all areas of metaphysics, physics, mathematics, chemistry, psychology, biology, and much more
  • Charles Lucy (do a web search for "Lucy Tuning")
  • ACOUSTICAL ALCHEMY, Therapeutic Sound Medicine for the Body, Mind and Spirit, By Terry Soroka


    Why are you doing this?

    I began my adult life expecting to be some kind of composer/performer/teacher.  However, it turned out that ...

  • ... my memory was too poor to be a good composer.  Hindemith wrote "We all know the impression of a very heavy flash of lightning in the night. Within a second's time we see a broad landscape, not only in its general outlines but with every detail. Although we could never describe each single component of the picture, we feel that not even the smallest leaf of grass escapes our attention. We experience a view, immensely comprehensible and at the same time immensely detailed, that we never could have under normal daylight conditions, and perhaps not during the night either, if our senses and nerves were not strained by the extraordinary suddenness of the event.  Compositions must be conceived the same way. If we cannot, in the flash of a single moment, see a composition in its absolute entirety, with every pertinent detail in its proper place, we are not genuine creators."  I think he's mistaken about the possibility of instantaneous perception, but he's right that to be a good composer, one must be able to remember all the details of a composition well enough to evaluate it from lots of angles easily.  It turned out that my memory was only adequate for writing short pieces at one sitting.
  • ... I was too self-critical to enjoy performing.  Even when the audience raved, I was disappointed and felt that it wasn't worth the trouble.
  • ... I lacked the necessary level of empathy to be a good teacher.

    Still, I loved music, enjoyed playing music in private if not in public, and felt that some of my musical ideas and experiences were worth sharing, so I worked on my sight-reading, learned to program computers, and started trying to make computer software to help people see what's inside my head (and theirs).