Home | Site Map | Watch | FAQ | History | Store | Contact


by Stephen Malinowski

0. Overview

Most tools for examining and measuring the world involve our eyes (the microscope, telescope, voltmeter, oscilloscope, ruler, etc.); our ears are relegated to a smaller and less important set of tasks technical tasks (alarm sounds, etc.). But not all data should be "seen and not heard," and in many cases an aural "view" of data reveals things that would be impossible to "see" other ways. Audification of data can be simple and inexpensive, and is fun! I hope this web page will get you started in using audification.

1. Advantages of Audification

Hearing is very different from vision; here are some of the ways these differences are advantages:

Audification is not ideal for every kind of data. But in some circumstances, the above advantages make it the presentation of choice.

2. Audification Techniques

The simplest audification technique is a straight signal-to-sound conversion, like you'd get by sending a signal directly into the input of an audio amplifier. For a lot of signals, this is all you need.

For some signals, the data is in the wrong time scale to fit optimally in the 20-20,000 Hz range of human hearing; in such cases, you need to change the time scale of the signal. Now that computer signal processing and A/D conversion are cheap and fast, the most practical approach is usually to record the signal digitally, then play it back at other frequencies.

In some cases, the signal is very noisy, and it's useful to filter the signal to hear a certain feature more clearly; typically, with a tunable band-pass filter.

Some data either (a) is not suitable for direct conversion to audio, or (b) contains information which isn't revealed by a direct conversion. In such cases, it may be useful to perform translations or extractions to get an audio signal that is useful. For example, if the data consists of a slowly changing series (1, 2, 3, 2, 3, 4, 5, 4, 3 ...), it might be useful to translate these into tones in a chromatic scale with the formula:

frequency = 440 * (2^(n/12))

An extraction is usually done by a computer program that searches for a certain feature, and creates a sound when it finds it.

The more you know about the signal, the more specific you can be in your audification of it. Therefore, it's appropriate to use the most raw signal dumps (with time scale changes if necessary) when you don't know what you're looking for, and techniques like filtering, translations, and extraction when you're looking for something more specific.

3. Sample Applications

The first audification I did professionally was to listen to the data communication traffic between a cluster of Sperry airline reservation terminals (SVT series) and a cluster controller made by InnoSys (my employer). In this setup, the cluster controller would exchange data with the terminals by polling each one in turn. There were two bugs I was trying to fix: (1) sometimes a given terminal would hog the controller, locking out the others; (2) sometimes a terminal would start spewing data, causing the controller to crash. We had a data-capture tool, but the bugs appeared infrequently, and the capture tool's storage was limited, so by the time we'd realized something had gone wrong, the trace data that would have shown us the cause was gone. Listening to the signal, we learned to recognize the sounds which led up to a crash, allowing us to stop the capture tool quickly after the problem occurred. There was also an unexpected benefit: we noticed various aural anomalies that weren't associated with any known bug but which led us to previously unrecognized problems with the polling algorithm, which we were therefore able to fix and tune before they became more problematically manifest.

Audification can be used to enhance a 2D scene. Let's say you have a dataset which consists of a 2D array of data elements, each of which has many parameters -- too many to show visually (ie: more than 3). Some of the parameters can be translated into sound, so that when you move a mouse pointer through the area, the sounds corresponding to the data in that area are produced. Using this, you can find hidden variations in the data, etc.

Computer software profiling (finding out how much of the computer's time is spent doing which actions) can sometimes be improved with audification. In real-time applications especially, audification can show how the efficiency of the software changes as the application does different things -- leading you quickly to the weak points.

4. Getting Started with Audification

There are two main ways to get started with audification: with a computer, and without.

Audification of existing signals. Many signals can be turned into audio with little or no trouble. You can start with a simple battery-operated audio amplifier and speaker; I got a cute combined unit (that looks like a miniature Fender guitar amplifier) at Guitar Center for about $40; you can probably find something just as good (for this purpose) for less money at Radio Shack, a garage sale ... or, you can just use your home stereo (though you want to exercise caution in proportion to how much money you have invested). To start with, you don't have to attach anything special to the amplifier; just a long wire will do. Turn the volume up until you can hear a 60-cycle hum when you bring your hand near the wire. Then, walk around your house, bringing the wire near things: electric clocks, the fuse box, stereo equipment, and so on. If you have computer equipment, connect it (through a 1-megaohm resistor to limit the current flowing into the amplifier) to any non-power-related component. It's very interesting to listen to the RS-232 data flowing between a modem and a computer while you do various things on the web.

Using a Computer. Many computer environments offer sound capabilities. Some programming languages have sound primitives built in (e.g. BASIC's "SOUND" statement). Some computers have easily accessible sound hardware (e.g. the IBM PC's "click" speaker). For getting started, it isn't important to have precise control; anything you can do that makes a noise will do. Take some program you've written, and add sound to it. You can use sound to generate a run-time profile of where the program is spending its time by having each subroutine entry point switch to a different sound; when you run the program, one sound will emerge as the most common. Or, you can have the program switch sounds for each iteration of a loop, to give you an idea of how fast the loop is running.

Whatever technique you use, you will begin to develop "audification consciousness" -- the awareness that the whole world is vibrating, and that if you know how to listen, you'll hear it.

5. References

I'm sorry to say that although I've read a few interesting articles on audification, I don't have any references at hand at the moment. However, Gregory Kramer edited a book on the subject, AUDITORY DISPLAY: SONIFICATION, AUDIFICATION AND AUDITORY INTERFACES, which has an on-line bibliography here, which may be of some help.

When I searched the web, I was surprised at how much new material had appeared since I first got interested in this subject. So, if audification/sonification sounds interesting to you, there's plenty out there to learn!