Music As Symbols Two ways to symbolically represent the musical events that comprise the Close Encounters lick; above in European musical notation, below in "piano roll notation" where notes are mapped with a keyboard. All the mapping strategies we look at in this article share one convention: time runs from left to right. [images from Cakewalk™]

      “Seeing is believing” at least according to residents of the great state of Missouri. But what is hearing? Our thoughts, the very way we make sense of the world, are mapped into language. Words are auditory - evanescent and invisible. Walking upright and wielding a quagga femur, an early hominid probably wouldn't be mistaken for a gorilla. But could any creature be considered fully human without a language with which to create and communicate a shared vision of the world? True, it’s possible to base a sophisticated communication system on gestures and signals, but sounds can penetrate into the darkest cave. And from your backside your clansman could shout “hey Grog, look out for that smilodon!”

      Not only is sound invisible, it is restless. We perceive it as a series of changes of amplitude, pitch and harmonic texture moving inexorably through time. This ceaseless motion makes it difficult to get a handle on the audio realm. It could be said that culture began with true language: it wouldn’t be unreasonable to say that true civilization began when our ancestors learned to map language into a fixed graphical format - writing. Later, musicians developed ways to map their audio art to help them control, study and perfect musical communication.

      Writing, whether words or a musical score, relies on the use of learned symbols. Written text works because we understand the language it represents. Written music in the classical European tradition, for example, is supported by a system of music education and musical directors that ensures performers interpret the notation appropriately.

      But what if we don’t already know where the meaning is? Bird songs, a Jimi Hendrix lead, or the sound of an undersea earthquake are examples of audio phenomena where the meaning is obscure. Today, you can map these sounds to an image - called a sonogram or audio spectrogram - with just about any computer. Having seen the sound, you’ll probably find you hear it better too.

Figure 1 digital sine wave fig. 1(a) amplitude wave view fig. 1(b) spectrogram view

      On an audio spectrogram time is frozen as the horizontal, or X-axis. Frequency (pitch) is depicted on the vertical, or Y-axis. Amplitude (loudness) is depicted by saturation, or color-coding. To illustrate what an audio spectrogram is, we’ve created a little demo using the ever-popular Close Encounters lick, played on several patches we set up on a Korg M-1 synthesizer. You can listen to the sound by downloading the mono hi-fi MP3 at the bottom of each figure. For each example we’ve included an an amplitude waveform view, and below that, an audio spectrogram. All of these were rendered by Cool Edit™ from Syntrillium. Click on any audio spectrogram to get a closer view including a time scale (in hundreds of milliseconds) on the bottom and a frequency scale (in Hz) on the right.

      In the first example (fig. 1), we hear the most basic possible sound, a simple digital model of a sine wave, with no effects. It’s not very pleasing to the ears, but it is highly instructive.

	Figure 2 electronic piano fig. 2(a) amplitude wave view fig. 2(b) spectrogram view

The energy of the waveform is almost all found in the fundamental pitch, and there are very few harmonic overtones. Therefore, we see a single bright line along the bottom of the audio spectrogram, and only the faintest of parallel lines representing the very weak harmonic overtones. This is what many people would call a “pure tone”.

      Contrast that with our second example (fig. 2), the same series of notes played on an electronic piano patch. Piano type sounds are rich in harmonic overtones and these appear on the audio spectrogram as parallel horizontal lines. Pianos are percussive, this one especially so. The string is hit by a hammer and therefore a piano is loudest at the moment the key is pressed. After that the volume begins dropping immediately. To add a bit of complexity, pianos also emit a momentary burst of noise when the hammer strikes the string. The bright vertical bar at the beginning of each note clearly shows this moment of non-harmonic noise.

      The third example (fig. 3) is a flugelhorn like sound. Note that, like the piano sound, it is rich in harmonics. The only obvious difference is what synthesizer players call “the envelope”. There is no hammer strike burst and the horn’s amplitude fades less because the player continues to provide an energy source for the tone by blowing air through the instrument.

Figure 3 flugelhorn patch fig. 3(a) amplitude wave view fig. 3(a) spectrogram view Figure 4 chorused string patch fig. 4(a) amplitude wave view fig. 4(a) spectrogram view

      In the final example (fig. 4), a rich, shimmering string-like patch, the fundamental pitch that dominated the simple sine wave (example 1) is mostly obscured in a sea of harmonics, echo and chorusing effects, which can be seen as small s-curves in the tones. While it’s difficult to pick out the five notes of the passage by looking at the spectrogram, most people would still hum the same notes if they were asked what it sounded like. It’s amazing, really, as the spectrograms make it obvious that the sounds are really not very similar.

      Our Close Encounters comparison demo is a bit trivial, but if you understand the basics of reading an audio spectrogram you can use it to help you see more mysterious sounds. The next two examples are of a wood thrush I recorded to DAT in the field earlier this year. By examining the variations in the wood thrush’s song on the spectrogram, details one might to miss while swatting mosquitoes in the woods start to become obvious. Note the pair of staccato starting notes on the first phrase (fig. 5(a)). You’d be likely to miss that part of the song among the many other birds singing in the woods in May. Notice also the structure of the trill in the second phrase (fig. 5(b)); there’s a strong piercing sound at about 5.3 kHz beginning about 0.8 seconds into the phrase, embedded in a broad band trill ranging from 3 to 9 kHz (and appearing again at 11 kHz).

	Figure 5 wood thrush fig. 5(a) thrush phrase 1 fig. 5(b) thrush phrase 2

      Wood thrushes sing a surprisingly rich and complex song, if you’re interested, download the long version [about 1.3 M]. Note that on all of the thrush recordings, filtering has been applied to remove extraneous noise: the portion of the spectrum removed appears black.

      If you’d like to try your hand at making some audio spectrograms we’ve included links to some low-cost or freeware downloads that will let you get started. If you’re interested in seeing more audio spectrograms I especially recommend the NOAA web site’s whale recording gallery. The whales are interesting, but the really good stuff bears the tantalizing label of “unidentified”. A couple of these unknown ocean sounds are perfect listening for Halloween.