Data beautification with MIDI::Realtime


Dirty data

Gardeners put great effort into removing beautiful daisies from their lawn. Pop music producers work hard to make songs as bland and unchallenging as possible. Programmers spend a lot of time boiling down, filtering and making data dull enough for management.

It doesn't have to be that way.

Whether you're averaging out website hits into graphs, analysing logs or munging survey results, it doesn't feel like an entirely creative process. You're making dirty data clean by filtering out any quirkiness that could make decision makers nervous. For a change, why not try highlighting that quirkiness? Why not throw the dull stuff away, and bring out the underlying beauty of the data?

(void) music

The first time I started thinking about data beautifying was when a good friend of mine came up with an interesting idea. We are fellow members of a very strange mailing list called '(void)', and her idea was to turn the list archive into music.

Hmm. Emails into music, weird. But hey, in previous lives I've hacked perl to turn databases into spreadsheets, and specification docs into C code, so why not mail into sound?

Hmm. Again. It was difficult, but possible... making beautiful code is always a big motivation, and the idea of making code that makes beautiful music was too hard to resist.

We dreamed up some more detail, and came up with two reasonably workable ideas about what the music should be. The first was that you should be able to hear each thread separately, so you could hear each one appear, intermingle with the others and die. The second idea was that the music should show how the list traffic changed over time; not only rises and falls, but also changes in texture. Different people post in different patterns, and over time, people from a wider range of timezones joined the list.

Great, two simple ideas that could work well together. I got to work.

Choices and non-choices

First came the choice of development platform. I use Perl and GNU/Linux for just about everything, and saw no particular reason to change my habits for this project.

The next question was what to make the sounds with. I'm afraid I didn't go through a rigorous decision process here either. I have this really nice Roland JV1080 synthesiser module sitting around on my desk. It cost a lot of money that I haven't finished paying yet. Basically, I had to use that.

So I had Perl, Linux and my synth. Now we're on to real problems. How was I going to make them all communicate?

Talking with MIDI

Most modern synths, mine included, use an old but well defined protocol called 'MIDI' - Musical Instrument Digital Interface. It was clear that my solution would have to involve that in some way. You have probably heard of MIDI. It allows musical instruments and programs (such as sequencers and Perl scripts) to send musical notes, controller changes (such as pitch shift and modulation) and other events to each other.

If you want to experiment with Perl music, you might think MIDI is no good to you without an external synth. But hold your horses, you might well have an excellent MIDI device without even knowing, inside your computer. Nowadays, standard consumer PC cards such as the SoundBlaster Live! and AWE series of cards are packed full of features, including very able onboard synthesisers. If you want to get into music, you might already have all the resources you need.

Anyway, back to my problems with getting Perl to talk MIDI.

A quick look at CPAN revealed a very nice suite of Perl modules called 'midi-perl', written by Sean M. Burke. They provide nice interfaces to the reading and creation of MIDI files. I had a lot of fun playing around with it, making very strange MIDI files that made my synthesizer sound like it was screaming with the pain of a thousand agonising deaths. I think they call it 'illbient' music.

However, while they have massive scope in areas such as algorithmic coposition, the midi-perl modules aren't designed for 'real time' work. You have to compose your music, save it to a MIDI format file, and then play it using a sequencer, or a program like elegantly named 'playmidi'. This was a big drawback for me - I wanted to display the data visually as well as musically, which a MIDI file wouldn't allow for.

MIDI::Realtime

So, as good as it is, midi-perl was not the tool for this job. Furthermore, after a lot of looking and asking around, I realised that nothing existed that could fit my needs. So with the MIDI spec in hand, I got hacking, and MIDI::Realtime was born.

MIDI programming was much easier than I expected. As long as you're not worried about synchronisation, it is just another protocol, really. Sending a note to an instrument is just a matter of writing a simple series of bytes, containing the note and velocity, to /dev/sequencer. Have a look at the code to see exactly how it all works (you can find it in the MIDI directory on CPAN). It has a nicely simple object oriented interface, and I've even put some documentation in there, with examples.

At the moment MIDI::Realtime only supports writing to a MIDI device, but reading should be supported soon. Let me know if you'd like to help me with the implementation, testing or application of this.

Playing the archive

Once I had Perl and my synth talking, the rest was a breeze. I hacked up a messy script to read the mailing list archives into a database, and a second one to read the database while playing sounds. I split up these two tasks to try to remove as much time consuming work as possible from the music playing part.

The music player itself works by scaling down the life of the list to 10 minutes, then stepping along that timeline, playing posts as they appear upon it. Each time a new thread (or subject) starts, the script randomly choose and plays a sound for it, and the subject line is scrolled onto the screen. Then, whenever a post appears from that thread, the sound is repeated. To give extra context, the author of the last mail to be played is shown on the bottom line of the screen, along with the date and time that it was posted. That's all there is to it.

The effect though, is spell binding. At first, the sounds start calmly and slowly, with occasional flurries of activity. Subject lines slowly scroll up the screen, with the authors flashing up below. As time goes on, the music increases in intensity, with more diverse sounds reflecting the greater number of threads running at the same time. The night-time pauses for sleep gradually become less pronounced as the subscribers cover more parts of the globe. The music becomes more manic as traffic increases, until becomes impossible to read the authors names as they flash by. And then, just as the relentless furiousity is becoming really far too much, it all stops. There are no more posts yet - we have reached present day.

I presented the music at a strange electronic music party in London called VXSLAB. We beamed the visuals on to a wall as the music played, and few list members who were present grouped around. It turned out to be quite an immersive and emotional experience, with the subject lines evoking many memories for us. We shared many grins. By the end, one list member was so overcome with emotion that he gave me a big hug. Mission accomplished. :)

You won't be able to see the visuals, but an audio mp3 recording of this performance is available for download. I'm not so proud of the scripts as I am of the perl module, but I can send you those on request.

Conclusion

The most important thing I've learned from this project is that data has many beauties that are usually ignored. A simple graph could have contained all the information that the music did, in a more precise manner. But it wouldn't have communicated any of the very real emotion behind the birth and growth of the list. I've plotted a few graphs in my time, but none of them have earned a hug from anyone.

I should also say that since doing this, I've discovered that there are many other people doing similar things. It seems that this automation of the creative process is a form of generative art. While this is not a completely new area, it has a great many unexplored depths. So if this has interested you, why not join in the fun?


(c) Alex McLean <alex@slab.org>

This article originally appeared in the January 2000 issue of perlmonth, an on-line journal for Perl programmers.