In 2006 I was approached by John Jacobs of ABC Radio fame. He had seen a VJ performance I’d done recently and thought I might offer some help to a project he was planning called Umami. The idea (expressed in my own words) went like this: Umami would watch the news feeds coming into ABC-TV all day and then condense them into the most interesting parts. Then late at night it would recall the images it had stored during the day as an ambient abstract flow that would be entertainment for late night viewers – something to have playing in the background along with the music and conversation that you may already have going. It would acknowledge and power the night-owl channel surfing that TV has offered for decades.
Now I am aware that there’s quite a few bits in there that raise questions, and we will come back to them all I promise.
John would be in charge of the idea and I would help out on the technical side of it. He wanted to get ABC-TV interested and hoped that we could get a demo going in a reasonable time frame. While John is considered a national treasure by some at the broadcaster there was much doubt about the project and the more I looked into it, the more doubt I had myself.
Then came The Pool, which is an ABC project to address shared media. Rather than my describing it here, you can see it all at the link. The thought came that rather than Umami watching the airwaves it could sit at the edge of the Pool and work with the media there, which would be already tagged and permitted and so on. I sat in on a few of the initial meetings at the ABC as an ‘interested artist’, but Pool had enough things to figure out without the added burden of our schemes, I fell out of that loop. John moved down to Melbourne and I had to find a new job and so things moved on.
But I knew that Umami needed some serious attention if it was going to ever happen and some of that involved the kind of research and development that I was not trained to do. On the advice of Norie Neumark I went back to university and undertook an undergraduate thesis, and despite feeling like the worst kind of idiot the whole time I managed to earn entry and a scholarship (which might come in handy some day). So now, four years later, I can begin.
Some of the issues that face this project.
Thinking back to the original idea, there are some words that gloss over processes that need a strong definition. By what rules does the system ‘condense’ the material – that is, by what rules does it delete or ignore? What do we mean by ‘interesting’ and how can a machine decide what is interesting or not? When it recalls the material by what rules does it arrange the work and what decides the abstraction and filters applied to the vision? All of this assumes that we can automate the business of the film editor or VJ, which assumes that we can define that in machine terms. I think that this is very uncertain.
To my mind Umami is a near fit for Freud’s concept of ‘dream work’. It takes material from the day’s experiences, condenses them according to their value in expressing unconscious themes and arranges them in a puzzle that slips by a censor in a late night recall – seemingly meaningless but heavy with unconscious meaning. Although Freud meant this to be an accurate description of a brain mechanism it is a pseudoscience more poetic than the basis of a real world device. Nevertheless the concept of the ‘dream work’ is for me the only interesting aspect of recycled art. Taking sections of existing media and reassembling them to imply a new thought is a kind of alchemy that turns video lead into gold. Alchemy, I am convinced by Duchamp, is the basis of art.
Diverging from the Umami idea.
Rather than make the machine act like a human, let’s move the other way. We assume a human operator and translate their (neurotic?) wishes into machine terms. An illustration: postscript is a computer language but it’s usually created via a high level tool such as Illustrator, very rarely does a human write the actual code. MIDI is a serial hexadecimal code but generally it is created as notes in a composing tool such as Pro Tools. We describe what we want to see or hear in human terms and the machine handles the translation. To answer one question above – ‘interesting’ is a decision made by a composer, and passed to the computer as guidelines for action.
Next question – how can we capture wishes as the kind of data a computer can parse? My idea is this: we need to survey how people do their work now, then form a pseudo code from that. Simple example – what really constitutes Picasso’s ‘blue period’? When Picasso grabbed the Cyan control and raised it, what was the link with his mind? This pseudo code would need to be different for each conductor. So the machine will present an abstraction layer into which the conductor’s meaning can be mapped. For Picasso it would be a surface that presented meaningful terms and for the machine a set of parameters that describe activity.
That is a very bald overview of the idea, and the next post will dig more thoroughly into the actual plan of action.
