Umami

The following article is part of my unfinished doctoral thesis. There are missing sections and errors. It may have some interest and so why not share. It’s going to be uploaded in stages so check back.

Abstract.

‘Visual music’ is the expectation that organised light has the same potential for aesthetic beauty as organised sound. That music made in each should share compositional structure seems reasonable, but despite a long history of theory and practice there is currently no equivalent to standard western musical notation for composing in light. For the practice to develop a more expressive and collaborative form, a generally useful notational system is desirable.

For scientists such as Newton the comparison involves ‘elegance’ – that the universe has minimal complexity, and where light and sound derive ratios and mixtures from a single underlying principle. As this direct connection has been disproven, artists and philosophers such as Rudolph Steiner have moved the argument to spiritualism, where sympathies between the mind and the universe are said to exist. This has unfortunately allowed a range of incoherent notions to conflict and many private notations to be developed and discarded.

Creative research cannot claim to discover scientific fact, but it can illustrate the ideas, and can employ useful internal logic in doing so. Given that spiritualism’s prominence in the history of visual music it should not be discounted when forming a new system, but needs to be reformed and updated to the best current model, that of individual psychology.

The performance system described here recalls moving images under the control of a musician. Unusually, it allows scoring and reproduction according to a notational system where the work is described by an emotional arc, but without narrative. This is based on the character of the source material as decided by the composer and measured using a widely documented psychological scale. The combination of open documentation, software and systems provides a benefit to artists working in this area, and opens discussion of a common problem for the art form.

Introduction

Isaac Newton’s Optiks was first published in 1704, being his account of experiments with light since the 1670’s. Here is the familiar demonstration of a prism splitting a shaft of white light into a spectrum of colours, then another collecting them back into white. Optiks is a revolutionary text in a number of ways; written in English to appeal to the growing audience for ‘natural philosophy’, it refutes the authority of Aristotle’s Physics of nearly two thousand years before. Newton declares his intention, ‘not to explain the properties of light by hypotheses, but to propose and prove them by reason and experiments’. Optiks reformed a tradition that preferred reasoning to experimentation and thus the book signals a paradigm shift in the history of science.

But Newton worked at a twilight between the philosophy of sympathies between disparate things, and the categorising of differences in similar things. Keen as he is to reform the methodology, he continues some of the thought processes of the ancient world.

In Book 1 of Optiks, figure 11 shows a circle, around the circumference of which are placed the now familiar named colours of the rainbow. Between them Newton has placed musical notes, in imitation of similar wheels then used to diagram musical harmony. The illustration accompanies text setting out the problem of mixing primary colours to a known result (page 114). The seven colours are not based on any stated physical principle; instead he divides the spectrum, ‘proportional to the seven musical tones, or intervals of the eight sounds’, specifying the portions unevenly to match a musical scale. His decision is not explained, and the impression is that his readers would find this correspondence familiar.

As we will see, the ideas behind this wheel inspired an art form known as ‘visual music’. In its simplest form, visual music is the expectation that organised light has the same potential for aesthetic beauty as organised sound. Here the term ‘music’ has an ancient meaning – being something ‘of the Muses’ and thus expressed equally across mathematics, poetics and so on. Some current terms in art continue this relationship – the ‘composition’ in visual art, its ‘harmony’ and ‘texture’. In reality, audible music is still closer to mathematics than most visual arts, a difference neatly summed up by Walter Pater’s 1888 quip that, ‘all art constantly aspires towards the condition of music’.

Visual music has that aspiration at its core. It is a realm of mechanical light machines and projections, colour symphonies and sounds painted on canvas, abstract films and psychedelic videos. It is vibrant, chaotic and seductive, but the closer one looks, the more obscure the rules of its composition seem to be. For a practice dedicated to light, the mechanism is largely occult – and the reasons are in the history of the practice.

It should immediately be differentiated from narrative. Visual music is non-objective, free of actors, events and locations. The emotional contour is not based on comedy or tragedy. It is supposed to affect the feelings of the audience through pure colour, form and sequence. The comparable audible form is not opera, but rather the sonata, where the abstract elements of pitch, timbre, rhythm and so on are required to act directly on our emotions.

Let’s now describe the specific problem addressed in this study.

The catalyst came in 2008 when I was asked to collaborate an automated system to perform visual music as a broadcast. The proposed system would collect daytime video imagery chosen by a composer and follow their instructions to re-arrange it overnight as a kind of Freudian dream work. That seemed to be a technical challenge, until I was surprised to find there is no universally accepted notation to compose or to document an abstract moving image composition without a recording. The question shifted from ‘how do I do this task in the expected manner’, to ‘why is there no expected manner?’

I could invent a unique language for the system, but how then could I convey my design to peers? How could I record the instructions in such a way that another artist (or system) could perform the composition later on, with different sources and yet covering the same emotional contour? In trying to realise this system, I encountered a problem for all visual musicians – how to score the work.

An immediate solution seemed to be in the field of signal analysis – speech recognition, pitch detection, video scene markers et al. These are technical solutions to the difficulty of imposing structure on a monolithic recording that produce ‘content descriptions’. MPEG-7 is one such international standard for describing media, but it is designed for analysis, not synthesis, and based around raw measures and narrative structures such as ‘who’, ‘where’ and ‘when’.

Consider the problem of describing one of Van Gogh’s Sunflowers. MPEG-7 can hold the ‘what’ of ‘sunflowers’, and the ‘who’ of ‘Van Gogh’. It may also describe the overriding yellow tint of the whole. But it cannot discern the painting from a generic photograph of sunflowers without knowing the artist. It holds nothing that abstracts the character of the work. A notation for visual music needs to describe emotional scale, to allow interpretation by another musician.

A lack of notation could be evidence that it is unnecessary, or inappropriate to the form. For example, in painting, the canvas is the document in itself, which promotes an emotion as a whole and about which even the artist speaks analytically. But a notation has been central in the aspirations of visual music history, which is replete with instruments, performances and systems for scoring, made even by the painters.

Here is a good moment to weigh the advantages of scoring. Any documentation allows review. A distinction is then needed between scores and recordings, between symbols arranged to represent the intended meaning, and a single definitive capture of the original communication. To borrow from Sontag’s On Photography – there is the difference between a ‘trace’, which is a facsimile of the unique object, and a ‘representation’, which is a recreation drawn from the original source.

A traditional oil painting, sound recording or motion picture is a whole that cannot be perfectly decomposed. It is understood that analysis of the work is necessarily incomplete, and made in reference to the original. For example Oskar Fischinger’s 1935 animation Composition in Blue is, by design, only available as a few authorised 35mm prints. It cannot be digitised, analysed, reformed or improved, and so its dialogue with the wider community is restricted.

Fischinger’s policy is in keeping with the period of recorded media. The late 19th century introduced recording technologies that promised greater fidelity. Stravinsky was notably delighted by sound recording, devoting numerous pages of his autobiography to raging against ‘interpretations’ and applauding recording as ‘the elimination of all chance elements’ and ‘music free from any distortion of my thought’. (Stravinsky 1962 p152.)

It was the ethnological composers such as Bartok, studying Hungarian folk music, or Grainger collecting English folk songs, who described the most limitation in notation. The music they transcribed involved intonations and phrasing exceeding the fidelity of traditional scoring and requiring new recording technologies. Grainger in particular was led to a famous pronouncement about standard notation being ‘absurd goose stepping’, and spent much of his later life building mechanical contraptions to perform music that exceeded the gamut of bars and notes (Grainger, in Simon 1983 p127).

Recording provides a master. In comparison, musical notation can be orchestrated in multiple ways and still be the same piece of music. Performance allows iterative development and improvement, which benefits the field as a whole. It can be seen as additional potential exercised by the artist. While Fischinger limited his work to masters, Sol LeWitt scripted his drawings so that others could perform them. As recorded media has declined in importance in the current century, artists are more likely to find balance between the unique object and openness to interpretation – with directors cuts, extended remixes, interactive art and so on.

Most visual musicians avoided collaboration. Solo ‘inventors’ populate the history. They patented ideas and refused to share them, or based their work on such idiosyncratic notions and neologisms that no one else could fathom them. In general they performed their own compositions on their own instruments rather than let any conductor reform it. They sought the same control and singular vision as painters.

The restrictions are evident even before mechanical recording became available. The problem seems not to be antagonism to notation per se, but a chaotic multiplicity of closed systems. And so, where audible music has enjoyed long periods of stability in which the art has been refined, visual music seems to have kept at the status of a ‘new media’ for centuries.

Because of this antagonism, patents are often more helpful than the written documentation in preserving process and intent. Following this pathway the history divides by technological leaps – experimental video is poorly connected with abstract film, which is not well linked with the mechanical light devices of the 20th century. None of these connect clearly with the efforts of ‘mobile colour’ painters, or composers that employed light in concert. There is a suspicion is that the various orchestrators of light have actively avoided a common language with which to negotiate their inspirations.

To avoid my adding yet another idiosyncratic system to the debris, it’s been necessary to study those that have already been and gone, and I have sought to find common themes in past artworks and theories, to relate that to what resources currently exist, and then present a plausible, open language. I will describe this survey in the first chapter that follows.

While notation is the focus, it calls up a more fundamental question. Why should visual art aspire to the condition of music at all? What is this condition? Music in its purest, most ‘absolute’ form is free of lyric and narrative. It elicits emotions purely by audible structure without representation and analogy; so it is ‘non-objective’. It approaches a hypothetical relationship between laws of nature and aesthetics.

Newton’s colour wheel illustrates what at one time seemed to be a direct connection between physical and creative structures. It is perhaps the last point where a science of light music seemed plausible, as afterwards came refutation and a retreat to the romantic and spiritual. Visual art became entwined with pseudo-science, especially that of the Theosophical Society, who become enormously important to the whole of visual art.

Spiritualism involves a yearning to know a simple relationship between all things in the universe, revealed by all manner of arts and sciences. It represents what I claim to be a useful pseudo-science, a coherent internal system that imitates science without external validation. It has been a partner to science throughout history, in alchemy and astrology – as when Kepler dispensed with the notion that the planets orbit in exact circles, only to then try place them by the Platonic solids. Pseudo-science has an unpleasant connotation when aligned with cancer cures and perpetual motion, but this misuse should not defame the whole idea.

Of course science itself is not entirely free of the spiritual. What the artists call ‘absolute’, the physicists call ‘elegant’. As Tsilikis describes it, “Elegance is the principle that postulates the adequate representation of a physical problem in mathematical formulae which bestow unity, symmetry and harmony among the elements of the problem.”

Elegance assumes that the universe is no more complicated than it needs to be. When Newton divided the rainbow according to a musical scale, he appealed to both scientific elegance and the artistic absolute. That colours and sounds should follow the same ratios is more elegant than two different systems. As for the absolute – the seven colours fit neatly with the known seven planets and seven metals, both symmetries of the medieval cosmos. If it seems odd that Newton should attempt to preserve a mediaeval framework, consider Einstein’s ‘cosmological constant’, which he introduced in general relativity to preserve Newton’s own framework. (We have since found that a very inelegant and almost occult ‘dark energy’ might explain this number that Einstein detested as ‘his biggest blunder’.)

I’d argue that this scientific elegance is the ‘condition of music’.

Much of what we understand of the universe is not the reality or even a mathematical model, but a conceptual illustration of the model. Quarks, black holes, and climate change – all are commonly known through illustrations that may be inaccurate, but still serve society by providing some understanding of nature. When an artist conducts research they are often concerned with this illustration of reality, and how to improve it.

Any creative practice based research requires that we allow the logic internal to an artwork to be indicative of the world at large. Useful pseudo-science is a necessary part of the proposed artwork and my task is to update and improve on the pseudo-science of previous artworks. The spiritual has not been a unreliable mechanism – it is after all occult, and it is hard to build a system to evidence the unseen.

Artistic abstraction draws on what are claimed to be universal symbols, forms, scales and structures that describe a greater reality. But it is also supposed to flow from the genius of the individual artist. To reconcile these two demands each artist needs to somehow carry about some internalised insight of the absolute. Despite its major historical role in visual music, the ‘soul’ must be replaced by something more pragmatic. We need to update ‘the soul’ to something that can be tested and found significant in repeated experiments, and this leads to depth psychology.

Depth psychology is a familiar tool in narrative media, where Jung’s notion of a ‘collective unconscious’ is common in story design. Jung claimed that ‘archetypes’ or symbolic personalities appear in myths, characters that have familiar identities and functions in guiding the moral instruction that myth provides. None of this is much of an advance on the esoteric, but it is has proven to be effective in designing an emotional contour in narrative.

Then there is Freud. One early proposal for my broadcast project was based on his concept of the dream work. Experiences throughout the day are stored in the mind; in sleep they drift ‘down’ into the unconscious where they become attached to repressed ideas. These ideas are then able float ‘upwards’, evading the censor that usually presses them firmly below the limit of consciousness. The proposed video system would have been fed video imagery throughout the day, at night, emotionally charged material would rise up to be presented on screen in a ‘stream of consciousness’.

Problems with this bad design surfaced faster than repressed imagery. What mechanism would decide on the sufficient emotional charge of an image? How can a machine detect that footage is significant and to be included in a programme? The answer was to assign a human conductor to describe what was valuable to them and have the machine interpret these rules. Which then inspired a much better question – by what means can a composer describe the emotions conveyed in a piece of video material?

A recent branch of psychology offers a solution. Individual Psychology is concerned with differences between people’s behaviour, and does not need any phantom mental apparatus. Nor does it deny individuality, as does Behaviourism. It is based on observable personality traits that have been found consistent over multiple studies, which have over time been reduced to five – the smallest set required to discern individual differences. This kind of psychology is quite elegant in the mathematical sense, and greatly reduces the complexity of the required mechanism.

Most important – it is a system that has already been peer reviewed and tested in practice. There is no need to reinvent and justify the validity of this measurement, so long as it is kept firmly in mind that it is being employed with the logic internal to this artwork. Instead of claiming that e.g. red always symbolises anger, the composer can describe the emotions they intend to be conveyed by a section of visual music, using a psychometric scale that has been shown consistent in making such measurements in people.

Obviously the measurement of ‘personality’ in things other than people is a pseudo-science, of the ‘music in a minor key is sad’ sort. I will devote some time in the second chapter of this paper to placing it within existing conceptions about art and creativity.

Once a system is designed it needs to be tested, and it is difficult to know the best venue for such a work. The gallery space, concert hall, the nightclub and so on each represent a culture that explains how to engage with a work. Visual music has been tried in all these spaces and in most cases became a synchronised light show explained by music. Even when a traditional artwork is examined in a gallery it can be difficult to grasp the intention without reading the exegesis.

Part of good design must be the simplest, most effective means by which an intended audience can be led to understand and appreciate the work. The first demonstration should remove unwanted variables and call upon the experience of the viewer, and if that succeeds then others can expand and challenge the system with less familiar content.

Past visual musicians have demanded that non-objective art and non-narrative video be privileged as a new medium, which has become increasingly hard to justify. In fact the anthropomorphic principle is older than writing. The gods and spirits that were said to animate the planets and seas are evidence that people will project personality on anything that sufficiently mimics motivation. This is at the heart of all animated media.

An obvious inspiration comes from representational language. It’s a reasonable expectation that an audience will know what is meant by ‘a calm sea’, ‘brooding clouds’ or ‘majestic mountains’, and that a natural formation may represent a projected emotional state. For this reason I decided to demonstrate the proposed notation by composing for a ‘sky’, with artificial light and clouds, hoping for acceptance of the same feelings as may be experienced from the real sky.

In chapter 3, I will detail the design of the system, and the demonstration made with it. In the final chapter I’ll describe the responses to the system by audiences, and how this leads further work on the notation.

Part 1: A Brief History of Visual Music

In this chapter I set out some of the history of visual music, so that the reader may see the current work in the context of the art form, the influences that act upon it, the traditions that it maintains, and some of the unresolved challenges it takes up. Visual music has too often been made arcane, and one objective of the current study is to lift the practice out of this designed obscurity. At times there will be diversions into the philosophy of science and spiritualism to convey the flavour of a period or a particular artist. I’ll keep most of that discussion in the chapter following, in which we deal with the principle of elegance that structures the work and informs the system of notation.

Visual music involves the arrangement of both colour and texture over duration, but most work has compared the frequencies of colour and musical pitch. That is the spine for this history; other elements will be addressed as they appear. By itself colour has multiple histories. My focus is on attempts to design systematic arrangements of colours in a scale, in analogue to the way that D is a higher musical pitch than C.

The utility of our colour perception is evident in the particulars of human sight. Our retina is covered mostly with ‘rod’ cells that respond to brightness and movement. A smaller number of colour responsive ‘cone’ cells tend to the middle of the retina. Three sorts of ‘cones’ are usual, colloquially ‘red’, ‘green’ and ‘blue’. Some people have four sorts, which may be an evolutionary investment.

The frequencies favoured by the red, green and blue cones are not evenly spaced, and their populations vary across individuals. The ‘blue’ cones are distinct in responding to wavelengths around 420 nanometres. The ‘green’ and ‘red’ cells are near neighbours, as the ‘green’ cells respond to greens around 534 nm and the ‘red’ to yellows around 564 nm. As they are so close, the ‘green’ cells are rarely stimulated without the ‘red’, and perception within warm colour involves discrimination between the signals sent by both.

Colour vision is not a simple math of equal primaries. The signals sum at some points along the neural network but subtract at others, so that some colours are perceived by the absence of their opposite. This difference between colour as a mathematical construct and a perceptual apparatus has caused confusion, as artists have generally assumed that there are fundamental colours that combine to create the range we see.

Ancient Colour Systems

Aristotle catalogued three existent theories of colour in his lecture on the Senses circa 330BCE. The third, which he finds least problematic, is that pure black and white blend into an indivisible mixture, the ratio of which defines the resultant hue. It is surprising that colour is supposed to mix from black and white – this stems from both translation and world view, and most relevant here, a long standing theory already in place about the relationship of light and music.

In explaining the confusion Sorabji (1972) starts with the translation from the Greek of leukon and melan – ‘bright’ and ‘dark.’ ‘Bright’ includes all sources of light, which at the time was the sun and fire, both having a warm hue. ‘Dark’ means absence of light, which is perceived as blue by the discrimination between red/green and blue by our perceptual apparatus. It is not as difficult to imagine that warm and cool make all hues, and this idea reappears with Goethe.

But Aristotle’s theory actually uses ‘bright’ and ‘dark’ as the amplitude of light that vibrates in patterns we see as colour. True to his philosophical method, he adapts an endoxa or prevalent best theory to the problem – that of musical harmony, traditionally assigned to Pythagoras.

An idealised string vibrating across its entire length produces a sound perceived as a single pitch. When the string is half the original length, it vibrates at twice the frequency, producing a note one octave above that fundamental pitch. Divide the vibrations of the string into thirds, quarters etc. and the harmonic series of the fundamental pitch are heard. These harmonics are perceived as pleasing combinations; more so, they are heard as joining into harmony. In-harmonic pitches do not combine in this way, and remain perceptible as tone clusters, and at very high frequencies, as noise.

Aristotle compares white to the simplest mode of vibration. As the vibration of light is subdivided there are ratios we see as basic colours, and then as the vibrations become more complex, more complex colours. Like many of Aristotle’s ideas it has a ‘just so’ appeal, but the maths is not well thought out – a ratio of light and dark allows two different colours to share the same basis. No one seems to have noticed for over a thousand years, or perhaps did not wish to challenge his authority.

The theory also requires some occult part of the universe to be ‘the string’, a part that can be termed a substance or a spirit as pleases the writer. Worse, it fails to explain why the experiment of mixing black and white pigment doesn’t actually make red. Aristotle was sophisticated enough to discriminate between colour and pigment – between ‘purple’ as an abstraction and ‘purple’ as something that comes from a particular source. But the writing that followed often slipped between these ideas.

Colour Arranged by Social Rank

Some colour scales have formed around relative value. Colours took on the attributes of the physical elements that best display them, onto which related qualities in the social or spiritual realm have been added. After the practical value is forgotten, the implied value remains.

The name ‘gold’ applies to both the metal and the colour in a wide spread of languages (for example 金 in Mandarin or thahab in Arabic). It is an extremely stable element and thus long lasting. It is a soft metal, more easily shaped and formed into coins and jewellery than most, and so a convenient means to carry and display wealth and status.

These physical qualities imply that gold in its purest state is the closest terrestrial substance to that which forms the seemingly incorruptible heavens. The shine and colour of gold compares to the sun and generally to those things that are high on a scale – gold is still the top level in prizes and coinage. In alchemical mysteries the desire to turn lead into gold is a metaphor for the purification of the alchemist’s own soul in alignment with a higher power. More practically it is difficult to create the appearance of gold in painting and the practice has been to use gold itself as a primary colour.

Purple is another colour with a complex value. In The Republic Plato remarks that purple is the most beautiful colour. It was a sign of rank around the ancient world, mentioned in both the Old Testament and The Odyssey. The colour was extracted from sea snails in Tyre (hence ‘Tyrian Purple’) at great expense and then shipped around the Mediterranean. Like gold, the dye is exceptionally long lasting and useful as investment property. The values ascribed to purple stem in part from this identification of purple with luxury and excess – ‘purple prose’ being a quick example.

The Romans thought badly of blue, as barbarians used woad to decorate their faces and hair. So a significant moment in western art is the arrival of ultramarine from mines in Afghanistan. As the name implies, this came by ship, was remarkably expensive, and so it was displayed in the most prominent places, particularly in the cloak of the Virgin Mary. Her attributes were those of tranquillity and fidelity – and ultramarine itself adopted this idealised character. The predominant aquamarine colour used in news and current affairs broadcasts is still a symbol of veracity.

So when, for example, we later look at the colours of ‘thought forms’ put forward by the Theosophical Society we should not be surprised to find these colours still taking on these long held representations in Western culture.

Two recent examples of artificially created colour meanings are illustrative. The first is the use of pink/blue as a feminine/masculine signal for infants’ clothing and toys, which has only been in place since a 1940 marketing campaign. Another is ‘hospital green’, which stems from the introduction of strong lights in operating theatres. The white gowns used by doctors to that point became too reflective, and in 1919 surgeon Harry Sherman replaced them and the wall paint with ‘spinach green’ to reduce glare.

While ancient society appears to have recognised a set of primary colours, the comparison is ahistorical. In AD77, Pliny’s Natural History describes pigments as either ‘sombre’ or ‘brilliant’; the latter being an extra expense for the patron (p.283). They then sort according to cost, origin and medicinal properties. He sternly advises that white, black, yellow and red were good enough for the “illustrious painters”, and that the contemporary taste for purple and blue in Roman society meant that, “… there is no such thing as high-class painting”. (p.299).

Gage spends an entire chapter in Colour And Culture tracing this quip about bad taste, through a relationship with the four humour theory, where red was blood, yellow urine and so on, up to the point where the correct colour primaries became a puzzle for practising artists to solve (Gage pp. 29-38). Prime colours have been interpreted as needed. For example, when artificial blue became both affordable in oils, it was retrospectively made part of Pliny’s system (Gage p.35).

Comparative Developments in Music

Before delving into the rise of light music itself it’s worth looking at developments in standard musical notation for a comparison. From the fall of Rome to the 11th century, notation was aligned with the performance of liturgy, directing the intonation of the singing voice by ‘neumes’ that are dots and squiggles taken from Latin accents. When written in a vertical arrangement these are able to indicate relative pitch changes, but not as a measure that would constitute a scale. The melody still needed to be learned by rote, and as there were many songs to know, that could take some time.

Guido of Arezzo is traditionally credited with collecting various inventions into a notational system in his Aliae regulae written around 1030. This introduced the precursor of the ‘clef’, which indicates a central pitch, that lines be ruled to place neumes on an equally divided scale, and other signals such as ‘accidentals’ – the sharp and flat notes that stray off the main key.

Guido was attempting to solve a very practical problem of communicating and teaching liturgical chants – that is, learning and teaching technology. If the pitch and duration could be indicated in a score, then the singer did not need to memorise as many songs, and could take their place in a choir more promptly. It was important enough that Guido was called in by Pope John XIX to explain his system, and encouraged to then document it.

Around 1260 Franco of Cologne set out a system for the durations of notes and so a scale for rhythm and timing. While there had been attempts before, he had the advantage of being the papal chaplain. Again this patronage was aligned with a sensible method, set out in a generally available text, the Ars cantus mensurabilis.

In these examples of music notation history we find two factors not often seen in colour music. Each invention responded to a recognised need in documentation and performance, which at the time connected with the functions of the political establishment. Both were communicated by a manuscript duplicated widely and freely (by a very effective monastic publishing system) and had the approval of a central authority. This combination of authority, dissemination and utility seems the key to music’s comparatively stable development.

Colour Harmony

Newton’s experiments in optics around the late 1600’s can now be positioned as distilling long-standing speculation into a principle. Having divided white light into a spectrum, Newton isolated individual coloured light beams and demonstrated that they cannot be further divided, unlike a visually identical mixed colour. Thus he differentiates physical properties from perception. He noted the similarity of colours at the edges of the spectrum and invented a colour that wraps it into a continuous circle.

Newton hoped to imply a direct correspondence by choosing seven colours in octave of musical pitches. But his circle instead came to describe colour harmony, analogous to musical harmony. As one traces the diameter from a colour on one side, it graduates through white to its complement, the colour that negates the first. Painters already worked with these complements – Leonardo wrote extensively about them – and the new circular scale neatly illustrated a known principle. Colour harmony can actually be any geometric relationship around the wheel that sums as white; a ‘triad’ is the triangular form such as between red, green and blue. Any triad will work as well, but the RGB system falls closest to our perceptional system.

Anyone can directly experience complements by staring at a colour for long enough to develop an after-image – e.g. staring at red causes an after-image of cyan. Despite that evidence, painterly doctrine remained that ‘red’ should be opposed by ‘green’, as mixed from ‘yellow’ and ‘blue’, and colour wheels continued to show the ideal instead of the experience. Jakob LeBlon is the probable author of an anonymous pamphlet setting out the first RBY tricolour system, and first to print full colour images around 1710 by overlaying ‘blue’, then ‘crimson’ and finally ‘yellow’. In reality these were not primaries at all. (Shevell p6.)

The Electro-Mechanical Age

Histories of visual music sometimes begin with Louis-Betrand Castel and his development of an ‘ocular harpsichord’ in 1725. I think it necessary to first mention the Jesuit polymath Athanasius Kircher, born 1602 and once credited with knowledge of all things. As well as maintaining the first science museum in Rome, he published over thirty popular illustrated books covering topics from the Emperor of China to the Magic Lantern. Kircher invented machines that illustrated his knowledge, and provided a role model of ‘mage mechanic’ that Castel took up, and passed on the artist inventors that followed.

Kircher drew much of his authority from the ancients, claiming to translate ancient languages, including Egyptian hieroglyphics. He invented many devices including a ‘Sunflower Clock’, a ‘Universal Magnetic Horoscope’, a ‘Cabalistic Mirror’ and so on. There was already doubt in his lifetime that they all existed, and those few that appeared in his museum were sealed from close inspection. Most relevant here is a section in his 1650 book Universal Music-making. Amidst engravings of automated music composing machines is Kircher’s claim that, “If, when a musical instrument sounds, someone would perceive the finest movements of the air, he certainly would see nothing but a painting with an extraordinary variety of colours.”

After his death Kircher became synonymous with encyclopaedic humbug. But Castel was a similar Jesuit philosopher, whose self-taught expertise in mathematics and physics led to his promotion to teaching at the Jesuit Collège Louis Le Grande in Paris. From 1720 he became the science editor for the religious Journal de Trévoux, which implies the Jesuits maintained their own version of scientific thought. He became a respected figure in French society, particularly as a critic of the Englishman Newton.

Castel saw physical laws as manifesting the willpower of God, and thus free will as the definition of humanity. This conceptualises all natural forces as enacting a single creative act, perceived mechanically across the senses. His proposal for the ‘ocular harpsichord’ was a thought experiment that sound and light should therefore, “equally consist in the insensible wigglings of the sonorous and luminous bodies” as he put it. He first adopted Newton’s colour wheel as it suited his purpose, but without Newton’s reticence.

Unfortunately for Castel, his 1725 announcement in the popular journal Mercure de France attracted great enthusiasm from the editor and readers, who immediately pressed for practical details. In no particular hurry to comply, he provided an updated proposal ten years later, shifting the colours around to make blue the tonic or fundamental pitch, with red and yellow equally placed by LeBlon’s recent RYB system. That conflicted with Newton, but the Mercure readers soon decided on a solution – Newton must be wrong. Problem solved.

The first machine was apparently shown around 1735, and it had changed very little by 1739 when the composer Telemann mentions it as part of a visit to Paris. Pushing keys on a standard harpsichord pulled strings that revealed brightly coloured boxes. In 1741 the machine was still in progress, now with small lanterns inside the boxes. But by 1752 Castel was near to bankruptcy, various machines unfinished and his anger at the readers of the Mercure spilling out in his writing; “The public has taken it too seriously”.

The scientific fraternity did not take it seriously. If anything came out of Castel’s experiments it was in prompting scrutiny of the relationship between pitch and colour. The French Academy of Sciences responded in 1737 with a number of troubling critiques. One paper delivered by Marian is enough. If a simple ratio of colour frequencies provides an aesthetic experience in colour music, we should be able to measure that ratio. Even without knowing the frequencies (as was true at the time), ratios can be determined by the angle at which the coloured light leaves a prism when refracted. Newton had shown that the largest ratio between the edges of the visible spectrum is a tiny 77:78 – the visible frequencies are a tight band of the whole spectrum, and no simple ratios akin to musical pitch can actually exist in visible light.

Any direct scalar relationship between sight and sound was now difficult to justify. Colour music retreated into toys and pastimes until the Enlightenment world view shifted to the Romantic, and the argument from the scientific to the spiritual.

A Move to the Subjective

As if summoned by the zeitgeist, the German poet Goethe arrived with a new way of viewing the problem. Goethe considered his 1810 Theory of Colour to be a greater work than his poetry, although much of it is poetical metaphor. In the preface he rudely compares Newton’s theory to an old castle in need of tearing down to let the sunlight in (p. xxxix). The introduction puts forward the circular idea that as the eye is formed by light, it is therefore the best means by which ideas of light can be formed.

In the Theory, Goethe holds that colour is purely subjective, and he methodically applies himself to experimentation with perception. His case studies include afterimages, how colours are perceived by their proximity and contrast to others, and particularly how brightness affects the apparent hue of the source. His method is thorough, but his observations did not cohere into a scientific theory.

He rightly disputed Newton’s seven primaries, and then claimed the spectrum formed by a prism to be a special case of a phenomenon where warm and cool colours are formed at the fringe of a central white area. While true, it is saying that a design around an independent variable is only a special case of all the possible permutations of that design, which is a fundamental disagreement with experimental method. Goethe failed to convince the scientific community of his ideas, but made a strong connection with artists. Systematic analysis by the great poet was vindication that their insights reflected principles of nature, and if the scientists thought little of his theory, then the painters thought less of the scientists.

The spiritualists were also pleased, and although the next chapter is about them, it’s useful to say how this was communicated. In 1882 Rudolf Steiner became an editor of Goethe’s works, and from them developed his own theosophy. He reinterpreted Goethe’s poetical notions of the eye as being actual physical evidence – for example, that an after-image is the perception of a real thing, which then became his argument for multiple ‘spiritual planes’ of existence. Covach (1992) traces how Steiner leveraged his study of Goethe into authority in ‘scientific spiritualism’, rising through the developing German romantic and occult community.

The concept of colour harmony allows the invocation of ‘sympathy’, which in its German form implies a kind of mutual influence that may be evident only to the aesthete. Romanticism involved a general distaste for scientific materialism aligned with a nationalistic, romantic notion of folk (völkisch) wisdom, an innate knowledge base stemming from contact with nature, diminished by society and machines. In 1849 Wagner had published The Artwork of The Future, in which he introduces a natural ‘inner necessity’ that drives art (Wagner 1849). Opera is a forgotten ‘total art’ that solves the ‘loneliness’ of music, dance and lyric, each unable to articulate itself unaided. Wagner has little time for painting, allowing only landscapes destined as backdrops for operas.

This romantic ‘total art’ employed symbolism and representation. For example throughout the 1880’s the German artist Klinger illustrated Brahms’ music scores with much approval from the composer, who eventually declared that, ‘I must conclude that all art is the same and speaks the same language’ (Frisch p.96). Klinger’s illustration is all operatic staging, full of high mountains and churning seas. When he once ventured too far into the abstract Brahms found it ‘too puzzling and incomprehensible” (p.94). Klinger retreated rapidly back to metaphor.

A rapid turnaround from the Romantic to the Modern took place around the start of the 20th century, in which this operatic symbolism gave way to abstraction. The Swedish painter Hilda af Klint first produced abstract paintings around 1905. She is poorly known – forbidding her work to be shown until 20 years after her death didn’t help. She too was a Theosophist, and her paintings directed by séances.

In her absence, Kandinsky claimed the mantle of first ‘non-objective’ painter. He was enthused by Rudolf Steiner’s ideas, as was the composer Schoenberg. Under the influence of Steiner’s spiritual planes they collaborated on what Schoenberg calls the musikalische Gedanke, the un-measurable ‘musical thought’ existing on a higher plane that directs the harmonies of both sight and sound. It is difficult to work out what the mechanism linking these planes might be, and it is fair to say that the occult had now become part of the design, although not one easily reproduced.

Kandinsky published his manifesto On The Spiritual In Art in 1911, which collected much of the current conversation around music and painting (Kandinsky 1946). He seeks to impose rules on expressionist painting, to align genius with ‘the universal’. He uses ‘absolute music’ as a model because he argues that it is able to convey emotional resonance and sympathy without dilution by representation of the physical world.

The first part starts by renouncing external sources of inspiration in favour of Wagner’s ‘inner necessity’. Kandinsky opines that the ‘nightmare of materialism’ will lift in the 20th century, replaced by non-objective art carrying the ‘internal voice’ (stimmung) of the artist (p10). Then follows a long and difficult section describing the ascent of a spiritual triangle and the primacy of the Theosophical world view (p.25), which can be safely skipped over here, as it is mostly a contemporary critique (sections II-III).

Section IV touches on the then current debate on ‘absolute music’ and applies it to painting. I need to quickly outline that debate to explain what makes music relevant. Kant’s Critique of Judgement in 1786 had placed genius in the distinction an artist creates from natural beauty. As we must impose order on raw sensory data, he says, art is located in the individual. In music, ‘mathematics has certainly not the slightest share’. But the growing Romantic Movement disputed his Enlightenment position, where for example in 1802 Schelling redefined creativity as the struggle for the genius to respond to universal natural laws. This is Kandinsky’s aim.

This view reflects developments in science and technology – including the musical sonata, in that it combines innovation in instrumentation, organisation of the orchestra, acoustics in architecture and so on. The sonata form is literate in that it has an introduction – the overture, arguments arranged in movements, and conclusion – the finale, which comments on the introduction. This strong resemblance to literary essaying bolstered an argument that music could represent a language.

By the mid 19th century ‘programme music’ became fashionable, being galleries of single movements, each painting a concept in music. This ‘pictorialism’ includes sonic portraits via a prototype musique concrete, emulating animals and natural sounds by careful orchestration. Music reached out to literature and painting, and the gesture was returned. Wagner despised pictorialism, while his friend and supporter Liszt saw it as portending a future art form.

In contrast, ‘absolute music’ is free of lyrics, of theatre, of any collaboration with external arts. It is appreciated as a transcription of an elegant mathematical formula that places the aesthetic experience with the listener. Kandinsky takes Wagner’s side that programme music is ‘an affected absurdity’, the ‘imitation in sound of croaking frogs’. The stimmung cannot be imparted by imitation – instead of music borrowing from painting, the opposite was needed. In painting there is a ‘modern desire for rhythm… for mathematical, abstract construction, for repeated notes of colour, for setting colour in motion.’ (pp.35-36)

Colour motion was a contemporary issue in painting, inspired by exhibitions of the time-lapse photographs of Muybridge and Marey. The Futurists painted multiple exposures directly; Kandinsky tried the same in abstract. Klee converted the static colour wheel into a dynamic motor where the secondary colours spin out of the primaries, and tried in a series of Fugues to emulate the time slices that were appearing in photography, X-rays and the like. (Shaw-Miller pp.152-154)

The second part of On the Spiritual attempts to describe a spiritualistic colour scale. He acknowledges that yellow might be made sour by association with lemons. But he brushes this aside and declares that ‘colour is a power that directly influences the soul’. For its part form is ‘the outward expression of… inner meaning.’ (p.43) His colour system is harmonic: From Goethe, yellow and blue are the main axis, yellow being warm and moving towards the viewer, and blue being a ‘heavenly colour’ because it recedes and turns in on itself. When they add to make green the motion stops and so green is restful. Because red is in quick motion, it becomes the opposite of green.

Kandinsky purposefully assembles his system only by internal logic, to purify it of any connection with the real, stating that, ‘The less the movement is motivated externally or intellectually, the purer, deeper, and more spiritual will become its effect.’ His colour is freed from any measurable justification.

Schoenberg’s reciprocal 1912 essay on The Relationship to the Text is perhaps more sober, but still holds that, ‘a pure mode of expression is denied to poetry, an art still bound to subject-matter’. There is only ‘pure perception’ by ‘men of calibre’ of non-objective qualities of a ‘true nature of art’ (in Schoenberg 1950).

Kandinsky was in tune with the avant-garde. Around the same time the Czech painter Frantisek Kupka wrote a related manifesto Creation in the Plastic Arts and exhibited his own colour motion – but also conducted professional séances, and was (as becomes expected) a Theosophist. There are many others, but we need to keep moving, as did Kandinsky and Schoenberg.

Both fled to America from the Second World War, where their musikalische Gedanke found a keen new audience. Schoenberg’s 12-tone music system became the academic mainstream of the late 20th century – and an inspiration in Hollywood horror film soundtrack. Kandinsky became entangled with Hilla Rebay and the Guggenheim ‘non-objective’ museum, a connection described in a later section on film and patronage.

The Electrical Age

The spread of electricity over the early 20th century brought colour machines with a wider gamut of brightness. Castel could only hope to amass enough candles to give a reasonable performance from his harpsichord. With electricity it became possible to modulate the intensity of the light source and so piano forte (soft and loud) techniques were made feasible. Electricity also inspired a flurry of pseudo-sciences around ‘electrical healing’ that rekindled spiritual discussion – particularly the influence that colours outside the visible spectrum might have on the human senses. Alongside these inventions came a growing dialogue about colour in music therapy.

We have a ‘souvenir’ pamphlet from Bainbridge Bishop from 1893 describing his pioneering work on various electrical colour devices. His most ‘satisfactory one’ was an attachment to a musical organ that allowed electrical light through coloured glass panes, with shutters opened by the keys. Bishop was first to describe a system to show octaves, where deeper notes spread their intensity over a wider form. The individual lights were angled such that a chord would not fully combine the individual colours, but show an artificial rainbow. Time spent in painting had taught Bishop the importance of traditional colour harmony – his colours emerge from, and balance around, the neutral grey of the projection screen. He writes, “Crude colours alone are barbarous” – something a great many visual musicians have failed to realise.

Alexander Wallace Rimington is better known, being a professor at Queen’s College with more expertise in publishing research. In his book Colour-Music of 1912 he describes colour as something ‘fixed’ by painting and the need for ‘mobile colour’ to rekindle our colour appreciation. In ‘the working class’, he says, ‘there is no evidence to be found of any real love of colour’, a matter he claims to harm their mental health (p10).

He makes the familiar comparison between sound and colour vibrations, but only hesitantly defends the resemblance of the colour wheel to the octave. Rimington argues that the subdivisions should not be locked to any specific musical scale and that any note can be made fundamental, so long as control of elegant ratios are available. Harmony and discord are left to further experiment – much as was happening in music at this time.

He is most concerned with time – the passage of colours, how they progress, their envelopes of brightness and so on. His idea is not that the light be aligned with pitch – instead the procession of light would in itself be a new kind of music with similar emotional power.

If the machines failed as evidence, then what was their point? One could be generous and say that they were prototypes, but Rimington signals a period where patents become the primary documentation and an investment in intellectual property – Edison was recently famous for the phonograph and the light bulb. A useful comparison can be made with the more recent battles of pharmaceutical companies and media conglomerates – as light music was considered both entertainment and medicine, the stakes were high. Rimington patented his mechanism in 1893, starting a battle between ‘inventors’ fought in courts rather than journals.

He demonstrated often at his home studio, but a public exhibition set for 1914 was abandoned in the face of World War I. His influence was in the growing field of theatrical lighting, and what was avant-garde for music became standard for stage plays. The Strand Lighting Company was by 1935 demonstrating a colour device at their showroom, a modified church organ (stepping around Rimington’s patent) that became the prototype for the modern stage lighting console.

Three more inventors may illustrate this electro-mechanical period. The Australian, Alexander Hector, is preserved mainly via a small box of news clippings in the NSW State Library. The Americans Thomas Wilfred and Mary Hallock-Greenewalt are better known through the support of private collectors and foundations. Patents describe much of their artistic process, and Betancourt’s two-part collection of Visual Music Instrument Patents is a primary documentation. They existed on the fringe of artistic and engineering societies, and their influence is often through their liaisons with better-known musicians and painters.

Hector’s light instruments were his evidence in a debate held in the years following the First War, between his claims to ‘science’ and the Sydney Theosophical Society’s mystical ‘thought forms’. He appealed to the aesthetics of naturally occurring proportions – art of the sort now called scientific visualisation. From around 1910 he sought ‘to correlate the essential principles of the various sciences’ and by 1912 held a light recital with music in Sydney Town Hall, which was well received (p.126 McFarlane 2012)

He filed his first British patent for “improvements … in colour music” in 1912.

In contrast, the American Mary Hallock-Greenewalt saw colour as an atmospheric accompaniment to sound, not a melodic element in itself. A prodigy pianist born 1871 in Syria, she started experiments with coloured gels in 1906, claims that she experimented with film, and performed her first piano concert with a self-composed light show in 1911. (HG 1919)

Her main fame is to have invented a notation and instrument for a ‘sixth fine art’ that she named nourathar, Arabic for ‘light flavour’. The instrument, the sarabet, required new controls and by 1934 she held eleven patents in electrical engineering. Throughout her life she became increasingly embittered with lawsuits against infringements by manufacturers and other light artists such as Wilfred – unfair given that nourathar is a slow procession of mood colours that accompanies music, whereas Wilfred’s lumia is a silent fountain of flame like swirls.

She was a resourceful self-taught engineer, who never received proper recognition for her work – partly as her inventions were collaborations with companies such as General Electric – but partly by being female. But it should be said that her relations with the DuPont Chemical family assisted much of her development.

Hallock-Greenewalt’s writing style can be a confused torrent in which one might drown. Her 1918 address to the ‘Illuminating Engineers Society’ is equal parts engineering and poetical neologisms. In it she sets out the need for light notation, “A hieroglyphic, a symbolism, a denotation designed for the orientation of artists is a necessary adjunct to any art of succession.” Her attempt at complete ownership of light music was self-defeating – nourathar is little known today.

Thomas Wilfred has become the best known of the three through patronage and location. He aligned with Theosophy, which brought contacts and support – starting with the wealthy patron Walter Kirkpatrick Brice, introduced to him via the successful Theosophist architect Claude Bragdon.

Wilfred claims to have experimented with coloured glass around 1905, but it would be 1919 before his first light machine was under construction. Bragdon had sought funds from Brice to found a global society and a laboratory for studies in light music. Once built, Wilfred bumped the other partners out, made the lab his family home and started on the Clavilux Model A. The first lumia performance in 1921 went well, and Wilfred began a world tour that lasted to 1925, at which time a lumia institute was founded lasting up to the second war. (Wilfred 1947).

Post war, Wilfred moved on to the exhibition of Convolux machines each presenting one composition housed in a picture frame, and attempted to market a domestic ‘Home Clavilux’. In 1951 the Museum of Modern Art exhibited three Convoluxes alongside prominent abstract expressionist paintings, to indicate their place in the lineage of American abstraction. Arguably Wilfred inspired some of the psychedelic light shows of the 1960’s. His Convolux Opus 161 recently appeared in Terrence Malick’s 2011 film Tree of Life, still serving as a spiritualistic framing device.

Non-Objective Motion Pictures

There is no reasonable way to cover all the film history that could be related to this project, and the best tactic must be to cover a set of indicative ‘absolute films’ by better-known artists that feed ideas into the current artwork. That generally means film that borrows ideas from the light machines, or those that provided inspiration for the synthetic video that followed.

Film production can be dated from 1895 with the Lumière Cinematograph camera/projector and Edison’s Kinetoscope soon after. Although patents competed from the start, their essential similarity encouraged inventors of individual devices to transition to end users of a technical standard. In 1906 Edison distributed the first animated short film Humorous Phases of Funny Faces, which is not often mentioned in histories of abstract film, but must surely have caused some inspiration among the non-objective artists of the time.

Around 1912 the Futurists Arnaldo Ginna and Bruno Corra report making a ‘chromatic piano’ with electric lights, and that finding the brightness inadequate moved on to try a film projector instead. They managed to paint a few one-minute ‘symphonies’, one of which featured a red star swelling on a green field. None were publicly shown or have survived. That same year Léopold Survage started work on a sequence of paintings intended for film called Coloured Rhythm, but was unable to produce enough cells.

Walther Ruttmann was first to exhibit what he called ‘painting over time’ with his 1921 Lichtspiel Opus 1, being approximately 1000 frames. He produced a few other ‘absolute films’, which (being no great spiritualist) he recycled as advertisements. Ruttmann had only a passing interest in non-objectivity, instead inspired by the Soviet montage represented by Eisenstein and Vertov. His 1927 Berlin: Symphony of a Great City crossed back from abstraction to symbolism of the Wagnerian sort, with documentary footage arranged according to an emotional contour in 5 acts. It toured with a large live orchestra with some success.

Viking Eggeling is a closer, if somewhat tragic example. He joined the Zurich Dada movement in 1915 but was inspired by the Swiss musician Busoni to search for a universal artistic language. Eggeling painted abstract syntax on paper scrolls, to be read sequentially as a pseudo code, and gave it musical names – Diagonal Symphony being one. Animation became feasible in 1920 when the wealthy artist Hans Richter ‘adopted’ both Eggeling and his ideas. The two gave an interview to Mondrian’s De Stijl journal in 1921. They mention rhythmic painting ‘as desired by Goethe’ and that the work represents a film-based language, formed from ‘an alphabet of polarities that represent the principle of composition in total art’.

After a bitter falling out, Richter claims (likely falsely) to be first to screen with Rhythmus 21. The severely impoverished Eggling screened his Horizontal Vertical in 1923. In 1925 the Diagonal Symphony was finally screened in public. Eggeling was too ill to attend, and died soon after. As his patron, Richter claimed the rights of the collaboration and the continuing fame.

As the Catholic Church had once encouraged a single system of musical notation as a technology for social order, Governments and private foundations now cultivated media arts that aligned with their needs. Their patronage determines whom we find significant in art history. Although a talented painter herself, the Baroness Hilla Rebay enters here as the courtesan of Solomon Guggenheim, the copper mining magnate. She originally named The Guggenheim as The Temple of Non-Objective Painting, and promoted the spiritualistic ‘non-objective’ work of painters Bauer, Kandinsky et al. Rebay also edited and republished Kandinsky’s On the Spiritual in Art for a new American audience.

The original plans for the Temple by Frank Lloyd Wright included a ‘light music institute’ in the basement, which by opening had become a film screening room. The refugee Hans Richter became the first of Rebay’s film artists, followed by Oskar Fischinger, Norman McLaren and the Whitneys.

Fischinger also found some commercial work, notably in Walt Disney’s Fantasia project. Disney had seen Wilfred perform in 1928 and Len Lye’s hand painted film A Colour Box in 1936 and was filled for a time with enthusiasm for visual music. Fischinger’s tenure was short lived, as Disney soon favoured a more symbolic and Wagnerian ‘total artwork’ for the film (Culhane p.37). His tastes then strayed to the surreal, with a short-lived project with Salvador Dali in 1944.

John Grierson was another significant patron, heading various UK government film units over the period – the Central Office of Information, National Film Finance Corporation and so on. Grierson was tasked with establishing both the Canadian and Australian Film Boards as a wartime expediency, and his influence eventually extended over a large sweep of the Commonwealth. He was dedicated to socialist documentary, but told in a lyrical and experimental way.

The career of animator Norman McLaren best illustrates this patronage in action – first employed by Grierson for the GPO Film Unit alongside Len Lye, he moved to New York in 1939, selling his films to Hilla Rebay, before passing back to Grierson in 1941, now head of the Canadian Film Board. Here McLaren produced a popular series of animated films including Neighbours, which won the 1951 Oscar. But it was for Rebay that he first painted sound on Dots and Loops – two abstract films that signal a change of paradigm in visual music.

To one side of a 20th century filmstrip is the optical sound track, where brightness indicates the volume of the sound at any instant. The pitch of a sound is defined by the frequency at which bands of light pass by a sensor. Fischinger had noticed a similarity between the sound track and ornamental shapes in 1932 and produced sound works by painting geometric forms on film. McLaren took this further by animating the sound to parallel an animated visual object.

Castel had proposed the universe to operate according to the generative force of a hidden creator, a force that he could visualise by the ocular harpsichord. McLaren’s films introduce the paradigm in which an artist makes critical use and misuse of a media platform and in doing so drives change in the function of that platform. Shelling’s definition of the creative genius was being reframed as a critical collaboration between an earthly engineer and artist, testing the rules of a technical system rather than natural order. This converts what was the spiritual into a model of spirituality. The clearest example of the engineer as deity of a small man-made universe comes in the next historical arc – synthetic video.

The Scan Line

Up to this point the mechanism of positioning light has been distinct to that of sound, and the canvas or frame a different topology to the sound source. Video imposes a universal, time based, mechanism for filling musical space – the scan line.

Analogue video forms an image by firing a beam of electrons against a phosphor screen from left to right as horizontal lines, scanned from the top to bottom of the screen 50 times a second. The amplification of the stream varies such that light and dark areas are sketched in the phosphor glow. A sound signal at an audible frequency will draw patterns, and much higher frequencies will be interpreted as colour. Video provides direct correspondences between pitch, form and colour, but only within the design of the platform.

A reasonable moment to start synthetic video would be Nam June Paik’s exhibition of televisions with abstracted displays as Exposition of Music – Electronic Television in March of 1963. ‘Abstracted’ meaning here a dislocation of the broadcast picture by physical damage to the magnetic yoke around the vacuum tube. Although the images were modulated by sound picked up by microphones in the installation, the music is formed in disrupting the intent of the machines’ design.

When presenting a retrospective of electronic art, the curators of the 1992 Ars Electronica chose the theme of ‘The Apparatus World’.

Abstract painting became an autonomous world of colors and forms… (and) briefly became a doctrine of signs for structures of existence, the soul, the cosmos. This exhibition “The Apparatus World — A World unto Itself” shows the wide range of these sound and image worlds, where the signal itself is no longer a carrier for depicting the object world but rather the image itself, autonomous worlds of sound and image that can be manipulated by both the observer and the machine. (Weibel 1992 p15)

This ‘apparatus world’ is the space in which a coherent internal logic operates – Kandinsky’s occult colour motion is the map of such a world.
Whereas Vertov had claimed the cinema camera to be a more powerful instrument to document the real, the synthetic video artists actively disputed television for making a similar claim.

Video is a real-time process, allowing feedback. Pointing the camera at a screen carrying the camera signal creates a swirling kaleidoscopic effect with a resemblance to both galaxies and mandalas. Like Hector back in 1929, the video artists were convinced that the resemblance revealed cosmic connections.

A 1974 announcement by Eric Siegel sets out the idea:
… I have just developed the first all electronic video synthesizer in the world.
It is the instrument of the New Television; the growing tendency of more artistic abstract television performed by beautiful enchanting people. Where conventional television seeks to inform and entertain the New Television will be engaged in expanding people’s consciousness and providing a way for constructive meditation.
The E.V.S. hypnotizes you and the person playing it controls your trip. So the way you see the E.V.S. will depend on who is playing it. (p.116)

In keeping with the general flow of psychological concepts over the 20th century, the spiritual now centred in the individual’s journey. Siegel is on a Jungian quest for ‘expanded consciousness’, and what Schoenberg once called ‘men of calibre’ are now ‘enchanting people’.

The 1992 Ars Electronica declared what seemed to be a generational progress from analogue to computer based art. That demarcation was never as well defined as the contemporary rhetoric would claim. The first computers were little more than versatile circuits – for example the PDP-8 has no microchips, only 8 basic instructions and a single accumulator. Around 1971 Doug Richardson created a ‘Visual Piano’ on a PDP-8 at Sydney University, which plotted geometry on a vector display captured to analogue film. One user was Peter Vogel, who would later go on to develop the Fairlight computer video instrument.

Digital circuits at first responded to the video scan line. Perhaps the best-documented platform is the Atari VCS, studied by Montford and Bogost at MIT in their Racing The Beam. This system has no graphics buffer and no pixels, and instead paints an image on the fly as the electron beam travels across the screen (p28). All other calculations take place in the moments where the beam moves into position to draw the next scan line.

We are still in a period of digital imagery, troubled by the success of this art form that has moved past the dreams and excuses of an infant technology. Artists difference themselves from the (now traditional) digital realm by making vinyl records and Arduino circuit boards. Analogue video synthesis tools are re-appearing as part of a general revival in bespoke analogue technology built by the individual. The end-users seek to return to the role of inventor.

This is the situation in which the current project takes place. Technical limitations such as faced by Castel in lighting his machine, or Ginna and Corra in painting enough film cells, are no longer a limiting factor. Instead the design question is how to enforce limitations to bring the right balance of clarity and functionality. Spiritualism is simply one aspect of this design, and the question is how to creatively limit spiritualism in the same manner as the other parts of the apparatus.

Summary

Visual music has appeared possible, desirable and just within reach despite a lack of progress over thousands of years. It is an ideal that has reshaped itself to the desires of individuals and societies, moving between science and pseudo-science as needed. It’s telling that the guiding principle is always situated just exterior to the visible mechanism, and I will come back to this in the next chapter.

The history of visual music is parallel to that of psychology, which starts with astrology and winds its way through animal spirits to arrive at Jungian archetypes. It’s illustrative how both Ruttman and Walt Disney reframed their interest from the abstract to the surreal in search of a storytelling device. It is by psychology that any notation for visual music needs to be framed. Like narrative or music it should be described by a contour of emotions while remaining agnostic about the style or symbolism.

As the proposed relationship between genius and engineer has moved from God, to higher planes, down to the media artist’s critical use of platforms, there has been the consistent expectation that the artist will discover a higher purpose than is evident in the design.

Material rewards for visual musicians have been few and far between. Some notoriety if one is lucky, as Wilfred was. Nevertheless the inventors have lined up and squabbled over the supposed riches. There is an implication of immaterial reward and that sensitive people will be refined through the art. Alchemy is a useful comparison, as it too involves the psychological projection of a change in the qualities of the person (from mortal to angelic) onto a change in a visible substance (from lead to gold). It’s also not far off the claims made for systems of narrative storytelling, that a facing a struggle will lead to the protagonist’s (and thus the reader’s) moral growth by the third act.

The crux of the building a creative interface is about acknowledging, defining and providing control over that part of previous systems which represents the psychological aspect of the apparatus.

(Go to next chapter).