"To remember an event is to experience it again, but not in the same way as the first time. Memory is a special kind of experience because it is composed of selected impressions, whereas actual experience is a welter of sights, sounds, feelings, physical strains, expectations, and minute, undeveloped reactions. Memory sifts all this material and represents it in the form of distinguishable events. Sometimes the events are logically connected so that sheer remembering can date them with respect to each other; that is, in a vivid recollection of (say) coming down a hill, the sense of being high up and of treading dry gravel has merged into that of accelerated motion, of the horizon's lifting all around, of places near the bottom of the path; and the whole series of changes may be remembered. Any special adventure along the way then finds its temporal frame in the memory itself. But most events are recalled as separated incidents, and can be dated only by being thought of in a causal order in which they are not "possible" except at certain times. The other items in this causal order are one's various other memories, but the order itself is an intellectual system."[1]

Susanne K. Langer, Feeling and Form: A Theory of Art, 1953

Introduction

In the above quotation Philosopher, Susanne Langer reiterates the connection between sense and memory. In her text “Feeling and Form,” she foreshadows the field of Affect Theory insofar as her ideas emphasize that sensations would guide the human spirit to act and react to various sensory inputs through emotions and encounters. Her emphasis on a philosophy of art and creative practice attracted me to her ideas. Indeed, it made me think of poiesis or bringing forth in the act of making– where the thought is brought into being from experienced events.

This introductory text does not necessarily praise everything Langer wrote, as she was also writing at a time during the 1940s and 1950s that was fraught with stereotypes and biases. She herself was largely ignored by the canon. This being the case, I still wanted to pick through the texts to understand how the fields of study that encompass Affect Theory and New Materialism might be compared if we consider (Langer’s text) as a philosophy that elucidates our relationship with nature/culture, mind/matter, and human/non-human entities. The only starting point regarding synæsthetics is that these are not mutually exclusive themes and are produced through various types of engagements, sometimes beyond human conceptions. More poignantly, materials have embodied expressive qualities - they do not need to be sensed to exist but become entangled in processes. By this, I mean biological, geological, symbolic, et al. Indeed Langer’s text and probably most prominent work, “Philosophy in a New Key,” looks at bridging various ways of “knowing reality.” These discursive “elements” arranged by thought processes are not necessarily logocentric; they are symbolic transformations of the experiential data that comes to the human mind through sense experience. In her work, “Philosophy in a New Key,” from 1942, Langer used music to articulate the bridge between symbol and feeling. When looking at the neurological condition of synesthesia, there is no question that musicians are often cited as seeing colour when sounds are produced. By considering hearing sense as a way to describe effects and emotional states, Langer foregrounded the epistemological conditions of Affect Theory. In the 1940s, computers were still in their infancy, mainly used to crack codes within a world upset by the trauma of war. In the ensuing decades, digital technologies and data provided a way of thinking about materials that could be used as a reconstructive or recuperative possibility as statistics and information gathering became more critical.

This, in turn, would fuel an ever-increasing interest in environmental concerns or social and political injustice that was more likely to be predominant in conceptual art practice. We see this in the explosion of eco-art and works that pinpointed inequality, for instance, in the work of artist Louise Lawler’s “Bird Calls,” 1971, which addressed the disparity of gender representation within gallery exhibitions. Although this still had the trappings of the binary between male and female participants through a lens of the second wave, it nonetheless signalled inherent bias in the curatorial world. Beyond that, it also spoke volumes about non-communication, translation, the role of humour in art and its subversive capacity. To cite another work, one might turn to Alexandra Daisy Ginsberg as having a kinship with Louise Lawler. In Ginsberg’s Machine Augeries, the piece becomes an environmental critique of technology, especially AI emulation; it evokes uncanny dred where reality is replaced by representations that are so highly defined that we can no longer detect a difference. Conversely, it elicits serenity if one only hears the birdcalls within the gallery installation.[2] This deconstructive contradiction between our senses and deception (the generative adversarial network) in the installation posits a cognitive dissonance that is politically engaged. So, synæsthetic work is crucial in the arts and for artistic practitioners as it often embraces the notion that work crosses boundaries and various research fields.

In what follows, I present a broad approach concerning questions surrounding Synæsthetics and its link to synesthesia, cognition and culture. This introduction highlights the differences (and some similarities) between terminology often exchangeable, like perception, sense, memory, awareness, sensation and consciousness. It argues for incorporating the senses into art historical discourse, as well as for evaluating the senses as agents of historical research. It's my goal to contribute concepts that consider sense technologies as they relate to everyday experience and show how they bridge gaps in our organic senses, hopefully without any favour of technological utopianism. I will try not to overlook the political implications of technological controls over the freedoms in which societies worldwide still struggle. As Langer states, “…the  philosophy  of  art  requires  the  standpoint  of  the  artist  to  test  the power  of  its  concepts  and  prevent  empty  or  naive  generalizations.”[2]

Over the last half-century, the art and technology arena has had various names and meanings. Writers and practitioners have attempted to supply concrete examples of what would fall under the umbrella of electronic or media art. Art using electronic, technical and software-driven means has been regarded as intermedia, multimedia, transmedia, interactive art, hypermedia, digital media and new media. These distinctions often relate to how the work is transmitted, received and produced through different communicative means. Having "media" associated with the name suggests a connection with popular visual culture and current events that ties this to the broader spread of what theorist/artist Hito Steyerl calls the “poor image.” Bearing this in mind, I intend to cast a wide net, including various projects that reflect the vast diversity of what might be related explicitly to Synæsthetic art. Infinite possibilities to develop technologically-based skills within these mediums are not unlimited as these mediums developed; they are subject to specific investigations, collaborations, and human pursuits that drive things forward, splintering into varied objectives and interests. Critiques within the field of media art offer a plethora of condemnations regarding the interplay of data, homogeneity of concepts, authenticity, copyright, and privacy rights - not to mention how progress is linked to the mental prosthetics of artificial intelligence and its inherent lapse in metacognitive ability. Rather than prioritizing digital constructs, I favour the notion that the digital and analogue worlds coexist. Digital forms and systems are human constructs of binary codes and numbers.

The Link Between Synesthesia and Synæsthetics

Synesthesia is a cross-modal experience within living organisms. Its meaning expresses a union of the senses and is often developmental or acquired genetically or epigenetically and embodied within the individual. In contrast, acquired synesthesia often arises from psychotropic drug ingestion or some neurological conditions not inherited. This condition may also operate on a sensory or conceptual level where a thought or sense consistently triggers a second perceptual experience, usually through a different sense. Another difference between acquired and developmental synaesthesia is that acquired synaesthesia does not demonstrate “either idiosyncrasy or automaticity or stability.”[3] Another distinction needs to be made for specific terminology related to ideas and sense. Put forward in a paper in 2009 on this topic, Dr. Danko Nikolić devised a more specific term for thinking and ideas realized as sense called ideaesthesia where inducers evoke perception-like sensory experiences called concurrents.[4]

In this paper, Synæsthetics is a preferable term because it focuses on aesthetics and attempts to account for the affective properties of materials rather than focusing on neurology and thought processes. By doing so, it has the potential to consider interspecies sense as well as the evolved organization as it relates to sense organisms and material environments. To describe its relationship to media art, I will sometimes cross the terminology to suit the psychological state and the electronic mechanisms and expressions of data to draw these relationships. For instance, a generator of experience may be referred to as an inducer. The conclusion that is being made here is that technologies take on new sensory possibilities as the spectrum of sonic, haptic, olfactory and visual possibilities are extended. It is a grave error to privilege visual and sonic inputs over other sense detection methods. This denigrates not only other ways of knowing or experiencing something but also omits the complexities accompanying events and descriptions.
An example would be infrared spectrum awareness through electronic sense and processing. These computational functions for processing sense data have the possibility of directly feeding into media art processes. One of the powerful capacities that Media Art promotes is that it counters many technological pursuits that aim to use information in disturbing ways – either through militarism or data mining. These new forms of creative research may shift academic and private developments to help city infrastructure and navigation and assist with environmental remediation, among many other things. At least amongst the multitude of prevailing dystopian futures, there may be a way for artists to insert themselves into a creative narrative that may counter more destructive outcomes. To this day, subversive works from makers and hackers attempt to do this as an example. In the vast arena of human imagination, one can at least hope that healthy doses of criticism and deconstruction will help to keep power and grand narratives in check.

Media Art, as it has been commonly known, has had multitudinous names over the years. Within its long history involving kinetic and electronic work, its varied multidisciplinary pursuits have enabled almost infinite combinations of projects through the equally diverse ideas of its creative practitioners. It has been labelled as intermedia, multimedia, transmedia, interactive media and hypermedia to name a few. More often than not, these distinctions relate to how it is transmitted, received and produced through different technological and material means. To prepare this text, I will be casting a wide net, including various projects that reflect the vast diversity of what might fall under the umbrella of Synæsthetic art. Philosophically, it goes without saying that the possibilities to develop technologically-based art within these mediums are not infinite. As these mediums developed, they were subject to specific investigations, collaborations, and primarily human pursuits that drive things forward, splintering into varied sub-genres.  To keep everything on point, I intend to focus on the research of Canadian Media Art practitioners whose work might integrate with the fundamental ideas of Synæsthetics. 

In preparing this book, the link between synesthesia, as a physical and sensory expression, and the use of technology in media arts became readily apparent. This articulation begs a critical research question. Were media artists interested in investigating jumping from one sense to another using sensory technologies? If so, what were some of the art historical examples? Finally, why have these inherent bridges between hardware and software sensory technologies been so predominant since the inception of media art practice? Is it because technologies are sense-driven? Before delving into this, explaining some of the concepts is essential. First, Synæsthetics is a neologism for any technology, especially technological artworks, that uses sensory data collection to affect future outcomes in various forms. By morphing into something else based on data or a phenomenon, past or present synæsthetic works create awareness within lived experience.[4] The future possibility or probability of cause-and-effect relationships may also play a part in how visuals, audio, or any other sense factor unfolds. The predictions of outcomes become part of the work by using algorithms, projections, statistics, or differential inputs that drive the complexity of the data to a live output. Formed between any living sensing organism, affective, emotional and intuitive relationships become entwined and embodied in the organism as systems of behaviour. It might be useful here to draw an association between the tangible benefits of Synæsthetic art and an internet of things that have responsive capabilities that could act as somatic bridges to other living organisms, as it will be discussed in-depth in this text.

As a Media Artist/Researcher, I find it apparent that sensors and electronics drive projects through the detection and processing of signals. The diphthong used in the title "Synæsthetics" conjoined "a" and "e" as a way to signify how arts and electronics share an affinity. It also alludes to the difference in the two spellings of esthetics and aesthetics used differently by authors explaining various theoretical constructs and resonates with Systems Esthetics, which came to the fore as early as the 1960s. As promoted by Jack Burnham, Systems Esthetics is still very relevant insofar as software has dominated our daily lives. In this way, Burnham was prophetic when it came to the insights about how creativity and software would become entwined with the complexities of everyday life. An example of this was when I did a quick search for the word “chrome,” thinking I would find the materials or processes associated with this; I instead encountered multiple listings deep in the search algorithm before coming across what I wanted. Not only was I looking at a computer to get the information, but I also realized that the language I used was being replaced by more pervasive systems taking on words, usually signifying something completely different.

Our current impetus to analyze and identify phenomena, movement, and material through electronic mediation brings us new horizons - at least in the sense that we move and negotiate what we think is spatial, physical or temporal reality. This mediation seems more predominant now than ever when the aesthetic use of electronics (at least in the early to mid-20th century B.C.E.) focused on electro/mechanical projects that included kinetic sculpture or communicated works through telephone lines. In works like Atsuko Tanaka's "Electric Dress," electricity was foregrounded and pushed multiple boundaries in fashion/anti-fashion with its unique and innovative way of discussing the body juxtaposed with an electrified world. When electronics became more accessible to artists, fundamental paradigm shifts in media arts ensued. At the same time (during the 1960s-1970s), there was also an explosion in studies of cognition, neuroscience, perception, psychoanalytic and gestalt psychology and information processing. Concurrently, artists, sociologists, cultural theorists and linguists were studying the affects and effects of new developments in electronic communications.  It took collaborations between industry and arts to create significant changes, as seen in events like Experiments in Technology's "9 Evenings" and other projects by its members that started to bring sense technologies to the fore. In this case, these dynamic 1966 performances bridged so many disciplines and ideas that their relevance will continue long into the future, ushering a continuance of creative pursuits only possible with the development of evermore instruments used to measure and record phenomena. 

It is essential to focus on the various trajectories that have had a bearing on the development of media art as it exists today. There are many trajectories at play with at least three lines of inquiry. The first iteration came from the study of perception and memory associated with sense and grew out of the visual and sonic art fields. One could use Cymatics as an example where particles (like sand) on the surface of a plate indicate regions of displacement. These patterns are affected by variables as wide-ranging as plate thickness, sound frequencies or the materials that comprise the particles. This differs from sonification, where data translates into sound, but it illustrates where Synæsthetics is perhaps more clearly linked. The proponents of these form-finding fields within the early part of 20th-century experimental practice became interested in the psycho-sensory mechanisms of the brain, which spoke to colour theory, materials and perception, reception and transmission dovetailing with Psychology and Psychoanalysis. These early forms of Bauhaus-inspired Eurocentric 20th-century scholarship were most evident in the Op Art and Abstraction analysis.
There were also kinships with morphology as pattern thinking was created to understand a commonality of characteristics that would bridge various cultural and aesthetic forms and as a way to traverse stylistic variations. Concurrently, these more formal concerns also became a large part of the pedagogy within institutions until the linguistic turn was introduced (mainly in the late 1960s) within Western pedagogy, ushering in more anthropocentric approaches at a time when environmental art was also being born. It is important to note that linguistics and computer programming share similarities as systems, and the natural leap from learning lines of code and language cannot be understated. More importantly, although perpetuating a binary between objectivity and subjectivity, this formal analysis (usually involving the elements and principles of design) helped lay foundations that traversed notions regarding creativity, emotion, intuition, and interpretation. Concurrently, the notion of how materials (and their interactions) became interactive, kinetic, and complex as the temporal experience was also identified through the sculptural works of Jean Tinguely and the more absurdist projects with direct lineage in Symbolism, Dada and Surrealism as Western Eurocentric constructs. Often borrowing from dreams and appropriating from various cultures and traditions, these works of flight and fancy became a way to disrupt the concrete narratives we see in logic and scientific progress. Imagine the historical implications of the recorded totality of what is possible at any given time - to the discipline of history! An example of such a thing lies with Google Street View when we come to understand the lineage of families residing in a genealogical context. This is also often missed in terms of the acoustic contexts. Imagine representations of streets in Google Maps!!! In this way, the other senses are entirely left out of the record.

In the second instance, the field of Neuroscience also played a role. By the 1960s, as computer science became quite sophisticated, there was a desire to link human thought and machine sense. As a result, cybernetics and more advanced A.I. became conjoined within the purview of artificially sensing the world. During the 1960s, systems art and decision-making processes emerged within conceptual art - now deemed art-techno-science collaborations. This meeting of minds happened through investigations by many independent artists and academics. However, it gained much more momentum through institutional organizations such as C.A.V.S. (Center for Advanced Visual Studies) at M.I.T. and more independent collaborations like E.A.T. (Experiments in Art and Technology) that collaborated with corporations and governments at all levels to secure these partnerships. One can emphasize this connection by examining two significant exhibitions, "Cybernetic Serendipity" and "Software," which featured elements that embraced the notion that biological intelligence could be augmented or integrated with computer processes linked to databases. The trajectories of these vast fields influenced O.C.R., A.I. visual and voice recognition, and cybersecurity insofar as predictive algorithms interact with human decision-making processes. This connection will be more widely discussed in chapter two regarding power, militarism and economic concerns. To reiterate, Synæsthetic creations are important to address the diffuse power structures that tend to proliferate their agendas by excluding others at the margins.[6]

The third iteration of how Synæsthetic media art grew as a field of inquiry stems from the world of fantasy, utopian projects and notions of social and political engagement within the discipline of architecture and the built environment. These proposed (often unbuilt) structures involved transportation systems and adaptable social space considerations with interconnectivity as part of the design. Tatlin's monument to the Third International is likely one of the most interesting proposed structures in Russian art. Simultaneously, we can also see the buildup of the organizational complex, computer-controlled transport, and sensory-controlled buildings (known more widely today as "smart" buildings). How synæsthetics is linked to networks, systems and grids, and information processing is evident here - not to mention industrial time management considerations, which grew out of Taylorism to increase signals sent to and from machines based on operations and material interactions. The connection here lies with efficiency and communication systems that allow instructions to be sent and received as quickly as possible or in real-time. The plan to electrify the world and bring energy supply to cities underscored the motivation of governments and corporations to create a socially integrated world that is interconnected and somewhat dependent on these supplies.  

[1] Langer, Susanne, Feeling and Form: A Theory of Art (New York: Charles Scribner’s Sons, 1953) 

[2] Alexandra Daisy Ginsberg. Accessed September 10, 2024. https://www.daisyginsberg.com/work/machine-auguries.

[3] Safran, A. B., & Sanda, N. (2015, February). Color synesthesia. insight into perception, emotion, and consciousness. Current opinion in neurology. Retrieved July 5, 2022, from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4286234/

[4] Nikolić, D. (2009) Is synaesthesia actually ideaesthesia? An inquiry into the nature of the phenomenon. Proceedings of the Third International Congress on Synaesthesia, Science & Art, Granada, Spain, April 26–29, 2009.

[5] Synaesthetics may also share some ideas with Synectics, developed in 1950s America to examine creative problem-solving methods in which the subject(s) may be unaware of the processes involved. The main difference is that rather than a process that specifically targets the psychology of creativity, Synaesthetics is a way for various transformative questions that are data-driven and can be output in many sense forms (visual, sonic, haptic, among others). 

[6] “Foucault: Power Is Everywhere: Understanding Power for Social Change: Powercube.Net: IDS at Sussex University.”, March 8, 2010. https://www.powercube.net/other-forms-of-power/foucault-power-is-everywhere/.