DHQ: Digital Humanities Quarterly
2016
Volume 10 Number 2
2016 10.2  |  XML |  Discuss ( Comments )

Experiential Analogies: A Sonic Digital Ekphrasis as a Digital Humanities Project

Viktor Arvidsson <viktoarv_at_ifi_dot_uio_dot_no>, Swedish Center for Digital Innovation. Department of Informatics, University of Oslo

Abstract

Humanistic uses of digital technologies have opened up new ways to think about, communicate, and discuss historical research. The common use of digital tools to visually represent ancient cultures and sites, however, has also introduced new issues. For example, critics have argued that digital visualisations, largely synonymous with reconstruction in 3D models, often attempt to represent a photorealistic-artificial vision of the past, and may often prove to be a way to communicate history to a large(r) audience [Forte and Siliotti 1997]. Against this backdrop, this article will discuss precisely how technology may help immerse researchers into historically situated life, and radically advance historical research. Adding to related criticisms of ocularcentric traditions of knowledge production, we contribute to this stream of research by arguing that contemporary visual representations of the past often concentrate on visual representations and seemingly maintain antiquity as a sanitised historio-cultural ideal [Westin 2012] [Tziovas 2014]. More specifically, this article seeks to demonstrate the potential of digital humanities to move beyond mere representations on screen and to mobilize other senses (specifically sound) as a historically situated component for research. For this purpose, we focus on the abstract principles and overall methodology for a recreation of the experience of sounds in the Roman amphitheatre.

That all our knowledge begins with experience there can be no doubt. For how is it possible that the faculty of cognition should be awakened into exercise otherwise than by means of objects which affect our senses, and partly of themselves produce representations, partly rouse our powers of understanding into activity, to compare, to connect, or to separate these, and so to convert the raw material of our sensuous impressions into a knowledge of objects, which is called experience.  [Kant 1996]

Epistemologies in the Humanities often draw upon Platonist frames of thought, the noumenal and the phenomenal, largely canonizing the former as essential in scientific outcomes, and disdaining the latter. Indeed, the noumenal (world known by intellect) is considered superior to the phenomenal realm, the material, physical world of immediate sensory perception [Howes 2005]. Experiencing phenomena as the initiating act towards examination, analysis and scholarly reflection is widely used in concrete scientific research, such as chemistry, mathematics, and physics. Experiencing as methodology has been particularly iterated by Emmanuel Kant, the early modern philosopher who is responsible for synthesizing both waves of rationalism and empiricism in modern epistemology.[1] Kant, in an attempt to explain the relationship between reason, scientific explanation and human experience, argued that our experiences are structured by necessary features of our minds. In his view, the mind shapes and structures experience so that, on an abstract level, all human experience shares certain essential structural features. Beyond such essentialist assumptions of knowledge production, we here argue that experiential and sensory arrangements mediated by digital tools may be used to advance and diversify the study of humanities, specifically the variability of historically situated human life.
Technology has opened up new ways to communicate and discuss historical research, often through conceptual recreation, with vision acting as the primary sense for engagement and reflection. A widely used method is visualizations. This, often intellectually refined methodology may provide the means to communicate history to a large(r) audience, in a situated space (cultural heritage exhibitions, museums, scholarly installations on display) or online (see Rome Reborn for example). The introduction of technology as both a lens and a methodology, however, has introduced new issues. Visualizations are "often synonymous with a photorealistic-artificial ideal about the past"  [Forte and Siliotti 1997]. They "often concentrate on visual representations and seemingly maintain antiquity as a sanitised historio-cultural ideal"  [Westin 2012] [Tziovas 2014] . While visualisations may facilitate an understanding of movement in (currently musealised) spaces or urban planning and architecture, they are, in their majority, deprived of other senses, beyond vision.
Against this backdrop, we examine how technology may help researchers in their quest to understand historically situated life, beyond mere vision. Our scholarly endeavour addresses the possibilities for radically advancing historical research through experiential analogy that may involve sensory immersion. Adding to related criticisms of ocularcentric traditions of knowledge production, we seek to demonstrate the potential of digital humanities to move beyond soundless representations on screen and to mobilize other senses (specifically sound) as a historically situated component for research. For this purpose, we focus on the abstract principles and overall methodology for a recreation of the experience of sounds in the Roman amphitheatre as a case study. Our aim is to demonstrate how experiential analogy may convert the raw material of our sensuous impressions into knowledge production.
That the study of senses has escaped our point of espial for centuries may, to an extent, be connected to deeper epistemological problems regarding the relative value of scientific and humanistic modes of knowledge creation [Smithies 2014]. In this article, we aim to target this issue and explore how Digital Humanities could facilitate a study of the senses in context and promote a more immersive and multifarious comprehension of historically situated life. Our case study is Roman entertainment, because of the challenges it entails for research. Our primary and raw data have their own limitations. Most amphitheatres are currently musealised monuments, inclusive of the soundscapes that spring from their surrounding environment, thus, out of their "original" context. Second, ancient narratives are composed by an upper echelon of literate Roman citizens, so the voices of microhistory (slaves, women, entertainers) are absent in our account of the Roman empire, making it partial. Later receptions of antiquity (especially during the 18th century) have been vastly re-iterated as idealized, thus blurring our contemporary understanding further. Recent research has delved more into micro-historic societal concepts, and consequently concluded that previous interpretations suffer from historically restricted assumptions, the outcome of adopting hierarchically stratified categories of analysis. If ancient performance is instead seen as a popular means to project concepts of identity, a sensory discourse may aid scholars in capturing and articulating the complex societal dynamics in a Graeco-roman backdrop.
The Roman amphitheatre as a material, architectural structure and a monument is present in the areas surrounding the Mediterranean sea, from North African countries to Asia Minor to continental Europe. The diversity of cultures composing the Roman empire seem to have adopted Roman spectacle as a form of popular entertainment. It has been argued that Rome was an "oral and aural" society [Toner 2014, 1–21]. Buildings, especially Roman amphitheatres, were designed to accommodate a live event, a spectacle that was inclusive of the sensory experience: it incorporated sound, smell, and movement. Latin and Greek texts describing this experience were originally read out loud to a largely illiterate population. To enhance the texts’ vividness, authors often used the rhetorical devise of ekphrasis, an evocative style of narrative that aimed to stimulate the senses, often simultaneously - in a synaesthetic manner. Synaesthesia is known generally as the condition for those who experience one kind of sensory stimulus simultaneously as another, most commonly sounds with colours. A broader definition of synaesthesia has been the "blending of all senses experienced by all readers, synaesthetes or not"  [Butler and Purves 2013]. By discussing the possibilities and problems of (a) digital recreation(s) of historically situated sound, this article aims to show how technology may help us not only redress the nexus of societal and cultural practices in relation to more immediate, pre-industrial forms of entertainment, but also provoke new questions about the popular rendering of antiquity within contemporaneity. Therefore, we hereby argue that historical accuracy, as a methodological criterion in the study of ancient performance, should be discarded in favour of experiential analogy. The noumenal then transforms from one "archetypal" ideal to a variation of experiences.
Through experiential methodology, rich sensory detail may aid scholars to move beyond ideals of accuracy, and to give us further insight into the lifeworld of antiquity. In particular, we examine how digital sound engineering can be used to re-enact the ancient experience of Roman entertainment. To address this question we suggest the incorporation of sound studies within the field of historical studies, and specifically a combination of text mining and archaeoacoustics: the study of the sonic experience of archaeological sites via digital acoustic modelling [Blesser and Salter 2006]. In so doing, we show that by using simple programmes for digital modelling of sounds scholars can readily layer audio onto the primarily visual components of conventional digital prototypes. However, we also show that soundscapes modelled around monuments are not enough: to understand Roman culture we must acknowledge sensory experience as ephemeral, therefore variable and historically situated.
To help scholars tackle this issue we thus articulate a methodological approach, termed synaesthetic prototyping. Inspired by post-processual thinking in archaeology, the proposed approach blends narrative with archaeoacoustic techniques to turn attention from sensory accuracy to experiential analogy. This allows scholarship to better account for the multiple interpretations possible of any sensory experience [Hodder and Hutson 2003] and to treat sensory data, in this case, sound, as materially situated in both space and time [Hodder 2012]. For example, repeated and frequent car horn sounds in the arctic town of Umeå, Sweden may be a sign of chaos, whereas in downtown Athens, Greece they indicate common traffic. By belabouring how to understand the ways in which senses interweave notions of place, society and culture through a deeper form of assessing monuments and reading primary narratives enabled by digital means, this article ultimately contributes to the sensory turn in archaeology and history research.
This article ultimately pushes toward a sonic digital ekphrasis [Lindhé 2013]. Similar to the rhetorical device of ekphrasis, which may be used to describe any experience, digital technology can be a means through which to recreate, experience and study the sounds within a historically situated occasion. Sonic digital ekphrasis becomes a methodological device in itself, enabling us to reassess the importance that senses and sounds in particular held for Roman culture. It is our hope that this article provides additional intellectual justification for a turn to studying the senses digitally. We wish to clarify up front that this topic requires further exploration, but is important to keep in mind: as digital humanities require conceptual anchors and an openness to theories and methods borrowed from other disciplines, scholars in a post-disciplinary world need to be reminded of the "importance of craft and the value of tacit knowledge"  [Smithies 2014], both as a tool and a methodology.

The Ephemeral Qualities of Sound #1: Archaeoacoustics

Digital prototypes in the form of 3D visualisations have found prominent use within humanities research [Drucker 2011] [Frischer et al. 2006, 163–82] [Forte 2010] [Nygren et al. 2014]. In traditional historical disciplines such as archaeology, history and classics, scholars have, for example, used digital technologies to "map out" and relate detailed geographic data to primary sources in order to perceive larger societal patterns, and to visualise places geographically and chronologically remote [Mahony and Bodard 2010, 1-14] [Barker et al. 2010, 185-200]. The ample use of web-mapping applications such as Google Earth have changed the ways in which scholars perceive data; the application of contemporary web-based methods towards visualisations and geospatial analysis challenges conventional concepts of space (see [Mahony and Bodard 2010, 7] and [Dunn 2010, 53–72]. The vast majority of scholarly attempts to digitally reconstruct ancient urban sites for entertainment or otherwise indeed rely on the visual representation of (physical) materiality such as buildings, bridges or roads through 3D and virtual reality models. These visualisations have provided much historical insight into aspects of urban development [Christopoulos et al. 2003, 61–71] and have facilitated critical discussions of the application of digital tools within the context of museology, or the narratives of digital heritage (see [Christopoulos et al. 2003], [Christopoulos et al. 2004], and [Giaccardi et al. 2012].
3D models then make evident how existing digital tools and related models carry assumptions of knowledge as primarily visual, thus neglecting other sensory detail and sustaining the ocularcentric tradition within humanities research [Howes 2005, 14] [Classen 1997, 401–12], as well as an idealised image of antiquity. Western intellectual traditions has indeed shown a marked preference for vision as the figure of knowledge (see [Evens 2005, ix]. The excuse is often that intangible artefacts, such as senses, leave no traces or evidence, so we cannot represent them in their entirety. The lack of evidence is in fact present in any historical research, sensory or not. As pointed out by several scholars across disciplines, sound is characterized by ephemerality. A scholarly exploration of sensory experience may provide gateways to both the mind and the body; it mediates cultural values between the individual and the social world[Favro 2006] [Betts 2011] [Witmore 2006]. Sounds and their meanings are considered as shaped by the cultural, economic, and political contexts in which they are produced and heard [Smith 2001, 7] [Ihde 2005, 61–6]. Indeed, sound studies have been referred to as

the interdisciplinary ferment in the human sciences that takes sound as its analytical point of departure or arrival. By analyzing both sonic practices and the discourses and institutions that describe them, it redescribes what sound does in the human world, and what humans do in the sonic world.  [Sterne 2012, 2]

Sound as experience may then help us understand concepts of identity and community. Haines-Eitzen, in her forthcoming, "Geographies of silence", discusses the affective potential of landscapes and their ties to the senses and imagination. Her case studies, like most sound studies available, investigate the acoustic signature of a place with regard to religious imagination and the shaping of religious experience. She also argues that ("Sounds of the Angels" forthcoming) "sounds may construct, inform, and shape identity; situate individuals in time and place, indicate seasons, and built environments, binding a community through shared experience".
The cultural connotations of sound then may be understood to extend beyond language and through sensory encounter. For example, the "mind the gap" message in the London underground brings to mind a particular context - that of a European metropolis and its sites. That in itself creates further connotations — technology, trade, traffic, and rush hour, to name but some. All these concepts are separate parts of the material culture of London and in a way, they come together in human consciousness in order to denote the very identity of the city, a sonic reflection of its life within its community. Similarly, in the Roman world, sound, community, and cultural identity intersected and culminated in a particular type of building — the Roman amphitheatre.
The two hundred and thirty Roman amphitheatres that have been found across the area of the Roman empire [Welch 2007, 9] are indicative not only of the prominent place of the spectacle within Roman life, but also the weight of sensory experience therein: Roman amphitheatres are audio-visually accomplished arenas whose architecture was designed with multisensory experiences in mind. Against this backdrop, it is highly problematic that digital representations of the past are largely devoid of human sensory experiences beyond the oft-fetishized visual. How then can digital technologies advance the study of the senses experience in antiquity? While any cityscape is composed by a spectrum of sensory cues, we here focus on sound. Thanks to modern digital equipment and software, sound can be more easily, reliably and relatively inexpensively recreated and mobilised in comparison with other sensory properties of the material world.
Archaeology is, by its very nature, an interdisciplinary science, and as such has been quick to adopt digital technologies (databases, electronic field surveys and mapping) into its research methodologies [Mattern 2013]. The majority of practical archaeologists now routinely include the use of digital methods in their work, and there is perhaps no longer any meaningful delineation between "digital" and traditional archaeology. However, digital visualization has become relatively common in the last decade, whereas archaeoacoustics is still a young field of study. One of the reasons to apply archaeoacoustics to studies of the Roman amphitheatre is to facilitate recreation of how sound was experienced in this arena, paying detailed attention to the shape and size of each building through measurements of their physical properties [Blauert 2013, 1–5]. Archaeoacoustics also offers a solid foundation for enquiries into the relevance of sound for ancient social life, as there is already a basis of measurement (and standards) – a metrology – for attending to the acoustic qualities of places and things. These studies have been carried out in the last few decades in relation to prehistoric sites [Watson and Keating 1999] and Greek and Roman monuments (see [Avgerinou and Dreni 2014, 233–42]). These studies show that digital technology could be used to capture the acoustic qualities of a particular site. They estimate and simulate qualities of sound in physical spaces such as reverberation, or decay time, and so forth. However, they never move beyond providing a soundscape, and towards a sensory reading of experience. These studies are yet to be carried out for Roman amphitheatres.
Within the study of archaeoacoustics the evaluation of acoustic properties of ancient sites is commonly achieved via digital sound modelling programs such as ODEON and CATT-Acoustic™. The ODEON software is developed for simulating the complex acoustics of buildings in a site. Given the geometry and surface properties of a particular space, the acoustics can be predicted, illustrated and listened to. To estimate the acoustic experience ODEON uses the image-source method, virtually building a room combined with ray tracing, in order to map the path of sound through pixels in an image plane and simulating the effects of its encounters with virtual objects, for example the audience, or the very material/surface of the building. Through such techniques ODEON and similar software allows scholars to recreate sound experiences in both indoor and outdoor areas with complicated geometry, such as Roman amphitheatres. Operating in a similar way to ODEON, CATT-Acoustic™ is typically used for acoustics research and education.[2] However refined and accomplished, the use of archaeoacoustics software and hardware raises both technical and scholarly challenges for researchers.
In addition to the issue of accurately recreating a particular sound in a given soundscape, there are also related scholarly challenges to address within any archaeoacoustic inquiry. The most significant scholarly challenge is that archaeoacoustics involves recording and recreating the full array of sounds, which has many permutations in any given scenario. This creates a methodological problem. Archaeoacoustic studies cannot be carried out in controlled laboratory situations like acoustics proper. Any study or recreation of sound should thus be operated within the interpretative paradigm of social and humanistic research. There are several parameters that affect how accurately we may comprehend the Roman perception of sounds. The average aural ability within preindustrial settings varied a lot from ours, since our contemporary sensory perception is now affected by stereo and quadraphonic sound; whatever measurements we make, the results may be deceptive [Avgerinou and Dreni 2014, 240]; yet they may, overall, facilitate the grasping of a more conceptual construction of the multisensory experience of the Roman amphitheatre, as long as one bears in mind these implications and limitations.
Technically speaking, in terms of accurately recreating the sound within ancient sites, archaeoacoustics is a more complicated subject of study than acoustics proper. To begin with, sound modelling techniques are often designed with other purposes in mind. For example, in building acoustics, which provides the basis of archaeoacoustic inquries, lower parts of the frequency range are often ignored as they are not as disruptive for the contemporary human brain that is used to quadraphonic sound. There are further technical challenges that researchers of sound ought to take into consideration, such as the immobility and fragility of standard acoustics equipment. Other issues encountered in acoustically focused archaeological studies is that one must use a variety of practices, noting atmospheric variables such as temperature, pressure and wind conditions. Due to these challenges, it is often necessary to involve acousticians with experience of working with a plethora of conditional parameters. Overall, acoustic techniques appropriate for archaeological contexts are a work in progress [Till 2014, 28].
Due to these technical and scholarly challenges, historical accuracy is rather utopian in both scholarship and digital representations. Because of the fragmentary evidence and spatiality of Roman entertainment we can never expect to recreate any (sensory) experience of the spectacle or any other social situation in its entirety. It is, however, perhaps possible to establish some of the cultural meanings that sensory experiences held for the Romans by using digital technology to craft a conceptual experience of Roman entertainment which is based on sensory data within the literature.

The Ephemeral Qualities of Sound #2: Text Mining and further puzzles

there was noise everywhere produced by the equipment of death; here a sword was being sharpened, there someone was heating metal plates, here rods where produces, there whips… the trumpets were blaring with funereal sound… everywhere there was wounds, moans and gore, you could only see danger. (ps.-Quint. Decl. 9.6)

Narratives about the Roman games offer a rich case study for discussions of the experiential potentials of multisensory prototypes of ancient sites but this is challenging in and of itself. Primary sources concerning the sensory experience of Roman entertainment vary chronologically and geographically, indicating the importance of space and place for the constitution of experience. Although most narratives are rich in sensory detail, some authors privilege particular senses over others: "Martial’s descriptions have a tendency to be tactile, Seneca favours sounds and smells, whilst Juvenal focuses on noise and sordid sensations"  [Larmour 2007] cf [Betts 2011, 129–30]. By contrast, Cicero favours sight, as does Plautus (Cicero On the Orator 2.357; Plautus The Two Bacchises 1023). Given the abundance of sensory information in narrative, deeper examinations of the sensory sphere in antiquity may offer key insights into the fine nuances of historical social life.
More than any other artistic endeavour, performance before an audience thrives on representing events within the entire sensory spectrum and, consequentially, further affects how senses mingle and mix. This form of synaesthesia is also present in many of the primary sources that detail the importance of entertainment for Roman life [Toner 2014, 13–16]. Accordingly, there is a need for methodological approaches that can also make use of this kind of "synaesthetic" quality of narratives to provide more holistic understandings of the experiences of Roman life, by reflecting its individual details.[3] Synaesthetic narratives appear in a variety of authors and contexts. Ancient narratives of Roman spectacle are of particular interest to this inquiry, as they often seem to blur all the senses (including vision), evoking them in quick succession. For example, in the heart of these multisensory events, there are typically questions of light and darkness, elaborate backdrops as well as special effects, which seem to engage all the senses almost simultaneously (Cic. Verr. 2.158 and 141 for decoration in the forum during the Roman games, SHA Car. 19). Caligula, for instance, held night-time performances that lit up the whole city (Suet. Cal.18; Tac. Ann. 14.20-1; cf. Suet. Dom. 4) and naturally, as in a preindustrial society torches and fire would be used, thus involving the sound of fire and the smell of smoke. The Roman games also began with a ritual sacrifice, which involved the sound of fire, the vision blurred by smoke as well as the burning of incense and the roasting of the sacrificial meat.[4] Sounds may then be described as separate entities in a text without necessarily being referred to as sound. Take for example Suetonius’ description of Caligula’s nighttime activities (Gaius 18), where the sound of the fire of the torches would have to be reconstructed separately from simultaneous songs or compositions for musical instruments (improvisational or not), and then arranged together.
In this way, narratives about Roman spectacle contain a variety of sounds yet mingled with other senses. Aside from the sound of fire, there was also music, mostly horns and trumpets, sometimes even a multitude of flutes (SHA Car. 19). We know of specific events and how these emphasise different sounds for the sake of narrative; Cicero is specific about the screams of victims: "An effective training for the ear in the endurance of pain and death" (Cic. Tusc. 2.20.46). There is also information about the slaying of 100 lions on the spot at Probus’ games, an event that generated loud roars that sounded "as loud as thunder" (SHA Brob. 19). The description of sounds also involves the use of tools: metal whips, hot irons, and chains to drive both animals and humans: "there was noise everywhere produced by the equipment of death; here a sword was being sharpened, there someone was heating metal plates, here rods where produces, there whips… the trumpets were blaring with funereal sound… everywhere there was wounds, moans and gore, you could only see danger" (ps.-Quint. Decl. 9.6). Narratives regarding the sensory experience of the Roman spectacle intermingle smell, sound, movement, and vision in quick succession; there is both multiplicity as well as concurrency of sensory experiences. The reader (or audience) who came to experience the Roman spectacle sensed as well as envisioned, therefore experience should be viewed as a totality of sensory elements and should be reiterated as such. Recreating this sensory paroxysm by digital means, from narrative ekphrasis to digital ekphrasis, poses further challenges. As illustrated, the isolation of soundscapes needs to take into consideration words that are beyond mere descriptions of sound. The abundance and complexity of sensory information in Greek and Latin texts is without a doubt; yet synaesthetic prototyping has not yet been conceived of as an option for 3D models. To begin with, the digital as lens is a relatively young tool, let alone methodology. While 3D modelling is continuously evolving, technologies of sound as a live and experiential enactment is a very recent field, currently restricted to sound and the film industry (see Auro3D at http://www.galaxystudios.com/sound/) with no further (read academic) publications yet considered. Beyond academia, popular culture, in the form of the film and television industry, has tried to recreate the experience of the arena involving both image and sound, but normally falls into the discipline of reception studies with minimum attention placed on how technology may affect our understanding of antiquity. These issues in connection with user experience with one medium (image) over the other (sound) is perhaps the culprit. If we are to reconstruct the sounds of the arena we may not be able to recreate every aspect of sound in its entirety, but we may use specific, thematically selected narratives to illustrate a point with reference to a specific sound effect. The section below attempts a methodology for a practical application of these issues.

Synaesthetic Narratives and the Recreation of Sound

This section presents a conceptual framework and methodology intended to help scholars to address the complexity that inevitably characterises sensory narrative about Roman entertainment. This "synaesthetic prototyping" involves two distinct but interrelated cycles of analysis. The first step is to undertake a thorough review of the Graeco-Roman literature which references Roman entertainment. Special attention should be paid to words that denote sensory elements, specifically those related to sound. The second step draws on recent methodological research in the digital humanities in order to creatively fill in gaps of knowledge and demonstrates that sound can be metaphorically "mapped" onto digital (visual) prototypes through the use of modern sound engineering schemes and tables. This process allows sensory experiences of ancient sites to be further explored through what can be described as imaginative fabrication. Synaesthetic prototyping is based on the premise that there is an audience/reader accessing the text, and that the narration of the experience has a sensory impact on them. It allows scholars to make use of available technological tools in order that through their digital recreation they may study sound in antiquity through a sonic digital ekphrasis.
The Roman spectacle offers a rich case study for discussions of the experiential potentials of synaesthetic narratives that detail the importance of entertainment for Roman life [Toner 2014, 13–16]. Consequently, there is a need for methodological approaches and outputs (such as multisensory 3D models) that can make use of this kind of "synaesthetic" quality of narratives in order that they might provide more holistic understandings of the experiences of Roman life.[5] Unfortunately, the technology is currently not fit for the purpose: sound can be modelled in a way that other sensory data cannot. So, for the purpose of this section, synaesthetic prototyping will focus on sounds described in Graeco-Roman literature, and attempt to arrange them according to their impact on the audience. In order to recreate and explore the possibilities of sensory experience in context, a deeper reading is required. The reader reads in order to sense, but does not necessarily separate senses from one another [Toner 2014, 3], and this creates difficulties for scholars seeking to mine texts for sensory data. The sounds we retrieve from texts may have been meant to be imagined, especially in the case of Roman spectacle. Thus, the isolation of soundscapes needs to take into consideration words that are beyond mere descriptions of sound. Recreating this sensory paroxysm by digital means, taking us from narrative ekphrasis to digital ekphrasis, is the primary challenge that the methodology herein proposed seeks to address. Although it is impossible to recreate every aspect of the amphitheatre’s soundscape, it is possible to extract specific sound effects. Take for example the sounds of hissing and applause described in a passage from Plutarch which refers to a specific event in the Roman arena in 67 BCE:

The people took this as a mark of dishonour to themselves, and when Otho appeared in the theatre they whistled (hissed) (syrigmos) at him insultingly, while the equestrians greeted him with loud applause (krotos).The people renewed and increased their hisses, and then the equestrians their applause[6].

How can we appreciate the experience of hissing and applause as situated in this particular historical and cultural setting? A synaesthetic understanding of narrative experience implies that we can examine this question by reading the sensory expressions together. Stated differently, we can begin to map out the social significance of hissing by relating it to other sensory material in the passages where applause occurs. By investigating these relations further, we can deepen our understanding of the function of hissing, but also start to reflect on nuances and varied forms of use of the same sound. That is, we trace a particular sound through its appearance in a greater sensory ensemble. Through such sensory investigation within the texts’ rhetorical ekphrasis we can begin to critically examine what hissing actually entailed rather than what hissing may have sounded like (the audio signals). This in turn allows for a situated (re)making of the experience of hearing hissing and applause within 3D models. In this instance, hissing entailed that the majority of the audience disapproved of Otho’s appearance. Plutarch does not state which of the two classes persevered in their manifestation of feelings, but it is likely that the equestrians were wealthier (since they could afford a horse) and perhaps fewer in number. In any case, in spite of given hierarchies within the Roman empire, the reaction of the audience and the sound it reproduced, confirm our understanding of spectacle as a place and an occasion were political actors and ideologies were exposed and (often) openly criticized - a form of social media/ popular culture of the past.
In order to conceptually reproduce and construct these sounds we might consider how sound engineers construct sound for contemporary film productions [Till 2014, 27].[7] When modelling the sounds from Plutarch’s passage we need to take into consideration a commonly used sound-engineering scheme, the modes of listening, which is a technique that aims to facilitate the conceptual creation of sound by engineers. Since 1995 our ability to identify and recreate sounds has been influenced by the awareness of multiple modes of listening. Chion (1994) was the first to introduce a more comprehensive scheme for modes of listening.[8] Tuuri, Mustonen and Pirhonen [Tuuri et al. 2007, 15–17] advanced Chion’s mode of listening into a table that contained and explained eight categories of framing: two Pre-conscious modes, Reflexive (causing reflexing responses) and Connotative (evoking freely formed associations); three Source-oriented sounds, Causal (identifying cause), Empathetic (state of mind or intentions) and Functional (purpose of sound); two Context-oriented sounds, Semantic (symbolic/conventional meanings) and Critical (importance for the situation); and a reduced sound (describing properties of the sound). The order of modes implicates their level of the audiences’ cognitive abstraction from low to high and can thus help conceptualise a soundscape which may help convey the experience of the ludi (Table 1).
Sound type and explanation Listening mode Example for Implementation
Pre-conscious (background, not intended)
Fragmentary, estimated and reconstructed from other sources.
Background sound of fire and tools(?), audience (talking, moving, and so on).
Reflexive Originating from various locations
Connotative Positive and negative connotations
Source-oriented
(organised, intended)
Applause then hissing.
Plebs and equestrians had very different seating arrangements, so this could refer to the source of sound, such as cheers coming from one area and hissing from another.
Casual
Equestrians
People
Empathetic
Approving
Disapproving
Functional
Support
Opposition
Context-oriented
(every sound or music)
Background sound of fire and tools(?), audience (talking, moving, and so on).
Applause then Hissing.
Semantic
Greet
Insult
Critical Political and Cultural connotations
Reduced Interchangeable
Table 1. 
Modes of listening for a soundscape. Source: Foka; based on [Tuuri et al. 2007, 16]
Plutarch delivers information about source- and context-oriented sounds. Pre-conscious sounds, such as background noise created by a crowd, can be estimated from architectural and other evidence for the amphitheatre. The cultural connotations are denoted through the sounds of hissing and applause; from the context we understand that one sound is disapproving, whereas the other is approving. The sounds of hissing and applause are also reflexive and causal: they trigger one another. Each sound originates from a particular area of the amphitheatre, an acoustic reflection of social status which is based on the seating positions of equestrians and people. But how can we better understand these sounds of hissing and applause beyond their being mere signals or binary emotions? How does extraction of these sounds from Plutarch’s text enable reconstruction of ancient events in contemporary digital media? If our goal is to translate the sounds from classical literature into a digital form, isolating a sound and conceptualising its level of cognition in sequence can indeed only be done with a provisional use in mind. This process offers only glimpses of the experience of Roman spectacle in its spatial and cultural contexts, and only hints at the importance these sounds may have held for the audience (and indeed the performers). Consequently, even if we manage to recreate the sound accurately, the contemporary reception or experience of that sound may not be the same – this cultural specificity of sensory experience is one of the challenges, but also one of the attractions, of sensory archaeologies [Hamilakis 2015, 4].
When combined with contemporary understandings and models of the acoustic properties of the Roman spectacle, narratives such as the Plutarch extract discussed here can be used to facilitate our comprehension of a variety of performative and communicative activities, both between audience and performance, and among spectators. Once we delve more deeply into our sources for their sensory data, as in the soundscape illustrated in Table 1, we see that the audience certainly held a strong sensory position within the Roman amphitheatre, and that was critical to the effects of entertainment (what Hamilakis describes as "affective"  [Hamilakis 2015, 6]. The crowd was involved in an intense physical and verbal participation and interaction: hissing, approving, disapproving and clapping. Once we reconstruct these sensory scenarios we are engaged in an iterative process of reconfiguration and conversation; then reflection begins. This process of exploring the various configurations and alternative possibilities is what we term as "synaesthetic prototyping".
Within the HUMlab project Digital Bread and Circuses (see more https://wiki.digitalclassicist.org/Digital_bread_and_circuses funded by the Baltic Group Foundation) and located at Umeå University, we[9] mapped the entailment of sound within a specific screen environment and tried to match this with real sounds (weather, fire, hissing, applause, the sound of weapons) that evokes the contemporary equivalent. Since there is the risk that looking for the semiotics of sound could lead one into exclusively non-audio territory we took into consideration that a signifier (sound) can be discarded as an arbitrary pointer to the significant, with the latter being of principal interest. For phase one of our synesthetic prototyping, we used six speakers to arrange a soundscape in Display Studio (see figure 1, below).
Figure 1. 
3D Visualisation of the Display Studio, HUMlab (credits: Mattis Lindmark)
With all our sound settings (reverberation, impact, etc.) set to analogue (as opposed to digital) we aimed for the sonic impression of a pre-industrial setting. We then divided natural sounds (birds, rain, etc.) from artificial (human-generated) sounds and we arranged them according to table 1 above, with the audience reaction as the largest level of abstraction followed by Pre-conscious sounds, such as background noise. We then placed sources (speakers in strategic position in our sound-isolated Display Studio (see figure 2). In phase two (2016) we intend to experiment with Auro 3D while sonic elements with the highest level of abstraction such as the reactions of the audience (hissing, talking, clapping, etc.) will respond with interaction design to the sonic reactions of the physical audience of the installation. This "making" recognises that, no matter the data, via the route of experiential analogy, information will always be incomplete and gaps therefore need to be filled imaginatively in order to explore modes of knowledge further. We aim to experiment with contemporary audience reactions from cultures currently found around the Mediterranean. We will use this method to examine phenomenal patterns of communal approval (if any), thus renegotiating social and ethnic diversity in the Roman Empire.
Figure 2. 
Display Studio, HUMlab floor plan disposition of sound inputs (credits: Anna Misharina), a 5:1 system of Dolby Surround. Further plans incorporate Auro-3D with a 9:1 system of speakers (www.auro-3d.com).

Conclusions: Towards a Sonic Digital Ekphrasis

In this article we have indicated some of the directions in which studies of sound in the Roman amphitheatre could be taken, and in so doing, we hope to address the need for new technological tools and theories to overcome the problems with recreating ancient sounds. Archaeoacoustics facilitates an understanding of sound and noise within a physical space, but has yet to be applied to Roman sites. Digital modelling allows for the partial mapping of Roman soundscapes. The study of soundscapes is a philosophical and conceptual activity which goes beyond the physical reconstruction of the acoustic properties of a space such as the Roman amphitheatre. Digital technology may facilitate the reconstruction, experience and reassessment of the sounds within a Roman amphitheatre, but the output is only as good as the data on which it is based, as advanced as the technology used to capture and recreate it. Graeco-Roman literature provides us with rich ekphrastic narratives and contexts, and a synaesthetic approach enables a deep reading of those texts. This deep reading of texts combined with digital modelling allows us to explore various sensory configurations, each of them hypothetical due to the fragmentary nature of the evidence and the cultural specificity of the experiences we seek to recreate. Nevertheless, this is a worthy enterprise. Starting with the assumption that during any live event there is an audience and a sensory (aural for example) impact on the spectators, experiential analogy through synaesthetic prototyping becomes a methodological device in itself. It aims to renegotiate the importance sounds held for the Romans and break down our understanding of the past as a mute, sanitized intellectual ideal.
The Roman spectacle was an experience that involved the entire sensory spectrum, for both its audience and its performers. In order to fully understand it, one has to engage in its multisensory vitality. Digital prototyping and installations of sensory data have the potential to bring out the cultural and historical significance of the senses through immersive experiencing. Beyond theoretical speculations, such an account of the senses in antiquity could enhance the experience of cultural heritage exhibitions[10], or add to peripatetic explorations of archaeology on-site. For example, the recreation of an opening sacrifice to the Roman games will incorporate both the sound and the visual effect of fire. It could also further inform more popular industrial applications of antiquity: film and games, for instance. Roman history and culture will no longer appear mute, or biased by older, sanitized and whitewashed receptions of antiquity. The adoption and canonisation of the noumenal as superior, bespeaks of our very own hierarchic stratification within conditions of knowledge production. Against the ocularcentric tradition, we finally argue that the potentials of digital prototyping of sensory artefacts should aim to bring out the cultural and historical significance of the senses for researchers and students. [Crary 1988], drawing on Latour, states how sight is central in scientific research of the past. With digital technology we may be able to learn to see and hear at the same time, or communicate the past to people who lack vision. And alongside its noise, we can further challenge its (digital) ancient neatness. Challenging idealised cultural stereotypes is certainly the first step to making Humanities relevant to the actual study of humanity. With the intermediation of technology, Digital Humanities projects offer a possible venue.

Acknowledgments

The authors would like to express their gratitude to the DHQ editors (especially Duyen Nguyen) and the anonymous reviewers of this article, for their acute professionalism and constructive criticism. We are also indebted to Kim Haines- Eitzen (Cornell) and Amy Papalexandrou (Stockton) for sharing their forthcoming articles and inspirational ideas, and Eleanor Betts (the Open University) for reading and commenting thoroughly on previous versions of this article. Last, but certainly not least, we wish to thank Anna Misharina, Johan Von Boer, Roger Mähler, Jim Robertsson, and Mattis Lindmark (Humlab), as well as Mattias Eklund (Toontrack) and Tom Van Achte (Galaxy studios) for discussing architecture, special arrangements for immersive environments and sharing their thoughts and valuable experience on sound technology in real time settings.

Notes

[1] See [Kant 1996].
[2] Aside from software, another common practice has been the estimation of sound within physical models such as concrete reconstructions (see [Till 2014, 31] on the reconstruction of Stonehedge in Maruhill, Washington State, US). The process often involves a theoretical prediction of acoustics as well as acoustical field-testing and measurement.
[3] Synaesthesia is known generally as the condition for those who experience one kind of sensory stimulus simultaneously as another, most commonly sounds with colours. A broader contemporary definition of synaesthesia, however, has been the "blending of all senses experienced by all readers, synaesthetes or not"  [Butler and Purves 2013].
[4] A poem from 460 CE describes games being raced by amateurs, rather than professionals. It gives a sense of what it was like to sit down and watch the races as they progressed. There are descriptions of sound, colours and movements in quick succession. The whistling cheers of the hordes, the bright gleam of the colours, white and blue, green and red on insignia. There is also a description of the horses themselves: their sounds and movements as they push, stamp, drag, struggle, rage, jump, fear and are feared (Sid. Ap. Carm. 23.307-427). In Dion. Hal. Ant. Rom. 7.72.13 there are similar indications of sound activity that is both source as well as context orientated: "After these bands of dancers came a crowd of lyre-players and many flute-players, and after them the people who carried the censers in which perfumes and frankincense were burned along the whole processional route". In the processional sacrifice of the games, the audience experiences a variety of senses. The sound of fire is both reflexive and functional (as well as the smell).
[5] Synaesthesia is known generally as the condition for those who experience one kind of sensory stimulus simultaneously as another, most commonly sounds with colours. A broader contemporary definition of synaesthesia, however, has been the "blending of all senses experienced by all readers, synaesthetes or not"  [Butler and Purves 2013, 1].
[6] τοῦτο πρὸς ἀτιμίας ὁ δῆμος ἔλαβε, καὶ φανέντος ἐν θεάτρῳ τοῦ Ὄθωνος ἐφυβρίζων ἐσύριττεν, οἱ δ᾽ ἱππεῖς ὑπέλαβον κρότῳ τὸν ἄνδρα λαμπρῶς, αὖθις δὲ ὁ δῆμος ἐπέτεινε τὸν συριγμόν, εἶτα ἐκεῖνοι τὸν κρότον. ἐκ δὲ τούτου τραπόμενοι πρὸς ἀλλήλους ἐχρῶντο λοιδορίαις, καὶ τὸ θέατρον ἀκοσμία κατεῖχεν (Plutarch Cicero 13.3).
[7] Music archaeologist Lund (2012) has suggested the use of the term sound archaeology rather than archaeoacoustics or music archaeology [Till 2014] as music is often defined as organised sound.
[8] It consists of causal, semantic and reduced modes. As causal (everyday) mode of listening refers to ecologically orientated evident denotations, there was indeed a place for a mode that addresses socioculturally shaped and learned (symbolic) codes - causal always refers to the source. Chion’s scheme of listening modes is comprehensive, yet one of its obvious shortcomings was its inability to consider the social and cultural connotations of sound and the level of cognitive abstraction in their recreation.
[9] The HUMlab team of inquiry is composed by (in alphabetical order) Anna Foka, Johan Grönskog, Roger Mähler, Anna Misharina, Fredrik Palm, Jim Robertsson, and Johan von Boer with Roman Spectacle as case study for a number of physical and experiential installations and a deep mapping prototype.
[10] It may be particularly relevant to the recording and experience of oral traditions such as epic poetry, or indigenous cultures (Australians, Sami, etc).

Works Cited

Agre 1997 Agre, P. 1997. "Toward a Critical Technical Practice: Lessons Learned in Trying to Reform AI". In Beyond the Great Divide: Social Science, Technical systems, and Cooperative work, edited by Bowker, G. C., Gasser, . L., Star, S. L. and Turner, W. Mahwah, NJ: Erlbaum: 131–58.
Avgerinou and Dreni 2014 Avgerinou, P. and Dreni S. 2014. "The Acoustics of the Eleusinian Telesterion". In Archaeoacoustics: The Archaeology of Sound, edited by Eneix L. C. OTSF: Florida 233-42.
Barker et al. 2010 Barker, Elton; Bouzarovski, Stefan; Pelling, Chris and Isaksen, L. 2010 "Mapping an ancient historian in a digital age: the Herodotus Encoded Space-Text-Image Archive (HESTIA)" Leeds International Classical Studies, 9
Barker et al. 2012 Barker, Elton; Bissell, Christopher; Hardwick, Lorna; Jones, Allan; Ridge, Mia and Wolffe, J. 2012. Digital technologies: help or hindrance for the humanities? (2012-02) Arts and Humanities in Higher Education, 11(1-2) (pp. 185-200)
Betts 2011 Betts, E. 2011. "Towards a Multisensory Experience of movement in the City of Rome", In Rome, Ostia and Pompeii: Movement and Space, edited by Laurence, R. and Newsome, D. Oxford: Oxford University Press: 118-32.
Blauert 2013 Blauert J. (2013) "Conceptual aspects regarding the qualification of spaces for aural performances" Acta Acustica united with Acustica: 99:1: 1-13.
Blesser and Salter 2006 Blesser, B. and Salter, L-R., 2006. Spaces Speak, Are You Listening? Experiencing Aural Architecture. Cambridge, MA: MIT Press.
Blesser et al. 2007 Blesser et al. (2007) "Spaces Speak, Are You Listening? Experiencing Aural Architecture", The Journal of the Acoustical Society of America (2007) 121: 4: 18-20.
Butler and Purves 2013 Butler. S. and Purves, A. 2013. Synaesthesia and the Ancient Senses, Durham: Acumen.
Chion 1994 Chion, M. 1994. Audio-Vision: Sound on Screen. New York, NY: Columbia University Press.
Christopoulos et al. 2003 Christopoulos, G. Gaitatzis, A. and Papaioannou, D. 2003. "Image-Based Techniques for Enhancing Virtual Reality Environments". Proceedings in Arts and Cultural Heritage 2nd International Workshop on ICTs, Athens: Art Nouveau: 61-71.
Christopoulos et al. 2004 Christopoulos, G. Gaitatzis, A. and Papaioannou, G. 2004. "The Ancient Olympic Games: Being Part of the Experience". International Symposium on Virtual Reality, Archaeology and Intelligent Cultural Heritage, Athens: VAST: 19-28.
Classen 1997 Classen, C. 1997. "Foundations for an Anthropology of the Senses". International Social Science Journal 49 (153): 401–12.
Crary 1988 Crary, J. 1988. "Modernizing Vision". In Vision and Visuality, edited by Foster. H. Seattle: Bay Press: 29–44.
Drucker 2011 Drucker, J. 2011. "Humanities Approaches to Graphical Display". Digital Humanities Quarterly 5:1.
Dunn 2010 Dunn S. 2010. "Space as Artefact: A Perspective in Neogeography". In Digital Research in the Study of Classical Antiquity, edited by Mahony, S. and Bodard, G., Surrey: Ashgate: 53-72.
Dunne 2005 Dunne, A. 2005. Hertzian Tales: Electronic Products, Aesthetic Experience, and Critical Design, Cambridge, MA: MIT Press.
Evens 2005 Evens, A. 2005 Sound Ideas: Music, Machines, and Experience (Minneapolis, MN: University of Minnesota Press, 2005), p. ix.

Favro 1996 Favro, D. 1996. The Urban Image of Augustan Rome, Cambridge: Cambridge University Press.
Favro 2006 Favro, D. 2006. "In the Eyes of the Beholder: Virtual Reality Re-creations and Academia". In Imaging Ancient Rome edited by Haselberger, L. and Humphrey, J., Portsmouth, RI: Journal of Roman Archaeology: 321-34.
Forte 2010 Forte, M. (ed) 2010. Cyber Archaeology. BAR International Series 2177, Oxford: Archaeopress.
Forte and Siliotti 1997 Forte M. and Siliotti, A. 1997. Virtual Archaeology: Re-Creating Ancient Worlds, New York: HN Abrams.
Frischer et al. 2006 Frischer, B., Abernathy, D., Giuliani, F. C., Scott, R. T., and Ziemssen, H. 2006. "A New Digital Model of the Roman Forum". In Imaging Ancient Rome edited by Haselberger, L. and Humphrey, J., Portsmouth, RI: Journal of Roman Archaeology, 163-82.
Giaccardi et al. 2012 Giaccardi, G. et al 2012. Digital Heritage and Social Media. New York: Routledge.
Haines-Eitzen (a) Haines-Eitzen, K. Forthcoming. "Geographies of Silence," in Knowing Bodies, Passionate Souls: Sense Perceptions in Byzantium, ed. Susan Ashbrook Harvey and Margaret Mullett. Cambridge MA: Dumbarton Oaks Publications.
Haines-Eitzen (b) Haines-Eitzen, K. Forthcoming. "The Sound of Angels’ Wings in Paradise: Religious Identity and the Aural Imagination in the Testament of Adam". in Jews and Christians in the Greco-Roman World, edited by Shira Lander et al. Rhode Island: Brown University Press.
Hamilakis 2015 Hamilakis, Y. 2015. Archaeology and the Senses: Human Experience, Memory, and Affect, Cambridge: Cambridge University Press.
Harvey 2006 Harvey, S. A. 2006. Scenting Salvation; Ancient Christianity and the Olfactory Imagination. Berkeley, CA: University of California.
Hodder 2012 Hodder. I. (2012) An Archaeology of the Relationships between Humans and Things, London: Wiley and Sons.
Hodder and Hutson 2003 Hodder, I. and Hutson, S., 2003. Reading the Past: Current Approaches to Interpretation in Archaeology. Cambridge: Cambridge University Press
Howes 2005 Howes, D. 2005. Empire of the Senses: the Sensual Culture Reader. Oxford: Berg.
Ihde 2005 Ihde, D. 2005. "Auditory Imagination". In The Auditory Culture Reader edited by Bull, M. Back, L. Oxford: Berg: 61-66.
Kant 1996 Kant, Immanuel; Kitcher, Patricia (intro.); Pluhar, W. (trans.) (1996). Critique of Pure Reason. Indianapolis: Hackett. xxviii.
Larmour 2007 Larmour, D. H. J. 2007. "Holes in the Body: Sites of Abjection in Juvenal’s Rome". In The Sites of Rome: Time, Space, Memory, edited by in Larmour, D. H. J. and Spencer, D. Oxford: Oxford University Press: 168-210.
Lindhé 2013 Lindhé, C. 2013. "A Visual Sense is Born in the Fingertips: Towards a Digital Ekphrasis". Digital Humanities Quarterly, 7(1): 376-82.
Mahony and Bodard 2010 Mahony, S. and Bodard, G. 2010. "Introduction". In Digital Research in the Study of Classical Antiquity, edited by Mahony, S. and Bodard, G., Surrey Ashgate: 1-14.
Mattern 2013 Mattern, S. (2013) "Ear to The Wire: Listening to Historic Urban Infrastructures" in Amodern 2: Network Archaeology.
Nygren et al. 2014 Nygren, T., Foka, A., and Buckland, P. 2014 "Digital History in Sweden". H-Soz- Kult, Humbolt: Humbolt University Press.
Schreibman et al. 2004 Schreibman, S., Siemens, R., Unsworth, J. 2004. A Companion to Digital Humanities. Oxford: Blackwell.
Smith 2001 Smith, M. M. 2001. Listening to Nineteenth Century America. Chapel Hill and London: University of Chapel Hill Press.
Smithies 2014 Smithies, J. 2014 "Digital Humanities, Postfoundationalism, Postindustrial Culture" DHQ Vol.8 n.1.
Sterne 2012 Sterne, J. 2012 "Sonic Imaginations" in The Sound Studies Reader, edited by Sterne J.. London and New York: Routledge: 2.

Till 2014 Till, R. 2014. "Sound Archaeology: Terminology, Palaeolithic Cave Art and the Soundscape". World Archaeology Journal 46(3).
Toner 2009 Toner, J. P. 2009. Popular Culture in Ancient Rome. Cambridge: Polity.
Toner 2014 Toner, J., 2014. "Introduction: sensing the ancient past". In J. Toner, ed., A Cultural History of the Senses in Antiquity. London: Bloomsbury, pp. 1-21.
Tuuri et al. 2007 Tuuri, K., Mustonen, M. & Pirhonen, A. 2007. "Same Sound – Different Meanings: A Novel Scheme for Modes of listening". In Proceedings of Audio Mostly 2007 Ilmenau, Germany: Fraunhofer Institute for Digital Media Technology IDMT. 13-18.
Tziovas 2014 Tziovas, D. 2014 Re-imagining the Past: Greek Antiquity and Modern Greek Culture. Oxford: Oxford University Press.
Watson and Keating 1999 Watson, A. and Keating, D. 1999. "Architecture and Sound: an Acoustic Analysis of Megalithic Monuments in Prehistoric Britain". Antiquity 73: 325-36.
Welch 2007 Welch, K. E. 2007. The Roman Amphitheatre: From Its Origins to the Colosseum‬. Cambridge: Cambridge University Press.‬‬ ‬‬‬‬‬‬‬‬‬‬‬‬‬
Westin 2012 Westin, J. 2012. Negotiating Culture, Assembling a Past: the Visual, the Non-Visual and the Voice of the Silent Actant. Gothenburg: University of Gothenburg Press.
Witmore 2006 Witmore, C. L. 2006. "Vision, Media Noise and the Percolation of Time: Symmetrical Approaches to the Mediation of the Material World". Journal of Material Culture Vol. 11(3): 267–29.