DHQ: Digital Humanities Quarterly
2018
Volume 12 Number 3
2018 12.3  |  XML |  Discuss ( Comments )

Ghosts in the Machine: a motion-capture experiment in distributed reception

Anna Foka <anna_dot_foka_at_abm_dot_uu_dot_se>, DH Uppsala and Humlab Umeå

Abstract

Digital reconstructions of classical antiquity are generally ocularcentric, appealing only to the sense of vision. We propose that new technologies may be used to engage the other senses in the act of reception, and specifically here we focus on kinaesthesia, or the sense of self-movement. This paper analyses a phase of the project Ancient Dance in Modern Dancers in which participants created performance pieces in a genre of Graeco-Roman dance for use in a motion-capture system. It was necessary for the performers to develop a range of translational strategies in order to communicate their movement to the system, entailing what we term “distributed reception”, in which the ultimate recipient of ancient source-material is not a human actor but rather the machine with which s/he is in collaboration.

Digital reconstructions of classical antiquity tend typically to use visualization as their primary sensory vehicle. As the “sensory turn” in historical scholarship moves beyond vision, however (e.g. [Butler and Purves 2013]; [Toner 2014]; [Betts 2017]), the question also arises of how alternative sense-experiences may be re-imagined, and how this re-imagining might be facilitated using digital media. Different forms of sensory engagement generate different forms of knowledge, creating what Foka and Arvidsson have termed the “experiential analogy”, whereby the ephemeral cultural practices of another era are translated into present-day sense experience [Foka 2016]. Foka and Arvidsson concentrate on sound, but equally crucial to knowledge-production is kinaesthesia, or the sense of self-movement. The vital contributions made by kinaesthesia to comprehending physical potentialities and spatial relationships as well as the abstract concepts derived from them have been widely demonstrated in the domain of cognitive science [Sheets-Johnstone 2011] (see also [Gallese and Lakoff 2005], [Gallagher 2005], [Noë 2004]). It is our contention that kinaesthetic engagement can also contribute to formulating conceptions of the ancient past, and that digital technology is an ideal tool for fashioning this analogical relationship. Even as self-movement offers the impression of immersive bodily contact with antiquity, virtuality presents a simultaneous reminder of its otherness.
Technology has opened up new ways to examine antiquity.[1] Digital technologies are used to relate detailed topographic data to primary sources in order to visualize place and space distant in time [Schreibman, Siemens, and Unsworth 2004] [Mahony and Bodard 2010, 1–14] [Barker et al. 2010] [Barker et al. 2012, 185–200]. Digital prototypes in the form of 3D visualisations have found prominent use within humanities research [Drucker 2011] [Frischer et al. 2006, 163–82] [Forte 2010] [Nygren, Foka, and Buckland 2014] [Foka 2017] [Vitale 2016]. As argued by Foka and Arvidsson, the vast majority of scholarly attempts to digitally reconstruct ancient urban sites for entertainment or otherwise indeed rely on the visual representation of (physical) materiality such as buildings, bridges or roads through 3D and virtual reality models. 3D models thus make evident how existing digital tools carry assumptions of knowledge as primarily visual, neglecting other sensory detail and thereby sustaining the ocularcentric tradition within humanities research (e.g. [Howes 2005, 14]; [Classen 1997, 401–12]), as well as idealised representations of antiquity. Western intellectual traditions have indeed shown a marked preference for vision as the figure of knowledge [Evens 2005, ix]. The excuse is often that elements of intangible cultural heritage such as dance and motion (see definition by UNESCO) leave no traces or evidence, so we cannot represent them in their entirety. This lack of direct experiential evidence is in fact present in any historical research, sensory or not.
Kinaesthesia, however, is becoming increasingly important to reconstructive interfaces. Digital visualizations, prototypes and reconstructions provide historical insights into aspects of urban development and facilitate critical discussions of the application of digital tools within the context of museology or narratives of digital cultural heritage [Giaccardi et al. 2012]. With the development of immersive technology, the recent trend is to move beyond static 3D models and to enable interaction with users navigating in virtual environments [Westin 2012]. Contemporary technologies such as Virtual Reality (VR) and Motion Tracking (MT) engines enable participation, observation and user-interaction; these in turn afford sensory engagements with space (environment, architecture) and artifacts of the past [Forte 2010]. Materials and environments are further digitally reconstructed as 3D Models (for a review see [Forte 2010], [Vitale 2016]). These are currently implemented beyond databases, and within gaming [Chapman, Foka, and Westin 2016] [Chapman 2017], or Augmented Reality environments [Astic 2011] [Westin, Chapman, and Foka 2018] as well as, less frequently, within Virtual or Mixed Reality environments. The rarity of VR or MR is an issue attributed to the complexities of local infrastructure [Foka 2017] as well as emerging upgrades and sustainability. Our knowledge about the intangible elements of the ancient world, such as movement, image, and sound, may be currently recreated in immersive laboratory environments.[2] However, beyond archaeology proper, research of ancient performance within immersive environments has not been conducted until recently.[3] Specifically, the creation of immersive interactive and multimodal environments for historical performance has been underexplored.

1.

The practice-led research collaboration Ancient Dance in Modern Dancers (ADMD) was set up in 2013 with the aim of translating the verbal and iconographic evidence for Graeco-Roman dance back into movement. Our initial hypothesis was that applying this evidence in the formulation of physical performances would provide an alternative mechanism for understanding dance from the performer’s standpoint, as a kinaesthetic process, rather than its usual treatment as spectacle or ocularcentric product. By drawing on the expertise of professionally-trained dancers, we have been able to produce a range of conjectures for the representation of selected scenarios, and at the same time to extrapolate a number of key principles for the art-form.
The dance genre we have concentrated on is tragoedia saltata, otherwise known as Graeco-Roman tragic pantomime, or simply orchēsis. This was a form of solo storytelling through dance and gesture popular between the first and fifth centuries CE across the Roman empire, especially in the Greek East. Much excellent scholarship over the last decade or so has established the attributes of orchēsis, which is now recognised as a sophisticated cultural discourse and an important vehicle for the late antique transmission of tragedy [Lada-Richards 2007] [Hall and Wyles 2008] [Webb 2008] [Macintosh 2010] [Garelli 2007] [Zanobi 2013]. Orchēsis scenarios could be comic or tragic, erotic or philosophical; a single masked dancer, the orchēstēs, played multiple roles, indicating character, setting and plot through a combination of hand gestures, choreograpic sequences, and iconic poses. He (usually he, although some female orchēstes are also known)[4] wore a flowing robe and cape, and was accompanied by musicians and a singer or singers who performed a libretto alongside.
No extant libretti have been definitively identified,[5] but one very probable Latin source is Ovid’s poetic compendium of myth, the Metamorphoses [Ingleheart 2008] [Lada-Richards 2013].[6] Transformation between characters, or from one state to another, was one of the skills for which orchēstes were celebrated;[7] moreover, the themes of orchēsis as summarised by the second-century satirist Lucian, “all of the mythical metamorphoses (muthikas metamorphōseis), those who have changed into trees and beasts and birds… and above all, the love-affairs of [the gods], even Zeus himself”, closely resemble the subject matter of Ovid’s poem.[8] For the current phase of ADMD, the two passages selected as our libretti were one episode in which sea-nymph Thetis attempts to evade a rapist by transforming herself into multiple creatures (Met. 11.229-65), and one in which the impious king Pentheus is torn apart by worshippers of the god Dionysus (Met. 3.699-729) (click here to listen; both texts may be found in the Appendix to this article).
Hitherto, ADMD has experimented only with live performance. Digitization has added new variables and raised a new set of questions regarding the relationship between the body of the dancer in the present, ever-mutable moment and the absent figure of the ancient orchēstēs, for whom the live dancer acts as a sēma, a marker. Live dance practice offered us insights into the vital symbiosis between the orchēstēs and the libretto, the orchēstēs and the musician, the orchēstēs and costume, character, emotions, mask/s and space; the additional layer of digitization placed more acute emphasis on the fundamental process of embodiment. Rather than the proprioceptive (internal sensorimotor) feedback which dominated the dancers’ choices in the development of their live performance pieces, the activation of an external avatar required ongoing interaction with this distorted mirror-image, this not-quite-self which both resisted and responded to the dancers’ manipulation. In setting up this experiment in motion capture, we shifted the point of reception from the human body to its virtual analogue.

2.

The objective of the pilot workshop run with ADMD was to examine how the dancers’ kinaesthetic translation of ancient data pertaining to orchēsis would be affected by the additional factor of digital interaction. For this purpose we used Motion Capture in order to create 3D avatars of the dancers. Once animated, the resulting videos will also enable each dance piece to be analysed from any angle in the context of a virtual ancient theatre environment.
Over the last few decades motion capture engines have been mostly enhancing our understanding of normal and pathological human movement and have been used in the context of medical practice and education, gaming and cinema.[9] Motion capture (or motion tracking) generally refers to the process of recording human actor or object movement and this information is used to animate digital character models in 2D or 3D user animation. Early techniques used images from multiple cameras to calculate 3D positions and assemble them to reconstruct the movement but not the actor’s visual appearance. Improved knowledge of locomotion drove the invention of new methods of observation. Currently, the most common method for the capture of three-dimensional human movement requires a laboratory environment inclusive of markers, fixtures or sensors. The movement of the markers is typically used to capture movement between two adjacent segments (e.g. knee joint, ankles, elbows etc) with the goal of near-precisely recording the movement of the joint (figures 1 & 2). For ADMD’s purposes, the crucial difference between 3D tracking and video recording is that video fixes a static viewpoint and coerces prospective users into a spectatorial role, with all the binary dynamics implied by this relationship.[10] In contrast, the interactive user plays an active role in determining the order in which already-generated elements are accessed. This is the simplest kind of interactivity, and it allows for variability of interpretations, as opposed to mere spectatorship.[11] In order to bring the user into the performance space, as it were, alongside the dancer, we found the polyfocal coverage of 3D capture to afford a less restricted medium.
For the purpose of recording the dancers we used the “Black Box”, an isolated motion capture laboratory space in the ground floor of the Sliperiet facility at Umeå University. In a space of approximately 20m2 (4x5m) we positioned a wooden flooring (3x3m), and initially twelve cameras on different angles to record the dancers. The hardware we used for the motion capture was an OptiTrack 13W system and the interface software was Motive. On the first day, we began warming up one by one the twelve motion capture cameras in by using the “wanding option” OptiTrack CW-500 (B= 250mm). We then tested the sensors with an OptiTrack three point sensor triangle (CS 200). The 90 degree triangle was rigid but served as a testing point to observe and to capture the two axes for the 3D capture. The software we used (Motive) offered several different interfaces including:
  1. Avatar, skeleton (no gender) that could be modified with default models that are male or female (figures 3–4).
  2. Motion tracking avatars with strings that display the direction and the velocity of the motion capture (figures 5–7).
Numerous problems were encountered while trying to set up and use the tracking system. The dancers’ feet could not be “captured” at first due to reflection from the concrete floor. The system would not pick up anything lower than 15cm, so the dancers’ feet remained undetectable. While the system was ready, the software was still not reading all the markers properly; it was picking up all 37 out of 37 sensor points, but informing us that it required more markers. This was solved by a combination of adjustments. We raised the floor surface to a height of about 3cm, and used non-reflective wooden boards. We also altered the angle of the cameras and moved the capture area so cameras could be attached to fixed points as opposed to being placed on moveable stands.
All this resulted in wasting some valuable time. By midday on the first day of laboratory work we still could not capture the whole skeleton as one, and we had to move on to test it with the dancers. While we got the 37 sensor points to show, there were further glitches in the system. For example, we would temporarily lose all movements of the arms and legs (because they had frozen or disappeared), but then manage to detect all the points again after a few moves. We were informed by Humlab and Sliperiet senior technician Jim Robertsson that glitches of that sort may be rectified with post-production animation. Another issue was the fact that we could not capture movement in detail – for example hands and feet. While there was no need to capture facial expressions given that historically pantomime dancers normally wore masks, there was absolutely no possibility to capture the movement of fingers or hands, or small, subtle movement. That put some constraints on how the dancers would perform.
The first question asked by the participating dancers was whether they should modify their performances in any way to compensate for these technological constraints. Many movements in the “Thetis” piece, for example, took place on the ground; but crouching, kneeling, rolling, or reclining brought the dancer perilously close to the plane below which the cameras could not reliably track. We instructed them not to make any initial modifications, but to wait until after reviewing the footage in order to identify problem areas. Feet proved troublesome, placing much of the footwork, the essential pedēmata of orchēsis, out of range, creating glitches in which they popped up reversed at the ankles or vanished altogether from the figure on the interface. Although our dancers typically performed their pieces barefoot, we found that wearing shoes under the suit’s felt slippers tightened the fabric and hence improved the consistency of capture. Another common issue was the concealment of bent limbs, particularly in some of these floor-level positions. Rapid or complex sequences were easier to track if executed in the centre of the performance area, where more of the cameras could track each marker. We added more markers, taking the total suit count up to the maximum of 50. It seems that for dance, and for this genre of dance in particular, more flexibility is needed in the options for positioning the markers on the body. The mask eliminates any need for capturing facial expressions, but the corollary is that a need for greater accuracy accrues to the hands, feet, and torso. (Click here to see an extract from one of the dances and the resulting avatar pre-animation).
Another of the issues we encountered had to do with veils, shawls and loose clothing. These costume items are integral to orchēsis [Wyles 2008] (figures 8–10) but could not be recorded in any possible way. We were advised to use small triangular “rigid body” markers attached to the edges of the cloth, and attached these rigid bodies onto a veil made of semi-transparent gauzy fabric. The rigid bodies we used had sensory properties but also made the thin veil the dancers wore significantly heavier. Moreover, they would create further problems for animators in post-production as they only marked the extremity of the cloth. The inability of the motion capture system to track the movement of the veil, however, presented a creative opportunity as well as a problem. Even if the material object could not be tracked by the sensors, its presence could still be perceived in the quality of the dancers’ movements, as they had to adapt to the constraints the prop imposed on them.

3.

The use of masks in itself has a profound effect on the way the dancers move. When facial expressions are hidden behind a mask, the dancers can only rely on body language to create meaning through their movements. This also allows them to embody with more ease a wider range of characters or other entities such as landscapes or elemental substances, as their individual self is made less visible. The dancers are aware that the focus of the audience shifts from the face to the whole body, and that they have to adapt their choreography accordingly. In the absence of any expression on the face to read, the effect of very subtle movements, such as those of a shoulder, a finger, or an inclination of the head, is magnified, thereby acquiring the potential to become very powerful and highly significant. Consequently, each movement has to be very precise and neat, as the smallest imprecisions are more likely to catch the audience’s attention and thus to blur the dancer’s intention. Working with a veil (a pallium) equally pushes the dancers to make clearer and more precise propositions. It also forces them to make more ample movements when dancing with it.
Working with the avatar turned out to have similar effects on the dancers’ choreography to those triggered by the use of the mask and of the pallium, and indeed actually amplified them. On the one hand, the avatar appears to become an extension of the mask, as it conceals the dancers’ own identity one step further. It promotes dissociation between the dancer and his performance. The body is perceived to be a kind of raw material that can be shaped at will to produce the desired effects, and is thus less emotionally invested. On the other hand, working with an avatar also imposes constraints which resemble those created by the use of the pallium. It requires that the dancers focus more on the smoothness and details of their movement, and less on their levels of energy and emotions. The movements have to be bigger in order to translate on the screen. Consequently, the dancers have to rely more on their technical skills as well as on the visual cues provided by the avatar, and less on their inner feelings. This also applies to the dancers’ energy levels. How much energy they put into each movement no longer reflects their emotional state, but instead is largely dependent on how much of it is needed to convey the desired effect through the avatar.
The performer of orchēsis is already in possession or occupation of dual, even multiple body-images, and the introduction of the on-screen avatar further complicates this perceptual relationship. Firstly, the dancer is both subject and object, skilled manipulator of the instrument that is her body. She is both spectacle and impresario. Second, the notorious plasticity of the orchēstēs means that by altering factors such as bearing, rhythm, tension, and speed, this instrumental body becomes the vehicle for a multitude of characters.[12] The articulate body which consists of a visible, exterior manifestation of motion – the shared sign-system of pantomimic gesture – coexists with the subjective, kinaesthetic evolution of the moving body-schema as experienced from within (on which distinction in dance more generally, see [Cohen Bull 1997]).
In antiquity, orchēstes had no means of seeing the whole of this outer body except as reflected in the movement of a trainer or rival performer. Their sensory feedback was purely proprioceptive. In the studio, our collaborating dancers make use of the mirror; previous ADMD research has embedded the use of video footage in the development process of pieces for performance. The 3D feedback of the screen extends this strategy of separating the visible (exterior) from the kinaesthetic (interior) body. The screen interface was available to our participants throughout, and although they did not typically refer to it during the execution of their performances, it was used in much the same way as a mirror when preparing. This mirror-image, however, was distorted, producing an appreciable disconnect between the motion of the dancer and its on-screen realisation. (A very basic example: because the markers were on the backs of the hands, they remained some 5cm apart on the avatar when the dancer clapped; in order for the avatar to clap, the dancer’s hands must cross.)
This raises the matter of translation. This has been central to ADMD from the beginning, as we define the activity of participating dancers as a form of intermedial translation, or translation across media:[13] performances witnessed in antiquity have been translated into the written texts and static images which are our source-material, and these we then translate back into movement using the kinetic discourses of twenty-first-century dance training and practice. One primary challenge addressed by ADMD has been that of making comprehensible a language of gesture that both references the extant data on orchēsis and resonates for present-day spectators. In the project’s initial phase, it was sufficient that participants invented an idiolect, a private language accessible to other members of the group. Subsequent phases involved public performance and consequently demanded expansion of the gestural vocabulary as well as refinement of its utterance: elements had to be at once precise, repeatable, and recognisable. Instead of addressing human spectators, on the other hand, the Umeå participants were addressing their performances towards the impersonal and inorganic but no less partial and interpretive gaze of the motion-capture system. As when a performance is transferred from an intimate venue to a large stage – which may certainly have been the case for Roman orchēstes[14] gestures suited to the studio had to be adapted in scale and orientation if their effect was to be comparable.
Ironically, the space in which the dancers were working (3m x 3m, as mentioned above) had shrunk to a fraction of their rehearsal room. In a similar fashion to the multiplication of bodies, the introduction of a screen environment added another dimension to the already-complex spatial dynamics operational in orchēsis. As well as representing character, the orchēstēs is responsible for establishing his setting: the sea-cave where Thetis is assaulted, the mountainside where Pentheus is caught spying. Without scenery or backdrop, space is transformed imaginatively through the interaction of movement and libretto. The musical soundscape is vital, as it fills up the space like liquid with the mood which the orchēstēs crystallises and channels. The other fictional setting, however, simultaneously being defined by the movements of our participants was the on-screen grid. In one sense featureless, in that it consists of nothing but the geometric axes that contain the stick-figure or “skeleton”, this anonymous screen-space in fact makes a powerful intervention into the conceptual location of the dance. No longer constitutively identified with the live dancing body and its material context, the dance is transposed into an intermediary, purgatorial zone where it is perceptibly neither the actions performed by the present-day dancer opposite, nor those of an ancient orchēstēs. A participant may be animating (dancing the role of) Pentheus on Mt Cithaeron, but he is at the same time animating (dancing the role of) the skeleton in the grid.

4.

From Humlab’s perspective and that of Digital Humanities in general, the collaboration with ADMD offered new insights. There is no comprehensive research about how the collaboration between artists, technicians, and academics may approach and appropriate technology as a critical tool, how technology changes conditions for participation in artistic (re)creation, and what is the value of technology as a medium and a reflective tool for knowledge processes within academic settings. Through the process of capturing the movements of Roman pantomime, we attempted to give at least partial answer to these questions. We examined first and foremost how the entanglements between technology, scholarship and performance art enable new knowledge production in multiple dimensions. Technology acts as a lens that affects the way we conduct empirical research. It brings new conditions and demands for research.
While the team of researchers, technicians and dancers initially saw this as a non-teleological experiment, the final implementation deliverable was to create a Virtual Reality prototype of a pantomime dancing avatar within a Roman theatre setting. With the aid of Humlab-based 3D artist Mattis Lindmark a simple, neutral avatar was first placed within a test space without a specific theatre background: (http://cultumea.com/testzone/MocapHistoryWeb/index.html last accessed 27.09.2017). At a later stage, the team borrowed a Virtual Reality environment of the theatre at Pompeii.[15] (https://www.youtube.com/watch?v=gApcaxSGqh0&feature=youtu.be last accessed 27.09.2017 see also figure 7). The avatar used was a synthesis of disparate gender and age features: a conventional female face on a young/early teenager male body with an olive skin complexion and African hair. The reason for the combination of these aesthetic elements was to reflect that professional dance training started from a young age in antiquity (Libanius, Oration 64). The final virtual reality prototype is intended primarily for HTC VIVE headmounts, while there are currently discussions for its incorporation within higher education as a study tool for the intangible cultural heritage of the Roman empire.
There is an advocacy for a gradually stronger integration of technology within arts and humanities research, and that is manifested in academic infrastructure, recognized by scholars as “inevitable change”  [Burdick et al. 2012]. In spite of being a relatively young direction, still on a stage of self-determination and struggling to situate itself in a bigger context [Crane, Seales, and Terras 2009], the digitalization of arts and humanities infrastructures has become an increasingly popular topic in academic spheres. Digital and computational tools promise new relationships between individuals and the knowledge production processes, the collapse of the long-standing opposition between the natural sciences and the humanities as well as freeing the humanities from anxious attempts to advocate for their worth and relevance in contemporary society [Nygren, Foka, and Buckland 2014] [Foka 2016]. We further argue in this article that immersion dictates that movement, auditory components and other forms of expression enable an interactive appreciation of materiality beyond the visual, thus enabling a simulation of what UNESCO defines as “Intangible Cultural Heritage”. Typically intended for education or for cultural heritage purposes, virtual worlds are gradually becoming popularized beyond the cultural or the creative heritage industry and have become tools for research, education and dissemination of the past (on mixed reality and CH see [Kolsouzoglou and Veneris 2006]). The complexity of building digital immersive visualizations, as well as the power of experiencing the past emotionally and physically ought to be addressed.
This discourse, however, is not unproblematic. First, although recent scholarship reflects the notion that humanistic study should correspond to the contemporary reality of the human experience, there are no relevant discussions for performance arts in relationship to its gradual reorientation toward technology. Second, there are no comprehensive studies about how new technological competences may critically relate historical performances to our current post-human, technologically intertwined condition. As a consequence of this digitization of disciplines that are (incorrectly) considered impractical and valueless, new directions have arisen: the importance of practical engagement, hands-on work, thinking-through-making alongside developments in digital software and hardware [Burdick et al. 2012] [Gold 2012]. A third issue that arises is that these directions remain equally under-studied and elusive. Against this backdrop, digital infrastructure for academia, recently reiterated as the material presence of technology that may bring praxis and theory together, is only discussed at the level of potentials of settings [Smithies 2014] as a core component for new, digital forms of knowledge production. The intersections of art with traditional humanistic research and the role of technology as infrastructure have only been studied in their theoretical potential, and not necessarily addressing organizational tensions related to the installation and use of digital research infrastructures [Foka et al. 2017].
While for ADMD technology as a medium facilitated interdisciplinary research and brokered collaboration between the artists, academics and technicians, there are some further, practical observations that need to be made. One issue was that the system required a lot of extra calibration and time to test cameras, sensor points and interface response to begin with. In the reality of academic experiments as such that require specific time and resources, it is good, in the future, to allow extra time so as to optimize the empirical part of the process. One striking example was the recalibration early in the workshop that lasted for a total of four hours. The further, practical engagement of scholars and practitioners benefitted greatly from keeping a diary of events, achievements, and problems.
In order to optimize the process of motion capture to record movement more accurately, there is not much to be done at the moment beyond post-production development of the captured avatars. We suggest that developers of motion capture may in the future consider how to ameliorate the issues identified above: glitches between sensor points and interface response, and the lack of ability to record items of clothing or motion intensity. Although early and preliminary, our very engagement with artists, text, motion, and captured image gave us new insights into both technological and humanistic inquiry. Humanities laboratories should ideally facilitate empirical engagements with text, accessed through the full range of sensory modalities, and slowly open up to recreating immersive portals to conduct humanistic research through crafting, experiencing and reflecting. Digitalized infrastructures for Arts and Humanities gain meaning and become innovative through equality of the socio-technical components and strong collaborative efforts between technology, art, and the humanities.
This collaboration between practitioners, scholars, technicians, and indeed with the technology itself can be defined as an instance of what we term here distributed reception. According to theories of distributed cognition more generally, cognitive (intellectual) activity takes place not exclusively within the human brain but rather within a more extensive, networked field of interactions incorporating both biological matter and inanimate objects (e.g. [Clark 2008 (1998)], [Damasio 1999], [Rowlands 2010], [Shapiro 2004], [Noë 2004], [Bennett 2010], [Sheets-Johnstone 2011], [Gallagher 2005]). In this instance, we relocated the cognitive act of processing movement from the locus of a human being to a (distributed) technological recipient. In translating ancient dance for a post-human world, for a cultural context in which digital media have become integral to everyday activities, the human actor-interpreter may not necessarily remain the sole target audience of ancient texts. Dance, it could be argued, is so intrinsically embodied an activity that it occurs only within the closed circuit of the human body; but we feel on the contrary that the technological can be incorporated productively into the terpsichoreal.
Performances, we found, had to be modified in order to convert the electrical firing of muscular innervation into the electrical signals of the motion-capture system and transmit a version of orchēsis meaningful to our many-eyed nonhuman spectator. A feedback loop was created as the dancers’ movements were informed in turn by the reactions of the figure on the screen. Subtle movements, for example, had to be rendered in a more explicit fashion if the system was to register them; hands must overlap to clap; a hidden foot glitches. The dancer dances their avatar like a marionette, but the marionette’s responses also affect the dancer. Agency was thus likewise distributed (as in [Bennett 2010]). Further modification and refinement of the performance could of course be deferred to post-production, but another option is to develop effective translation techniques for meeting the medium halfway and adapting this dance form to make it comprehensible not just to human viewers but also to the alternative sensory faculties of a machine. Dispensing with the unattainable goal of ever more isomorphic capture, we might instead exploit the given conditions of this unique partnership, creatively accommodating the demands of this obedient yet exacting collaborator.
In doing so, the dancer’s relationship to the antiquity s/he represents becomes manifest. When the dance is live, it is possible to elide the present-day performer with the absent ancient dancer, to mistake the translation for the source-text and to forget its duplicity, or (less pejoratively) its doubleness; but when, on screen, the skeleton simultaneously appears in all its inhuman smoothness and misapprehended disjointedness, we are reminded of how these performances are inescapably haunted by their predecessors. If and when the avatar appears in its fully reanimated Graeco-Roman guise, the effect will be even more pronounced. The figure in the grid, the imagined orchēstēs, comes into being only as the ghostly body-double of the dancer on the floor: visible as an external analogue of the kinaesthetic metamorphosis experienced within, its separateness supplies an imagined point of origin, a parallel text created in the act of reception.

Figures

Figure 1. 
The capture area.
Figure 2. 
A dancer experiments with the suit, while watching her 3D image on screen.
Figure 3. 
The “skeleton” view.
Figure 4. 
A more acrobatic version of the “skeleton” view.
Figure 5. 
The “marionette” view.
Figure 6. 
The “marionette” view in action.
Figure 7. 
The Virtual Reality environment and avatar in 2D.
Figure 8. 
Dancers rehearsing with mask and pallium (veil / cape).
Figure 9. 
Dancer rehearsing with mask and pallium (veil / cape).
Figure 10. 
Dancers rehearsing with mask and pallium (veil / cape).

Appendix: texts and translations

Est sinus Haemoniae curvos falcatus in arcus:
bracchia procurrunt, ubi, si foret altior unda,
portus erat (summis inductum est aequor harenis);
litus habet solidum, quod nec vestigia servet
nec remoretur iter, nec opertum pendeat alga.
Myrtea silva subest bicoloribus obsita bacis
et specus in medio (natura factus an arte,
ambiguum, magis arte tamen), quo saepe venire
frenato delphine sedens, Theti, nuda solebas.
Illic te Peleus, ut somno vincta iacebas,
occupat: et quoniam precibus temptata repugnas,
vim parat, innectens ambobus colla lacertis.
Quod nisi venisses variatis saepe figuris
ad solitas artes, auso foret ille potitus.
Sed modo tu volucris (volucrem tamen ille tenebat),
nunc gravis arbor eras (haerebat in arbore Peleus),
tertia forma fuit maculosae tigridis: illa
territus Aeacides a corpore bracchia solvit.
Isque deos pelagi vino super aequora fuso
et pecoris fibris et fumo turis adorat,
donec Carpathius medio de gurgite vates
"Aeacide," dixit, "thalamis potiere petitis!
Tu modo, cum rigido sopita quiescit in antro,
ignaram laqueis vincloque innecte tenaci.
Nec te decipiat centum mentita figuras,
sed preme, quidquid erit, dum, quod fuit ante, reformet!"
Dixerat haec Proteus et condidit aequore vultum
admisitque suos in verba novissima fluctus.
Pronus erat Titan inclinatoque tenebat
Hesperium temone fretum, cum pulchra relecto
Nereis ingreditur consueta cubilia saxo.
Vix bene virgineos Peleus invaserat artus,
illa novat formas, donec sua membra teneri
sentit et in partes diversas bracchia tendi.
Tum demum ingemuit, "neque" ait "sine numine vincis"
exhibita estque Thetis. Confessam amplectitur heros
et potitur votis ingentique implet Achille.
There is a bay in Haemonia, curved like a sickle:
it extends its arms, where, if the sea were deeper,
there would be a port (the water lies flat over sand-banks);
It has a solid shore, which is neither marked by tracks
nor confined by a road, nor covered by dangling seaweed.
There is a myrtle-wood beneath mottled ivy
and in the middle, a cave (whether made by art or nature
is uncertain – more likely art), where often,
seated on your bridled dolphin, Thetis, nude, you come.
Here Peleus, as you lay overcome by sleep,
assailed you: and when you tried to resist him by pleading,
he became violent, and wrapped both arms around your neck.
And if you had not assumed many different shapes
as is your accustomed art, his audacity would have succeeded.
But first you became a bird (he still held the bird)
then an imposing tree (Peleus hung onto a branch).
Your third form was a striped tiger: and that one
made the terrified son of Aeacus loose his arms from your body.
He addressed the gods of the sea, pouring wine on the waves
and the innards of sacrificial beasts and the smoke of incense,
until the Carpathian prophet rose from the depths.
"Son of Aeacus," he said, "You seek to be able to bed her!
Well now, when she lies quiet, asleep in her rocky cave,
tie her up while she’s unconscious and confine her in tight bonds.
And don’t be deceived by her hundred deceptive forms
but hang on, whatever she becomes, until she turns back to herself!"
So saying, Proteus sank his face into the water
and his final words were submerged in the flowing tide.
The Sun was setting, and turning his chariot,
set a course for the west, when the beautiful sea-nymph
returned and reclined on her accustomed rocky couch.
Immediately Peleus assaulted her virgin limbs.
She assumed new forms, until she felt her body restrained
and her arms stretched in different directions.
Then at last she groaned, "You have not won without divine help!"
and showed herself as Thetis, defeated. The hero embraced her,
achieved his desire, and fathered the invincible Achilles.
Table 1. 
Ovid, Metamorphoses 11.229-65
Perstat Echionides. Nec iam iubet ire, sed ipse
vadit, ubi electus facienda ad sacra Cithaeron
cantibus et clara bacchantum voce sonabat.
Ut fremit acer equus, cum bellicus aere canoro
signa dedit tubicen, pugnaeque adsumit amorem,
Penthea sic ictus longis ululatibus aether
movit, et audito clamore recanduit ira.
Monte fere medio est, cingentibus ultima silvis,
purus ab arboribus, spectabilis undique campus.
Hic oculis illum cernentem sacra profanis
prima videt, prima est insano concita cursu,
prima suum misso violavit Penthea thyrso
mater.
"Io, geminae" clamavit "adeste sorores!
ille aper, in nostris errat qui maximus agris,
ille mihi feriendus aper." Ruit omnis in unum
turba furens; cunctae coeunt trepidumque sequuntur,
iam trepidum, iam verba minus violenta loquentem,
iam se damnantem, iam se peccasse fatentem.
Saucius ille tamen "fer opem, matertera" dixit
"Autonoe! moveant animos Actaeonis umbrae."
Illa, quis Actaeon, nescit dextramque precantis
abstulit: Inoo lacerata est altera raptu.
Non habet infelix quae matri bracchia tendat,
trunca sed ostendens deiectis vulnera membris
"adspice, mater!" ait.
Visis ululavit Agaue
collaque iactavit movitque per aera crinem
avulsumque caput digitis complexa cruentis
clamat "io comites, opus haec victoria nostr’ est!"
Non citius frondes autumni frigore tactas
iamque male haerentes alta rapit arbore ventus,
quam sunt membra viri manibus direpta nefandis.
The son of Echion persisted. He no longer ordered others to go,
but went himself to Mt Cithaeron, where the followers of the god
performed their rites and raised their voices in Bacchic singing.
Like a war-horse foams at the mouth with desire for battle
when the war-trumpet sounds the advance with its sonorous bronze,
so Pentheus was struck by the ululations that made the air vibrate,
and as he heard their cries his rage burned hot again.
In the heart of the mountains, on a peak surrounded by forest
but clear of trees, visible from everywhere, is a field.
Here he came to lay profane eyes on the sacred rites.
The first to see him, the first to halt her wild running,
the first to strike Pentheus with her rod like a spear…
…was his mother.
"Hey, sisters," she shouted, "come here, my sisters!
This huge boar, that has been causing havoc in our fields -
I have to smash him!" They came running as one,
the furious crowd; they all came together and pursued him, terrified,
- oh, terrified now, now speaking words less violent,
now cursing himself and admitting that he had done wrong.
Wounded, he still said, "Oh, help me, Aunt
Autonoe! Be moved by the spirit of Actaeon."
She knew nothing of Actaeon, and as he held out his hand to her
she ripped it off: Ino got the other one.
The poor man had no arms left to extend to his mother,
but thrust his maimed torso at her, stripped of limbs.
"Look, Mother!" he said.
Agave howled exultantly at the sight
and tossed her head, and whipped her hair,
and tore off his head, gripping it in blooded fingers.
"Hey, girls," she cried, "This is a great victory for us!"
Autumn leaves, touched by the frost and unable to cling
to the tall tree, are not more quickly scattered by the wind
than the limbs of that man were dismembered by terrible hands.
Table 2. 
Ovid, Metamorphoses 3.699-729

Notes

[1]  See precisely previous work by [Schreibman, Siemens, and Unsworth 2004] and [Forte 2010]; for 3D visualizations as emerging from popular culture see [Foka 2017].
[2]  Many universities have programs that research and develop immersive technology. Examples are Stanford's Virtual Human Interaction Lab, USC's Computer Graphics and Immersive Technologies Lab, Iowa State Virtual Reality Applications Center, University of Buffalo's VR Lab, and Teesside University's Intelligent Virtual Environments Lab, Humlab at Umeå University and http://immersiveeducation.org.
[3]  With the exception of Bozia “Reviving Classical Drama: Virtual Reality and Experiential Learning in a Traditional Classroom” in this special issue.
[4]  See e.g. epigraphic inscriptions such as IG XIV 2342 (= GV 675); epigrams such as Greek Anthology 16.284, 16.286 & 16.287; Apuleius Metamorphoses 10.29. For discussion of the evidence for female pantomime artists, see [Starks 2008].
[5]  pace [Hall 2008]. In the literature these are referred to as fabulae salticae. Famous poets such as Lucan and Statius are recorded as having composed them, but they do not survive.
[6]  Greek plays were also adapted for pantomime: see e.g. TrGF 1, p344 ad 14a; Suetonius, Gaius 57.9; Greek Anthology 16.289.
[7]  Lucian, On the Dance 19.
[8]  Another inclusion in the canon of pantomime subjects is Pythagorean philosophy, with its principle of reincarnation (Lucian, On the Dance 70; cf. Athanaeus Deipnosophistai 629d-e), which is also treated in Ovid’s Metamorphoses (Book 15).
[9]  See [Foka 2018].
[10]  See [McMillan 2002, 271–91] and [Dixon 2007] especially in relation to the study of interactive platforms for the study of performance arts, [Kilteni, Groten, and Slater 2012, 373–87] regarding VR.
[11]  For interactivity and interactive media see [Manovich 2001].
[12]  Lucian, On the Dance 19.
[13]  Influential in this respect has been the work of [Scott 2012].
[14]  On the range of performance venues used for pantomime, see [Webb 2008] and [Garelli 2007].
[15]  Credit: Jeffrey Jacobson, Public VR.

Works Cited

Astic 2011 Astic, I., et al.: A Ubiquitous Mobile Edutainment Application for Learning Science through Play. In: Museums and the Web 2011, Philadelphia, PA, USA (2011)
Barker et al. 2010 Barker, E., Bouzarovski, S.; Pelling, C. and Isaksen, L. 2010 “Mapping an ancient historian in a digital age: the Herodotus Encoded Space-Text-Image Archive (HESTIA)” Leeds International Classical Studies, 9.
Barker et al. 2012 Barker, E. Bissell, C., Hardwick, L., Jones, A., Ridge, M. and Wolffe, J. 2012. Digital technologies: help or hindrance for the humanities? (2012-02) Arts and Humanities in Higher Education, 11(1-2) (pp. 185-200).
Bennett 2010 Bennett, J. 2010. Vibrant Matter: a political ecology of things. Durham: Duke University Press.
Betts 2017 Betts, E. (ed.) 2017. Senses of the Empire: multisensory approaches to Roman culture. Farnham: Ashgate.
Burdick et al. 2012 Burdick, A., Drucker, J., Luenfeld, P., Presner, T., & Schnapp, J. (2012). Digital_Humanities. New York: MIT Press.
Butler and Purves 2013 Butler, S. & Purves, A. 2013. Synaesthesia and the Ancient Senses. Durham: Acumen.
Chapman, Foka, and Westin 2016 Chapman, A. Foka, A., Westin, J. (2016) “What Is Historical Game Studies?” Rethinking History vol.20:4, Taylor and Francis. 
Clark 2008 (1998) Clark, A. & Chalmers, D. (2008) [1998] “The Extended Mind” in Clark, Supersizing the Mind: embodiment, action, and cognitive extension, Oxford: Oxford University Press.
Classen 1997 Classen, C. 1997. “Foundations for an Anthropology of the Senses”. International Social Science Journal 49 (153): 401–12.
Cohen Bull 1997 Cohen Bull, C. 1997. “Sense, meaning and perception in three dance cultures” in Desmond (ed.) Meaning in Motion: new cultural studies of dance. Durham: Duke University Press.
Crane, Seales, and Terras 2009 Crane, G., Seales B., and Terras, M. (2009). “Changing the center of gravity: Transforming classical studies through cyberinfrastructure”. Digital Humanities Quarterly, 3(1).
Damasio 1999 Damasio, A. 1999. The feeling of what happens: body and emotion in the making of consciousness. New York: Harcourt Brace.
Dixon 2007 Dixon, S. 2007. Digital performance: a history of new media in theater, dance, performance art, and installation, Cambridge, Massachussets: MIT Press.
Drucker 2011 Drucker, J. 2011. “Humanities Approaches to Graphical Display”. Digital Humanities Quarterly 5:1.
Evens 2005 Evens, A. 2005. Sound Ideas: Music, Machines, and Experience (Minneapolis, MN: University of Minnesota Press, 2005).
Foka 2016 Foka, A. & Arvidsson, V. 2016. “Experiential Analogies: A Sonic Digital Ekphrasis as a Digital Humanities Project”. Digital Humanities Quarterly 10.2.
Foka 2017 Foka A. &Gelfgren, S. 2017, “Visualisering som verktyg och metod för historieforskning”. Stockholm: Daedalus.
Foka 2018 Foka, A. (2018) “The Digital Aesthetic in Atlantis: the Evidence” in Hobden, F. and Wringley, A. Ancient Greece on British Television, University of Edinburgh Press. 187-202.
Foka et al. 2017 Foka, A., Misharina, A., Arvidsson A., and Gelfgren, S. (2017) “Digitalized Infrastructures: Towards a viable socio-spatial model for Arts and Humanities”. Digital Scholarship in the Humanities: Oxford University Press.
Forte 2010 Forte, M. (ed) 2010. Cyber Archaeology. BAR International Series 2177, Oxford: Archaeopress.
Frischer et al. 2006 Frischer, B., Abernathy, D., Giuliani, F. C., Scott, R. T., and Ziemssen, H. 2006. “A New Digital Model of the Roman Forum”. In Imaging Ancient Rome edited by Haselberger, L. and Humphrey, J., Portsmouth, RI: Journal of Roman Archaeology, 163-182.
Gallagher 2005 Gallagher, S. 2005. How the body shapes the mind. Oxford: Clarendon
Gallese and Lakoff 2005 Gallese, V. & Lakoff, G. 2005. “The brain’s concepts: the role of the sensory-motor system in conceptual knowledge”. Cognitive Neuropsychology 22.3-4, 455-79.
Garelli 2007 Garelli, M-H., 2007. Danser le Mythe: la Pantomime et sa Réception dans la Culture Antique. Louvain: Peeters.
Giaccardi et al. 2012 Giaccardi, G. et al 2012. Digital Heritage and Social Media. New York: Routledge.
Gold 2012 Gold, M. K. (ed.), Debates in the Digital Humanities (2012). Minneapolis: University of Minnesota Press: 490–509.
Hall 2008 Hall, E., 2008. Is the “Barcelona Alcestis” a Latin pantomime libretto? In: E. Hall and R. Wyles, eds. New Directions in Ancient Pantomime. Oxford: Oxford University Press. 258-82.
Hall and Wyles 2008 Hall, E. and Wyles, R., eds. 2008. New Directions in Ancient Pantomime. Oxford: Oxford University Press.
Howes 2005 Howes, D. 2005. Empire of the Senses: the Sensual Culture Reader. Oxford: Berg.
Ingleheart 2008 Ingleheart, J., 2008. "Et mea sunt populo saltata poemata saepe" (Tristia 2.519): Ovid and the pantomime. In: E. Hall and R. Wyles, eds. New Directions in Ancient Pantomime, Oxford: Oxford University Press. 198-217.
Kilteni, Groten, and Slater 2012 Kilteni, K., Groten, R. And Slater, M. 2012 “The Sense of Embodiment in Virtual Reality” MIT Presence. vol. 21, No. 4, Pages 373-387.
Kolsouzoglou and Veneris 2006 Kolsouzoglou, A., Veneris, Y. 2006 Studying Antikythera Mechanism in Mixed Reality. In: International Conference on Decoding the Antikythera Mechanism - Science and Technology in Ancient Greece.
Lada-Richards 2007 Lada-Richards, I. 2007. Silent Eloquence: Lucian and Pantomime Dancing. London: Duckworth.
Lada-Richards 2013 Lada-Richards, I. 2013. “Mutata corpora: Ovid’s changing forms and the metamorphic bodies of pantomime dancing.” Transactions of the American Philological Association 143.1.105−52.
Macintosh 2010 Macintosh, F., ed., 2010. The Ancient Dancer in the Modern World: Responses to Greek and Roman dance, Oxford: Oxford University Press.
Mahony and Bodard 2010 Mahony, S. and Bodard, G. 2010. “Introduction”. In “Digital Research in the Study of Classical Antiquity”, edited by Mahony, S. and Bodard, G., Surrey Ashgate: 1-14.
Manovich 2001 Manovich, L. (2001). The Language of New Media, MIT Press.
McMillan 2002  McMillan, S. 2002 “Some cyber-places are more interactive than others : A four-part model of cyber-interactivity”, New Media and Society. Vol 4(2): 271–291.
Noë 2004 Noë, A. 2004. Action in Perception. Cambridge, Mass.: M.I.T Press.
Nygren, Foka, and Buckland 2014 Nygren, T., Foka, A., and Buckland, P. 2014 “Digital History in Sweden”. H-Soz- Kult, Humbolt: Humbolt University Press.
Rowlands 2010 Rowlands, M. 2010. The New Science of the Mind: from extended mind to embodied phenomenology. Cambridge, Mass.: MIT Press.
Schreibman, Siemens, and Unsworth 2004 Schreibman, S., Siemens, R., Unsworth, J. 2004. A Companion to Digital Humanities. Oxford: Blackwell.
Scott 2012 Scott, C. 2012. Literary Translation and the Rediscovery of Reading. Cambridge: Cambridge University Press.
Shapiro 2004 Shapiro, L. 2004. The Mind Incarnate. Cambridge, Mass.: MIT Press.
Sheets-Johnstone 2011 Sheets-Johnstone, M. 2011. The primacy of movement (2nd edn.). Amsterdam: John Benjamins.
Smithies 2014 Smithies, J. 2014 “Digital Humanities, Postfoundationalism, Postindustrial Culture” DHQ Vol.8 n.1.
Starks 2008 Starks, J.H. 2008. “Pantomime actresses in Latin inscriptions” in Hall & Wyles (eds.) New Directions in Ancient Pantomime. Oxford: Oxford University Press.
Toner 2014 Toner, J. 2014. A Cultural History of the Senses in Antiquity. London: Bloomsbury.
Vitale 2016  Vitale, V. 2016, “Transparent, Multivocal, Cross-disciplinary: The Use of Linked Open Data and a Community-developed RDF Ontology to Document and Enrich 3D Visualisation for Cultural Heritage.” In: Bodard, G & Romanello, M (eds.) Digital Classics Outside the Echo-Chamber: Teaching, Knowledge Exchange & Public Engagement. London: Ubiquity Press 147-68.
Webb 2008 Webb, R. 2008. Demons and Dancers: Performance in Late Antiquity. Cambridge, Massachussets: Harvard.
Westin 2012 Westin, J. 2012. Negotiating Culture, Assembling a Past: the Visual, the Non-Visual and the Voice of the Silent Actant. Gothenburg: University of Gothenburg Press.
Westin, Chapman, and Foka 2018 Westin, J., Chapman, A. and Foka, A. (2018) “Special issue on Technology and Cultural Heritage for Histories of the Oppressed,” International Journal of Heritage Studies: Routledge, 24, volume 3: 283–286.
Witmore 2006 Witmore, C. L. 2006. “Vision, Media Noise and the Percolation of Time: Symmetrical Approaches to the Mediation of the Material World”. Journal of Material Culture Vol. 11(3): 267–29.
Wyles 2008 Wyles, R. “The symbolism of costume in ancient pantomime” in Hall & Wyles (eds.) New Directions in Ancient Pantomime. Oxford: Oxford University Press.
Zanobi 2013 Zanobi, A. 2013. Seneca’s Tragedies and the Aesthetics of Pantomime. London: Bloomsbury.