DHQ: Digital Humanities Quarterly
Volume 7 Number 3
2013 7.3  |  XML |  Discuss ( Comments )

Visualizing Theatrical Text: From Watching the Script to the Simulated Environment for Theatre (SET)

Teresa M. Dobson  <teresa_dot_dobson_at_ubc_dot_ca>, University of British Columbia
Stan Ruecker  <sruecker_at_id_dot_iit_dot_edu>, Illinois Institute of Technology


The Simulated Environment for Theatre (SET) offers an alternative to static, two-dimensional visualizations of theatrical text by modeling the process of moving from text to performance in the Western, text-based tradition of live theatre production. Its interface links a three-dimensional ‘Stage View’ populated by character avatars to a ‘Text View’ enhanced with text visualization tools by means of a central ‘Line of Action’ analogous to a timeline. All three parts of the interface can be edited; edits in any individual section appear in the others; and all three can be played back simultaneously. While emphasizing that the traditional medium of theatrical performance is the actor’s body moving in time and space, and providing a digital analogue for that medium, the system also frees users from the limitations of the single, time-bound spectator’s perspective. Potential applications include theatre analysis, theatre pedagogy, and preparation for theatrical production.

Introduction: Visualizing Theatrical Text

This paper describes the design evolution of our Simulated Environment for Theatre (SET), a visualization system that conceives of theatrical text as a flexible process or technology [Worthen 2010] occupying time and three-dimensional space, rather than a fixed entity that can be captured in two-dimensional media. The impetus for the project was our curiosity about how digital media might enable uniquely theatrical uses of theatrical text; that is: we began with the question, "what do scholars and artists do with scripts, and can any of these things be done better in digital media than they can be done in print or manuscript"? We are less interested in solving a problem with existing uses of text than we are in extending the affordance of digital media. As Brett D. Hirsch [Hirsch 2011] has recently pointed out, digital editions of theatrical text ought to be able to do things that print editions cannot do; yet to date, most digital editions of plays (largely as a result of anxiety about their scholarly reception) adapt print conventions only very conservatively. To describe this trend in functional terms, existing digital editions of theatrical texts help readers to conduct textual scholarship: to understand, analyze, interpret, and trace the bibliographic and interpretive histories of printed texts; but we wondered, by contrast, how we might help users conduct theatrical creation and scholarship: to analyze a text for theatrical action; to conceptualize a production of a text; and to visualize the relationships between text and other spatial and temporal elements that function semiotically in theatrical performance.
It will be obvious that our understanding of "theatre" has been grounded throughout this project in the live, Western "dramatic" tradition, because the role of text in that tradition is so central. Reaching all the way back to Aristotle as our earliest point of reference, we think of this kind of theatrical text as a means of recording a central theatrical action, a major change in the world of a play, which is in process throughout a performance. This action begins at a play’s beginning and ends at its end; in performance, it is intelligible in the play’s plot, which is in turn expressed through the arrangement of material elements of production (bodies, objects, sound, light) in time and space. Aristotle’s tragedy – in performance – is hence a " mimesis of an action" [Aristotle 1997, 67]. We have found Canadian theatre director Richard Rose’s explicit application of Aristotelian analysis in production particularly helpful in elucidating the role of theatrical action in a theatre creation process [Rose 2009]. Rose describes a play’s "line of action" as its organizing principle, on the basis of which all the material manifestations of the performed work can be selected and deployed. He encourages his directing students to define a line of action in terms of a single verb that describes as precisely as possible the change in the play world effected by the action between its beginning and its end. On the basis of that concrete description of an overall change, directors can develop design and choreographic metaphors to materialize it, and can guide actors to choices that further its incremental progress from moment to moment in a production. Rose’s methodology has been especially useful in illustrating for us how the Western "dramatic" theatre both privileges text as a primary means of access to the organizing principle of a stable work of art and also positions it in performance as only one among a range of adaptable and manipulable semiotic elements of production. Among contemporary theorists of theatrical text and performance, Rose’s approach aligns most closely with that of William Worthen [Worthen 2010], who argues that this tension in the nature of a theatrical text is a consequence of its simultaneous identity both as tool for communicating a work of theatre and as a technology for creating it:

Tools always have an immediate purpose, are used to accomplish a specific task. Like tools, texts can be made to function in socially accredited ways, in illicit ways, and in ways that require a new technological adaptation, require us to rethink the technology and the work it accomplishes. You can use a screwdriver to drive a screw, or to open a paint can... Driving a screw, opening a paint can, ballasting a sculpture, stealing a car, killing someone: each use of the tool engages a different sociability, a different conduct of technology….the utility of dramatic writing lies in the perception of what kinds of activity, performance, doing something, those properties might afford in the social technology of the theatre. [Worthen 2010, 21–22]

Our aim, then, in designing the SET system, has been to enable the analysis and application of the technological function of theatrical text in the social context of the theatre, rather than to represent any individual theatrical text, material witness to it, interpretation of it, or historical perspective on it.
The approach we have taken to designing SET differs considerably from our first approach to theatrical text visualization, a prototype system called Watching the Script, which was designed primarily as a reading tool to enhance students’ understanding of play-texts. In what follows here, we explain the ways in which Watching the Script, despite its use of moving "character" dots on a "stage" seen in ground plan, nonetheless parallels four major print-based trends we have encountered in digital text visualization (we categorize these as print analogy, analytic visualization, literal illustration, and production illustration). Our departures from these trends in the new design for SET include the system’s navigable three-dimensional Stage View, which accommodates a variety of production elements linked directly to the central organizing principle of a Line of Action, visualized as a kind of time line. Text, which is linked to the Stage View through the Line of Action, is as flexible and penetrable as any other performance phenomenon in the Stage View. The interface also incorporates a set of specific sets and stages, consisting of three-dimensional models of real spaces created to scale, so that the adoption of the SET tool involves an awareness of an actual performance environment rather than a single stylized space. Users can move avatars around in the Stage View’s three-dimensional spaces and play text and action back in real time. While the new design has the potential to be useful to three constituencies of users – theatre researchers, theatre students, and theatre creators – our pilot user study has indicated that a more intuitive interface and a broader range of performance-related functions would enhance the system. In future development, we are particularly interested in exploring the system’s capacity to represent non-textual records of theatrical production, as well as less traditional theatrical works, including works that emphasize non-textual elements of performance, mediated performances, and other works that might be described as "post-dramatic" theatre [Lehmann 2006].

Watching the Script (WTS): a two-dimensional visualization

Our first prototype, Watching the Script, was conceived primarily as a reading tool. Its interface visualizes theatrical text in the three Views illustrated in Figure 1.
Figure 1. 
Our original text-centric design for Watching the Script showing a visualization of Shakespeare's Romeo and Juliet.
The "Overview" panel on the left side of the screen represents an entire theatrical text in a single microtext column [Ruecker, Homich and Sinclair 2005]. The Overview can be used to illustrate important structural patterns at a glance. For example, each character’s lines can be rendered in a different colour to illustrate the relative importance of characters in terms of line-count, frequency of appearance, or presence in different parts of a play, or to reveal patterns in the co-occurrence of characters that might suggest the development of relationships over the play as a whole. Other tools permit users to select and view patterns in scene locations and act and scene breaks.
The "Reading View" panel, second to left, shows portions of the text at a scale appropriate for reading; the context includes the moment currently visible on the primary "Stage View" on the right. The same visualization tools available in the Overview can also be applied to the Reading View, so that, for example, speech headings for a character's lines are rendered in the same colour used to represent that character in the Overview. Users are thereby offered visual cues to help situate the textual details of a particular passage in the context of the larger patterns of the play as a whole.
The "Stage View" separates the text into speeches and associates them with coloured dots representing the characters that deliver them. On a separate interface, users can define a movement for each character-dot around a stylized stage seen in ground-plan, as the text of that character's speeches scrolls; one movement is possible for each speech. By liberating speeches from their typographical sequence and associating them instead with mobile character dots, the Stage View focuses attention on even smaller units of text and emphasizes the relationship between text and speaker. Non-speaking characters remain on stage, but are grayed out in order to retain focus on the current speaker. In combination, the features of the Stage View create a visual argument [Galey and Ruecker 2009] that the speech – the coherent, discrete utterance of a single character, regardless of length – is the basic unit of a theatrical text. The Stage View provides a visual metaphor for the act of speaking in turns.
The premise that one discrete, continuous utterance of a single character (regardless of its length) is the basic unit of a theatrical work is fundamentally typographical in nature. It extrapolates the aesthetic structure of a theatrical text from its representation in print and manuscript reproductions: because the words on a page are separated by character names, the smallest unit of visual space is a complete utterance; because speeches are delineated by changes in names, speeches are ontologically defined by character; hence character utterances are the basic unit of a theatrical text. This two-dimensional approach to visualizing theatrical text as "the act of speaking in turns" is limited in obvious ways: even on its own terms, it cannot represent more complex verbal interactions between characters, such as overlapping dialogue or eavesdropping; but more importantly, it is not flexible or sensitive enough to performance conditions to engage the technological nature of theatrical text.

Watching the Script and traditional approaches to visualizing text

Most digital visualizations of text – theatrical and non-theatrical – are similarly concerned with the arrangement of information in configurations analogous to those found in familiar material media. We have encountered four major strategies or streams of approach.

Digital analogues for print editions

First, and most common, is the ubiquitous digital analogy for print or manuscript most obviously represented by documents prepared using word processing software. This approach imitates the visual appearance of print or manuscript: the default setting of a word-processor's window in the English-speaking world, for example, yields a two-dimensional image of black letters arranged sequentially in lines from left to right and top to bottom on a white rectangle; the relationship between words and background can be set to represent the relative proportions of various fonts and margins on various sizes of paper. Often the digital analogy for paper is enhanced with navigational tools (such as hyperlinks, search functions, and zoom features), and sometimes non-print media are embedded into paper-analogue frames. YouTube is an obvious example, in which windows containing time-based media appear in a two-dimensional, static white background populated with black text; Sophie, a software system for creating, "reading", and annotating multimedia digital "books" demonstrates a more sophisticated version of the same principle (it similarly uses a static, two dimensional background with designated areas for other kinds of media, including hyperlinking and collaborative authoring tools); Wysiwyg Web Builder provides a 'what-you-see-is-what-you-get' system for creating web pages that can also accommodate forms, PayPal, and RSS functions. But these features are enhancements to print format rather than violations of it, operating like traditional tables of contents, indexes, annotations, or figures. Digital editions of theatrical works such as the Internet Shakespeare Editions' annotated As You Like It [Bevington, ed. 2007] and the Canadian Adaptations of Shakespeare Project's multi-media Interactive Folio edition of Romeo and Juliet [Fischlin 2007] take advantage of opportunities like these to illustrate textual variants, provide historical context, or even present performance records from a range of productions. The Reading View of Watching the Script also operates in this way. It divides a text into passages of manageable length on a readable scale, minimally interfering with the integrity of the text as a whole in order to make it more accessible to readers, and in doing so maintains print's typographical visualization of characters speaking in turns.

Analytic visualization

A second approach to digital text visualization deconstructs and re-assembles texts to reveal patterns not visible within the confines of a single page or window and then communicates the emergent patterns metaphorically in the form of graphs, images, or concordances. This is essentially an analytic approach to text visualization, described by Brent Nelson [Nelson 2008] as the digital evolution of the disassembly and reassembly of material artifacts that has been historically typical of humanities research. An example from a theatre-related discipline is ScriptGeist, a film pre-production software system that extracts mentions of props, costumes, locations, and other material and logistical details from scripts and re-assembles them into lists for distribution to various production departments [McKie 2007]. More innovative digital visualizations communicate textual patterns in intuitive, challenging, and aesthetically pleasing ways. Mandala, for example, a rich prospect browser[1] for exploring text collections more intuitively than traditional interfaces can, allows users to upload texts (including theatrical texts), weight criteria for word-searches, and view results in circular patterns [Sinclair, et al. 2009]. But the analytic approach nonetheless maintains the analogy to print and manuscript formats. Like those formats, its essential paradigm is the arrangement of information in two dimensional space. It deconstructs the arrangement of words on a two dimensional page and re-arranges the same words into new patterns in similar two-dimensional spaces, patterns familiar from print and manuscript such as tables, figures, graphs, two-dimensional images, and so on. Although the goal of re-arranging the original text is to offer an alternative representation of the same content, emphasizing patterns not readily discernible but nonetheless present in the original text, the result of the process is an entirely new text, often non-narrative or even non-syntactical in nature.
Watching the Script takes an analytic approach to visualization in its Overview panel, its features for associating character and colour, and its features for highlighting or excerpting certain kinds of text such as all text belonging to one character. While there are clear benefits to this approach, the casualties to the intended technological function of a theatrical text are especially apparent in this last case: if the actor playing Romeo has only his own lines for the Capulet ball scene to study, he doesn’t know, for example, that he and the actor playing Juliet are together speaking a sonnet, an elevated poetic form which signals that this new romantic connection extends beyond the whimsical romantic intrigues of Romeo’s past (1.4.206-223). Analytic visualization reduces rather than enhances the intended functionality of a theatrical text as a theatre technology.

Literal illustration

A third common approach to digital visualization of both theatrical and non-theatrical texts participates in the long tradition of literalizing images described in or suggested by a text. Sometimes the elements illustrated are individual characters, objects, or events, as in Michael Lahanas' interpretive drawing of the shield of Achilles posted with a translation of Homer’s description of it from book 18 of the Iliad [Lahanas 2006]. Sometimes literal illustrations are symbolic or emblematic schemas synthesizing a variety of moments or ideas in a single two-dimensional frame, visual "confections", to use Edward Tufte’s term #tufte1997, such as the promotional posters for Baz Luhrmann's Romeo + Juliet [Bazmark Films 1996]. A third strategy selects and extracts a sequence of events or movements of characters or objects as described or implied by a text, and plots them through two-dimensional space over time, providing a kind of kinetic summary of a textual work. Homer’s Trojan Theatre, for example, allows users to watch icons representing ships and armies move through the of battles of the Trojan war on a simplified map almost in ground-plan [Clay and Kenan 2007]. Since they re-organize their originals, we might even think of these latter two forms of literal illustration as literal, illustrative analytics. Multi-media illustrations -- movie trailers, for example -- often combine literal illustrative techniques to create analytic, kinetic confections. Most software designed to assist in motion picture pre-visualization or storyboarding operates as illustration: images representing shooting locations defined in the equivalent of stage directions, and sometimes also the arrangement of characters against these backgrounds, are organized in visual sequences analogous to the traditional storyboards used to plan the visual progression of a film. One example of such software is Celtx [Celtx 2010]; FrameForge Previz Studio [Schafer 2010] is a more sophisticated version, incorporating 3-D images embedded in two-dimensional frames; and the movie-making simulation game Moviestorm [Moviestorm Ltd. 2012] operates on similar principles.
The Stage View in Watching the Script attempts a limited stylized literalization of textual images in its association of character dots with characters’ speeches. In supplanting the typographical arrangement of the text it represents, it, like other literal visualizations, distorts its source text at the same time as enriching and augmenting it. Magnificent though a drawing of Achilles' shield can be, it cannot capture the lines and letters that indicate the rhythms and other sound-patterns of Homer's verse and through them the connotative visceral and intellectual meanings essential to the work as a whole. Nor, taken out of context, can a drawing of the shield effect the sudden shift in a reader's imaginative experience from syntagmatic progress through the epic to paradigmatic expansion of the lyric moment in which the symbolic object is encountered. On the other hand, the visualization adds specificity and detail not necessarily included in the original, and helps a novice reader understand the detail that is included in it. Similarly, animated maps and movie trailers do not necessarily aim to create rich epic or theatrical experiences of the inexorable accumulation of horrors – visitations from gods, violations of the bodies of heroes, illicit sexual encounters, murders of cousins, concealment in tombs, resurrections from the dead; instead, their concerns are traffic patterns and release dates.
Any literal, illustrative approach to text visualization can represent a part or an analysis or an interpretation of a printed textual work, demonstrably faithful to something in the original work, whether that be its material appearance (type, page, layout) or its contents (word, commentary, image, event, pattern), or both. No method of literal illustration of a theatrical text can faithfully represent a work of theatre, however, neither its material appearance nor its contents. Theatrical text is clearly intended to lead to a visualization of some kind, a material manifestation of something suggested by the text, but not to a literal visualization of the world of the play. Rather, a theatrical text is intended to lead to another material manifestation of the work, a theatrical production in which characters embodied by living actors move around on a three-dimensional stage in real time and space before an audience. That visualization is in turn intended to suggest the world and action of the play, making the work of theatre imaginatively present in the minds of an audience. Since theatrical texts rarely literally describe stages, but instead describe the fictions that staging is in turn intended to describe, a literal illustration of a theatrical texts skips over its performativity altogether; it treats text not as a performance technology, but as another, non-technological form of literature.[2] To the extent that theatrical texts are intended for performance, they, unlike other kinds of texts, require the addition of representational elements other than those that can be supplied imaginatively by readers. Even a theatrical work whose primary mode of expression is verbal must necessarily incorporate elements other than text. Literal illustration alters theatrical text at the ontological level.

Production illustration

In response to this challenge, modern print and digital editions take a fourth approach to visualizing theatrical text, incorporating images of theatrical production rather than literal illustrations of the text. Jill L. Levenson's introduction to the recent Oxford edition of Romeo and Juliet [Shakespeare 2000], for example, includes eight illustrations of stage productions and two of films [Zeffirelli 1968], [Lurhmann 1996]. Other print series such as Cambridge's Shakespeare in Production interpolate notes on and images of stage production into the text itself (see [Shakespeare 2002]). The online Internet Shakespeare Editions provides a broader collection of records in a searchable database embedded in a separate section of the site; it functions somewhat like an appendix to the text [Prescott 2009]. A number of Peter S. Donaldson's projects, including the Shakespeare Electronic Archive, Shakespeare in Asia, Hamlet on the Ramparts, and his Cross-Media Annotation System (XMAS) explore new ways of linking and integrating performance records and texts [Donaldson 2011a], [Donaldson 2011b], [MIT Shakespeare Project]. By representing a variety of productions in relation to an individual text, these approaches to visualization implicitly acknowledge that no individual illustration of an individual production should be privileged. Accepting the premise that different productions of Romeo and Juliet can represent the same work, they provide a variety of alternative visualizations for comparison.
Nonetheless, no group of contrasting production records can be said to visualize the work any more than any individual production record does. Production records participate in the same conventions as other material media, and perhaps more importantly, by providing a set of alternative illustrations for the same single text, groups of contrasting production records privilege the printed text as the locus of the work. Even editions that include extended representations of alternate texts (see Levenson's inclusion of the entire first Quarto of Romeo and Juliet [Shakespeare 2000], for example, or the Internet Shakespeare Editions' inclusion of early variants of the plays [Best 2005]) treat texts as coherent entities derived from print sources. Production records, on the other hand, are fragmented excerpts, arguing that coherence lies outside or between those records rather than in any individual production that may be represented by one or another of them. Visualized in this way, the theatrical work is still, ultimately, conceived as a complete and stable textual entity.

Re-visualizing theatrical text

Watching the Script, our initial prototype for visualizing theatrical text, employs methods of visualization analogous to those used in familiar material media in its Stage View; consequently, it cannot visualize the technological nature of theatrical text. To create a digital visualization of theatrical text without misrepresenting its defining characteristic, we have attempted instead to visualize the process of moving from text to performance.


There are some precedents for a technological approach to visualizing theatrical text in digital media. Michael Best, Stewart Arneil, and Martin Holmes' Scenario [Best, Arneil and Holmes 2002] enables students to create blocking tableaux of scenes from Shakespeare's plays by placing characters in fixed positions on a two-dimensional perspective-drawing of a stage similar to the Globe theatre. Each frame can be annotated with explanatory notes. Larry Friedlander's TheatreGame [Friedlander 1991] allows students to make audio recordings of scenes, program synchronized stage action in a 2.5D graphical interface (a pseudo 3D or ¾ perspective such as that used commonly in first-person shooter computer games), and play back the virtual 'scene' created. Both systems acknowledge the spatial medium of theatrical performance by offering a visual analogue for three-dimensional space; Scenario invests more fully in the specificity of the performance space, whereas TheatreGame acknowledges time as a performance medium. Both also engage users in activities that imitate a theatre artist's evolving use of theatrical space through a rehearsal process, Scenario emphasizing the analytical and interpretive activities of a director in its annotation function, and TheatreGame emphasizing the actor's work by having students record audio performances of scenes. Credo Interactive's DanceForms system for choreographing dance implies a relationship between tools and technologies of performance by presenting a view of a choreographic record in the same interface as a view of avatars executing the choreography [Credo Interactive 2009]. However, none of these systems makes an explicit visual argument about the nature of the relationship between theatrical text and performance in its interface design.

The Simulated Environment for Theatre (SET)

Our new prototype adopts some of the same principles as these systems, but offers a different range of functions in response to our thinking about the nature of theatrical text. The Simulated Environment for Theatre (SET) is shown in Figure 2.
Figure 2. 
Our re-visualized three-dimensional Simulated Environment for Theatre (SET) emphasizes the line of action rather than the text, and situates the characters on scale models of actual venues and sets.
SET offers a centrally-placed visualization of a line of action in the form of a horizontal image analogous to a timeline. The Line of Action links a Reading View in the lower half of the screen to a Stage View consisting of a three-dimensional scale model of an actual venue and set populated by abstracted character avatars in the upper half of the screen. The user’s agency is emphasized in that users can upload, edit, and annotate all aspects of the Stage View, Line of Action, and Reading View, including the texts and stage and set models used, and the movements of "characters" on the stage. The Line of Action can be broken down by users into smaller units of action, ranging in size from acts to scenes to French scenes (delineated by the entrance or exit of a character) to speeches to individual "beats" of dramatic action. Users define divisions between units of action by associating them with words, stage directions, or intervals between words and stage directions in the Reading View, or with character movements in the Stage View. All three views can be edited in the main interface; edits made in one view affect the others; and all three views scroll simultaneously during playback. In other words, the system treats units of action, rather than individual characters' speeches, as the basic units of the works visualized.
In each individual view, additional features serve the new paradigm. The Reading View shifts primary ontological integrity to the line of action rather than the text. The text in the Reading View can be cut, augmented with stage business, expanded with textual additions, and annotated; but any and all of these alterations are linked to (and hence metaphorically justified by) actions in the continuous Line of Action. The Reading View also emphasizes the embodied nature of text by placing it next to a Character Panel, which visualizes the presence or absence of characters during the passage shown in the Text Panel, whether they are speaking or silent. Our revised Stage View attempts to visualize the materialization of a theatrical text in fuller and more concrete terms than our first model did. Instead of offering a map-like aerial view of hypothetical traffic-patterns, our new Stage View uses three-dimensional models of actual theatre spaces.[3] Models are populated by abstract, three-dimensional avatars representing five classes of beings: adults, children, babies, large animals, and small animals, as shown in Figure 3. Each has a "nose" to indicate the direction in which it is facing. The avatars are distinguished from one another by colours keyed to names in the Character Panel, and their user-defined movements are optionally visualized in colour-coded "paths" showing each character's progress and stopping-points [Ruecker, Grotkowski, et al. 2013]. Speeches also appear in the Stage View, in colour-coded bubbles, visually associating characters' bodies and voices. Alterations to the Stage View are, like alterations to the Reading View, linked to the Line of Action.
Figure 3. 
SET provides five classes of avatars with noses: adults, children, babies, large animals, and small animals.
An additional advantage of the new model is that it permits users to move through three-dimensional space to view the stage or the audience from a variety of perspectives, including actors' perspectives, as demonstrated in Figures 4, 5 and 6. So the system can represent the multiple perspectives of audience members in different parts of the theatre, a phenomenon that must be accounted for both in planning and in analyzing theatrical production. The Stage Map in the upper right corner can be used to define predetermined viewpoints, for ease of navigation.
Figure 4. 
SET permits users to view theatre spaces from any point in the audience. This is a view from house left. The viewing position is highlighted in yellow and red on the Stage Map at upper right. Alternative viewing positions are visible on the Stage Map in grey.
Figure 5. 
This is a view from house right.
Figure 6. 
SET permits users to adopt the perspectives of performers. Here, we see through the eyes of the actor playing Juliet.
Finally, the visual emphasis on any individual element of the interface is determined by the user, by adjusting the sizes and positions of panes, as shown in Figure 7.
Figure 7. 
With the Character Panel and Stage View minimized, and the tear-away option for text panels utilized, this user-defined configuration of the SET interface emphasizes text.

Next steps: usage cases, usability, and multiple modalities

During the first stage of development of the SET system, we conducted a pilot usability study with scholars as participants [Roberts-Smith 2010] and consulted informally with theatre creators about the system’s potential applications. In our next phase of development, we are exploring potential differences in the needs of the three constituencies of users we envision for the SET system – theatre researchers, theatre students, and theatre creators — and experimenting with new functions tailored to each group's needs. We are assessing new functions and design ideas in a full-scale user study with twenty participants who have advanced expertise in one or more of our three target fields. Our methodology combines recording participants’ interactions with the SET interface [Nielsen 2000] and conducting semi-structured interviews [Guha and Saraf 2005] that include questions about the potential for the tool to be an important resource for participants in future [Ruecker 2006b]. The findings of our study will be integrated with our ongoing participatory design process, which includes interactions between designers, programmers and domain experts who are themselves potential end users of the system [Gabriele 2010].

Usage cases and usability

There are four main areas of focus to this new phase of development. For theatre researchers, we have added an annotation function that permits users to transparently acknowledge and document their research sources, as illustrated in Figure 8.
Figure 8. 
A visualization of the University of Waterloo Drama program’s 2008 production of Shakespeare’s Julius Caesar, directed by Jennifer Roberts-Smith, annotated with technical drawings of the university’s Theatre of the Arts and an image of the maquette of William Chesney’s design for the production’s set.
Because our research-oriented usage case makes non-textual performance records explicit, we have come to think of it as a historiographical iteration of SET [Roberts-Smith 2013]. For students of theatre, by contrast, we are conscious of the need to embed pedagogical guidance into the design of the system, so that students’ experience as users is a learning activity in itself. In our first tentative exploration of a pedagogical iteration of SET, we have analyzed the ways in which our current design implicitly guides users through the process of answering a theatre-historical research question [Roberts-Smith and Harvey 2013], by first presenting users with an empty environment to which theatre-historical records must be added before the records can be interpreted in a visualization like the one shown in Figure 8 above. This step-by-step process is illustrated in Figures 9, 10, and 11.
Figure 9. 
The empty SET.
Figure 10. 
Empty SET and source: the technical drawings for the Theatre of the Arts.
Figure 11. 
SET, source, and visual interpretation of the source: our model of the Theatre of the Arts.
We hope to develop our pedagogical usage case into an iteration that can guide students through a process of theatrical text and performance analysis, as well as theatre-historical research. We are beginning to develop our third, creation-oriented usage case by re-imagining our avatars as sketching tools for theatre directors to use in preparation for rehearsal. This process is taking the form of an open-ended, iterative movement back and forth between an analysis of preparatory sketches in Richard Rose’s rehearsal scripts [Rose 2011] and the SET team’s collaborative generation and review of a range of potential approaches to visualizing the creative functions of Rose’s sketches. We have come to think of this design methodology as a "research discourse of alterity" [Gabriele and Ruecker 2013]; it is typical of our development process to date. Lastly, the fourth strand of our current exploration of use cases and usability addresses the more intuitive interface that was indicated as a priority by participants in our pilot user study. To that end, we have begun to explore the potential applications of game interface design to our system [DeSouza-Coehlo and Roberts-Smith 2012], and we are seeking responses to our initial re-conceptions of the interface, along with our approaches to research annotation, pedagogical organization, and creation-oriented avatar design, in our user study.

Multiple modalities and digital performance editions

In addition to these four main areas of focus, however, we are also exploring an interesting theoretical question that arose during the first stage of SET development, namely: can non-textual records of production be visualized as elements of a technological process, like theatrical texts, or are their modalities different enough to warrant a different approach to visualization? It is obvious, for example, that some records – such as a set design like William Chesney’s in Figure 8 — are intended to directly to lead to material manifestations that we might think of as literal illustrations of their contents. Since they are not necessarily used literally, however (for example, the set actually built for the University of Waterloo production of Julius Caesar made a significant change to the horizon line indicated in the set model), they may arguably become technological in the process of their application. Other production records – such as Richard Rose’s pre-rehearsal sketches – are designed to be technological, since they record the deployment of material production elements whose spatial and temporal semiotics can be analyzed for theatrical action [Rose 2011]. Still other records operate primarily as production illustrations, as do production photographs that record moments in a live performance event (though these need to be carefully distinguished from publicity photographs, which are often artificially set up during dedicated photo shoots). Lastly, a final category of production records seem to intend a hybrid of literal, technological, and illustrative/documentary functions. Our curiosity about this final category of record has led to some tentative experimentation inside the SET system with the well-known map appended to the only surviving manuscript copy of the medieval morality play, The Castle of Perseverance [Castle of Perseverance], shown in Figure 12. It records a set design, which apparently is not only intended for future literal materialization, but also describes possible alternative future materializations, as well as seeming to record how the set was built in a particular past performance. The map’s stage directions seem similarly intended for future technological application while simultaneously recording how the play has already been staged.
Figure 12. 
The Castle of Perseverance Map.
Figures 13 and 14 show our early experiments in visualizing the multiple modalities of this text in the SET system.
Figure 13. 
A visualization distinguishing between some of the Castle of Perseverance Map’s technological and literal functions while maintaining its integrity as a single research source in SET’s Stage View (2D for technological function; 3D for literal illustration).
Figure 14. 
A visualization distributing some of the different functional modalities of the Castle of Perseverance Map into different areas of the SET interface.
These initial, incomplete experiments have opened a new avenue of inquiry for us, namely: might it be possible to conceive of an entirely new approach to editing plays as digital performance editions? By performance edition, we mean a visualization that takes on the traditional responsibilities of communicating the transmission and interpretive histories of a text, as well as acknowledging its extant material witnesses, without sacrificing access to its technological functions. How might we acknowledge the multiple modalities of our research sources, their historical development, and their continuing present technological instability, all in the same virtual space?

Conclusion: scope and ambition

SET's new approach to visualizing theatre allows users to engage with the process of analyzing a theatrical text for action and materializing action through embodied movement in time and space. Each tool (text, space, character, and so on) is concrete and specific, but is also treated as a participant in the technology of the theatre through its relationships both to other tools and to the organizing principle governing the deployment of them in an individual work of theatre. That technology is further represented as active and processual, generated at all levels by the user through his or her selection and deployment of the tools of production. Design challenges now centre on integrating the theoretical premises and visual arguments made by the system with the needs and interests of users in three primary user-groups — theatre scholars, students, and creators — working in the live, Western, text-based tradition.
The current version of SET has some flexibility for representing works that fall outside this tradition. An Aristotelian understanding of theatre as dramatic action need not be limited to works whose "emphatic production element" — the element that bears proportionally the highest level of responsibility for generating theatrical action [Rose 2009] — is words. We might as easily have a prop-based tradition or a sound-based tradition as a text-based tradition of theatre, as long as these elements of production generate embodied action in space over time. (Samuel Beckett's Breath [Beckett 1974], which has no actors at all, is an example of a form such a prop- and sound-based tradition might take.) The verbal text in SET’s Reading View could hypothetically be replaced by a musical score or a choreographic record or a storyboard. Similarly, users could begin by moving visualizations of characters around in the Stage View and define a line of action based on these visual patterns. Since the system will accept any consistent three-dimensional model of a stage space and/or set, any individual line of action might also be viewed against two or more venues; any venue or set model might be viewed against a number of different texts. This flexibility offers the opportunity to analyze the impact of changes in the deployment of individual production elements upon our perception of a particular line of action, the impact of a change in technological application, in Worthen's terms, of textual and non-textual theatrical elements. SET also offers some freedoms from the hermetic linearity of Aristotelian theatre. Although its timeline provides a linear perspective, users can also move forwards and backwards in time, focus on individual moments or individual characters, visualize choreographic traffic-patterns over time, and access multiple audience and performer viewpoints, rather than being limited to the single, time-bound viewpoint of the live spectator.
Among the limitations we acknowledge in the current design of the system, is the fact that it does not represent works of theatre that take non-Aristotelian forms. Our singular Line of Action, for example, may not be adequate to represent the multiple, simultaneous actions in a medieval mystery cycle, or the cyclical action in a work like Beckett’s Play #beckett1970. Additionally, since the system takes virtual, three-dimensional space as an analogy for material space, it cannot account for performances that originate in virtual environments, nor can it accommodate multi-media performance elements such as video projections in its virtual sets. Furthermore, since those sets are treated as hermetic fictional worlds, the system is only relevant to works that do not participate in the "post-dramatic" [Lehmann 2006] interest in interactive performance environments, whether live, as in the work of such such artists as Improv Everywhere [Improv Everywhere]; mediated, as in Stern and Mateas’ "one-act interactive drama", Façade [Stern and Mateas 2005]; or in the hybrid forms of such theatrical marketing games as 42 Entertainment’s I Love Bees [42 Entertainment 2004] or Year Zero [42 Entertainment 2007]. Consequently, in addition to our short-term interests in developing usage cases, enhancing the system’s usability, and exploring the possibility of a digital performance edition, a longer-term ambition of the SET research team is to consider how – or even whether – constructions of theatricality in non-Aristotelian forms might be visualized in digital media.


We gratefully acknowledge the support of the Social Sciences and Humanities Research Council of Canada. Thanks also to our anonymous reviewers for their extremely helpful suggestions; to Peter S. Donaldson for directing our attention to some other digital systems for visualizing theatre; to Cassandra Silver for introducing us to 42 Entertainment; and to participants in the Implementing New Knowledge Environments 2009 Birds of a Feather Conference for their comments on an earlier draft of this paper.


[1]Stan Ruecker [Ruecker 2006a] defines rich-prospect browsing interfaces as "those in which some meaningful representation of every item in a collection is combined with tools for manipulating the display."
[2] There are exceptions, of course. 'Closet dramas', for example, are not intended for performance. Some theatrical texts are choreographic rather than verbal: Samuel Beckett's Act without words [Beckett 1958] is a canonical example by a playwright deeply engaged in language. George Bernard Shaw's stage directions notoriously define the appearance of his characters in terms unlikely to be reproducible except in the physiognomy of actors; see Man and Superman [Shaw 2000], for example.
[3]Some forms of non-traditional theatre are produced in vertical audience-performer orientations: in the Argentinian dance theatre company De La Guarda's 2010 touring show, for example, the audience stands in the centre of a large room while performers travel up the walls and across the ceiling; and some of Australian-based performance artist STELARC's early work involved hanging his own body from meat-hooks suspended from a ceiling or outdoor structure, with audience looking up from below (photographs are available on both artists' web sites) [STELARC 1994]. But these were nonetheless both embodied performances presented in relationship to architectural spaces of some kind.

Works Cited

42 Entertainment 2004 42 Entertainment (2004). I Love Bees [online]. Microsoft Corporation, Aug. 2004. Available from: http://ilovebees.com [Accessed 22 Feb. 2013].
42 Entertainment 2007 42 Entertainment (2007). Year Zero [online]. Nine Inch Nails, 12 Feb. 2007. Available from: www.42entertainment.com [Accessed 22 Feb. 2013].
Aristotle 1997 Aristotle, 1997. Aristotle’s Poetics. George Whalley, trans. John Baxter, ed. Montreal: McGill-Queen’s University Press.
Bazmark Films 1996 Bazmark Films and Twentieth Century Fox Film Corporation, 1996. Posters for Romeo + Juliet. In: Romeo + Juliet. Internet Movie Database [online]. Available from: http://www.imdb.com/title/tt0117509/posters [Accessed 11 January 2011].
Beckett 1958 Beckett, S., 1958. Act without words: a mime for one player. Endgame, a play. New York: Grove Press. 85-91.
Beckett 1970a Beckett, S., 1970. "Play." Cascando and other short dramatic pieces. New York: Grove Press. 43-63.
Beckett 1974 Beckett, 1974. Breath. First Love and Other Shorts. New York: Grove Press. 89.
Best 2005 Best, M., 2005. "About the Internet Shakespeare Editions." Internet Shakespeare Editions [online]. University of Victoria. Available from: http://internetshakespeare.uvic.ca [Accessed 13 July 2010].
Best, Arneil and Holmes 2002 Best, M., Arneil, S., and Holmes, M., 2002. Scenario. Half-Baked Software. University of Victoria Computer-Aided Language Learning Laboratory.
Bevington, ed. 2007 Bevington, D., ed., 2007. William Shakespeare. As You Like It (Modern). In: Michael Best, gen. ed. 2009. Internet Shakespeare Editions [online]. University of Victoria. Available from: http://internetshakespeare.uvic.ca [Accessed 13 July 2010].
Castle of Perseverance [The Castle of Perseverance Map.] The Castle of Perseverance. Ca. 1440. MS. Folger Shakespeare Library, Washington. Shelfmark V.a.354. 191v. Image ID 120742.
Celtx 2010 Celtx [online], 2010. Available from: http://celtx.com [Accessed 13 July 2010].
Clay and Kenan 2007 Clay, J. S. and Kenan, W. R., 2007. Homer's Trojan Theatre [online]. Institute for Advanced Technology in the Humanities, University of Virginia. Available from: http://www.homerstrojantheater.org/index.html [Accessed 13 July 2010].
Credo Interactive Credo Interactive, n.d. LifeForms. Credo Interactive [online]. Available from: http://charactermotion.com/index.html [Accessed 23 July 2010].
Credo Interactive 2009 Credo Interactive, 2009. DanceForms. Credo Interactive [online]. Available from: http://charactermotion.com/index.html [Accessed 23 July 2010].
DeSouza-Coehlo and Roberts-Smith 2012 DeSouza-Coelho, S., Roberts-Smith, J. et al. "Control Freaks: An Iterative Story of Designing a Scholarly Environment in 3D." Society for Digital Humanities/Société pur l'étude des médias interactifs, Congress, May 26-28, 2012.
Donaldson 2011a Donaldson, P. S., Shakespeare Performance in Asia [online]. Available from: [Accessed 11 January 2011].
Donaldson 2011b Donaldson. Cross-Media Annotation System [online]. Available from: [Accessed 11 January 2011].
Fischlin 2007 Fischlin, D., 2007. Interactive Folio. Canadian Adaptations of Shakespeare Project [online]. University of Guelph. Available from: http://www.canadianshakespeares.ca/folio/folio.html [Accessed 13 July 2010].
Friedlander 1991 Friedlander, L., 1991. "The Shakespeare Project: Experiments in Multimedia." In: George Landow and Paul Delany, eds. Hypermedia and Literary Studies. Cambridge: MIT Press. 257-71.
Gabriele 2010 Gabriele, S., et al. 2010. "From Surface to Space: Adding a Dimension to a Theatrical Visualization Interface." Society for Digital Humanities/Congress of the Humanities and Social Sciences. Montreal, May 2010.
Gabriele and Ruecker 2013 Gabriele, S. and Ruecker, S., forthcoming 2013. "The Emergent Discourse of Alterity in the Extended Collaborative Making of the Simulated Environment for Theatre." Designa 2013 International Conference on Design Research, Universidade da Beira Interior, Portugal, 21-22 November 2013.
Galey and Ruecker 2009 Galey, A. and Ruecker, S., 2009. "Design as a Hermeneutic Process: Thinking Through Making from Book History to Critical Design." Digital Humanities 2009 Conference, University of Maryland, June 22-25, 2009.
Guha and Saraf 2005 Guha, T. K. and Saraf, V., "OPAC usability: Assessment through verbal protocol." Electronic Library. 23.4 (2005): 463–473.
Hirsch 2011 Hirsch, Brett D., 2011. "The Kingdom Has Been Digitized: Electronic Editions of Renaissance Drama and the Long Shadows of Shakespeare and Print". Literature Compass [online] 8 (9): 568-591. Available from: http://onlinelibrary.wiley.com/doi/10.1111/lico.2011.8.issue-9/issuetoc [Accessed 27 September 2012].
Improv Everywhere  Improv Everywhere | We Cause Scenes [online]. Everywhere Productions, Inc., 5 Feb. 2013. Available from: http://improveverywhere.com [Accessed 22 Feb. 2013].
Lahanas 2006 Lahanas, M., 2006. The Shield of Achilles. Hellenica [online]. Available from: http://www.mlahanas.de/Greeks/Mythology/AchillesShield.html [Accessed 13 July 2010].
Lehmann 2006 Lehmann, H.-T., 2006. Postdramatic Theatre. Trans. Karen Jürs-Munby. London: Routledge.
Lurhmann 1996 Lurhmann, Baz. Romeo and Juliet. Starring: Kate Winslet, Leonardo DiCaprio. Twentieth Century Fox. Film. 1996.
MIT Shakespeare Project MIT Shakespeare Project [online]. Massachusetts Institute for Technology. Available from: http://web.mit.edu/shakspere/index.html [Accessed 23 July 2010].
McKie 2007 McKie, S., 2007. ScriptGeist [online]. Available from: http://www.scriptgeist.com [Accessed 13 July 2010].
Moviestorm Ltd. 2012 Moviestorm Ltd., 2012. "The Virtual Movie Studio." Moviestorm [online]. Moviestorm Ltd., 2012. Available from: http://www.moviestorm.co.uk/hub/home [Accessed 13 April 2012].
Nelson 2008 Nelson, B., 2008. "Introduction: Reassembling the Disassembled Book." CHWP (2008), rptd. Digital Studies/Le champ numérique 0.12 (2008) [online], 1. Available from: http://www.digitalstudies.org/ojs/index.php/digital_studies/issue/view/12 [accessed 25 February 2013].
Nielsen 2000 Nielsen, Jakob. 2000. Designing Web Usability: the practice of simplicity. Indianapolis, IN: New Riders.
Prescott 2009 Prescott, P., 2009. "Shakespeare in Performance." In: Michael Best, gen. ed. 2009. Internet Shakespeare Editions [online]. University of Victoria. Available from: http://internetshakespeare.uvic.ca [Accessed 21 July 2010].
Roberts-Smith 2010 Roberts-Smith, J., et al., 2010. "Literary Theory and Theatre Practice: Watching the Script and the Simulated Environment for Theatre" [poster]. Digital Humanities Conference 2010. King's College London, UK, 7-11 July.
Roberts-Smith 2013 Roberts-Smith, J., et al., forthcoming 2013. "Visualizing Performance History: Two Productions of Judith Thompson’s White Biting Dog in the Simulated Environment for Theatre " (SET). Special Issue: Digital Inquiry Into Textual Relations: Research and Teaching with Online Tools. Ed. S. Brown and S. Ruecker. Digital Studies/Le champ numérique.
Roberts-Smith and Harvey 2013 Roberts-Smith, J., Harvey, K. et al., forthcoming 2013. "Is There an Archivist in the Sim? Teaching Literacy Through Experience in a Post-Positivist, Mixed-Media Virtual Theatre Archive." Canadian Theatre Review 156.
Rose 2009 Rose, R., 2009. Directing Intensive [DVD]. Tarragon Theatre, Toronto, Canada, February-March 2009. Unpublished. Used with permission.
Rose 2011 Rose, R., dir., 2011. Director’s script with sketches for Forests. By Wajdi Mouawad. Tarragon Theatre, Toronto, 2011. Private collection of the artist.
Ruecker 2006a Ruecker, S., 2006a. "Experimental Interfaces Involving Visual Grouping During Browsing." Partnership: the Canadian Journal of Library and Information Practice and Research [online], 1 (1). Available from: http://www.criticalimprov.com/index.php/perj/article/view/142/177 [Accessed 11 January 2011].
Ruecker 2006b Ruecker, S., 2006b. "Proposing an Affordance Strength Model to Study New Interface Tools." Paper presented at the Digital Humanities 2006 conference, at the Sorbonne, Paris, France. July 5-9, 2006.
Ruecker, Grotkowski, et al. 2013 Ruecker, S., Grotkowski, A. et al., forthcoming 2013. "Abstraction and Realism in the Design of Avatars for the Simulated Environment for Theatre (SET)". Visual Communication 12.4 (November).
Ruecker, Homich and Sinclair 2005 Ruecker, S., Homich, E., and Sinclair, S., 2005. "Multi-level Document Visualization." Visible Language, 39 (1). 33-41.
STELARC 1994 STELARC [online], 1994. Available from: http://www.stelarc.va.com.au/images.html [Accessed 13 July 2010].
Schafer 2010 Schafer, K., 2010. FrameForge Previz Studio [online]. Innoventive Software. Available from: http://www.frameforge3d.com [Accessed 13 July 2010].
Shakespeare 2000 Shakespeare, W., 2000. Romeo and Juliet. J. L. Levenson, ed. Oxford: Oxford University Press.
Shakespeare 2002 Shakespeare, W. 2002. Romeo and Juliet. James N. Loehlin, ed. Shakespeare in Production. Cambridge: Cambridge University Press.
Shaw 2000 Shaw, B., 2000. Man and Superman. London: Penguin Books.
Sinclair, et al. 2009 Sinclair, S., et al., 2009. Mandala [online]. Humanities Visualization Project. Available from: http://www.mandala.humviz.org [Accessed 13 July 2010].
Sophie 2010 Sophie [online], 2010. Astea Solutions AD. Available from: http://www.sophiproject.org [Accessed 12 July 2010].
Stern and Mateas 2005 Stern, A., and Mateas, M., 2005. Façade [online]. Available from: http://www.interactivestory.net/ [Accessed 11 January 2011].
Tufte 1998 Tufte, E. R., 1997. Visual Explanations: Images and Quantities, Evidence and Narrative. Cheshire, CT: Graphics Press.
Worthen 2010 Worthen, W., 2010. Drama: Between Poetry and Performance. Malden, MA: Wiley-Blackwell.
Wysiwyg Web Builder Wysiwyg Web Builder 7 [online], 2010. Available from: http://www.wysiwygwebbuilder.com [Accessed 12 July 2010].
Zeffirelli 1968 Zeffirelli, F., director. Romeo and Juliet. Film. 1968.