It was a rainy fall night in October 2014. I finally sat down to play the game P.T. (7780s Studio, 2014), a survival horror game that games media had widely publicized in the past couple months.[2] I sat on my couch in the dark — perhaps not the wisest decision — and began to play. In the game I found myself in a dimly-lit hallway. I looked around at the different objects it contained, such as pictures, notes, a phone, and a radio, and paused for a while on a door eerily cracked open, until it suddenly slammed shut. I reached the end of the hallway eventually and opened the door there to find myself in the exact same hallway. This happened each time I opened that door, and each time I encountered copies of the hallway with changes ranging from the small (an object in a different place) to the major (suddenly the entire hallway was red with flashing and spinning lights). Finally, I got stuck: I seemed to be encountering the same version of the hallway over and over, and nothing seemed to change. I looked up guides and walkthroughs for the game to figure out what I was missing and learned that I needed to do something to get the in-game phone to ring, though no one seemed to know what that something was. So I started trying everything I could: interacting with every item, walking down the hallway backwards, and turning the radio on and off countless times. I even tried whispering into the PlayStation 4 microphone, per a Reddit suggestion, but still the phone didn’t ring. I’m still not sure what I did to trigger the phone-ringing event––perhaps the game just pitied me for taking so long to figure it out.

This brief narration of my experience with P.T. is an illustration of a larger point, namely that players have different experiences and thus different stories in the games they play. On the one hand, my story is unique. No one else has had my exact experience of playing P.T. in my apartment, navigating P.T.’s hallway, encountering the game’s events (some randomly generated), or getting stuck where I did. On the other hand, my story is quite common in that all players of P.T. have their own versions of it. While we all encountered different parts of the game at different times and in different ways, there are still many images, objects, events, and feelings that we all experienced sooner or later. This shared-but-varied phenomenon in game narrative is the result of player interaction with the game, and is both afforded and limited by what the game system (hardware, software, interfaces, etc.) and the players themselves make possible. Games achieve a sense of individuality through interaction by accommodating (intentionally or otherwise) a wide variety of play styles and allowing players to make different decisions in order to keep the game interesting [Atkins and Krzywinska 2007, 5]. Matthew Miller, one of the creators of the MMORPG City of Heroes, describes this dilemma: “We wanted players to have unique experiences as they played through the game, so that when a group of players got together, they could talk about [their exploits], and it wouldn’t be identical to everyone else in the room” [Miller 2009, 125]. Play demands the freedom and potential for difference, and as players navigate games they create their own unique (and yet similar) narratives.

Of course game studies scholars have long noted that the player’s ability to affect a narrative through gameplay is the defining characteristic of game narratives.[3] In this framework, game developers and theorists such as Chris Crawford have often discussed interactivity as something of a problem for game narrative and experience, even going so far as to say that there is an “apparent incompatibility between plot and interactivity” [Crawford 2013, 51]. The issue that Crawford and others have identified is that narratives need to have structure and coherence on some level, whereas interactivity and play are more about the flowing emergence of possibilities that often buck against the limitations of narrative’s structure. This tension, while not insurmountable, is why we seem to be less likely or willing to call a sequence of representations a narrative the more variable that sequence is. Thus when players discussing the same game relate drastically different experiences with the same content, such as puzzles and bosses solved or beaten differently; different paths taken or choices made; or even entirely different characters and plots, we’re less likely to call that content narrative. The reality of shifting and unstable content poses a difficulty for games scholars of all sorts (players are always doing unexpected things), but is particularly challenging for those attempting to locate central themes, meanings, and stories in a given game and play experience. How does one begin to account for all of these possibilities for variation? Or, to put it another way, how does one stabilize a game’s narrative enough to begin to make claims about it, without ignoring the emergent qualities of that narrative?

In response to these difficulties and questions, a number of scholars have proposed ludonarrative as a concept that unites storytelling with the emergent qualities of play. As Tison Pugh explains, ludonarrative is a “hybrid hermeneutic” that looks at the shared structures of games and narratives, including the intersections between them in video games and other media [Pugh 2019, 1]. The experience of narrative in video games is one where “you, the player (or reader, one could say), are suddenly thrown into someone else’s story and are expected to continue the tale,” and this experience reveals “the notion of the text as being informed by the process of play and as being characterised by multiplicity” [Mukherjee 2015, 1, 9]. In other words, video games demonstrate how narrative can be playful, variable, and divergent, and this can be true of other narrative texts such as novels, films, or comics that invite their readers or viewers to form their own interpretations and experiences. Within video games, ludonarrative approaches seek to understand the “whole” of a game by viewing gameplay, story, and the player in their many relationships with each other, rather than viewing narrative and play as inherently opposed, conflicting elements as previous scholarship often did [Toh 2017, 2].

While ludonarrative provides a framework for understanding the interrelated natures of narrative and play, there remain many questions about what forms ludonarrative can (or could) take and how players use them. Most scholars working with game narrative, including ludonarrative ones, have generalized the issue of divergent player experience by pointing to the possibilities for it, and perhaps exploring a few of those major possibilities in a given genre or game. At best, some scholars have utilized social scientific methods for player interviews, forums research, and community studies to understand how different players engage with and interpret a game.[4] These studies are extremely valuable in how they account for the player’s role in game narrative and experience, but they are also limited by sample sizes and what players remember or are consciously aware of in their gameplay. Such issues are largely unavoidable, and are not a significant detriment to the studies. However we have yet to develop a method that allows for the assessment of moment-to-moment experiences of a game and its story, including both what is common to different player’s playthroughs and what is different. In other words, while ludonarrative helps account for how narrative is interactive and variable in games and other media, we have only begun to assess the types and amounts of variance that are possible in particular game genres and individual games. One way to start doing this is to examine recordings of different players’ playthroughs of the same games, such as Let’s Play videos on YouTube and other streaming services. As Roth et al. note, these videos are “qualitative samples for the evaluation of the interactive narrative user experience” that have hereto been ignored in game narrative scholarship [Roth et al. 2018, 12].

This article provides a new method for using video recordings of gameplay to study similarity, difference, and variation in narrative caused by player interaction. It does so utilizing emerging digital humanities tools for distance visualization of visual media, such as the ImagePlot macro for ImageJ. ImagePlot is software that takes a collection of images and plots those images on a basic graph where the x and y axes correspond to different, quantifiable characteristics of the images. I will explain this process and its affordances and limitations in more depth shortly, but for now note that ImagePlot allows for the analysis of many types of visual media, including paintings, photographs, film, and of course video games. For the purpose of visualizing narrative variation in games, ImagePlot allows one to take a video recording of gameplay, extract the images from it, and plot those on a graph so an entire playthrough of a game is visible in one image (Figure 1). This distance visualization allows for storing, representing, and analyzing player experiences on a scale previously impossible: one can view a player’s entire playthrough of a game in one image, and then compare that playthrough with tens or hundreds of other playthroughs at a time. Further, rather than relying on player experience abstracted from actual play or reported from memory afterward, this distance visualization relies on video recording of play as it happened. By plotting the images extracted from the playthroughs of games by different players, one can begin to trace how variable a given game’s play experiences can be. A comparison of playthrough imageplots reveals the exact what and when of similarities or differences between a large number of player experiences with the same game. The imageplots also connect with exact measures of time and image qualities (such as brightness, hue, or even objects in the images), allowing the assessment of the magnitude of difference in a game, and providing some answers as to just how variable the variability of a game’s ludonarrative is. This article will focus specifically on variance in time (at what point players encounter something, and how long it takes them to play) and sign (what players encounter, the content of the images in the imageplot).

Developing these methods and answering the question of how variable a game’s ludonarrative is presents an opportunity to reconsider narrative form. Despite attempts to expand narrative in recent decades, such as the theories that space itself is narrative or that human perception of reality is dependent on narrative, the concept of narrative has remained limited to being a linear or at best multilinear form.[5] Even scholars of cognitive narrative processes such as David Herman and Marie-Laure Ryan, whose theories of narrative emphasize how it generates “possible worlds” with many potential outcomes (similar to the multiplicities of ludonarrative), have yet to account for how much difference and variation is possible in narrative experience, interpretation, and construction [Herman 2004]. It has remained difficult to assess just how much is possible in making possible worlds, or how far a narrative can change or stretch before it becomes either another narrative entirely or something unrecognizable as narrative. Yet if we can build on these theories of interactivity and possible difference by focusing more on specific lived experiences and instances of variation, we can arrive at an understanding of narrative form that is less focused on linear or multilinear outcomes and more attentive to the procedural and emergent construction of narrative similarities and differences. Such a ludonarrative understanding of narrative would emphasize the relationality of signs and open up room for their change and motion, rather than seeking to fix them in set times, places, and interpretations.[6] Shira Chess points toward this potential in her theorization of video game narratives as queer narratives that forego normative teleological structures of climax; instead games present narratives that are non-reproductive and continuous [Chess 2016, 84]. Imageplots further the conception of narrative as an ongoing, shifting, and possibly queer process by visualizing narrative as it unfolds and providing a means of assessing difference and continuity throughout the playing of a game.

In pushing us to consider narrative as an active and changeable form that requires play and emergence, the imageplots and the distance visualization they provide contribute a method to the renewed discourse on forms, perhaps best captured in Caroline Levine’s Forms. Levine defines form as “all shapes and configurations, all ordering principles, all patterns of repetition and difference,” and goes on to say that narrative is the form that best captures how different forms “collide”: “the strange encounter between two or more forms that sometimes reroutes intention and ideology” [Levine 2015, 3, 20]. The collisions in forms that Levine identifies are observable in the play of similarities and differences in player experiences that are captured in imageplots. As players encounter content in a game, their actions affect the form of that content — its duration, its arrangement, and its relation to other forms in the game. Imageplots capture the effects of this interactivity in the form of visual signs and suggest that narrative in games is best understood as a particular sequence or arrangement of moving and colliding signs. The signs that are common to all playthroughs of a game can be taken together as the primary or central narrative of the game: they are the core experience that causes the game to cohere as one game, one system of rules and possibilities for players to play with. The signs that are different from playthrough to playthrough indicate the places and parts of the game where the player can deviate from the central narrative and create their own unique experience of it. Taken together, the differences reveal how variable that narrative is, or how much play there is to it: more differences mean greater opportunity for play and variation, and fewer differences mean less.

The overarching goal of the distance analysis method proposed here is to locate patterns in the visual signs between playthroughs, and to thereby identify both the common and different meanings, experiences, and possibilities in a game. In terms of ludonarrative, this means being able to establish a central narrative to a game that is not limited to the game’s scripted elements, but can also accommodate player input and individual experience. This method can provide new insights on how players influence game narrative, and what degree of emergence comes with that influence. Furthermore, such a study of game narrative can contribute to moving game studies and narrative theory beyond accounting for interactivity and variation in abstraction, and gain a more solid grasp of narrative as a form that is flexible, emergent, and yet limited, always in a playful relationship with its audience. By doing so, we can better perceive the possibilities for difference that exist in current games, and how we might play within games and other visual media in order to produce new and transformative narratives. If we can see narrative forms in a given text, then we can better understand how those forms operate, how we can change them, and what we can imagine through them.

This opportunity to rethink visual narrative form is made possible by ImagePlot, a macro for the software ImageJ, which is usually used for the analysis of large quantities of images (such as brain scans) in medical science. ImageJ is developed and updated by the National Institutes of Health, and the software allows for a variety of plugins and macros for different types of image analysis. ImagePlot is one such macro, and was created by a team led by Lev Manovich as part of the Software Studies Initiative.[7] ImagePlot takes a collection of images and plots it on a graph according to preset data points. For example, the imageplots included here are plotted with a x-axis of time, and a randomly generated y-axis (Figure 2).[8] In effect, this means that the imageplots represent the temporal progression of images in the collection, or in other words they present the images in the order that they appear in the video. The progression from the left to the right side of the graph mirrors the progression of the video, with the left side of the imageplot being the beginning of the video and the right side its end, and each column on the graph contains images from the same moment in the video. This mapping of images onto a graph provides a distance visualization of a video, allowing one to change the scale of the data and see all of the video at once, instead of only seeing individual parts at a time. In this sense, ImagePlot is similar to many other digital humanities tools (such as those used for network analysis, topic modeling, etc.) that help us zoom out from a particular moment in a text or corpus and see larger trends across many moments and texts. ImagePlot allows us to do the same for visual media, and this method is only one of its many potential applications in the analysis of photos, paintings, film, and video games.

This project uses ImagePlot to visualize entire playthroughs of video games, such as P.T., the game I narrated earlier. The process does not require any coding knowledge, but rather involves using a series of files in conjunction with ImageJ. In order to run the ImagePlot macro, one first needs a collection of images and an associated file or spreadsheet with the filenames of the images and any other necessary data, such as a value for hue or saturation in the images. To generate these, a number of video files were downloaded from YouTube, where each video was a complete playthrough of a game recorded and posted online. Next, images were extracted from each of the videos, and organized in separate folders. A data spreadsheet for each video was populated with its corresponding image filenames, and each image was assigned a random y-value. Finally, ImagePlot was used in ImageJ with the collections of images and spreadsheets to plot the images on graphs. In sum, one could describe the process as downloading the videos, extracting the images, organizing the images, and then plotting the images.[9] This entire process is in every way mediated by software, and the end result is a particular form of data and knowledge quite different from simply watching a video of a playthrough on YouTube. ImagePlot and ImageJ thus fit into a larger technological trend that Manovich identifies as software creating new media, which he defines as “a combination of particular techniques for generation, editing and accessing content” [Manovich 2013, 335]. Imageplots are a new way of viewing and accessing existing media through distance visualization, and in this regard they are a form of new media. Yet they are also their own new form of content, making ImagePlot and ImageJ (the software that creates them) new media as well.

The imageplots included primarily come from one game — P.T. — in the survival horror genre, which tasks players with playing and surviving while being frightened by horror graphics and settings. I have limited my analysis to one game and one genre in order to isolate narrative variation coming from player interaction and experience, rather than coming from the differences between different games or game genres. While any game genre could work for this analysis, I selected the survival horror genre for its tendency to limit player actions in order to create a sense of fear. Survival horror games often take place in constrained spaces and supply few options for players in order to keep them on set paths and force confrontation — after all, monsters are not as scary if one can completely avoid them [Chien 2007, 64]. As such, survival horror games have hypothetically fewer variations between playthroughs, making those variations more consistent and noticeable. I selected P.T. in particular because it is a uniquely excellent case study for narrative variation due to its content and cultural context. P.T. is an abbreviation of Playable Teaser, and was originally intended, as its name suggests, as a teaser for a new Silent Hill game being directed by Hideo Kojima (perhaps most famous for the Metal Gear franchise). The game is relatively short, containing only about an hour’s worth of play time, and consists of repeatedly exploring the same hallway in a dark house with different encounters on each repeat. Thus repetition and variation are inherent to the game’s structure and narrative, and the game provides an exemplary space for using those concepts to measure player interactivity and differences between playthroughs.

The playthroughs of P.T. visualized in these imageplots are taken from various YouTube videos and Twitch streams and include playthroughs by popular streamers and regular players.[10] There are a large number of them available, mostly due to P.T.’s notoriety–after Konami cancelled the upcoming Silent Hill game and removed access to P.T. in April 2015, the game became an instant cult classic. Each playthrough plotted is a complete playing of P.T., in some cases accompanied by visuals associated with the YouTube channel at the beginning or end of the video. I have specifically excluded playthroughs that are heavily edited, as the additional visual elements added to those playthroughs introduce a bias toward similarity in the images. Heavy editing or removal of parts of the play experience also defeats the purpose of capturing and visualizing actual play, and instead creates its own entertainment media further removed from the game itself. Three of the playthroughs contain images of the player superimposed on the gameplay (as with many Twitch streams), but in the resulting imageplots the images from the player cameras are small enough to be unnoticeable without zooming in.

There are several limitations to this process worth mentioning before diving into analyzing the imageplots. The ability to produce imageplots of game playthroughs is limited by having (or not having) access to video recordings of the game in question. This means that because less popular games have fewer recorded playthroughs available on YouTube or Twitch, it is harder to produce many imageplots of them to compare. Even if one sets out to produce their own recordings, there are still constraints of time and hard drive space that make it difficult to create many playthroughs to use. An additional challenge is finding complete playthroughs of a game that are not cut into smaller sections on YouTube, though one can get around this by splicing the smaller videos together into complete playthroughs for plotting. The length of a game is also potentially a limiting factor, as complete playthroughs of many games are significantly longer than the average playthrough of P.T., and take more time and computing resources to plot.

In order for the use and comparison of imageplots to be meaningful, careful attention to the type and source of the playthroughs that generate them is required. On the most basic level this means identifying exactly what the plotting of images is revealing regarding the relationship between data points. To put it another way, ImagePlot is capable of generating very interesting imageplots that contain very little meaning and insight for the analysis of the object they are plotting. An imageplot of the brightness and hue of a game reveals very little about that game’s narrative structure, and likewise an imageplot of time yields only the most basic observations of color usage in a game. Thus one must ensure that what one plots with ImagePlot matches the questions they are asking. This project focuses on questions of narrative variation in games, so the imageplots included here all deal with time, signs, and narrative structure. I have chosen not to plot the games according to qualities like hue, saturation, or brightness because all of the playthroughs being plotted are of the same games, so plotting those qualities would yield similar (if not identical) results.

The imageplots that result from this process are distance visualizations of players’ playthroughs of a particular game, and as such they effectively provide an archive of player experiences in that game. When looking at an imageplot of a playthrough, one can see everything that a player did in that game from start to finish. This means one can see both the player’s choices and actions, and the game system and environments in which they took place. Imageplots are thus excellent examples of archives as Foucault defines them: as “the general system of the formation and transformation of statements” and “the system that governs the appearance of statements as unique events” [Foucault 2010, 129–30]. The archive as “system” is key to Foucault’s understanding here; it is the system that actually produces the objects it contains. In terms of the P.T. imageplots of this project, the archive is the system of the game and the player: together they fashion the player’s experiences, encounters, and interactions (their “statements”) with images while playing. Plotting these images creates distance visualizations that are objects of the archive: they are produced by the game system, and also record that system’s possibilities.

Conceptualizing the imageplots of P.T. playthroughs as archival objects helps clarify their relationship to the game and their significance for analysis, but it does not explain how they function. In order to understand how imageplots operate and capture play, one can consider them as diagrams, drawing on the idea of the diagram as explained by Deleuze in his reading of Foucault. Deleuze quotes Foucault in saying that a diagram is more than an image, it is “a functioning, abstracted from any obstacle,” “a map, a cartography,” “a machine that is almost blind and mute,” and “a spatio-temporal multiplicity” [Deleuze 2012, 34]. In essence, to Deleuze and Foucault a diagram is a frozen image that still contains motion and possibility in their purest, most ideal (and thus stillest) forms. Imageplots of game playthroughs are static images, yet they are images of play — they are snapshots of a system in operation, and evidence of that system’s potential. As such, they constitute diagrams of play. Each imageplot maps out a play experience with all of its movement and variation, yet the recording and plotting of images arrests and captures these same forces. The overall diagram of a given game is constantly changing according to the player, and yet the archive of the game imposes limits on this variation as well. Within this conceptual framework, this project is a construction of an archive of diagrams that contain the multiplicity of play possibilities, and that thus enable a tracing of the limits of interactivity and emergence in a game’s narrative and play.

ImagePlot provides a new methodology for assessing play experiences on a scale previously impossible. The imageplots of P.T. demonstrate how this method works, and what we can learn from it about narrative form.

While there are many types of variance between the playthroughs that emerge in the imageplots, the observable differences in narrative form between playthroughs fall into two broad categories. The first type of variability is variance in time, measured in both the duration of particular content in a game and in points in time when an individual player reaches a specific part of the game. The different times of the playthroughs reveal how their respective players engage with P.T., including how factors like skill, experience, and intent specifically affect the playing of the game and the experience of the game’s narrative. Measuring time in a playthrough using Imageplot also reveals how variable the durations of particular game events are and establishes how long a portion of the narrative is for an average player.

In order to measure variation, however, one first needs something to measure from. The points of similarity and sameness between imageplots are therefore a crucial place to start, and thankfully they are also some of the most immediately apparent aspects of them. All of the playthrough imageplots are plotted to be the same size, regardless of the amount of overall time each one visualizes. In effect this means that an imageplot of a shorter playthrough, such as PT3 (00:29:44), is visually similar to a longer playthrough such as PT5 (00:51:11). While this effaces the visualization of exact time, it is helpful for the comparison of time relative to different overall playtimes. In other words, one can see that events happen at relatively the same place in the playthroughs of PT3 and PT5, even if they happen at different time markers because the two players took different amounts of time to finish the overall game. Establishing that relative sameness between playthroughs then enables the observation and analysis of the variance created by player interactivity. Furthermore, one can interpret the similarities between imageplots as the visualization of the game’s basic structure and narrative–the common foundation that the game provides and all players experience.

The most obvious example of variation in time in the playthroughs is the fact that every playthrough has a different overall playtime. Of the ten playthroughs plotted here, the longest one (PT10) has a playtime of 1:24:50, and the shortest one (PT9) is 0:20:00 long.[11] At first glance there appears to be a wide discrepancy between the two that would indicate the possibility for almost infinite variation, but taken together with the other playthroughs some clear trends emerge. For example, the average time of all the playthroughs is 0:39:22, with a standard deviation of 00:18:12. This means that 67% of the playthroughs in this dataset fall between 00:21:10 and 00:57:34 in playtime, and that that range of time accounts for the majority of play experiences with P.T. Of course there is still the possibility for outliers of longer or shorter playtime (including PT9 and PT10 already mentioned), and the addition of more data would inevitably alter these numbers. Yet these playthroughs seem fairly representative of extent available playthroughs on YouTube and Twitch, and perhaps even of player experiences not present through those platforms. As such, data collected from these playthroughs reveals a measure of how player interactivity shapes playtime on average, which affects narrative time and experience in P.T. A player who spends more time in an area will have a fuller experience of that area, and likely uncover parts of the game narrative that a player who spends less time there might miss. In this way variance in time is intricately linked with variance in sign covered in the next section.

Beyond measuring the overall length of a playthrough, imageplots also indicate the specific times when players encounter different parts of a game. In P.T. the most prominent example of this is an iteration of the hallway that is tinted red and is common to every playthrough and corresponding imageplot (Figure 3). Even without measuring the exact time the player reaches this version of the hallway, the imageplots show that there is a general variance in the time different players do so. Most of the players reached this point at roughly halfway through their playthroughs, but imageplots PT2, PT6, and PT7 enter the red hallway closer to the end of the game. PT6 is especially noteworthy in this regard, and the brevity of the playthrough after the red hallway is surprising when compared to the overall playtime, which was actually longer than average (00:40:00). It may be that the player in PT6 was more familiar with the later parts of the game and was able to complete them quickly. In any case, PT2, PT6, and PT7 reveal that time in the later stages of P.T. is especially variable in ways that it is not in the first half of the game. This variance in playtime is actually significant in the narrative as well. The versions of the hallway found in the first half of the game are relatively stable in terms of time and space–there are no tints to the lighting, the player moves at normal speed, and overall experience is more realistic (to the extent that that is possible in a world with strange monsters and supernatural occurrences). Yet all of this changes as P.T. progresses, and the game’s space and time begin to break down in the sections where playtime becomes most variable. The imageplots thus reveal that the narrative themes of P.T. are present even on the levels of game structure and playtime. This supports the ludonarrative position that narrative and play are not readily separable in games, but rather that they are constantly interwoven in the experience of a game.

Potential explanations for the time differences between the playthroughs are found in the individual player’s level of experience with the game, their playstyle, and their intentions for recording their play experience. While it is difficult to account for all of these factors of player experience with a game without contacting them, the way the player describes their playthrough can provide some answers. Four of the P.T. playthroughs included here (PT1, PT3, PT4, PT6) have been labeled “Walkthroughs” by their players, indicating that they show viewers the correct way to play and complete the game. As a result, one can assume that these players have played the game before and know each part of it well enough to conduct the walkthrough. In contrast, the other six playthroughs are variations of the “Let’s Play” format on YouTube and Twitch, which usually has players playing a game for the first time for an audience (sometimes with commentary). The lack of player experience in Let’s Plays relative to Walkthroughs means that playtimes in the former are on average longer, though not drastically so: the average playtime for the six Let’s Plays was 0:41:45, and the average for Walkthroughs was 0:35:47, with a difference between them of 0:05:58. These numbers indicate that while experience and intention do have an effect on variation in playtime, they alone do not fully explain it. The relatively small difference between Walkthroughs and Let’s Plays also may suggest that there is a central narrative and experience to a game that averages can help trace.

Experience and intention are by no means the only possible explanations of time variation, but unfortunately imageplots do not (and cannot) account for all factors. For example, players in different playthroughs may have been distracted by other things happening in their personal environments while playing, thus delaying their progress. Some players also may have more experience with different games that are similar to P.T., making them more familiar with the game’s conventions and better able to navigate them. Finally, players also may have played with different hardware setups that could subtly lengthen or shorten their playtime (this is especially common with older games like Silent Hill 2). These and other possibilities do not appear directly in the imageplots, and because of this it is impossible to account for them without additional information. Future work with ImagePlot and playthroughs could remedy this with more direct contact with players, or by adding methods for analyzing the images from player cameras found in some playthroughs.

Imageplots of another game in the Silent Hill franchise, Silent Hill: Downpour (Vatra Games, 2012), provide useful comparisons, both to other games in the series and to games with more room for exploration than P.T.. Silent Hill: Downpour was advertised as containing discoverable and variable content, largely to promote its replayability. The imageplots of Downpour are of very different playthroughs: SHD1 is a regular playthrough of the game with a playtime of 4:59:19, and SHD2 is a speedrun (where players attempt the shortest playtime possible) with a playtime of 2:52:20 (Figure 4). Interestingly, the two imageplots reveal fewer points of relative comparison between the playthroughs than with P.T., indicating that there is much more time variance in more open games like Downpour. Presumably this is due to the presence of more content, meaning differences in playstyle will accumulate greater time differences by the end of the game. More importantly, this demonstrates how it is more difficult to locate a central narrative and player experience in longer games with more choices and player interactivity. It is not that a central narrative is not present, but rather that that central narrative is spread out and potentially obfuscated by the multiplicity of optional content interwoven with it. As seen in the imageplots, the narrative diffuses across the variability created by player interactivity, and the measures of variance in playtime (average, standard deviation, etc.) become looser as a result.

The durations and times captured by the imageplots are the results of thousands of moments of gameplay, and each moment presents the possibility for at least slightly different choices and actions on the part of the player. In this sense, the temporality of the imageplots is directly related to the multiplicity of emergent possibilities that are present in gameplay, and now captured in distance visualization. This suggests that the temporality of game narrative and play is inextricable from its emergent capabilities, similar to the temporality of cybernetics and neural nets described by Orit Halpern. Halpern writes, “the temporality of the net is preemptive, it always operates in the future perfect tense, but without necessarily defined endpoints or contexts. Nets are about T+1” [Halpern 2014, loc 3114]. Time in systems like nets or games operates on sets of uncertain future conditions, meaning that each moment in a game opens on new possibilities for emergence and alteration. The accumulation of these moments as seen in the imageplots accounts for measurable differences in time between playthroughs. They also involve the second category of difference: variance in content, or sign.

Beyond encountering objects at different times or in different places, players can also simply encounter different objects. There are several prominent examples of this in the P.T. playthroughs, including encounters that are similar (like the red hallway already mentioned), those that are similar and different (every player encounters a crash screen, but there are different crash screens), and those that are entirely different (such as a jump scare only some players encounter). The presence or absence of these events in a playthrough alters a player’s experience of the game, including their perception of the game’s narrative. I refer to these differing experiences of content as variance in sign because they involve the presence, absence, or rearranging of visual, auditory, and haptic signs that affect the game’s narrative.

While the red hallway, the crash screen, and the jump scare could each be fruitful moments for close analysis, I will only cover them briefly here for their significance to narrative structure and variance in games.[12] The red hallway is perhaps the most immediately apparent similarity between all of the P.T. playthroughs, and the imageplots reveal that it varies more in its location in time than it does in its form. This does not mean that its form is not significant––if anything it means quite the opposite. The form and meaning of the red hallway are crucial to the progression of the story, as evidenced in the fact that every player must encounter it. The red hallway is where the reality of P.T.’s game world begins to bend and break, evident in how time speeds up in that version of the hallway and many of the normal objects are replaced with abstract and nightmarish versions of themselves. For example, all of the paintings and family pictures in the red hallway are replaced with images of flashing and revolving lights. This warping of the game’s reality is crucial to the central narrative of the game and its conclusion: as the player discovers more and more about a series of grisly murders, their first-person experience of the game’s environments becomes increasingly unstable, representing a growing psychosis. The red hallway is a breaking point in this process, and every player must encounter it in much the same way.

The imageplots demonstrate how the game seems to return to normal after the red hallway by presenting players a brightly lit version of the hallway. However the player quickly notices that the screen seems to break and glitch as they move around, and eventually the game seems to crash. Every player encounters the crash screen, indicating its significance as a moment of complete breakage in the game’s reality. Yet the type, color, and message of the crash screen varies, including red, yellow, white, and black versions of the screen seen in PT9, PT7, and PT2 (Figure 5. The black version in PT9 can be hard to distinguish from the other dark images of the imageplots without zooming in). The variance in the crash screen’s form introduces the possibility for different meaning and interpretation depending on the version the player encounters. Examples of the message on the screen include “I’m heading there [to hell] now,” “Development halted due to inexplicable bug,” and “I’ll call later.” Which screen the player sees alters the meaning of the central narrative, creating a story of paranormal horror, a metafiction about game development, an unresolved mystery caller, or other potential interpretations.

Finally, some occurrences are limited to individual experiences, and are different on every playthrough of a game. For example, the player in P.T. is haunted throughout the game by Lisa, the ghost of a murdered wife from the radio reports that the player hears. However the encounters with Lisa are randomly generated, and there is a jump scare where Lisa attacks the player character and breaks their neck that only some players experience (occurring roughly half way through PT6). If this happens to a player, that player must start the entire cycle of looping hallways over again. The addition of these signs serves the same function as most jump scares: to frighten the player with sudden action. Yet they also alter the game’s narrative by changing the individual experience of the player, introducing elements of failure and becoming trapped in the seemingly endless hallway. Rather than being entirely secondary or separate, the variance of even a small number of optional and random signs can significantly affect a game’s central narrative.

The differences in signs between the playthroughs and imageplots of P.T. reveal the especially performative and playful nature of signs in games, which contributes to the overall mutability of narrative form in the medium. Each image or object in a game is a sign that the player interacts with–receiving it, interpreting it, deploying it, altering it, etc. This understanding is not new; C.S. Pierce recognized a similar interactivity with signs in identifying what he calls the “interpretant” of a sign, a mirror sign created in the mind of an individual encountering a sign [Pierce 2011, 99]. Yet visual signs such as those found in games function somewhat differently from linguistic signs, and they operate by what Johanna Drucker calls distinct “basic codes of visual form” [Drucker 2014, 42]. Drucker describes these basic codes as forms such as lines, shapes, and movement, and all of these are certainly present in games [Drucker 2014, 43]. Games are their own visual form, however, and their visual signs have media-specific codes as well. For example, objects in games are often marked by specific game symbols or highlighting to notify the player that the object is interactive. Similarly, the layout of space and objects can give players visual cues for where to go and what to do there, or in other words how to play. These visual signs are basic codes particular to games and are essential to how players navigate and play with game narrative. ImagePlot is capable of capturing the various outcomes or traces of these codes and players’ interactions with them, and in this sense an imageplot is itself an indexical sign of player activity in game. The relationship between an imageplot and player activity is indicative of how visual signs in games are often directly linked to functional purposes, and interacting with them during gameplay is a performance of meaning with variable and emergent outcomes.[13]

The visual and playful signs present in an imageplot become further legible with the help of film semiotics. Film scholars such as Jean Mitry have theorized the relationship between visual and linguistic signs since the 1980s and have noted that different types of signs can have similar structures while functioning differently: “any likeness exists in the structures not in the forms” [Mitry 2000, 24]. Just as there are important differences between signs in film and language, there are also differences between signs in film and signs in games. One such difference is the fact that the player takes on an active role in the construction of signs in games. With P.T., for example, it is up to the player to interpret the signs in the game and interact with them in the correct ways and sequences in order to progress. Because different players perform this role differently, there is variance in time and sign between imageplots of the same game when there isn’t for imageplots of the same film. Christian Metz’s concept of the “imaginary signifier” further explains the player’s role and presence in the imageplots. In short, in film the imaginary signifier is the transparent or absent signifier that nonetheless represents the viewer in the film, acting as their “prosthesis for [their] primally dislocated limbs” [Metz 1982, 4]. A similar phenomenon takes place with player characters in games, particularly first person games like P.T., where the camera mimics the player’s first person experience of reality and the player character is not shown on screen. The physical embodiment of the player is nowhere represented in an imageplot, yet their presence in the form of their choices, actions, and encounters are ever-present. In this sense, the imaginary signifier of the player is the first person perspective, and it is found in every image plotted in the imageplots.

Perhaps the greatest insight film semiotics can lend to games and imageplot analysis is how signs and narratives are fundamentally fragmented, even as they give the illusion of being unified and complete. Film semiotics suggests that visual signs are fragmentary in the sense that they are always temporary and incomplete, lacking an actual referent and becoming distributed across images and time. Peter Wollen notes this in describing film as the “fragments of raw reality, multiple and equivocal in themselves” [Wollen 1969, 132]. The meaning of this fragmentation has profound implications for game narrative in that perfectly unitary narrative becomes elusive if not impossible. Imageplots reveal how signs in games are always fragmented and distributed across the overall experience of the game depending on player interactivity; players encounter, interpret, and sequence signs differently. As they do so, they generate similar but different realities, and in every moment of play there is the potential for difference to emerge. Importantly, the variability of signs and narratives in games does not render them ineffectual or secondary to play, but rather demonstrates how they are omnipresent and necessary to the construction of meaning in games. Metz emphasizes the crucial function of signs when he writes, “Nevertheless, it is in [the symbolic’s] wake that we can find hope for a little more knowledge, it is one of its avatars that introduces ‘understanding’” [Metz 1982, 4]. ImagePlot helps us visualize how fragmentary signs come together in different configurations in play to create unique narrative realities, and film semiotics provides a framework for theorizing this process and its possibilities.

The play of game signs evident in imageplots reveals a crucial aspect of visual signs that they share with all other signs–as much as they are identifiable and understandable, they are never fully stable. As a visual sign enters into the virtual world and thus into play with other game signs of all sorts, it will continue to change and evolve in a way similar to the change over time Saussure describes in diachronic linguistics [Saussure 2011, 81]. One type of evolution in a game sign is formal, as a particular sign evolves according to player interaction and progress. An excellent example of this is the paper bag in the opening room of P.T., which over the course of the game goes from being a simple bag to being a narrator of sorts with a mouth. With each of its evolutions the form of the sign will change somewhat, and consequently the way the sign will be interpreted by the player and others will change as well. However such signs also change informally, in that their relationships to the player and to other game signs will constantly be in flux. There are limits to these changes; after all each game sign remains beholden to the other signs in its system, especially those that make up the central narrative and experience. Still, variance in sign means that narrative form in games is never static or fully stable.

Taken together, the signs in a game create the narrative of a player’s playthrough, and the presence or absence of particular signs can fundamentally alter that narrative. Variances in encountered signs constitute measurable differences in game narrative between playthroughs. Paying close attention to these points of semiotic difference allows one to trace the effects and magnitude of player interactivity in a given game.

The archive of player experience created by imageplots allows for measures of variance in both time and sign in game analysis. There remain a number of limitations to this archive––for example, the imageplots cannot fully account for individual players’ embodied reactions to a game or exactly how they interpret game elements, though a future analysis of player actions and facial expressions caught on camera for Twitch streams (such as those found in PT8) may be able to move towards that capability. In sum, the strengths and weaknesses of this approach are:

Many of these weaknesses could be addressed through studies that recruit players to play a given game (or games) and record new playthroughs, controlling for play environment, player experience and intent, and other factors. Yet the advantage of the method as-is is that it captures a limited picture of how players are already playing on their own and experiencing ludonarrative.

What the imageplots already account for is a sense of the central narrative experience for each game, making it clearer which parts of a game are common to every player’s experience and which ones are specific to individual playthroughs. The central narrative of a game consists of these common parts, or in other words what the developers coded as the essential and necessary elements of the game’s characters, events, and overall world. In P.T., the imageplots reveal that the central narrative is found in the red hallway, the crash screen, and many copies of the hallway that are seemingly identical. Zooming in, one can see that the central narrative is told through documents and objects that all players interact with, and it is the scripted narrative of a man murdering his family and killing himself. It is not that the other parts of the game that are randomly generated or optional do not matter, or that they are not meaningful. Such parts are often still significant for altering or enhancing a game’s narrative, and they are crucial for building space for play and individual experience. Yet being able to distinguish between a game’s narrative core and its variable content allows for analyses that account for both the central narrative and for interactivity and specific differences in player experiences. It provides a way to record and trace the limits of player interactivity, and thus to assess where and how that interactivity leads to different interpretations and meanings for the game. These types of analysis are valuable to the study of games, ludonarrative, and other visual media for multiple reasons: they hone our focus on the types and effects of interactivity in games; they help us get more specific about similarity and difference between players’ experiences of a game and aid in the analysis of individual games as texts; they highlight new understandings of narrative; and they point to possibilities for playing and designing for difference in video game worlds.

The greater attention to variation and similarity in player experiences in imageplots furthers our understanding of interactivity in games by helping us see exactly where and how different games are interactive. While interactivity in games usually refers to the interaction between player and game, such as choosing what to do, what to use and interface with, and where to go in the virtual world, there are other types of interactivity present in games that become apparent in imageplots [Walsh 2011, 84]. For example, Daniel Punday argues that games are really a form of multimedia in that they are a point of interaction between many media, including music, film, and text [Punday 2011, 31–1]. In this sense, media interact with each other as well as with people, particularly in automated ways enabled by software and digital networks. Narrative scholars such as Marie-Laure Ryan have identified similar forms of interactivity between different categories and aspects of games, such as the interaction of a game’s “semiotic substance,” “technical dimension,” and “cultural dimension” [Ryan 2014, 25, 29]. Players interact with game systems, other players, and even other media, and all of this takes place on various levels of physical or abstract interaction as forms collide, combine, and emerge. All of these processes are in different ways interactive, as Ryan argues: “digital texts are like an onion made of different layers of skin, and that interactivity can affect different levels” [Ryan 2011, 37]. Ludonarrative in games is dependent on these many forms of interactivity, including the interaction of different narratives, such as those laid out by developers, created by players, or that emerge in collective discourse inside or outside of a game [Mejeur 2019]. All of these types of narrative and interactivity are present in some way in imageplots: the narratives scripted into game content by developers (in P.T., the story of a man murdering his family) are seen in the objects in the images, the narratives players create themselves are observable in the sequence and duration of content they choose in their playthrough, and even some of the collective narratives that emerge from players interacting are discernible in the player’s webcam as they stream for an audience. Imageplots thus allow us to isolate how particular interactive and variable forms contribute to player experiences and meaning in specific moments throughout a game. They do so by providing observable measures of these effects — one can identify and examine how the player in PT1 played differently from the player in PT2, and how those differences change the reality of the game.

Imageplots make it possible to identify and get specific about variations in different players’ experiences with a game, such as my experience with P.T. that I narrated at the beginning of this article. P.T. ends with the phone in its hallway ringing and the player answering it, which shortly thereafter triggers a final cutscene that is a trailer for the cancelled Silent Hill game. I struggled to get the phone-ringing event to trigger, and as a result I spent a great deal of time in the final version of the hallway desperately trying possible solutions while hoping to avoid the randomly generated Lisa jump scare death event. The P.T. imageplots reveal that my experience, while shared with several other players (PT4, PT5, PT10), was not universal, and even seems to be in the minority of players’ experiences with the game. This difference in our playthroughs significantly alters the meaning and affect of the game’s ending. For most players, the ending is a short exploration of a familiar version of the hallway that constitutes an escape from the chaos of the preceding parts of the game. For me and other players who were trapped in the final hallway for greater amounts of time, the ending was a frantic, stressful scramble to find a way out before possibly dying and having to start over. Despite both experiences ending in the same place, the variance in time and sign profoundly affects the narrative pacing and structure, potentially even leading to divergent interpretations of the game’s messages and meanings.

Imageplots thus aid in the analysis of individual texts by allowing us to see the structure of an entire visual text at once, and to assess how that structure contributes to the text’s meanings. In other words, ImagePlot allows us to see the shape of a text, much the same way that sentiment analysis allowed Matthew Jockers to see the shape of plots in novels, that topic modeling allowed Ben Schmidt to identify fundamental plot arcs in popular TV shows, or comparing different plots across media allowed Chess to identify the queer potentials for narrative in the play spaces of games.[14] Using imageplots of playthroughs, one clearly sees how long or short a segment of a game is, the colors and aesthetics used in different portions of a game, and what trends or cycles exist in the course of a game. For example, the imageplots of P.T. included here reveal the structure of the game’s narrative that contributes to its overall meanings and determines what all players will experience and what different players will experience differently. In particular, the imageplots demonstrate that every player will spend roughly the first half of the game exploring more or less similar versions of P.T.’s hallway, evidenced in the lighting and coloring of the images that remains the same throughout those sections of the game. However around halfway through their playthrough (more or less, depending on the player), the player will encounter a red version of the hallway that radically changes the player experience and sets off a descent into instability and incoherence in content that dominates much of the rest of the game (see imageplots). While the red hallway starts this for each player, the descent looks different for each player as well: they encounter different versions of the crash screen after the red hallway and randomly generated events from that point on. This structure is essential to the narrative of the game — the narrative of a man who wakes up with no memory of his surroundings, gradually discovers that a man murdered his family in the house, and confronts the growing terror that he might be that man. What is at first a casual, if cautious, exploration of a strange house gives way to the desperate attempt to escape an increasingly random, broken, nightmarish reality, and this narrative progression is visible in the game’s structure. Distance visualization allows us to see a game’s overall layout and plotting, and this perspective lends new evidence to game analysis that links close analysis of specific scenes to larger forms, structures, and patterns.

Beyond the analysis of individual texts, imageplots and distance visualization contribute to a ludonarrative understanding of narrative form that is nonlinear, emergent, and playful. Imageplots of games demonstrate the constant connection between narrative and play in that they show how the traditionally narrative elements are interwoven with segments of play. In other words, the “cutscenes and backstories” that game studies scholars such as Markku Eskelinen identify as the only narrative parts of a game still need play in order to connect them together into a narrative, and cannot be isolated or abstracted away from the player pushing them forward [Eskelinen 2012, 224]. Even moments in P.T. where the player is simply moving down the hallway or looking at objects —the mundane actions that make up the majority of the P.T. playthroughs and imageplots —are progressing toward the ultimate fulfillment of the narrative, and it seems arbitrary to completely divorce those moments from their narrative function. Instead of considering narrative as a linear or multilinear progression of events separate from gameplay, what if we considered narrative as a system of signs that coheres into a core experience, with variations in sign and time emerging and impacting that center? The crucial difference here is that narrative becomes less of a static and stable form, and more of an active, evolving, and mutable one. Hanna-Riika Roine argues for a similar definition of narrative by pointing out how narrative forms are affected by “the participatory nature of digital media:” “they now appear to us as phenomena to be cut, pasted, reassembled, and distributed” [Roine 2015, 83]. Imageplots visualize these changing and colliding forms in games and other visual media, and help capture what narrative looks like as players and viewers experience it differently.

Exploring the collisions of many forms, structures, and semiotics in visualizations like imageplots can yield new insights on how ludonarrative texts operate and create multiplicities of meaning. This is not completely new work, and there have been calls for the use of semiotics in game studies, such as Paolo Ruffino’s work with semiotics and the social contexts of games, or Brian Upton’s work with the procedural generation of meaning in games.[15] However we have yet to fully account for the immense multiplicity of forms and possibilities for variation in games, and there is still no consensus in game studies (particularly the emerging study of ludonarrative) as to a methodology for game analysis. ImagePlot makes it increasingly clear that ludonarrative in games relies on the complex interactivity of language and play to create both similarity and difference between player experiences. Yet how exactly game signs are formed, laden with meaning, and interpreted by players and developers in different contexts remain rich topics for further research, and ImagePlot is one method for conducting this research on a much greater scale than individual analyses of single playthroughs. Identifying the central narrative and using differences in time and sign to measure player interactivity is a starting point. It provides a few of the analytical tools needed for the theory and praxis of a critical, playful interpretation of games that can also account for player interaction and difference. But the ability to visualize a game’s forms, signs, and central narrative with ImagePlot also has many profoundly interdisciplinary implications and potential uses that are unexplored: imageplots could help assess where and how players spend time in games, which game mechanics and designs are helpful or harmful to player experiences, or how players learn and adapt through the course of a game, among many other questions. For example, the P.T. imageplots provide an archive of player experiences in a popular survival horror game, and they can help answer questions of how and why players engage in settings that create unease, fear, and even terror. In this sense, ludonarrative, semiotics, and the study of game forms can contribute to work in the other disciplines of game studies, even as it pursues its own projects.

By better understanding how forms operate in games through distance visualization and analysis, we can ultimately find ways to use them in the creation of new story worlds and even new realities. For example, Ryan argues that critical attention to interactivity can reveal how media form new spaces and experiences for players of games [Ryan 2011, 40–1]. Alice Bell capitalizes on this observation and other parts of Ryan’s work to suggest that hypertext and interactive fictions (including games) constantly generate “Possible Worlds” that are “ontological domains,” and thus different spaces and conditions of being [Bell 2011, 79]. Players repeatedly choose, explore, or foreclose upon these possible worlds, and they do so by interacting with the changeable forms that comprise a game’s system and can be captured in imageplots.

Of course there are limits to which possible worlds can emerge that are largely tied to the limitations of our current systems. As Ryan writes, “we do not really have a story-generating system sufficiently sophisticated to produce a wide variety of interesting stories out of data internal to the system” [Ryan 2011, 48]. A story system capable of adapting to any user input has long been the goal of game designers such as Chris Crawford, but even after decades of attempts such systems are only capable of generating basic procedural narratives. It might be that the creation of a fully interactive system can only happen once we have reassessed our understanding of forms ranging from signs to narratives and beyond, and developed computing, programming languages, game engines, and other assets for better utilizing variable and emergent forms like ludonarrative. ImagePlot and other digital humanities tools for visualization can help get us there by visualizing forms, and making it possible to compare how they collide, flow, and shift. In games, they reveal what emerges as the common core of a game’s possible worlds and establish what most possible worlds for a game will look like. In effect, they show us a game’s pieces, and how those pieces can relate to each other and to us. With that knowledge, we can envision how our interactions with a game — along with all of the variations they generate — could be different or better. Now let’s play.