Viktoria Brüggemann is a research associate at the UCLAB at the University of Applied Sciences Potsdam. As a cultural scientist, her research emphasis is on cultural history and museums, with a focus on different ways of knowledge sharing in the (digital) present. ORCID: https://orcid.org/0000-0003-3858-0269
Mark-Jan Bludau is research associate at the UCLAB at the University of Applied Sciences Potsdam. His main field of interest lies in information visualization with focus on interaction techniques and the visualization of cultural heritage data. ORCID: https://orcid.org/0000-0001-6300-8833
Marian Dörk is a research professor for information visualization at the Institute for Urban Futures of the University of Applied Sciences Potsdam. He co-directs the UCLAB, a transdisciplinary research space at the intersection between interface design, computer science, and the humanities. ORCID: https://orcid.org/0000-0002-3469-7841
This is the source
We propose the philosophical notion of the fold as an evocative vocabulary for the design and critique of interactive data visualizations. An expanding range of application areas, such as digital art history and literary studies, illustrates the potential of data visualization for research and education in the humanities. Coinciding with the increasing currency of data as evidence in the humanities, this research addresses a growing interest in data visualization for visual analysis and argumentation. For example, cultural collection visualizations promise a range of possibilities for visual and interactive representations of digital cultural heritage, used both for free exploration and focused research. Based on the concept of the fold, prominently advanced by Gilles Deleuze, this paper outlines a critical framework that draws attention towards the complexity of the underlying data. The fold offers a way to analyze and conceptualize visualizations through the lens of three integrated operations: explication, implication, and complication. It is an opportunity to think of interactive visualizations as portals into coherent, elastic, and ultimately infinite information spaces. Accordingly, it rejects the separation between interactivity and visual encoding and draws attention to the transitions between multiple states of a visualization. We identify design patterns of the fold in data visualizations, devise a framework for the critical interpretation of interactivity in data visualization, and demonstrate the implications for the digital humanities.
Outlines how the Deleuzian concept of the Fold offers a novel means of analyzing and conceptualizing data visualizations
... its envelopments and developments, its implications and explications, are nonetheless particular movements that must be understood in a universal Unity ...
While questions of visual encoding – the way visual variables, such as position,
size, shape, color, or density, are used to represent data in information
visualizations
This is especially problematic when considering complex and multidimensional
datasets, as of cultural collections true,
which has sparked a debate over the
role of the humanities in the design and interpretation of visualizations,
understanding data as interpretive and calling for more ambiguity in their graphical
expression
We propose the notion of the fold, advanced by French philosopher Gilles Deleuze
With this research, we transfer Deleuze’s notion of the fold to data visualization and derive three concrete qualities for visualization design and critique in the humanities: coherence, elasticity, and infinity. In a next step, we characterize the manifestation of the fold’s operations and qualities in a range of exemplary visualizations with the help of interpretive illustrations and concrete examples from digital humanities projects. Lastly, we formulate a critical framework consisting of principles and questions for the design and interpretation of interactive visualizations and discuss open questions for future work at the intersection of data visualization design and humanistic inquiry.
This research is largely motivated by a recent surge of work on the visualization of
cultural heritage data, for which numerous visualization approaches have already been
proposed, yielding an abundance of visual interfaces and representation techniques
Overview first, zoom and filter, then details on
demand
To this end, a range of techniques and methods have been proposed. The visualization
of complex datasets, among other, calls for the careful choice of visual variables
and encodings
The iterative analysis of data can be thought of as an ongoing information practice
that exhibits the characteristics of flow
, i.e., high
levels of concentration, immersion, and motivation
In all this, we are guided by a growing critical awareness about graphical interfaces
In the following, we briefly introduce Gilles Deleuze’s concept of the fold, in which
he links Leibniz’ baroque idea of monadology to postmodernism The Fold
and abstracting
vocabulary and concrete functional illustrations. Philosophy, in Deleuze’s sense,
creates concepts, and concepts produce an orientation or direction for thinking
In
a way to differentiate matter without introducing discontinuity
allegory of the worldwhich in his sense contains the fold as an element of infinite iteration
pleats of matterand the
folds in the soul,which are distinct and still continuously interwoven. This metaphor also refers to the human body and soul, making the distinction important for the understanding of the processes of information interpretation and accumulation: following Deleuze, the monad (or human) already holds all information, twisted into many folds. If it was confronted with a new question or information on the level of the pleats of matter, through the senses, the folds in the soul would automatically begin to twist and turn, after which the resulting answer or links to other information would suddenly become visible. The answer was already there, it was only hidden in the manifold twists of the soul.
While Deleuze’s treatment of the fold is rather extensive and, in parts, elusive,
his description of the unfolding of information offers a useful vocabulary for the
conceptualization of (digital) information spaces. The related notion of the monad
has already been proposed as a unique perspective on social systems by Bruno
Latour and colleagues actual
and the virtual,
which are mutually dependent and connected through folds, just as the monad is
divided into pleats of matter and soul, but still continuously interwoven.
Although Deleuze may not have meant virtual
in the sense of digital,
the concept of the fold holds a powerful metaphor for the structure and
understanding of digital information spaces, in which only certain parts of the
available information are perceptible at a time, while the whole information space
remains invisibly present. Following Deleuze, the folds of information exist in
the virtual world mostly, because they are potentially infinite and offer an
unlimited range of capabilities. Still, everything within the monadologic system
is connected, although not always ascertainable Because the world is
in the monad, each monad includes every series of the states of the world;
but, because the monad is for the world, no one clearly contains the
‘reason’ of the series of which they are all a result, and which remains
outside of them, just like the principle of their accord.
For Deleuze, the three operations Explication–implication–complication
form the essential triad of the
fold The Fold
(see Figure 1). Explication describes the process of
unfolding, e.g., opening a book, dividing something into its subsections, or
fanning out its multiple facets (see Figure 1,
left). On the opposite, implication refers to the commonly known process of
folding that reduces something in size and detail, e.g., closing an open book or
folding a paper. Deleuze draws our attention closely to these processes within
monadic systems: If something inside the monad is hidden through the process of
implication, what is folded still comprises everything else, though this is not
always perceivable with the human senses. Through the process of explication,
hidden connections become visible again, but the (un)folded entity nevertheless
possesses connections to all others. Unpredictable outcomes and new connections
can occur during folding and unfolding, as a fold can generate the sudden
juxtaposition of formerly opposite (and also opposing) points.
At the intersection of the fold and monadologic theory, Deleuze introduces
complication as the third operation of the fold (see Figure 1, right). Here, he draws from medieval philosopher Nicholas of
Cusa (1401-1464) to explain the all-embracing function of folds for the mind and
the matter. For Cusa, everything that was outside the perceivable was divine
simplicity, which he defined as a true folding
of the
opposite and contradictory, and named it complicatio
complicated
god that was able to explain
things outside of causal connections and rational thought. Deleuze shifts the
process of complication from the divine sphere as outside to the inside of human
information processing via his theory of the fold. The operation of complication
thus explains the process of information accumulation and connectedness of
everything perceivable, while also addressing its arbitrariness. Because
everything inside the monad is folded to infinity, every possibility already lies
inside of it, yet, it cannot expose itself in its entirety at any moment.
Surprising occurrences are thus not more than a complicated
folding
process, in which connections are rearranged and only a part of the connected
universe becomes visible at a time.
The perpetual incompleteness of any knowledge is what makes this theory so relevant for information representation, which aims to communicate complex matters, but needs to make its omissions and reductions transparent. Folding processes can disclose and even simplify ‘complicated’ occurrences: the moment of collapsing an angle from 1 degree to 0 degree, when the eye can visually follow the angle turning into a line, describes this process best. After having observed the transition, the resulting line is not only a line but consists of two layers and could still be seen as an angle; it comprises multiple folds of information that we can only capture if we are able to follow and understand the operation of the fold.
... an elastic body still has cohering parts that form a fold, such that they are not separated into parts of parts but are rather divided to infinity in smaller and smaller folds that always retain a certain cohesion.
In addition to the concrete operations of the fold, Deleuze’s opus constitutes a unique and global proposition about the form and function of information spaces. To relate the fold theory with visualization practice, we have abstracted three overarching qualities – coherence, elasticity, and infinity – that align the fold with interactive data visualizations in the context of humanistic inquiry.
Instead of considering information ‘points’ as discrete objects, the fold expresses the coherent quality of monadologic systems that are defined by context and relations. Accordingly, the single data point in a visualization stands for more than one expression, it is part of a system and defined through its connections to other points of the network. As stated by Deleuze in the opening quote of this section, this quality multiplies itself infinitely: No matter how far the information space is passed through, each finding, output, or information point possesses a relation to the beginning of the journey and the whole universe. The quality of coherence thus includes outcomes and object pairings which might seem opposed, since every connection forms an everlasting part of the fold.
This quality captures the ongoing change in an information space, the twisting and turning of information and connections, with one impulse following another. With regard to thought processes, elasticity becomes apparent: Thoughts sometimes stretch and flow unconsciously and casually, sometimes quickly and targeted, but they are continuously changing their form and direction. The same principle should be applied to visualization elements: Through their ability to fold, individual elements and entire arrangements can assume multiple possible manifestations, are flexible in their position and appearance, but never lose the first principle of coherence. Elasticity thus describes a material quality that is situated between fluidity and hardness, i.e., neither lacking coherence nor being fully rigid.
The fold surprises with its infinite possibilities; the outcomes of folding
operations appear to be unlimited. Smaller and smaller threads can be unfolded
in a visualization, or connections between elements can be found and followed
in a multitude of possible combinations. This does not imply that the actions
and events in information spaces are not also repeatable or retraceable, but
rather that folding operations are never final or completed. The same holds
true for the underlying datasets that may never be sufficiently comprehensive
to represent an artefact or collection. There are always further data
perspectives that could, in theory, be invoked for interpretation. The quality
of infinity is especially connected with the operation of complication and
reminds us that comprehensive information spaces might seem structured and
transparent at first glance, but are highly dependent on the perspective of the
viewer and a multitude of data dimensions. Because the fold offers
(potentially) infinite possibilities of twisting and turning, folding processes
remain unpredictable and serendipitous
In the context of information visualization, interactivity has been largely viewed
through the lens of users tasks and intents folds
or folding
in
their project descriptions or prototypes [e.g., Bach et al.
2016, Dörk et al. 2014, Riehmann et al. 2018, Zhao et
al. 2014], we will demonstrate that current visualization practice actually
provides a variety of interaction patterns that align with the principles of the fold
and can serve as an inspiration for digital humanities scholars. Accordingly, with
this research, we do not present new visualization techniques, but explore a new way
to conceptualize interactivity in data visualization.
Designing visualizations along the fold means to understand information spaces as
elastic, coherent, and potentially infinite systems. Instead of focusing on static
snapshots of visualizations, which would favor their visual encoding, the fold sheds
more light on the in-between
states of folding processes, emphasizing the
transitions between visualization states as meaningful views that need to be
considered throughout the entire design process. In a design study, we have already
described this process by means of a book collection from the 19th century, focusing
on a set of functions which enable what we call scalable
exploration
This section presents examples of folding operations in data visualizations through abstracted illustrations and will subsequently demonstrate how the fold’s qualities can form the basis of a critical framework for their design and analysis. We reference existing visualization examples, which have inspired us in the creation of the illustrations and encourage readers to visit them to gain a deeper understanding of the various mechanisms of the fold.
Arguably, explication and implication as the first two operations of the fold are
common mechanisms in data visualizations, implementing the fold in the sense of
increasing or decreasing detail, aggregating or separating, clustering or
dispersing, etc. In particular, semantic zoom
unfoldsinto multiple sub-items and thus provides a more granular view on a selection [e.g., Morris 2018]. Network graphs like this are popular in the digital humanities, for instance when visualizing relationships between authors as in this visualization of Johann Wolfgang von Goethe’s social network: https://www.deutsche-biographie.de/graph?id=sfz53095
Alicein an interactive word tree version of the book
In contrast to explication and implication, which correspond to established interaction techniques, the operation of complication introduces a conceptual approach to high-dimensional data visualization that is relatively seldom considered. Dimensionality reductions and force-directed layouts can be misleading, error-prone, or difficult to grasp, yet, the gradual build-up of a multidimensional visualization can be viewed as a continuous complication (see Figure 3):
Contrary to visualization techniques that separate dimensions into multiple
coordinated views or enforce an abrupt display change, complication suggests
traceable transitions between successive visualization states, which gradually
integrate additional data aspects into the same visualization. To think of
multidimensional data visualizations as dynamic processes of data complications
instead of a static image of reality can help identify the various factors
influencing the visual representation and make interactive functions more
comprehensible. For instance, a complicated
timeline which positions
similar data points next to each other, forming a curve while allowing users to
follow the process of bending, sparks questions about the linearity of timelines
per se and the underlying structure and dimensions of the dataset.
Considering the examples above, the fold’s operations help to identify the interactive mechanisms of a visualization and examine their role in transforming the appearance and arrangement of visual elements. Furthermore, the three operations can be used to spot the lack of interactive capabilities or conceive dynamic behaviors for visualizations being created. Here the fold does not only offer principles for the design of and interactions with visualizations, but also enables us to think about the potential and challenges of humanistic data. Building on Deleuze’s writing and related visualization research [e.g., Elmqvist et al. 2011], we now formulate tentative design principles and questions, thereby following the operations of the fold but especially relating its newly defined qualities to the critical enquiry of data visualizations.
The fold’s coherence manifests itself in the deep contextualization and connectedness of all elements. In order to realize this high degree of coherence in interactive visualizations, the visual encoding and interactive features need to be consistently coupled across all views. Coherence means, for example, that one information point stands in relation to every other point within the visualization, and the context of those relations is both visually and interactively represented. For example, in cultural heritage collections, artifacts might be related to each other through many different explicit (e.g., same author) or implicit (e.g., sharing a subject) relations over a multitude of possible data dimensions (e.g., attributes, visual similarity, temporal sequences). Integration of coherence in visualization is not only dependent on abstract relations but mainly visible in the form of visual cues and linkages that reveal how elements are connected and invite the viewer to follow them.
Question 1: How are the connections between data elements,
visual encodings, and interactive features exposed?
Furthermore, the quality of coherence is particularly promoted by consistent design decisions across the entire visualization, regardless of its dynamic state. As a person interacts with the visualization, the coupling of visual representation and interactivity consistently holds. Similarly, the behaviors of interaction techniques function consistently across all views. Visual variables, such as color, shape or position, that are added to encode additional dimensions should not stand in conflict with existing encodings. Humanities scholars conceptualizing visualizations should therefore pay attention to different states of the visualization (e.g., overview – detail), which interactive functions can be used at which states, and watch out for the application of them throughout a visitor’s journey – for instance when applying filters to a dataset and changing to a different zoom position afterwards.
Question 2: How consistent are visual encodings and
interactive techniques across all views, throughout multiple continuous
states?
When reducing many points into a single point, elements should not be removed, but be folded while preserving the respective relation to the remaining elements. The implication or explication does not only influence a discrete element but also has an effect on the whole visualization, indicating the overall coherence of the information space. When visual features are reduced or when elements are collapsed, the design of the respective transitions should be meaningful and consistent. Similarly, elements being added should appear from logical positions in the interface.
Question 3: Do the arrival and departure of elements in the
display convey the concept of object permanence?
A high degree of elasticity means that elements are flexibly embedded into a complex visual appearance and arrangement and that they are able to change their shape and leave their position to appear elsewhere. They are not static and can show themselves anew repeatedly – and, also in unforeseen representations. For example, objects in visualizations of cultural collections can have multiple relations to each other but are often assigned a static position in web-based interfaces. Instead, it could be helpful to include flexible positions and to consider a range of visual representations, moving between abstract shapes to more concrete images of an object. However, the fold is neither erratic nor random. The changes carried out through folding operations should be comprehensible and meaningful.
Question 4: To which degree are fluidity and rigidness
balanced so that the elements and arrangements accommodate all possible
values and relations in a dataset?
The spectrum of dynamic changes to individual elements and entire arrangements needs to be carefully considered. To do this it is paramount to view visual encodings not as static mappings between data dimensions and visual variables, but as complex sequences modulated by the fold’s operations. Accordingly, like a blooming flower, the additional data dimensions in Figure 3 b) are already comprised in the data points and their revelation demonstrates their elasticity.
Question 5: Are the elements and arrangements designed with
regard to their dynamic behaviors, i.e., their intermediate states?
Humanistic data is oftentimes shaped by various forms of uncertainty,
subjectivity, and ambiguity
Question 6: Does the representation of the data through the
visualization constrain interpretive qualities of the data, such as
uncertainty or ambiguity, that could be dissolved through interactivity and
elasticity of the encoding?
The quality of infinity relates to a multitude of possible representations of data, the insights they may evoke, and circular or open-ended navigation mechanisms. Specific combinations of visual form and interactive functionality in a visualization can evoke new, surprising, and inspiring expressions of a dataset and a variety of possibly unexpected insights, surprises, and serendipitous discoveries. The quality of infinity can be observed, for example, in Figure 3 b) through the seemingly infinite combination possibilities within the weightings of the multiple dimensions.
Question 7: Do all visual elements in the interface afford
interactivity to transform the view and generate new insights?
An interactive visualization is never complete, but always in progress. If a data visualization could be thought of as a densely connected network of linked perspectives, at any given point in the network of possible visualization states, users ideally should never be led to dead-ends. For instance, unfolding detail in Figure 2 b) and c) will always lead to an impression of other information while preserving the context and therefore allowing for open-ended interactions without the need to resort to the browser’s back-button.
Question 8: How is the incompleteness of views embraced and
endless traversal of the data encouraged?
Arguably, a lot of thought during the design process goes into the visual representation of a visualization, i.e., the mapping from various data dimensions to a limited number of visual variables, whereas interaction techniques and animated transitions are oftentimes only an afterthought, sometimes even used as a last fix to solve issues of a visual encoding. Similarly, the interpretation of data visualizations is almost exclusively focused on the visual representation – the rules of turning data qualities into graphical elements – often disregarding the dynamic interplay between display parameters and interactive capabilities. With the fold we put forward an approach that emphasizes a need for simultaneous and coordinated consideration of interaction and representation in data visualization. In the process, we identified the connections between visual encoding and interactivity as crucial for the implementation of the folding operations and emphasized them in the critical framework, arguing for a close consideration of visual appearance and its dynamic behavior.
However, prototyping interfaces may in parts still be restricted by static forms of prototyping techniques, a lack of suitable tools, or a massive amount of data. Additionally, thinking of representation and interaction in all states of a visualization in unison with the intention to achieve meaningful transitions may turn out to demand significant conceptual, technical, and intellectual effort. While realistically seeing the complexity that the fold brings to visualization practice in the digital humanities, we believe that its notion can have a profound effect on the way we think about data, and even in its smallest implementation can contribute to more profound and purposeful visualizations.
Proposing the use of the fold and its operations to approach a gap in the development of data visualization, we recommend the consideration of the qualities coherence, elasticity, and infinity as fruitful starting points for both creation and critique. Nevertheless, these qualities only describe an ideal environment for the fold. In practice, their degree of implementation might always be partially limited. While we have provided a first framework for their implementation in digital humanities projects and research, we are aware that the applied methods are highly dependent on the actual data and project setting. Furthermore, the interpretation of data visualization remains deeply subjective. Especially machine-generated arrangements, for instance, based on similarity, might need a high amount of testing and editing, or even a narrative layer, to arrive at comprehensible, yet “complicated” visualizations in the sense of the fold. Additionally, we would like to see more research into the perception and necessities of different user groups in digital humanities visualizations.
Although this research mainly focused on the implementation of the fold through a new
perspective on the design and interpretation of visualizations, we also see the
potential of the fold when it comes to their very foundation: data. Thinking of
Deleuze’s infinite folds, the common approach to data as given
As data visualization continues to expand its relevance in the digital humanities,
there is a growing need to come to terms with interactivity as one of its most
fundamental aspects. While the challenges of cultural heritage data and the
complexity of their implementation into dynamic visualizations has gained critical
attention, the prospects of interaction techniques to this end are not discussed as
intensely. With this research, we proposed the notion of the fold as a productive way
to jointly consider interaction and encoding in data visualization. While generally
understood as an essential component of data visualization, interactivity is often
treated in separation from the visual encoding and as a second step in the design
process, when all decisions about the visual variables have already been made. This
relegation of interactivity is perpetuated in the critical interpretation of data
visualization that is similarly focused on the visual encoding and lacks the
vocabulary to make sense of the provided interactive capabilities. Drawing from
Deleuze’s writing on the fold, we formulated a critical framework for interactive
data visualization consisting of operations, qualities, and questions for their
design and interpretation. In this context, we do not only hope to encourage
consideration of folding operations for the visualization of data, but especially see
the importance of understanding information spaces and the data themselves as a
manifold space. In order to treat interfaces as subject to critique and
interpretation
To investigate the viability of the fold for the critical consideration of existing visualizations, we abstracted new, explanatory illustrations from various visualization examples. It was our aim to examine in which functions the fold was already manifested in data visualization practice. During this process, we noticed that a considerable number of the visualization techniques exhibited the characteristics of the fold, while certain aspects are still underdeveloped and could be explored in future research and design. While the pair of implication and explication can be seen in numerous visualization examples, we see a necessity and, through the notion of the fold, an opportunity for designing visualizations that offer insightful complications, more specifically through meaningful transitions. This is particularly important when visualizing digital cultural heritage data, where multidimensional and multifaceted collections or datasets offer intriguing opportunities for the joint design of interactive exploration and visual representation. Based on the operations and qualities of the fold, we have formulated a critical framework as an invitation to jointly consider the interactive capabilities and visual representations of data in visualization techniques in the digital humanities. Coherence, elasticity, and infinity are valuable qualities for the design of interactive visualizations that can trigger unexpected and surprising insights – the raison d’être of data visualization.
We would like to thank Linda Freyberg, Sascha Freyberg, Rabea Kleymann, Francesca Morini, Arran Ridley, Jonas Rogge, and Fidel Thomet for their comments, advice, and feedback along the way. Furthermore, many thanks to the reviewers of
Search, Show Context, Expand on Demand: Supporting Large Graph Exploration with Degree-of-Interest