Here, we describe the process by which we turned the raw Hansard parliamentary debates into machine-readable material for downstream steps in the pipeline. The process described contains choices specific to the Hansard dataset, but it is modular and easily interchanged with a different suite of pre-processing steps.

We download Hansard from the web as a series of XML files[4] and parse full text and metadata from the XML into a tabular, tab-separated value (TSV) format using the Python programming language. The input data could be any collection of documents (news articles, tweets, political speech, etc.) in any text format (cannot be images or scans of documents), so long as it is converted into a TSV structured such that each row is a document. In the Hansard TSV file, each row is a debate and the columns are debate ID, full text, and metadata. We additionally append four written reports crucial to the historical analysis to our corpus. The result is a corpus made up of 45,585 speakers who uttered 294,203,233 words, 1,033,536 speech acts, and 111,689 debates.

Figure 1 shows what parliamentary speech and the written reports look like over time. The number of debates per year correlates closely with the number of speakers per year, and they both increase over the course of the century with a notable spike in the last quarter century. The average length of our documents shows a weak inverse correlation. The number of speech acts and the number of unigrams per debate are both somewhat reduced in the last quarter of the century. In other words, as the number of debates and number of speakers increase, the speech acts tend to get less verbose over time.

As is standard in Natural Language Processing, we took steps to pre-process the corpus to prepare the text for downstream analysis. These steps include tokenization, removing non-alphanumeric characters, lowercasing, spell-checking, replacing both common stopwords and custom stopwords with a substitute word, and stemming. We walk through each step in detail during the rest of this subsection.

By the end of the process, the original 294 million words are reduced to 97 million. Almost 200 million words are replaced by the substitute word because they are common stopwords (for example: “a”, “an”, “the”, “I”, “you”) or misspelled words. While the reduction seems significant, substitute words would contribute very little to the topic modeling results and thus minimal information is lost during the process. The final step in pre-processing is to create a document-term matrix, which is a numerical representation of the corpus that can be read and analyzed by a machine. Each document is represented by a vector of word counts. Different weights and normalizations can be applied to the document-term matrix, given the needs of the data or research.

The goal of applying NLP methods to Hansard is to answer historical questions about the shifting concepts of land ownership and property in the nineteenth-century British empire. Traditional historical scholarship involving close reading and analysis is not suited to this task because the text corpus is too large to be read by a single historian or even a team of researchers. Instead, we offload the task to a computer.

Topic models are probabilistic models for discovering the abstract topics that occur in a large corpus of documents through a hierarchical analysis of the text [Blei and Lafferty 2009]. The idea is that a corpus is made up of topics, topics are distributions over a fixed vocabulary of terms, and documents are mixtures of topics in different proportions. The vocabulary of terms can be words or pre-processed tokens like unigrams or n-grams. Topic modeling can answer questions that would otherwise be intractable with a large corpus: (1.) What is a large, prohibitively long corpus of documents about? (2.) Can we extract a human-readable sub-corpus of semantically-related documents? A random process is assumed to have produced the observed data, which are the words that make up the documents in the corpus. Given the observed data, the posterior distribution of the hidden variables (word distributions for each topic, topic proportions for each document, topic assignment per word for each document) determines the hidden topical structure of the corpus which tells us what the corpus is about, answering the first question we seek to solve with topic modeling; the posterior estimates can be applied to tasks such as document browsing and information retrieval, answering our second question. In fact, topic modeling has been used as the basis for “trawling” for a smaller corpus [Tangherlini and Leonard 2013].

The evaluation of topic modeling results is subjective, requiring manual inspection by subject matter experts and cannot replace the traditional textual analysis. The benefit, however, is that the subject matter experts need only to read through a handful of terms per topic rather than a significant part of the corpus. The drawback is that we lose vital information encoded in natural language when we numericize it. For example, the bag-of-words representation of a corpus loses context entirely by treating documents as unordered collections of words [Harris 1954]; n-grams cannot preserve word relationships past their immediate neighbors [Manning and Schutze 1999]; and while vector models like word2vec and doc2vec preserve far-apart word relationships, they are not suitable for identifying a relevant sub-corpus within a large corpus [Mikolov et al. 2013] [Le and Mikolov 2014].

We use a particular flavor of topic modeling called Latent Dirichlet Allocation (LDA) [Blei et al. 2003]. The model does not have a prior notion about the existence of the topics; it is given a hyperparameter, k, which describes how many topics are associated with the corpus, and discovers the k topics from the observed data, the words in the documents. The algorithm is implemented in a number of programming languages and is simple to use with few tunable parameters, but requires hand labeling of topics by subject matter experts. We use a Java implementation of LDA called MAchine Learning for LanguagE Toolkit (MALLET) [McCallum 2002]. In developing the pipeline, we made topic models with varying k values (50, 100, 200, 500, 1000) and different topic modeling methods like Non-negative Matrix Factorization [Berry and Browne 2005] and Dynamic Topic Models [Blei and Lafferty 2006] available to the users.

Within HaToRI, the process of extracting a subcorpus starts with LDA. LDA not only tells us the topics that are discussed in a corpus, but also which ones each document contains. Knowing what documents are about allows us to narrow our focus to a smaller sub-corpus from the whole of Hansard, reading the debates and speech acts that are about our topic of interest — land and property. Sub-corpus extraction using topic modeling allows us to find a richer set of documents than keyword search alone because the topics are discovered from the data, rather than imposing domain-specific and subjective knowledge on the text. Keyword searches and term frequencies work well when the topic of interest can be unambiguously identified by one word or a short phrase. Documents in a corpus can be clustered based on similarity to one another and subject matter experts need to analyze each cluster to decide if the clusters make semantic sense and if any are about their topic of interest. This can only be done if the corpus is not too large and the experts can at the very least scan through each document. Sometimes it is already known that one or a few documents in a corpus are about the topic of interest and the question is whether there are other so far undiscovered relevant documents in the corpus? The goal then is to find documents which are similar to the seed documents. A search based on seed documents works if most words and terms in the seeds are about the topic of interest and not other general topics. This is not often the case. Our proposed method for sub-corpus extraction is a topic-model-based document ranking process where users choose multiple topics of interest, rank documents by how closely they match the topics of interest, explore the results, and iterate by toggling on or off particular topics. We harness a code pipeline to build a highly customizable web-app that makes available to the ordinary user a process of Critical Search characterized by iterative interaction with large corpora [Guldi 2018].

We use three steps in topic modeling: import, modeling, and post-processing. The first two steps are fairly straightforward parts of the MALLET program. In modeling, we tune the number of topics, keeping other hyperparameters (e.g. alpha and beta (priors), number of iterations, sampling method, etc.) constant. We created topic models with the number of topics equal to 100, 200, 500, and 1000. In post-processing, we asked a group of humanists to read the ten most heavily weighted words in each topic, give the topic labels based on those words, and qualitatively evaluate which k number of topics is ideal for Hansard (see the Results section for more details).

Sub-corpus extraction is done by ranking the documents in a corpus using their relevance to one or more topics of interest. The simplest ranking algorithm counts the relative frequency of topic words, or word occurrences assigned to at least one topic of interest, and ranks the documents with the highest frequencies as most relevant. However, this algorithm can incorrectly down-rank longer documents. We improve on this simple ranking algorithm by using the lower bound of Wilson score confidence intervals for a Bernoulli parameter, z, as the sorting value [Agresti and Coull 1998]. The parameter z is tuneable and a larger z sporadically introduces longer documents into the top-ranked sub-corpus; in our web app, the user can visually explore how increasing z changes the rankings by reading the document titles and linking to the full text of the documents. Different values of z may be optimal for sub-corpus extraction depending on the topics of interest and the user’s goals.

We argue that topic-based sub-corpus extraction is a highly effective way to identify a relevant sub-corpus and highlight this point in the context of exploring past land use and eviction in the British Parliamentary debates.

An early choice made in the pipeline is what the ideal number of topics (k) is for the Hansard corpus. We pre-computed topic models at different increments of k (50, 100, 200, 500, 1000) for review. Humanists determined that the optimal number of topics is 500 because of the uniqueness and specificity of the topics. Some topics are difficult to label distinctly from other topics in the 1000 topic model, while most topics could be given unique labels in the 500 topic model. Five distinct land-related topics, their ten most heavily weighted words, and their designated labels are shown in Table 2. They are sorted by the year(s) in which they peaked, with early peaks on the left and late peaks on the right. The language used to describe land in Hansard mirrors the historical shifts of the period, from an agriculturally valued plot in the first half of the century to a rental relationship between landlords and tenants in the middle of the century and to legal protections that favored tenants in the latter half of the century — protections that included rent restrictions, freedom from arbitrary evictions, and land redistributions [Readman 2008].

We build this process of preparing, reviewing, and labeling different k values into the code pipeline. While the computational cost of preparing different models is not reduced, the human effort of maintaining code and documentation on many models is greatly reduced.

Figure 3 shows topic 35, which we named “Negotiating the interests of landowners vs. tenants who seek to improve their rentals”. This topic peaks twice in the late 1840s and early 1850s and then peaks again in 1870 and 1880, mirroring the historical arc of this topic. The Devon and Bessborough Commissions were ordered and written in 1845 and 1881 because of calls for land reform from peasant farmers and tenants [Sartori 2014]. This topic is one of several that point the scholar to the land question, and in combination they can paint a fuller, richer picture of the topic than any single excerpt of Hansard or written report.

We present a land and property example that illustrates how topic modeling goes beyond word frequencies for historical analysis. Figure 4 shows word frequencies for the most commonly occurring words in topic 35. The word frequencies are normalized by the total number of unigrams in each year and are shown per 10,000 words. Four peaks stand out: “person” in the early part of the period and “land” in the late part of the period. “Land” and “properti” have a few smaller peaks throughout the period. The pattern does have some historical significance, but it is certainly noisier than the peaks in topic frequency seen in Figure 3. There is no clear peak in the middle of the century where we would expect to see massive activity about this topic, which could potentially lead us to miss some interesting and important documents. Furthermore, if we were to search for documents of interest in Hansard using these frequently occurring keywords, we would likely incur many false negatives that are skewed towards the years in which the word counts peaked.

The screenshots shown in Figures 3 and 4 are taken from the HaToRI web app. The tool allows humanists to interact with their corpora in a user-friendly, visual, iterative, and exploratory process and avoids the time and resource penalties that traditional search and close reading of corpora incur. We will walk through the various features of the tool with accompanying screenshots in this next section with an emphasis on how the tool enhances digital humanities research by incorporating some of the values of code pipelines discussed in the introduction.

The tool approaches visual topic exploration at different levels or views: word, document, and topic. Taken together in a single interface, these levels give the user a three-dimensional view of their corpus that is less possible when searching for specific words, documents, or topics in silo. The n-gram plot in Figure 5 shows word frequencies per year for the top six words in a topic. Each line in the plot shows frequencies for a given word and each word in the plot is also shown on the right as a button; they can be removed from the plot by clicking on the words and the plot will dynamically re-scale based on the frequencies present in the current plot window. Words can be added by searching the vocabulary in the search bar. The search results filter down as the user types a query because corpora are often pre-processed differently, resulting in a vocabulary made up of stems or lemmas rather than actual words. The frequencies can be shown in absolute terms or relative, normalized to the total number of words in the year. The word view gives the user a finer-grained view of the corpus, but leaves out the greater context and interaction of the words.

Figures 6 and 7 show topic views of the corpus. The home page shows the first view, where each topic is a word cloud showing the top words in the topic. The size of the word indicates its probability within the topic; bigger words are more likely to be sampled from the topic than smaller words. The page scrolls and loads one chunk of word clouds at a time. The user can filter to a smaller subset of topics by searching for unigrams in the search bar at the top right corner of the page. The alternate view shows a table view of the topics with an additional sort functionality that will sort the topics based on any of the table fields. They can be sorted by their ID, by mean peak year, alphabetically by words, or by their proportion in the corpus. The recommended navigational flow of the corpus exploration part of the tool is to start at one of these two topic overviews and drill down closer into a single topic. Figure 8 shows a detailed topic view. The view includes more top words than the overviews, a plot of the topic over time, a link to the n-gram plot, and a sorted table of documents that are most associated with the topic. The detail view allows the user to explore the topic from various different vantage points, but does not allow multi-topic exploration. All topics are shown in the plot on the left in Figure 9, and they are arranged based on their similarity to each other. Topics with very similar word distributions cluster together and topics with very dissimilar word distributions are very far apart.

Figure 10 shows ranked documents relevant to the five land-related topics shown in Table 2 for z = [0, 100, 500]. For land topics, larger z values produce a better ranking than smaller z values. There are documents from the early and late periods of the century, near the peaks of land-related activity that we expect. Also, three out of four written reports, denoted by the prefix “seed-”, are in the top-ranked documents list when z = 500; we expect the written reports to be ranked highly because the historical literature confirms that they are key documents summarizing land use in the nineteenth-century. The low proportion of topic tokens out of total tokens in the written reports — 0.031, 0.066, and 0.156 for the Bessborough, Richmond, and Devon reports, respectively — indicates that the seed reports contain a lot of noise and shows the weakness of an earlier document-based approach compared to our topic-based approach [Lee et al. 2018] [Tangherlini and Leonard 2013].