1. Project Overview

1The German writer Theodor Fontane (1819–1898) produced many novels, poems, essays, and other types of texts, including the notebooks which we are currently editing (Radecke 2010, Radecke 2013). Fontane’s notebooks constitute one of the last bodies of his manuscript materials that is still not entirely published; only a few excerpts have been published, and these do not comply with standards of textual criticism and editing (Radecke 2010). From 1859 until the end of the 1880s, Fontane filled notebooks with a mass of miscellaneous materials, such as diary entries, letter drafts, plans for poetic writing, prose sketches and drafts, lecture notes, drafts for theater and arts reviews, book excerpts, and also notes and sketches accumulated on his journeys. A total of 67 of these notebooks are known to exist, most of them measuring approximately 10 × 17 cm and containing between 64 and 180 leaves. Some pages have been left blank, so we are dealing with less than 10,000 pages with writing on them. Fontane’s handwriting is often hard to decipher, which means that we need to put a lot of effort into transcribing it. Once published, though, this transcription will be a considerable benefit to the reader.

3All of the examples in this paper use data from one of the 67 notebooks (shelf mark C07). C07 contains some of the notes taken during or shortly after Fontane’s second journey to the German regions of Thuringia and Franconia in the summer of 1873. Fontane visited several different places there, including the battlefields of the Napoleonic Wars of the early nineteenth century, and sites related to the sixteenth-century Protestant reformer Martin Luther. Some notes in C07 refer to a book about Thuringia that Fontane planned to write, but apparently never did (Wüsten 1973).

2. Encoding References to Entities in Theodor Fontane’s Notebooks

4Obviously, a notebook about Thuringia and its history is bound to contain many references to entities, primarily places and persons, and also dates. In order for these references to be analyzed, they first had to be verified by the philologists and then tagged as references to entities in our TEI code. Despite the progress in automatic Named Entity Recognition technology (Wettlaufer and Thotempudi 2013), it turned out to be less useful for our purposes as the dataset we are dealing with is relatively small and we aim for high precision in identifying references to entities, including indirect references such as pronouns. Therefore we are identifying and tagging all references manually. For notebook C07 we have already completed this task, using the TEI element <rs> (referencing string) to point out words in the notebooks which refer to entities.

Example 1. Example <rs>.

<line>5. <seg><rs type="direct" ref="#Luther">Luther</rs></seg> tritt als Mönch</line>
<line>in das <seg><rs type="direct" ref="#Kloster_EF">Auguſtinerkloſter</rs></seg></line>

5Reference attributes (@ref) point to nodes located elsewhere in the TEI dataset. It should be noted that the organization of the TEI dataset and the location of the entity notes therein is of no importance to the reference linking mechanism described here. At the current stage of our project, the TEI code for each of the 67 notebooks is stored in its own TEI document, with the entity indexes stored within each TEI header. However, combining the 67 TEI documents into one large document with the entity information stored in a single header, or placing all entity data in a separate TEI document altogether, would be just as feasible. In the entity data nodes, additional information is provided on the entities, such as hyperlinks to authority file records, or a classification into person or place.

Example 2. Example of <person> and <place>.

<sourceDesc>
  <listPerson>
    <!-- ... -->
    <person xml:id="Luther">
      <idno type="GND">118575449</idno>
    </person>
    <!-- ... -->
  </listPerson>
  <listPlace>
    <!-- ... -->
    <place xml:id="Erfurt">
      <idno type="GeoNames">2929670</idno>
      <place xml:id="Kloster_EF">
        <idno type="OpenStreetMap">164587492</idno>
      </place>
    </place>
    <!-- ... -->
  </listPlace>
  <!-- ... -->
</sourceDesc>

6We will show further entity types in a later example. In the case of the authority files, the pointers from the TEI code can be either simple URIs (or just authority file identifiers stored in <idno> elements from which URIs can be easily built), or qualified hyperlinks (e.g., in <relation> elements) using a vocabulary such as CIDOC CRM (Le Boeuf et al. 2013) in order to express relations which may support a Linked Data structure. The processing of these links to authority records as described in this article works in either case, as long as valid URIs are provided. The philologists validate the entries, and if there is an error within the hyperlinked authority data we will store corrected values or extended information in our TEI dataset. Chronological references are encoded in a simpler way, using the <date> element and a direct normalization within attributes of the att.datable class.

7Our method of encoding references to entities is fairly similar to that employed in other TEI editions. For instance, in the edition of William Godwin’s Diary (Myers, O’Shaughnessy, and Philp 2010), the more specific elements like <persName> and <placeName> are used instead of the <rs> element. In the Godwin edition, a @ref attribute points to an HTML website, which however is in most cases based on a TEI document consisting of a <person> element. Within this element, the normalized name of the person and biographical information are provided. Many other TEI projects use entity referencing mechanisms like this, even though the elements, attributes, and attribute values may vary. In many cases, tools for entity analysis and visualization can be applied to different data sources with minimal adaptation effort, so no standardization of the respective TEI source code is required. A problem we did encounter is that some TEI projects do not provide direct access to their XML files, which makes them harder to process automatically.

3. Semantic Analysis

8Many TEI edition projects already encode references to entities, but semantic analysis entails far more than just knowing which entities have been mentioned by an author. Semantic analysis as we understand it is a methodological approach that builds upon such tagged entities. Applications query these entity data, aggregate them, possibly enrich them with or link them to external data, perform calculations on them (e.g., sorting algorithms), and generate different kinds of output. This output can be in the shape of text, images, moving images, or potentially any other medium, and can be either simple and concise or complex and rich, but may provide new insights into the data present in the TEI-encoded material. This kind of analysis shifts the focus from the written signifiers to the actual signified entities, and tries to make claims about the mentioned persons, places, dates, works, events and other entities, and their interrelations. This in turn allows conclusions about the TEI-encoded work in which these are referenced, as we show in the examples below.

10The semantic analysis methods described here should not be confused with Linked Data processing (Berners-Lee 2006), which relies on explicitly qualified relations between entities (typically in the form of RDF triples). Such data can be derived from the TEI data discussed here, but no Linked Data as such is present in the datasets used in our project.

3.1 Timeline

12The SIMILE Timeline widget consists of an HTML document that uses JavaScript to process an XML file written in a simple XML markup language specific to SIMILE.

Example 3. Example of SIMILE XML.

<data>
  <event durationEvent="true" title="Luther, Martin" start="1483" end="1546"/>
  <!-- ... -->
</data>

14The resulting timeline (figure 1) consists of two bands, the lower one aggregating the upper one on a larger scale. In the upper band, light blue bars indicate that the date of birth (that is, the beginning of the bar) is an estimate, and only the death date (the end of the bar) could be fetched from the GND record. Our timeline begins in the fifth century CE, when Thuringia was under Frankish rule and the Frankish king Merowech defended Thuringia against the attacks of Attila the Hun. In the Middle Ages we see the long line of Thuringian landgraves (most of them called either Ludwig or Friedrich) who ruled Thuringia until the year 1440 when it became part of Saxony. The timeline ends in the early nineteenth century with Napoleon and other military commanders who fought in the battles of Jena and Auerstedt during the Napoleonic Wars. This timeline enables us to tell which historic periods are covered in the notebook, if we assume that the mentioned persons are a suitable indicator for that. We can also see that, at least in this notebook, Fontane did not mention any of his contemporaries.

15It can be very interesting to compare the timeline of one of Fontane’s notebooks to a timeline created from a different TEI data source. For this purpose we again turn to the edition of William Godwin’s Diary, as it resembles Fontane’s notebooks in both the time of creation and the nature of the material. Of course, a diary is a different medium than a notebook, but both contain a sufficient number of references to person entities that we can display on a timeline. To narrow the material down, we selected a single year’s worth of diary entries, for the year 1835, which is the last complete year within the scope of Godwin’s diaries. The XSLT code to create the timeline has to be adjusted to the Godwin TEI code, though only slightly. The main difference is that birth and death dates are already contained within the <person> elements, and do not need to be retrieved from elsewhere. Again, a filter was applied for missing birth and death dates. Overall, this XSL document is even shorter and simpler at only 75 lines.

16The resulting timeline (figure 2) shows us many people who were all either alive in 1835, or recently deceased, such as a John Curran. (Godwin refers in his diary to the removal of Curran’s body from London to Ireland at the time of writing.)

17The comparison of the two timelines shows a marked difference: while Fontane exclusively mentions historical figures in his notebook C07, Godwin’s diary entries of 1835 are concerned with the present, as can be expected from a diary. The advantage of this method is that it may give an overview of some aspects of the content of large amounts of textual data in a short time, without being prone to the bias of human annotators and indexers.

3.2 Geospatial-Temporal Data Aggregation

18The referencing of places within our encoded documents is also realized with <rs> and a corresponding node within the <teiHeader> connected via @ref. Sixty-eight different places are mentioned in notebook C07.

21To find dates corresponding to the named places, we selected an interval from eight preceding or following elements starting from the matching <rs> of the place. We prefer the following dates if both the nearest preceding and the nearest following element are at an equal distance. The idea behind this matching criterion is that the proximity of place and time references in the notebooks suggests a semantic link. Ideally, they describe where and when one single event took place. The chosen interval of up to eight steps was determined by trial and error and yields the highest number of meaningful matches. In the future, users should be able to specify this interval as well as the priority of the date, if there is one with the same distance left and right from an entity. In the case of notebook C07, 24 items appear in the Geo-Browser’s table. The Geo-Browser only displays 17 of our 24 items because the others refer to more abstract terms like the Holy Roman Empire (“HRR”) or Thuringia, and the authority files we use are not able to provide the required data, especially not for the desired time.

22The dates returned by this algorithm are also used to select a background map provided by the Geo-Browser. The arithmetic mean of the temporal data in this example is 1451.25 CE and 11 out of 17 dates with spatial reference specify the early sixteenth century. A suitable background map with the borders from 1492 can be selected via parameter (&currentStatus=mapChanged=Historical+Map+of+1492). Another possible solution is to use the information about the notebook’s date of creation, which can be found on the book cover in most cases. Then the background map of 1880 is the best selection for notebook C07.

23Placenames corresponding to a selection in the timeline are displayed in a table below, where backlinks to the edition can be placed as well as any additional information. The table also includes the terms and values we are not able to show in the map, like the name “HRR” we described above.

3.3 Network Analysis

25The transformation script matches all entities and generates the required documents. The first document is the HTML file, which contains the needed JavaScript and a reference to the external D3.js library. The second is a JSON file, which contains one object per entity and one associated array per object that includes a list of connected entities. The tree-like structure of XML allows the transformation of any document to a network graph by selecting elements that share the same ancestor. The only requirement is that the XML input consists of at least two elements. For example, the <surface> element (which is used in the Fontane TEI data to encode a notebook page) is a common ancestor of the entities. This produces a network of co-occurrences; the (undirected) edges mean that the connected nodes are both descendants of a <surface>; both appear on one page, if the condition is defined to match only the <surface> elements that are direct children of <sourceDoc>, because more than one surface may be part of a single page, for example where there are glued-in newspaper articles.

26As in the Geo-Browser example above, we assume that the proximity of two references to entities suggests a semantic connection. Naturally, such a connection may also exist between two entity occurrences separated by a page break. Therefore, better criteria for connectedness could be proposed, such as co-occurrence within one sentence, but this requires linguistic markup which is not part of our notebook edition. The notebook page as a unit of semantic coherence is still a relatively meaningful criterion, and the most feasible due to our usage of the <surface> element.

27The Force-Directed Graph algorithm creates a network starting from randomized positions of the nodes and applies a weight for a single node and a link strength. Based on these values, a network is rendered. The result for C07 (figure 4) is a reliable network that shows Thuringia in a centered position with the most edges; it is the most frequently occurring entity and it is mentioned on several pages together with the connected ones. The other places with many links are Erfurt, Weimar, and Kapellendorf. Erfurt and Weimar were the cultural centers of Thuringia; Kapellendorf is a village where the last of the battles of Jena and Auerstedt during the fourth part of the Napoleonic Wars (War of the Fourth Coalition) took place. There is a part of the network where ethnic groups (green) appear together, which represents a page of notebook C07 (6 recto) on which Fontane describes the Thuringians in opposition to their neighbors, the Franks, Cherusci, Saxons, and others. This page’s headline is “Thüringens Geschichte” (History of Thuringia), which is also the topic of the following pages. The benefit of the network is that a major topic can be identified with a single view.

28The output of this D3.js application is an SVG graphic which can be further transformed. <svg:title> elements are used to store the node names, which modern browsers should display on mouseover. To get a better overview of the entities in the notebook, the node names should actually be inserted as nodes, but since there is not much space available in the Force-Directed Graph, a different design might be a better choice. The Hierarchical Edge Bundling example (figure 5) provides a circular layout with the nodes in alphabetical order on each level of hierarchy. Again, the hierarchy is based on the <surface> element, on which the attribute @n determines the level and will be expressed as the first part of the object name within the JSON file. In our case this attribute contains the leaf number with a letter "r" for recto and "v" for a verso side and this attribute is transformed in a html:id, so we can go back from a single entity to the leaf of its first occurrence by generating a hyperlink with the help of JavaScript. If this part is left out, the objects will be sorted in alphabetical order and the network will contain more edges to link those entities that co-occur on one page. Applying the hierarchy allows these edges to be deleted, because the categorization lets the nodes appear together and a bigger gap between the categories marks the border. This is one of the rare cases in which adding more information to a visualization simplifies and refines the output at the same time.

29The result is an interactive graphic in which the appropriate edges are highlighted when the cursor is placed over a node. If one selects the node “Luther,” all links and nodes that appear together with a reference to Martin Luther on a page will be highlighted. When one does this, the items within the first and third clusters change their color to red. Thus we get the same information as in the Force-Directed Graph: the topic of Martin Luther is confined to the first part of notebook C07, while Thuringia is the central topic of the rest of the document.

30Both networks show an outlier. The personal entity of Lucas Cranach, a German Renaissance painter and a good friend of Luther and his wife, is located outside of the network. He appears as the only entity on one page. Furthermore his name is the only inscription on this page at all and it is followed by three blank pages. One possible way to integrate this entity into the network might be the use of another apportionment, as the fact that two entities are referenced on the same page may be regarded as artificial. Only minor changes would have to be made to the XSLT code to use other divisions of text, such as chapters. As notebooks are not typically organized in chapters, we can use paragraphs (encoded with <milestone> here) instead. Instead of one specific element, a distinctive number of blank surfaces in series can be interpreted as a marker between two sections. Based on this information, we can build a different network. This is experimental and aimed at defining the best clusters according to the purpose of our examination.

31The network graphs above take advantage of the precisely marked up text and focus on all named entities. We seem to take these entities out of their contexts, but actually we make their contexts more clearly visible as we group them together where Fontane has written about them on the same page in his notebook. The networks still represent every surface, or represent the notebook, but the visualizations just pay attention to entities. It would not be difficult to apply these scripts to any other element to aggregate or summarize given information with the option to browse to the respective lines of text where the entity occurs. As our dataset grows, we might be able to categorize the notebooks or to distinguish literary manuscripts from other notes. At some point we will be able to apply methods from network theory, for example to measure the centrality of nodes and compare the values from different networks for a single entity. This could provide insights into the structure of the analysed texts, e.g., regarding the identification of topics.

4. Conclusion

32All the example applications we have presented in this article build upon existing tools and services. The additional effort required to make them work with the TEI data from the forthcoming edition of Theodor Fontane’s notebooks (and from the edition of William Godwin’s Diary) was minimal. The necessary scripting (mainly XSLT) and code customization were easily carried out in addition to our regular work within the Fontane edition project. These efforts were facilitated by a spirit of openness shared by all parties involved: both the D3.js library and the SIMILE Timeline widget are open-source software released under a BSD license; the data sources GND, GeoNames, and OpenStreetMap have permissive licenses—Creative Commons Zero (CC0), Creative Commons Attribution (CC BY), and Open Data Commons Open Database License (ODbL), respectively; and the data from William Godwin’s Diary is released under a Creative Commons Attribution-NonCommercial license (CC BY-NC) and, just as importantly, is offered directly in TEI/XML.

33The benefits of such an approach are undeniable: it enables researchers who are concerned with the meaning within textual material to explore questions that would otherwise be difficult and/or tedious to tackle, thus fulfilling the promise of the digital humanities. Therefore, we wonder why this kind of semantic exploration is not applied more often within the TEI community (or the digital humanities community as a whole). The lack of similar work is particularly regrettable because the true power of this approach would become apparent if there were more results to compare, so that a scholarly dialogue might ensue. Likewise, the availability of more datasets would increase the validity of the analyses performed on them, while more available tools would increase the possible research problems that could be examined through semantic investigation. Furthermore, this approach is a possible answer to the call for markup analysis that Fotis Jannidis started at a panel discussion during the DH2012 conference (Bauman et al. 2012). The work of scholars in the field of literature is more and more digital, but rather than using the richly encoded TEI document, plain text is still the most common format for text analysis.

34The underlying question here is: who is responsible for carrying out the required work to develop, maintain, and customize tools for semantic exploration? It is naïve to believe that ready-to-use applications would emerge from “the Web” or “the community” on their own. Rather, all practitioners in the field ought to ask themselves what their contribution to this chain of development, use, sharing, and re-use could be. Instead of waiting for someone else to develop a generic tool that fits all purposes, anyone can make a small effort towards such a development by providing interchangeable data, using common standards, and building on pre-existing work. And with the philosophy of open source and sharing ideas in mind, we provide all our scripts as well as interactive examples at http://fontane-nb.dariah.eu/​tei-conf/​.