This work has received funding within the CHIST-ERA Programme under the following national grant agreements: 1. Spain, PCIN-2017-064, MINECO; 2. Poland, 2017/25/Z/ST6/03045, National Science Centre; 3. Austria, FWF – I 3441 – N33, FWF Austrian Science Fund; and 4. Ireland, Irish Research Council.
1The management and display of uncertainty in computational processes has recently gained interest within the scientific community. As the volumes of data and algorithms at play grow in complexity, many scholars are becoming aware of the importance of communicating uncertainty as a means to enhance the explainability and accountability of computational processes in a wide range of domains (Hullman 2020). As opposed to previous approaches, which mainly focused on the systematic elimination of uncertainty by applying different statistical constructs, modern solutions tend to provide faithful representations of the underlying statistical models with the aim of enabling better comprehension by end users, ultimately allowing them to make more informed decisions based on the reality of the observed data.
2In this regard, the digital humanities (DH) community has approached the problem from various angles, such as the development of data quality metrics for digital cultural collections (Windhager et al. 2019), the quantification and categorization of uncertainty in DH analysis processes (Martín-Rodilla and González-Pérez 2019; Rocha Souza et al. 2019), the improvement of existing strategies for the development of digital research tools and environments for history (Edmond 2019), and from the perspective of confidence (Franke et al. 2019). In the course of this work, manual and automatic annotation have emerged as one of the junctions in DH workflows where uncertainty can be introduced. Despite its importance, the consideration of uncertainty in the annotation task has not been thoroughly addressed in the literature. Thus, this situation has brought the attention of interdisciplinary teams of researchers seeking to understand the implications of bringing uncertainty to the surface of DH projects. This is the case with PROVIDEDH (PROgressive VIsual DEcision-Making in Digital Humanities),1 an EU-funded four-year project that aims to provide DH scholars with an online space to consider uncertainty related to the completeness and evolution of digital research objects in order to enhance the development and corroboration of hypotheses in a wide range of use cases. One of the project’s outcomes is the development of a collaborative platform centered around the collaborative, uncertainty-aware annotation of TEI texts.2
3Fortunately, the identification and tracing of certain types of uncertainty in the life cycle of a textual corpus can be partially achieved with the use of TEI tags (TEI Consortium 2020, ch. 21, Certainty, Precision, and Responsibility).3 However, these methods are not commonly used. The motivation of this paper is to enable a broader landscape of use cases in which these tags can be utilized by exposing them to the end user in a bespoke user interface. In particular, the platform facilitates a use of the <certainty> tag that gives the possibility of annotating the degree of uncertainty with an attribute @cert that allows the assignment of broad categorical values (“high” / “medium” / “low” / “unknown”) or @degree to allow for more granular numerical values. The platform provides a user-friendly interface for annotating entities and uncertainties related to them, so it mainly exploits @cert , which is intended more for humans. However, it also allows users to upload already annotated data sets in which @degree attributes are filled in algorithmically (because it is likely impossible for humans to ascertain a difference between, for instance, 0.67 and 0.68 degrees of confidence).
4The ultimate goal of this implementation of <certainty> and its attributes is to provide, for teams which collaboratively edit texts, an instrument to explicitly document their assumptions and conclusions regarding uncertain passages in their shared text. Alternatively, these tags can assist in making TEI-encoded texts reusable beyond the FAIR (findable, accessible, interoperable, and reusable) criteria for data. This deployment of <certainty> can instead make the data more epistemically available, with the provenance or prior interpretive effort that has been applied to it, integrated into the editing process that shaped it.
5One of the first tasks of the PROVIDEDH project was to describe and to define the various sources of uncertainty in text-based DH research objects, with the aim of compiling a taxonomy of uncertainty for visualization and representation of uncertainty in DH. This taxonomy is now stable, though it remains under development as a living document. In its current iteration, it is divided into two main categories: “user-recognized” and “machine-generated” uncertainty. The former describes the manner in which users perceive and categorize uncertainty in digital texts and is related to previous work in visualization and the humanities (Fisher 1999; Simon, Weber, and Sallak 2018). The latter concerns cases when the annotation (including about uncertainty) is done by an algorithm. For a complete explanation of the motivations behind the proposed taxonomy and its categories we refer the reader to Therón Sánchez et al. (2019). Although initially the taxonomy proposed a closed, fixed set of categories, we are currently departing from this approach because of a series of recent findings resulting from interactions with real users. During these preliminary evaluations, we discovered, for example, that the optimal naming of the categories fluctuates depending on the project’s specific humanities content and its end user’s academic background (e.g., certain users prefer to employ “imprecision” while others refer to the same category as “vagueness”). Furthermore, recent findings are pointing us to think that the number of human-assigned categories (high, medium, low, and unknown) could, in some cases, not be exactly four but rather vary slightly around this figure. Finally, and in order to augment the expressive capabilities of the taxonomy, we are also supporting a flexible (or fuzzy) implementation that allows users to combine two or more categories in a single statement. In summary, although the capture of user-perceived uncertainty in digital interfaces is a very sensitive matter that necessitates implementing important validation strategies, in this paper we report on preliminary strategies at the data format level that we know will be required for our platform to meet the identified user needs.
6Finally, the taxonomy and platform are currently being tested in two separate scenarios: (1) free annotation of texts, and (2) the normalization/unification of entities, which involves the assertion by a human actor, with some degree of certainty, that two or more entities are indeed the same. In our tests, we employed two historical, TEI-encoded data sets that are described in the next section.
7We are aware that marking uncertainty levels and categories is a functionality unlikely to be used by many individual scholars working within current paradigms, and indeed we would imagine many potential users might not take advantage of the affordances of such an option set. Most scholars may just want to mark identifications or readings as uncertain but will not want to qualify this uncertainty further. But our approach emerged within a project and platform context where the convergence of a number of different emerging research paradigms demonstrated a strong case for the inclusion of such values. First, they can play a role in the support of collaborative and indeed distributed scholarship. The characterization of something as uncertain always implies a contextual richness that markup is ill-equipped to capture, in particular as uncertainty is ultimately an epistemic state, and therefore resides in the individual consciousness of the scholar. Where that richer tacit information may not be directly available because, for example, two scholars are working together without regular direct communication, markers of the degree or categories of uncertainty can take on a significant signaling function without needing to capture too much detail. Second, the decision to include these tags emerged within a project that was testing the potential of progressive visualization as a research tool for historians. Considering and assigning degrees and categories of uncertainty may not seem to be a particularly valuable exercise in and of itself, unless or until a tool such as a visualization can allow these attributes of uncertainty to be viewed in a comparative framework and used to weigh alternative explanations, distinguish norms from outliers, and prioritize future efforts.
- 4 Online Depositions Website, Trinity College Library Dublin, accessed December 1, 2020, http://1641 (...)
8The 1641 Depositions (Trinity College Dublin, MSS 809–841)4 are witness testimonies mainly by Protestants, but also by some Catholics, from all social backgrounds, concerning their experiences of the 1641 Irish rebellion. The testimonies document the loss of goods, military activity, and alleged crimes committed by the Irish insurgents, including assault, stripping, imprisonment, and murder. This body of material is unparalleled anywhere in early modern Europe, and provides a unique source of information for the causes and events surrounding the 1641 rebellion and for the social, economic, cultural, religious, and political history of seventeenth-century Ireland, England, and Scotland.
- 5 Rezeptdatenbank der Gastrosophie, History Department, University of Salzburg, accessed December 1, (...)
- 6 Accessed December 1, 2020, http://gastrosophie.sbg.ac.at/salzburg-zu-tisch/.
- 7 Zentrum für Gastrosophie, accessed December 1, 2020, https://www.gastrosophie.at.
- 8 See the ACDH Salzburg Recipes GitHub repository, accessed December 1, 2020, http://github.com/pr (...)
9A second case study used to test our annotator was a data set of historical recipes of the Baroque era from the former area of Austria and beyond.5 The recipe collection was previously established and pertained to the citizen science project Salzburg zu Tisch,6 carried out and led by our cooperation partners, staff members of the Center for Gastrosophie,7 the History Department at the University of Salzburg, Austria. The Gastrosophie recipe collection counts around ten thousand historical recipes from different cookbooks and different authors, with the majority being in nonstandard German and a small number in other languages such as English. During this previous project, the recipes were digitized, entered into a WordPress instance, and annotated with the help of interested citizens. In this project, they the recipes have been converted to TEI8 and collectively further annotated (see figure 2).
10Given the broad spectrum of named entities that a project may require, based on the data set being used and the variation in what different users consider to be a meaningful certainty-category naming, we identified a need to allow the user to specify the entities and uncertainty categories within the project being created in the platform. Such a specification determines what entities and their properties will be available and how users will assess the correctness of the documents. This must be set up before working with the TEI documents, during project creation.
11For this purpose, we provide an interactive interface (see figure 1) that abstracts how the corresponding TEI formatting will be done, and which allows the user to specify both the entities and the taxonomy as well as the colors and icons that will be used across the application to encode this information.
Figure 1. Screenshot of project creation settings.
12Working with existing TEI entities (such as “person,” “place” or “event”) and user-defined ones (such as “ingredient” or “utensil”) is tightly integrated into the proposed system, therefore making the whole process effortless to the user, which in turn makes the platform usable in a broad range of different DH research fields.
13The configuration of the project requires handling the definition of new entities and the creation of a taxonomy. The first is accomplished by using the TEI <object> element with its @type attribute. Custom certainty categories are handled by creating a project-specific taxonomy which is then referenced in annotations. In addition to the name, optional descriptions can be added for each category to help users understand how uncertainty is being conceived and used in their project. Such information results in the creation of a declaration like the following (which varies based on the project settings):
Example 1. Uncertainty taxonomy generated from default settings.
<taxonomy>
<category>
<catDesc>User recognized uncertainty</catDesc>
<category xml:id="ignorance"><catDesc>Ignorance</catDesc></category>
<category xml:id="credibility"><catDesc>Credibility</catDesc></category>
<category xml:id="imprecision"><catDesc>Imprecision</catDesc></category>
<category xml:id="incompleteness"><catDesc>Incompleteness</catDesc></category>
</category>
<category>
<catDesc>Machine generated uncertainty</catDesc>
<category xml:id="algorithmic"><catDesc>Algorithmic</catDesc></category>
</category>
</taxonomy>
14The uncertainty taxonomy generated from project settings is available at https://providedh.ehum.psnc.pl/api/projects/{project-id}/taxonomy/ on the platform and, as we have mentioned, it is referenced in uncertainty annotations. Other settings (such as colors or icons) are stored in the internal platform database. Storing them in TEI files is not necessary as they are only used for visualizations inside the platform.
15After creating a project, the creator can assign other users to the project and give them read or write permissions. The creator starts working on a project by uploading a data set to the platform. The platform supports files in UTF8-encoded TEI P5 format; if a user prefers to start with plain text or use another encoding, the files will be converted to UTF8-encoded TEI P5 during the upload. If TEI files already contain annotated entities, the platform will assign them @xml:id so that they can be referred to in further work. Files changed in this way are stored in the platform as a second version of the uploaded files, so the user can track the history of files. In addition, existing annotations and entities are entered into the database, making them easier to manage.
16In the aforementioned data sets, we identified a large number of cases where uncertainty can be annotated. This implies that there are a large number of users with some amount of prior knowledge of XML editing and the TEI format, and these uses cases provide a good opportunity to make use of interactive interfaces to ease working with data sets and collaborating within the documents.
17We, therefore, designed and integrated into the platform an annotator tool that allows for both the reading of TEI documents in a user-friendly manner and the annotating them using simple text-selection interactions. This interface strongly emphasizes collaborative editing, making use of visual encodings to easily distinguish the authorship of annotations at first glance.
Figure 2. Screenshot of the Annotator. In the right pane, the text is displayed with tags colored according to the type of uncertainty annotated by the users. Changes in the uncertainty tags over time are represented in the chart at the top right. In the left pane, the details for each annotation are shown, including the original text to which the annotation refers, the uncertainty degree and type chosen by the author, and other useful information.
18This collaboration is possible thanks to the use of the TEI format, enabling precise targeting of the source of uncertainty not only regarding the tag name, attributes, or values for entities but also for other people’s annotations.
19The Annotator adds all the changes related to uncertainty to the TEI header. Each user can annotate the same pieces of text in their own way; if we put different users’ tags in the same places in the text, it would make the document quite difficult to read. We decided to place all the uncertainty annotations in <certainty> elements in the header and only refer to them from the text. We further decided not to use other TEI options for tagging uncertain alternative texts, like <choice>, in order to manage uncertainty annotations in a coherent way within the platform. We also considered storing the uncertainty annotations in a <standOff> element, which we use to store entities occurring in the text, but ultimately, we decided to store them together with the annotators in the <profileDesc> element, as shown in example 2.
20Next, we describe in depth the use cases for annotation that we identified, and how they are managed within the platform.
21First, let’s present a modal window that appears after the user selects a text fragment and chooses the “annotate uncertainty” option. It is shown on the left side of figure 3 below.
Figure 3. Screenshots of the window for uncertainty parameters of the text.
22For case A, the user selects “Value” from the “Target” combo box and optionally can select the “Certainty level” and “Categories” (from the defined taxonomy of the project). As a result, the selected text is annotated as a segment and the corresponding uncertainty annotation is added in the <profileDesc> element. Simultaneously, the annotator (the author) is added to the list of annotators, if missing (see example 2).
23Case B is a simple extension of A. The user provides an asserted value in the “Value” text input field, which in their opinion should replace the uncertain piece of text (see the right side of figure 3 and the result in example 2).
Example 2. Sample uncertainty annotations made by two users referring to questionable text.
<profileDesc>
[…]
<textClass>
[…]
<classCode scheme="http://providedh.eu/uncertainty/ns/1.0">
<certainty xml:id="certainty-A" locus="value" cert="unknown" target="#seg-1" resp="#annotator-1"/>
<certainty xml:id="certainty-B" locus="value" cert="medium" ana="https://providedh.ehum.psnc.pl/api/projects/1/taxonomy#credibility" target="#seg-1" resp="#annotator-2" assertedValue="many"/>
</classCode>
</textClass>
[…]
<particDesc>
[…]
<listPerson type="PROVIDEDH Annotators">
<person xml:id="annotator-1">
<persName>
<forename>Michał</forename>
<surname>Kozak</surname>
<email>mkozak@man.poznan.pl</email>
</persName>
</person>
<person xml:id="annotator-2">
<persName>
<forename>Alejandro</forename>
<surname>Benito</surname>
<email>abenito@usal.es</email>
</persName>
</person>
</listPerson>
</particDesc>
</profileDesc>
[…]
<body>
[…] with Donnogh Magwire of Rossbegg in the said County gent, and <seg xml:id="seg-1">a number of</seg> armed Rebells, who with swords drawen […]
</body>
24Use case C, along with the remaining types of annotations, concerns the entity types chosen during project creation. For these types of annotations the Annotator uses not <seg> but <name>. It creates an entity of the provided type, adds it to the list of annotated entities, and annotates the selected text as <name> with a @ref attribute that points to the created entity. This entity list (or rather these lists, because each entity type has its own list) makes it possible to keep track of named entities in a document and spot possible duplications, and allows opportunities for unification and the easy exploration of the corpus. These lists are added to the <standOff> element and are visible to users directly above the text in the Annotator (see figure 2).
25This use case focuses on text annotation without uncertainty, illustrated with the following scenario: a user selects “Ambrose Bedell” in the following TEI document and annotates the name as a person.
Example 3. An excerpt of one of the testimonies from the 1641 Deposition data set.
<body>
[…] <pb n="fol. 105r" pagenum="32"/><p>Ambrose Bedell gent sonn to the late reuerend father in God William Lord Bishop of Kimore in the country of Cavan (by the crueltie of the Rebells deceased sworne and examined deposeth and sayth […] </p> […]
</body>
26The person entity is associated with some predefined properties (predefined subnodes and attributes) that the user can provide. They all are added to the created entity in the dedicated list of annotated entities (in this case people). For instance, from the context the following properties can be extracted: sex, role name, forename, surname, and occupation. Compare the modal window for person properties (figure 4) and the annotation result (person-1 in example 4).
Figure 4. Screenshot of the window for entity properties.
Example 4. Sample lists of annotated people from the 1641 Deposition data set.
<standOff>
[…]
<listPerson type="personList">
<person xml:id="person-1" sex="male">
<persName>
<roleName>Deponent</roleName>
<forename>Ambrose</forename>
<surname>Bedell</surname>
</persName>
<occupation>Bedel</occupation>
</person>
<person xml:id="person-2" sex="male">
<persName>
<roleName>Victim</roleName>
<forename>William</forename>
<surname>Kimore</surname>
</persName>
<occupation>Bishop</occupation>
</person>
<person xml:id="person-3">
<persName>
<roleName>Victim</roleName>
<forename>Mr</forename>
<surname>Cloghy</surname>
</persName>
</person>
</listPerson>
[…]
</standOff>
27In the text, we use <name> to annotate the selected text and @ref to refer to the entity.
Example 5. Sample annotation of an entity name that refers to the entity.
<body>
[…] <pb n="fol. 105r" pagenum="32"/><p><name xml:id="name-1" ref="#person-1">Ambrose Bedell</name> gent sonn to the late reuerend father in God William Lord Bishop of Kimore in the country of Cavan (by the crueltie of the Rebells deceased sworne and examined deposeth and sayth […] </p> […]
</body>
28These two cases are combinations of the previous ones (A, B, and C) with the same modal windows but shown to the user as a wizard.
29When annotating with uncertainty a piece of text which becomes a reference to an entity, the corresponding uncertainty annotation points to the added <name>. This allows the user to express their uncertainty regarding the spelling of the entity name. The user does not express any doubts regarding the type or properties of the entity.
Example 6. Sample uncertainty annotations referring to questionable entity name without and with asserted entity name.
<textClass>
[…]
<classCode scheme="http://providedh.eu/uncertainty/ns/1.0">
<certainty xml:id="certainty-D" locus="value" cert="low" ana="https://providedh.ehum.psnc.pl/api/projects/1/taxonomy#credibility" target="#name-1" resp="#annotator-3"/>
<certainty xml:id="certainty-E" locus="value" cert="medium" ana="https://providedh.ehum.psnc.pl/api/projects/1/taxonomy#ignorance" target="#name-1" resp="#annotator-4" assertedValue="#val-E"/>
<val xml:id="val-E">Ambroze Bedel</val>
</classCode>
</textClass>
30It should be noted that, contrary to example 3, the “asserted value” provided by the user consists of more than one word. However, the schema of @assertedValue attribute does not accept whitespaces. In such cases, we propose using the <val> element and pointing to this phrase in @assertedValue. This is also not fully in line with the TEI Guidelines, which only allows identifiers of <anchor> elements (TEI Consortium 2020, ch. 21, Certainty, Precision, and Responsibility).9 We have, however, submitted a feature request to the TEI-C so that @assertedValue can also take a teidata.pointer value.10 We use this extension in a number of subsequent cases, to annotate with uncertainty that a piece of text is the name of an entity or that two (or more) entities are in fact the same entity. In these particular cases, @assertedValue must be a pointer, as it takes an attribute value of either @ref or @sameAs (see examples example 10 and example 15). Ultimately, the TEI Consortium is expected to introduce the @assertedValueTarget attribute in a future version of the TEI specification for this purpose. And pointers are to be banned in @assertedValue.
31This use case is similar to previous use cases D and E, but an instance of the entity already exists in the text (it exists in the dedicated list of entities and its name is annotated in the text). Another user can select the annotated text and add an uncertainty annotation with their doubts and an optional alternative (asserted) text. Modal windows are again presented in a wizard form.
Figure 5. Screenshots of the wizard windows for adding uncertainty annotation.
Example 7. Another sample uncertainty annotation referring to a questionable entity name.
<textClass>
[…]
<classCode scheme="http://providedh.eu/uncertainty/ns/1.0">
<certainty xml:id="certainty-F" locus="value" cert="high" ana="https://providedh.ehum.psnc.pl/api/projects/1/taxonomy#ignorance https://providedh.ehum.psnc.pl/api/projects/1/taxonomy#incompleteness" target="#name-1" resp="#annotator-2" assertedValue="#val-F"/>
<val xml:id="val-F">Ambrose Beadle</val>
</classCode>
</textClass>
32In this use case, the Annotator can annotate a selected fragment even if it crosses hierarchical boundaries in the underlying markup. We encountered such a case in the data set of historical recipes.
Example 8. An excerpt from one of the recipes in the historical recipes data set.
<body>
[…] Mell vmb, Pache <gap/> sye in butter, das <lb/> sye Zimblich Resch <gap/> Werden, thue daß <lb/> Gehackte in ein Zinnene <gap/> schisßl <gap/>, Gies <gap/> <lb/> ein Coppauner […]
</body>
33Annotating “Zinneneschisßl” as a utensil with a gap inside its name leads us to the following annotations according to use case C:
Example 9. Sample annotation of an entity name divided by a tag.
<body>
[…] Mell vmb, Pache <gap/> sye in butter, das <lb/> sye Zimblich Resch <gap/> Werden, thue daß <lb/> Gehackte in ein <name xml:id="name-2" next="#name-3" ref="utensil-1">Zinnene</name> <gap/> <name xml:id="name-3" prev="#name-2" ref="utensil-1">schisßl</name> <gap/>, Gies <gap/> <lb/> ein Coppauner […]
</body>
[…]
<standOff>
<listObject type="utensilList">
<object type="utensil" xml:id="utensil-1"/>
</listObject>
</standOff>
34If another user would like to express their doubts about the type of this entity, they must proceed as in figure 5 but they must choose “Tag name” in the “Target” combo box. Then the “Asserted value” input text changes to the combo box with defined entity types:
Figure 6. Screenshot of the wizard window for adding uncertainty annotation regarding the entity type.
35The request sent from this modal window is resolved into the following uncertainty annotation:
Example 10. Sample uncertainty annotation referring to questionable entity type.
<textClass>
[…]
<classCode scheme="http://providedh.eu/uncertainty/ns/1.0">
<certainty xml:id="certainty-G" locus="name" cert="high" ana="https://providedh.ehum.psnc.pl/api/projects/1/taxonomy#credibility" target="#name-2 #name-3" match="@ref" resp="#annotator-1" assertedValue="#utensil-1"/>
</classCode>
</textClass>
36As we can see, the “Asserted value” "utensil" is only a request parameter that is resolved in the TEI annotation to the @match attribute with the value "@ref", i.e., the XPath expression that identifies the @ref attributes of two <name> elements. In the TEI annotation, @assertedValue points to "utensil-1" in the list of utensils, but a @locus attribute value of "name" indicates the tag name (i.e., type) of "utensil-1".
37Use case H is similar to G. In this case a user wants to directly assign an asserted value to a property of an existing entity about which they have concerns, or simply to express those concerns. In the wizard they have to choose “Attribute” in the “Target” combo box. Then they will be able to provide the property name and an optional asserted value for this property.
Figure 7. Screenshot of the wizard window for adding uncertainty annotation regarding an entity property.
Example 11. Sample uncertainty annotation referring to questionable entity property.
<textClass>
[…]
<classCode scheme="http://providedh.eu/uncertainty/ns/1.0">
<certainty xml:id="certainty-H" locus="name" cert="medium" target="#person-2" match="persName/roleName" resp="#annotator-2"/>
</classCode>
</textClass>
38Case I is very similar to case C, where we annotate a piece of text which becomes a reference to the entity added to the list. Here we want to annotate that another piece of text refers to the same entity, as in example 12:
Example 12. Sample of a partially annotated fragment of the historical recipe data set.
<body>
[…] Nimbe ein Guetten <name xml:id="name-3" ref="#ingredient-3">Coppaun</name> <gap/> siede <gap/> <lb/> ihme marb <gap/>, nimb die Prust von dem <lb/> Coppaun, hack sye garkhlein, Nimb […]
</body>
[…]
<standOff>
<listObject type="utensilList">
<object type="utensil" xml:id="utensil-1">
<objectIdentifier>
<objectName>Zinnschüssel</objectName>
</objectIdentifier>
</object>
</listObject>
<listObject type="ingredientList">
<object type="ingredient" xml:id="ingredient-3">
<objectIdentifier>
<objectName>Kapaun</objectName>
</objectIdentifier>
</object>
<object type="ingredient" xml:id="ingredient-4">
<objectIdentifier>
<objectName>Zimt</objectName>
</objectIdentifier>
</object>
<object type="ingredient" xml:id="ingredient-5">
<objectIdentifier>
<objectName>Zitronensaft</objectName>
</objectIdentifier>
</object>
</listObject>
</standOff>
39This is like use case C except that a user has to toggle the switch from “This is a new entity” to “This refers to an existing entity” (cf. Figure 4) and then select one of the existing entities from the file. The platform resolves the request by adding another <name> tag referring to the same item on the list:
Example 13. Sample name annotations referring to the same entity.
<body>
[…] Nimbe ein Guetten <name xml:id="name-3" ref="#ingredient-3">Coppaun</name> <gap/> siede <gap/> <lb/> ihme marb <gap/>, nimb die Prust von dem <lb/> <name xml:id="name-5" ref="#ingredient-3">Coppaun</name>, hack sye garkhlein, Nimb […]
</body>
[…]
<standOff>
<listObject type="utensilList">
<object type="utensil" xml:id="utensil-1">
<objectIdentifier>
<objectName>Zinnschüssel</objectName>
</objectIdentifier>
</object>
</listObject>
<listObject type="ingredientList">
<object type="ingredient" xml:id="ingredient-3">
<objectIdentifier>
<objectName>Kapaun</objectName>
</objectIdentifier>
</object>
[…]
</listObject>
</standOff>
40This case is a simple extension of case I, where assigning the <name> to the entity can be annotated with uncertainty.
Example 14. Sample name annotation referring to the same entity with uncertainty.
<textClass>
[…]
<classCode scheme="http://providedh.eu/uncertainty/ns/1.0">
<certainty xml:id="certainty-J" locus="value" cert="high" target="#name-5" match="@ref" resp="#annotator-2" assertedValue="#ingredient-3"/>
</classCode>
</textClass>
41Case K describes entity unification, a process that allows specifying with some degree of uncertainty that two entities may be the same. For this use case, we assume that the data set is already annotated manually or automatically, so there are already lists of entities in each TEI file. In a similar manner as for cases I and J, a user can annotate with some degree of uncertainty that two entities are the same. The difference is that we use @sameAs instead of @ref. For instance when a user unifies a person with the identifier "person-3" from example 4 with a person with the identifier "person-100" from the file "file-1", then the following uncertainty annotation is created in the TEI file with "person-3":
Example 15. Sample annotation that unifies two entities with some degree of uncertainty.
<textClass>
[…]
<classCode scheme="http://providedh.eu/uncertainty/ns/1.0">
<certainty xml:id="certainty-K" locus="value" cert="high" ana="https://providedh.ehum.psnc.pl/api/projects/1/taxonomy#imprecision" target="#person-3" match="@sameAs" resp="#annotator-1" assertedValue="file-1#person-100"/>
</classCode>
</textClass>
42Similar but reflexive annotation is created in the file "file-1". In cases when the same user ("annotator-1") had already unified "person-3" or "person-100" with other entities, then the number of annotations will grow exponentially, taking into account that the "sameAs" relation is transitive.
43Here, it is worth mentioning that these and other annotations are not added directly to TEI files, as we present them above, but they are stored in a database in the form of commits (batches of changes in the project state). These commits are assigned to the file versions they are based on. This way, while rendering the project state to the user, the platform can use XML content from a given file version. In addition, the platform renders the changes from commits associated with this and previous versions, effectively showing to the user the state of the project from any given moment in time.
44Entities and their properties stored in the database are also the source of data for the automatic unification algorithm we deployed on the platform. It employs a neural network and weighted multigraphs to recognize entities appearing in the data set under different identifiers and to suggest unifications when certain similarities are detected. The platform presents each of the proposed unifications together with the context of the occurrence of the entities being unified, where the user can reject or accept it (see figure 8). In case of acceptance, the user must express their confidence about the unification. The Visual Disambiguator learns from users’ actions and subsequent runs of the algorithm take the rejected and accepted unifications into account.
Figure 8. Screenshot of the Visual Disambiguator.
45This last case is all about annotating with uncertainty other uncertainty annotations, a big part of how users can interact with each other in the process of working with TEI files. This enables a process of collaborative disambiguation and editing that starts with a user selecting an uncertain fragment from the list of annotations and choosing “Annotate this.” After filling uncertainty parameters (“Certainty level” and “Categories”), the following annotation is created:
Example 16. Sample uncertainty annotation referring to another uncertainty annotation.
<textClass>
[…]
<classCode scheme="http://providedh.eu/uncertainty/ns/1.0">
<certainty xml:id="certainty-H" locus="name" cert="medium" target="#person-2" match="persName/roleName" resp="#annotator-2" assertedValue="Perpetrator"/>
<certainty xml:id="certainty-L" locus="name" cert="low" ana="https://providedh.ehum.psnc.pl/api/projects/1/taxonomy#incompleteness" target="#certainty-H" resp="#annotator-1"/>
</classCode>
</textClass>
46Identifying and tracing uncertainty sources and types is an important part of the task of communicating uncertainty in DH research objects, and a process in which the use of the TEI standard can be of great help. Therefore, we identified the need for promoting the use of TEI encoding and providing the tools (both formal and technological) to describe and manage uncertainty throughout the lifecycle of a project. In this paper, we have also presented our approach to creating a taxonomy, the data sets used for its testing, and a collaborative platform that incorporates tools for working with the TEI standard with an emphasis on making uncertainty more present and facilitating collaborative work within the project. We also presented and described how TEI can be used to approach a broad spectrum of use cases where uncertainty can be specified, and how the specification of uncertainty can be modeled in TEI.
- 11 Renato Souza, food network project in the ACDH-DH “exploration space,” GitHub repository, accessed (...)
47Finally, another branch of the PROVIDEDH project developed by the Austrian Centre for Digital Humanities and Cultural Heritage is based on annotations of such aspects of historical recipes as ingredients, preparation time, and spiciness. Recipe similarities are calculated based on the co-occurrence of ingredients, their quantities, and their associations. Ingredient unifications are made with the use of mapping tables, where simple and canonical forms of ingredients are preferred rather than fancy mentions like “crisp potatoes,” or “fresh meat.” The project aims to investigate the similarities and interactions between cuisines and, therefore, cultures.11 This is a good example of using entity annotations and their unifications. Another example of a software system that manages uncertainty (about the location of toponyms in North Africa as they appear in historical sources of medieval and modern times) is described by Martín-Rodilla, Pereira-Fariña, and González-Pérez (2019).
SVN keywords: $Id: jtei-provideh-201-source.xml 1136 2022-09-09 16:22:20Z ron $