1Letters are an essential source for many research fields in the Humanities, including history, literature, musicology, and linguistic studies. In contrast to published works such as books and articles, private letters entice with the possibility of a more intimate and authentic insight into the author’s thoughts, since they belong to those text types that are not usually intended to be published. Besides the (more or less presumed) privacy there is one even more important feature of letters: in contrast to publications, the content of a historical letter (either private or official) generally cannot be anticipated by a modern reader before actually reading it, because the content is extremely heterogeneous. Letters may contain information about a variety of people, publications, events, and issues from the author’s working and living environment. Scholars therefore can make use of letters not only to study a person or a certain event, but also to investigate more systematic research questions, for instance: “How was the music business organized in the nineteenth century?” That is why letters are an important source that are often collected in scholarly editions, which prepare and provide these texts for subsequent research. Furthermore, a letter is usually not a “nomadic single object” (Bunzel 2014, 234), but part of a larger correspondence network, allowing insights into connections and networks between correspondence partners. For scholars, questions emerge such as: “Who is in contact with whom and to what extent?,” “Where do circles of correspondence partners evolve and who are the central players within these circles?”
2For editorial and practical reasons, however, scholarly editions of letters are usually focused on partial correspondences: either they concentrate on a certain person or they deal exclusively with the correspondence between two specific correspondence partners. The motives for this approach are usually straightforward and reasonable, considering methodology and work efficiency. But for the research questions mentioned above that involve a broader examination of correspondences it is often necessary to perform time-consuming searches across various editions of letters. For example, historians might be interested in the reactions of contemporary people to certain historical events. To study them, they would have to examine multiple scholarly editions and other publications. Furthermore, the number of letters that are available in scholarly editions is so large that often even experts cannot keep track of them—let alone the whole amount of letters preserved as archival documents. Thus, the majority of edited letters remain unused and uncited (Bunzel 2013, 117).
3Since the early 2000s the scientific community has been discussing different approaches to solve this problem with the help of information technology. One early attempt was the BRIEFkasten project of the Institut für Textkritik in Heidelberg, which began in 2005.1 The aim of BRIEFkasten was to establish a database for printed or edited letters between 1750 and 2000. Some further projects arose around the research subject “Republic of Letters” (ca. 1550–1750), for example, the database Early Modern Letters Online2 or the project Frühneuzeitliche Ärztebriefe3 (Physicians’ Correspondences from the German-Speaking Territories), which gather metadata from both scholarly editions and archives to achieve better insights into the early modern scientific community. But these databases have some restrictions: first, they are created to address specific research questions and therefore focus on specific time periods, geographic areas, or topics. On the one hand, this is a legitimate approach considering the research questions they address, but on the other hand it excludes letters that might be interesting for other research. Second, these databases do not offer an open, standard-based, and documented way to provide and update data. And finally, the data cannot be retrieved via an open Application Programming Interface (API) under a free license for subsequent use.
4A general problem seems to lie with the conception of a single, centralized database, into which all the letter data have to be imported and where the data must be maintained. Considering the huge numbers of letters and potential research focuses, this concept may not be realistically pursuable. This could be one reason, why the BRIEFkasten project was discontinued after a couple of years. In contrast, an approach where decentralized data is simply aggregated via a standardized interchange format seems more promising (Rapp 2009, 205–6; Herres and Neuhaus 2002, 18)—especially if we would like to allow editors to use their own appropriate data model (e.g., in digital scholarly editions, the specific underlying TEI encoding) for their different purposes. This consideration also led Romanticism specialist Wolfgang Bunzel to request
the creation of a decentralized, preferably open digital platform, based on HTML/XML and operating with minimal TEI standards, which is extensible in different directions and allows for existing web portals and websites to contribute at the lowest possible cost. This doesn’t require some kind of superstructure which covers the entire amount of letters from the Romantic era (the size of which could not be estimated exactly, anyway) but rather an intelligent linking system, which associates existing documents with one another. The creation of such a nexus will naturally lead to research options reaching from searches for persons and places to specific keyword-based searches …
(Bunzel 2013, 123, my translation)
5The web service correspSearch takes a step in this direction, collecting the letter metadata from various editions by using a standardized interchange format and making the data available through open interfaces, based on TEI XML and under a free license. The initiative for this web service arose from the workshop “Editions of Letters from around 1800: Finding and Connecting Interfaces,” which was organized in February 2014 by Anne Baillot and Markus Schnöpf.
6The web service correspSearch aims to help scholars by offering a central place to search for edited letters, and by guiding them to the original publications. Only the metadata is aggregated from letter indexes of scholarly editions. Legal problems which could arise while retrieving full texts can thus be avoided; furthermore, scholarly editions found only in print form can be included without expensive digitization of entire publications. CorrespSearch is conceptually open in the sense that there is no focus on a particular research subject, time period, or region. This allows for new kinds of research questions to be explored.
7The web service correspSearch is designed to aggregate correspondence metadata from digital indexes of letters which are hosted elsewhere on the web. These digital indexes of letters are created and provided by various scholarly edition projects according to a specific TEI XML–based interchange format (see section 3 below) and were registered to the web service with their URL. The metadata provided via these letter indexes is then retrieved by the web service in periodic intervals. Thus, it is very straightforward and easy to add indexes of letters to the web service and update them if necessary. The data not only can be searched via a website, but also can be queried automatically via an Application Programming Interface (API) that is open, well documented, and supports different formats. As the data gathered is provided under free licenses, the web service offers all aggregated data under a CC-BY 4.0 license as well, thus making the data available for further reuse.
8There already exist a variety of digital infrastructures regarding correspondence. The web service correspSearch is therefore designed to fit into the existing infrastructure landscape and complement existing services in a useful way. For information regarding purely archival resources, for example, there are already digital formats and services available, such as the Encoded Archival Description (EAD) format and the German digital union catalogue Kalliope.4 This is why correspSearch focuses more specifically on edited letters—correspondence which has undergone scholarly editing, meaning that there is a published abstract or transcription, maybe even including a commentary. With further developments we envision that the data aggregated by correspSearch can be combined with data from other resources.
9The technologies behind correspSearch are the open source software eXist-db5 and the programming languages XQuery and XSLT.
Figure 1. Technical concept of correspSearch.
- 6 TEI P5: Guidelines for Electronic Text Encoding and Interchange, Appendix C Elements, accessed July (...)
10In order for correspSearch to be able to aggregate the metadata of letters from diverse sources, these metadata have to be provided in a standardized and machine-readable format. The TEI XML format seemed appropriate in this context, because it had been used in digital editions for several years, and has now become a de facto standard. Furthermore, it is possible to rely on the work of the TEI Correspondence Special Interest Group (SIG) which developed the element <correspDesc> (“correspondence description”) for the TEI Guidelines. This extension of TEI provides a tagset for the recording of correspondence-specific metadata within the TEI Header, such as sender, addressee, and place of writing (Stadler, Illetschko, and Seifert 2016; Stadler 2014). After going through a few changes, <correspDesc> was integrated into the TEI Guidelines in April 2015.6
11It therefore seemed straightforward to base the exchange format needed here on TEI XML and to use the new element <correspDesc> in this context (Stadler 2014). In order to allow for the provision of digital indexes of letters, the “Correspondence Metadata Interchange Format” (CMIF) is currently being developed within the TEI Correspondence SIG.7 The essence of a CMI document consists of multiple <correspDesc> elements, each describing one letter. However, the <correspDesc> elements are used in a significantly reduced and restricted manner in order to allow for further automatic processing. For instance, the <correspAction> element may only be of types "sent" and "received" and can only contain the elements <persName>, <placeName>, and <date> as children. In addition, if applicable, each <correspDesc> element may contain a reference to the corresponding digital edition which is available online. Only a few elements from the TEI Header are relevant for the CMIF besides the <correspDesc> element, for example, a bibliographic reference about the corresponding edition of the letters. Using a very reduced and restricted subset of the TEI for the CMIF is necessary in order to enable interoperability without any human intervention (besides registering the URL of the CMI file with correspSearch).8 However, this restriction applies only to the CMI file, but not to the TEI encoding of the texts and their metadata in the scholarly edition itself.
Figure 2. Schematic rendering of the different parts of a <correspDesc> element in the CMI format. The example originates from an example provided by the TEI Correspondence SIG.9
12At this point in the development of the web service, only basic metadata about letters can be recorded: information about sender, addressee, places of writing and receiving, and dates. In the future, it will be possible to record additional metadata such as people or events mentioned in a letter, or whether some references are uncertain, or the type of textual basis of a letter edition (e.g., manuscript or draft). It will then also be possible to browse through these categories in the correspSearch web interface. We also plan to allow the encoding of the corresponding archival resource in CMIF with the help of official URIs from digital archival catalogues (in Germany, for example, from Kalliope). These further developments raise the question whether CMIF should continue to be fully conformant to the TEI Guidelines or if it might be necessary to stray slightly from the TEI in certain aspects. These and other issues regarding the further development of CMIF are discussed in detail in “Perspectives of the Further Development of the Correspondence Metadata Interchange Format (CMIF)” (Dumont 2015).
- 10 On the use of authority-controlled IDs, see Stadler 2012.
- 11 Library of Congress Authorities Help Pages, Frequently Asked Questions, accessed July 31, 2017, htt (...)
- 12 Specification of BEACON link dump format, GitHub, accessed July 31, 2017, https://github.com/gbv/be (...)
13The CMIF specification strongly recommends the usage of IDs from authority files to identify persons and places across projects and publications.10 Authority files are databases which were usually created and are still maintained by cooperative consortia of libraries, often headed by a major or national library, such as the Library of Congress or the German National Library. They are designed to “establish forms of names” and other metadata “used on bibliographic records … to provide uniform access to materials in library catalogs and to provide clear identification of authors and subject headings.”11 For each entity there exists a record with a unique, project-independent and persistent identifier, which can be referenced by everyone and is widely used as a reference in scholarly communities. In the context of correspSearch, the use of authority files avoids the limitations that would be created by referring to specific strings. Names can occur in different variants (e.g., without a certain first name) and vary orthographically. Furthermore, they can be ambiguous (e.g., “John Smith”). Thus, correspSearch searches are based on the IDs from authority files as encoded in the CMI files. The benefit of using authority-controlled IDs was proven by the German scholarly community via the application of the simple BEACON12 format in diverse encyclopedias or digital editions (Baillot and Busch 2016).
- 13 Last updated October 21, 2016, http://www.dnb.de/EN/Standardisierung/GND/gnd_node.html.
- 14 Accessed July 31, 2017, http://authorities.loc.gov/.
- 15 Accessed July 31, 2017, http://www.bnf.fr/fr/professionnels/autorites_bnf/s.autorite_bnf_presentati (...)
- 16 Accessed July 31, 2017, http://id.ndl.go.jp/auth/ndla.
- 17 Accessed July 31, 2017, http://viaf.org/.
14The web service correspSearch supports several authority files for persons. Currently, these are the Integrated Authority File (GND) of the German National Library,13 the Library of Congress Authorities,14 the Autorités de la Bibliothèque Nationale de France (BNF),15 as well as the Web Authorities of the New Diet Library (NDL) in Japan.16 The Virtual International Authority File (VIAF) hosted by OCLC is also supported.17 With the help of VIAF, correspSearch maps IDs from one authority file to the IDs of the others. In the end, it does not matter which one of the five supported authority files is used in a CMI file to identify senders or recipients: the web service automatically looks up all available IDs of a person.
15Using IDs from authority files is not entirely unproblematic. First, in some cases entities may have been recorded multiple times, which means that there are several IDs available for one person. This complicates the selection of the appropriate ID, but orientation can be gained by assessing the validity of the different options: oftentimes, one of the records available is more enriched than the others and should therefore be used. Second, it also happens that entities, especially persons, that are of interest to research in general do not have any record in an authority file. In such cases it is necessary to add new records to an authority file, which usually has to be done by the staff of the maintaining library or organization. Because of limited human resources in libraries, completing this step can take some time after a new record has been suggested. Despite this lack of optimization, authority files remain the best approach currently available to connect information about persons, places, etc., across projects and institutions.18 Especially in the context of semantic web technologies, the usage of standardized unique identifiers (i.e., URIs from authority files) for common entities is crucial in order to ensure the sharing of research data as linked open data. We can expect the maintenance of authority files to be enhanced in the future.
16For the identification of places, the support of GeoNames was implemented in correspSearch. GeoNames is a free licensed database, which consists of over ten million geographic names that are categorized in nine “feature classes” and 645 “feature codes.”19 For each place name, additional data such as geographic coordinates are available. Place names that are not already available in GeoNames can be added by any registered user. If necessary, it is possible to note alternate names for a place, for example, names in other languages or historical names. It is also possible to categorize as a “historical site” a place that no longer exists. Despite these features, the handling of historical places in GeoNames is limited because it is a contemporary gazetteer: that is, it is not possible to record the time period during which a place (or place name) existed. It is a major limitation that the data about state boundaries or the state to which a city belongs only reflect the current situation and have no historical depth whatsoever. This is challenging, but not impossible to address, when it comes to correspondences since the places mentioned in the context of historical correspondence are often names of cities or villages. They may today be part of a larger municipality but remain addressable because they still exist as a separate data record with a geographic point and not with a certain area, which can change over time. All in all, the usage of GeoNames in CMIF and correspSearch up to now has been beneficial and raised only limited issues.
- 20 Carl-Maria-von-Weber-Gesamtausgabe (WeGA), accessed July 31, 2017, http://www.weber-gesamtausgabe.d (...)
- 21 Briefe und Texte aus dem intellektuellen Berlin um 1800, edited by Anne Baillot, accessed July 31, (...)
- 22 Digitale Briefedition Alfred Escher, edited by Jopseh Jung, Zürich 2015. accessed July 31, 2017, ht (...)
- 23 Accessed July 28, 2017, http://www.hin-online.de/.
- 24 PhD diss., Technische Universität Berlin, 2012, published online 2014, http://dx.doi.org/10.14279/d (...)
17As of the beginning of February 2017, correspSearch has gathered the metadata of more than 26,000 letters from almost 110 publications. Letters from digital editions (e.g., Carl Maria von Weber—Collected Works,20 Letters and Texts: Intellectual Berlin around 1800,21 or the project Alfred Escher’s Correspondence22) are not only registered to the web service but also directly available to the user via a link. Additionally, many letters from printed scholarly editions have been recorded in CMIF and are thus available through a digital catalogue for the first time: for example, the letters of the natural scientist and anatomist Samuel Thomas Soemmerring or the correspondence between Karl August Varnhagen von Ense and Friedrich de la Motte Fouqué. Besides scholarly editions, the web service records edited letters that are published in appendixes of monographs or in journal articles. Examples are the open access journal HiN: HiN. Alexander von Humboldt im Netz. International Review for Humboldtian Studies,23 where edited letters of the famous scientist are published periodically for the first time, or the monograph Studien zum geistlichen Werk Otto Nicolais,24 where Klaus Rettinghaus published a number of edited letters in appendix B.
18The aggregated data is growing and will continue to grow in the future; multiple scholarly editions have announced their upcoming contributions. Such participation is crucial: it is only through the willingness of relevant projects to provide correspondence metadata that a web service like correspSearch can be valuable to the scholarly community. Every scholarly project is welcome to provide its metadata in the CMI format. Interested scholars and projects can find step-by-step instructions, documentation of the CMI format, and a FAQ section on the website25 as well as templates and examples in the GitHub repository of the TEI Correspondence SIG.26
- 27 For example, the use of the wrong ID from an authority file.
19Providing a CMI file has multiple advantages for the participating research projects: First, it enables them to make their edited letters more easily available to the community and to connect their editions of letters (automatically—see below) with other scholarly editions. Second, they help to facilitate research that involves a broader examination of correspondences, as mentioned above. Furthermore, by publishing their metadata, they can get useful feedback to correct and refine their metadata, as the experience with CMIF and correspSearch shows so far: more users read and use the metadata and therefore errors can be more easily discovered.27
- 28 For the available output formats and search parameters, see the overview of the correspSearch API, (...)
- 29 Which has been done, for example, by the Deutsche Biographie, edited by Historische Kommission bei (...)
20Researchers—and of course editors working on their own scholarly editions—can search the correspSearch data pool easily via the web interface. The search functionalities are currently restricted to the essential fields, but they will be extended in the future. The aim from the very beginning, however, was to provide the data not only in a human-readable but also in a machine-readable way in order to allow for automated queries and subsequent use. Therefore correspSearch offers multiple application programming interfaces. The APIs support all search parameters available in the web interface.28 The results can be retrieved in several formats. First, correspSearch offers all data as TEI XML in the CMI format. Second, to enable as much reuse as possible, the output is also provided in a JSON serialization. Mixed-content XML elements cannot be well converted into JSON, but such markup almost never occurs in the CMIF. Besides TEI XML and TEI JSON, the web service offers, third, an experimental CSV output, which is intended for individual use of search results (which is why this API is also implemented in the graphical web interface). Finally, a BEACON API enables the cross-linking of search results to a specific person in correspSearch.29
- 30 The visualization is available in an interactive format, accessed July 31, 2017, at http://corresps (...)
21Via the APIs, scholars can exploit the aggregated data using the technologies and software they favor. Therefore, with sufficiently extended data and suitable software it will be possible to perform research on, for instance, social or correspondence networks based on correspSearch. The developers from LAB110, for example, have imported correspSearch data into their visualization tool “nodegoat” to visualize a correspondence network (figure 3).30
Figure 3. A visualization in “nodegoat” from correspondence metadata gathered by correspSearch.
- 31 edition humboldt digital, edited by Ottmar Ette. Berlin: Berlin-Brandenburgische Akademie der Wisse (...)
22Thanks to the API, it is also possible to automatically refer or even link from one digital letter edition to related letters provided by other editions. This function has already been implemented in the edition humboldt digital31 (figure 4). Whenever a user visits a certain letter on the website, the digital edition automatically performs a query via the correspSearch API to find out if there are any other letters from Humboldts’s correspondent (in the pictured case, Samuel Thomas Soemmerring) in other scholarly editions. In the pictured case the results from the correspSearch API offer—among others—a hint to a letter from Georg Christoph Lichtenberg, which was sent to Samuel Thomas Soemmerring two days before.
Figure 4. Screenshot of the digital scholarly edition edition humboldt digital, which presents among other manuscripts some letters to and from Alexander von Humboldt. In the top right corner, links to letters in other scholarly editions are presented with the help of the correspSearch API. Source: http://edition-humboldt.de/H0002729.
23This feature helps researchers avoid methodological problems when interpreting a piece of correspondence: when analyzing a letter they usually consider the preceding and following letters in the correspondence between the sender and addressee as well. Their interpretation often does not include the letters which the correspondents sent to or received from other persons within the same context, although the content of those letters may be highly relevant to the letter they are interested in. With this feature, the broader correspondence context of historical letters can be explored in a greater depth than ever before.
24Besides TEI XML, TEI JSON, and CSV, the correspSearch API also offers a preliminary RDF version of the data by using the web service XTriples developed by the Academy of Sciences, Humanities and Literature in Mainz. With the help of XTriples, the CMIF is converted into RDF statements.32 Providing the aggregated correspondence metadata as RDF enables users to analyze the data with the help of semantic web technologies. Furthermore, the RDF data can be used as Linked Open Data in new contexts and in combination with data from other sources in order to answer questions beyond the correspondence-specific issues (Grüntgens and Schrade 2016). For instance, with data from multiple universities’ catalogues of professors, lecturers, and students, scholars could research whether or not a correspondence network (or part of it) reflects a certain “school.”
- 33 “Digitale Briefeditionen im Semantic Web: Der XTriples Webservice—Semantische Aussagen aus XML,” pr (...)
Figure 5. This diagram, created by Torsten Schrade,33 illustrates how the correspondence metadata aggregated by correspSearch could be used as Linked Open Data (LOD) with information from other resources like authority files (here GeoNames and the Integrated Authority File (GND)). This simple example illustrates the idea of using data from correspSearch with further LOD resources.
- 34 Besides a complete correspondence between Humboldt and one of his correspondents, scholars also pub (...)
- 35 The aim of completeness is still a valid ideal for scholarly editions of letters. The problem that (...)
25Alexander von Humboldt is a good illustration of how a decentralized approach such as the one proposed here can provide a major contribution to research. In the course of his life Alexander von Humboldt wrote about 50,000 letters. Today we know about the content of around 12,400 of them. The number of letters addressed to Humboldt probably exceeded 100,000, even though only 3,400 of those letters are known today (Schwarz 2002, 194–95; Suckow and Schwarz 1998, 119). From the beginning, this extraordinarily large number of documents has deterred Humboldt researchers from planning an edition of Humboldt’s complete letters. They have concentrated instead on providing editions for certain parts of his correspondence.34 The focus on specific correspondents of Humboldt is legitimate from an economical and methodological point of view. However, these selective editions also entail problems (Suckow and Schwartz 1998). First, the choice of criteria applied to guide the text selections is of decisive importance, as well as the choice of whose correspondence to prioritize, since, after all, the aim of completeness will never be achieved, though it still exists in principle.35 Secondly, users of selective editions may lose sight of the fact that individual letters not only form part of a correspondence between two people, but rather are embedded in a much larger network of letters. This is the case in particular when it comes to Humboldt, who wrote various letters to different people at the same time, and often his statements on similar issues or people vary depending on the addressee. Humboldt possessed great diplomatic skills and knew very well that he might risk making enemies had he expressed critical opinions too openly towards the person concerned (Werner 2000, 2).
- 36 edition humboldt digital, edited by Ottmar Ette. Berlin: Berlin-Brandenburgische Akademie der Wisse (...)
26Collecting the edited letters of Humboldt in correspSearch (a task that has not yet been completed) helps overcome a methodological disadvantage of selective editions, since it enables the contextualization of individual letters within the entire correspondence of Humboldt: on the one hand for the editor during the creation of a scholarly edition, on the other hand for the user while exploring and analyzing edited letters. By proceeding like this, the practice of selective editions, which is straightforward with regard to other aspects, may be continued. Therefore, the possibility of automatically connecting letters in scholarly editions with the help of correspSearch, as shown above, was implemented within the edition humboldt digital.36 The connection is especially useful for this edition project because the selection of letters and texts considered for the edition is partially based on thematic categories. In addition, further development of correspSearch will allow for improved accessibility to the various topics of Humboldt’s correspondence.
27However, the web service correspSearch is not only meant to provide renewed access and contextualization for historical persons who are already known. It is also intended to draw attention to less well-known correspondence partners. The aggregation of correspondence metadata from diverse editions enhances the research options and makes it possible to evaluate networks and their presumably central figures. This method not only draws attention to persons who have not previously been taken into consideration by a scholarly edition, but it also facilitates the evaluation of their importance, because it focuses not only on one or two persons but makes it possible to display a whole network of, for example, academics.
- 37 Kritische Ausgabe der Werke von Richard Strauss: Online-Plattform. München: Bayerische Akademie der (...)
- 38 Briefe der Fruchtbringenden Gesellschaft und Beilagen. Die Zeit Fürst Ludwigs von Anhalt-Köthen 161 (...)
- 39 “Participate,” correspSearch, http://correspsearch.net/index.xql?id=participate.
28Especially for editions of letters, the digital age has brought significant progress that allows scholars to address a series of known challenges like incompleteness and contextualization of letters, or the discovery and disclosure of correspondence networks. The web service correspSearch is a building block for improved solutions in this context. We plan to develop correspSearch further in order to enhance its search functionalities (e.g., for persons and publications mentioned in the text) and to make use of the extensions planned for the CMI format. In the course of this development, support for further authority files will be implemented. Besides the search functionalities, we plan to simplify the creation of CMI files through diverse tools. The aggregated data should grow further. Recently, new metadata records have been added: among others, records from the online platform of the critical edition of the works of Richard Strauss Werke37 and from the digital edition Briefe der Fruchtbringenden Gesellschaft und Beilagen.38 The more correspondence metadata is published and may be included in correspSearch, the more useful the web service will become to creators and users of scholarly editions. We welcome additional contributions!39
RSS feed
