Navigation – Plan du site

AccueilNuméros38Stylistique et traductologie de c...Giving a translator’s touch to th...

Stylistique et traductologie de corpus

Giving a translator’s touch to the machine: Reproducing translator style in literary machine translation

Harun Dallı, Olgun Dursun, Tunga Güngör, Sabri Gürses, Ena Hodzik, Mehmet Şahin et Zeynep Yirmibeşoğlu

Résumés

Cette étude interdisciplinaire examine dans quelle mesure un système de traduction automatique peut reproduire le style d’un traducteur, s’il est formé à un corpus de traductions existantes du même traducteur. La méthodologie d’analyse stylistique de Leech et Short (2007) a été adaptée pour déterminer les spécificités stylistiques d’une traductrice turque majeure, Nihal Yeğinobalı. L’analyse qualitative des données fut réalisée via une lecture détaillée de traduction, et l’analyse quantitative sur des mots-clés et des données morphologiques via des outils d’analyse de corpus et Python. En comparant le corpus de Nihal Yeğinobalı et un corpus de référence représentatif de la traduction littéraire turque de la même période, des spécificités stylistiques furent observées aux niveaux lexical et morphologique. Une analyse d’attribution d’auteur montra alors que les spécificités stylistiques peuvent aider à distinguer le corpus de Nihal Yeğinobalı du corpus de référence. Le corpus de Nihal Yeğinobalı fut ensuite utilisé pour former et tester un ensemble de modèles de traduction automatique pour obtenir un modèle affiné au corpus de la traductrice, qui fut comparé avec un modèle de traduction automatique préformé au domaine général. L’analyse d’attribution d’auteur du résultat de la traduction automatique a distingué le modèle de traduction automatique affiné, l’attribuant à Nihal Yeğinobalı, du modèle préformé, le classifiant comme référence. L’analyse des mots-clés du modèle de traduction affiné montre un haut degré de similarité au niveau lexical entre le modèle de traduction automatique affiné et Nihal Yeğinobalı. Ces conclusions apportent des preuves de transferts de styles, suggérant qu’un système de traduction automatique peut reproduire le style d’un traducteur une fois formé sur un corpus de ses traductions.

Haut de page

Texte intégral

1. Introduction

1Recent developments in large language models have increased the visibility of machine translation (MT) as it has started to aim for more challenging texts. Literature, an area that machine translation once had almost no use in, has become an object of research for both Translation Studies and Computer Engineering scholars investigating the extent to which technology can be integrated into the translation process. Studies have so far tested the quality of literary translation using domain-adapted MT models as well as freely available online MT tools, usually based on scores calculated by automatic metrics such as the BLEU (Bilingual Evaluation Understudy) metric (e.g., Toral & Way, 2015; 2018). BLEU is an algorithm for evaluating the quality of a text which has been machine-translated from one natural language into another. Such metrics take into account a reference translation usually produced by a human translator and compare the MT output against this reference translation. The scores are higher for outputs that are structurally closer to the human translation. Yet, as can be observed clearly in retranslations, literary translation allows more room for different wordings or sentence structures. Consequently, such automatic evaluation metrics can miss some important aspects of literary translation, including style.

  • 1 This research is funded by the Scientific and Technological Research Council of Türkiye (TÜBİTAK) (...)

2In this study,1 we focus on style in machine translation and examine the traces of a human translator’s style in the outputs generated by an MT engine trained with translations of the human translator in question. We follow Saldanha’s (2011) conceptualization of translator style as a consistent configuration of distinctive characteristics that are identifiable across multiple translations, and which exhibit a discernible impetus not explicable solely in terms of authorial style or linguistic limitations. Since translator style is considered independent of the original author’s style, the present methodology will rely on target text analysis only, in line with Baker’s methodology (2000). In adopting this definition and methodology, the present study explores translator style as an existential factor, foundational to authenticity and the cultivation of a unique voice in translated texts. If trained to produce a specific translator’s style, MT might be endowed with a personalized imprint, ensuring consistency in linguistic features throughout the translated content, while potentially facilitating reader engagement, fostering a sense of familiarity, and contributing to the branding of the translator’s reputation and cultural heritage.

1.1. Style in corpus-based translation studies

  • 2 This constitutes the ratio of different words (types) to the total number of words (tokens) in a (...)

3The use of corpus tools has provided translation studies researchers with an insight into patterns of stylistic choices rather than an analysis of isolated examples of choices. For example, type-token ratio,2 word and sentence lengths, frequency of morphemes and lexical categories, etc., have been analyzed as indicators of vocabulary richness and lexical and syntactic complexity (Baker, 2000; Bosseaux, 2001; Frankenberg-Garcia, 2022; Li et al.,, 2011; Saldanha, 2011; Vajn, 2009). Patterns are determined by comparing features in a translation corpus and a reference corpus representative of more general linguistic trends (e.g., see Baker, 2000). Baker’s (2000) methodology is based on an analysis of the target text, although other approaches also take into account the role of the original author’s style (Boase-Beier, 2011; Malmkjær, 2003; Munday, 2008; Saldanha, 2011; 2014).

4Other studies (e.g., Olohan, 2004; Mastropierro, 2018; Mikhailov, 2021) have employed keyword analysis, which is based on the keyword function of a corpus tool listing all keywords (i.e., lexical items) used in a translation corpus considered against the same type of information obtained from a more general reference corpus of multiple translators and/or non-translated works. Keywords are not merely the most frequently employed items; they can also constitute rare or specialized vocabulary. Importantly, their concordance (i.e., use in context) is examined to discern their function on grammatical, semantic, and discourse levels. Frankenberg-Garcia (2022) conducted a quantitative analysis of keywords on grammatical and lexical level (further analyzed based on word class) in a corpus of translations from Portuguese into English. A qualitative analysis of concordance was conducted subsequently to examine the use of particular word classes given the wider discourse context (including features like proper nouns, foreign words, register, etc.). A comparison of the (human) translations in the corpus with machine translations of the same source texts revealed significant differences in the use of keywords, including higher level of lexical consistency in line with register, particularly in the translation of proper nouns and foreign words, as well as a higher degree of explicitation in the human translations. The author attributes this to translators being strategically target oriented for the purpose of achieving better comprehension among their readers.

1.2. Style in literary machine translation

5Literary machine translation has attracted researchers in the last decade, starting with studies by Besacier (2014) and Besacier and Schwartz (2015). The former study has explored the contribution of a domain-adapted MT model to the translation of literary texts which have not yet been translated into a particular language, while the latter has used the MOSES statistical MT tool to train a model with 25 million sentences as out-of-domain data. The output of this MT model was post-edited by readers and professional translators. The researchers found that post-editing took about half of the time required for translation from scratch and that the output was acceptable for readers but not very satisfying for professional translators. For the English-Turkish language pair, Şahin and Dungan (2014) investigated the use of Google Translate in the statistical machine translation (SMT) paradigm for different text genres, including literary texts. Toral and Way have tested the capability of both statistical MT (2015) and neural MT (NMT) (2018) for literary translation using domain-adapted systems. The researchers found a significant difference between the two, NMT with better results, and highlighted the importance of the amount of data needed for NMT models to increase the quality. Sluyter-Gäthje et. al. (2018) have also emphasized that the use of literary data for MT training, in the case of English-German literary translation, for both SMT and NMT would generate better results. The number of studies on the use of MT in the literary domain has been increasing as MT has already proven successful and effective in domains such as technical and general texts, but those studies mainly have focused on performance, usually measured with the help of automatic metrics.

6Style in literary machine translation, on the other hand, has been approached only peripherally in the last few years. For instance, creativity in machine-translated literary texts was a topic of interest for Şahin and Gürses (2019) who investigated whether using Google Translate (GT) for the translation of an English novel into Turkish would limit novice translators’ creativity. The researchers provided some evidence regarding the negative impact of using MT in literary translation. Kuzman et al (2019) and Matusov (2019) conducted studies on the comparison of GT with customized NMT models for literary translation. The NMT model by Kuzman et al (2019) could not outperform GT but their fine-tuned models produced better results than their base models, as in Matusov (2019). Creativity in MT was studied further by Guerberof and Toral (2020), who found out that creativity is only visible with human intervention in the translation process. In their study, the researchers used an MT engine customized for literary texts for the English-Catalan language pair.

  • 3 The possibility of using MT to reproduce a translator’s style.

7There are also few studies focusing on a particular translator or a particular genre in MT research. Kenny and Winters (2020) focused on the extent to which translator’s stylistic features can be maintained in post-editing literary machine translation completed by DeepL. The researchers asked a prominent translator to post-edit the machine-translated versions of his previous translations and found that such a workflow diminishes his voice as a translator. Another example is Hansen et al., (2022), who found some evidence of a customized English-French MT system generating translations with comparable levels of complexity to a human English-French translator, despite the lower quality of the MT generated translations. However, the complexity parameters analyzed in this study were general statistically measurable ones that did not take into account the particular features of the translator’s style. There have been very few studies on the affordances3 of an MT engine for reproducing a human translator’s style, such as Michel and Neubig (2018) and Wang et. al (2021), who looked at non-literary texts.

1.3. Style in (literary translation into) Turkish

8The study of style is fairly recent in Turkish literature. The late westernization of literature in the 19th century and the language reform in 20th century resulted in radical transformations in literary style and the conception of style. Today it is accepted that Leo Spitzer, who worked at Istanbul University for a short period between 1933-35, promoted the study of Turkish stylistics. A further inspiration was Erich Auerbach, who worked at the same university between 1936-47 and wrote his classical work of criticism and stylistic analysis, Mimesis: The Representation of Reality in Western Literature. Spitzer’s and Auerbach’s students and colleagues conducted the first studies on individual literary styles of Turkish authors. Of importance to stylistic analysis are also Süheyla Bayrav, Berna Moran, and Tahsin Yücel, who focused on Western and contemporary Turkish literature, as well as Kaya Bilgegil, Şerif Aktaş, Kazım Yetiş, and Ahmet Çoban’s works on old Turkish-Ottoman literature and rhetoric. Still, it is difficult to say that Turkish literary criticism during the 20th century developed a theory or perspective on style. Critics mention individual styles of authors, but they mostly rely on their impressions. More recently, style in translation has been a subject matter of several studies on Turkish (Alkım, 2002; Dinçel, 2010; Durmuş, 2021) and there have been attempts to create digital corpora of Turkish literary language (Aksan et al.,, 2012). The present study constitutes a novel attempt at computational stylistic analysis of a literary translator and a recreation of her style.

1.4. Present study

9The purpose of this study is to conduct a quantitative and qualitative analysis of stylistic features in the complete corpus of English-Turkish translations by the literary translator Nihal Yeğinobalı (NY). An existing methodology of stylistic analysis in English (Leech & Short, 2007) was adapted and applied to our analysis of translator style. The latter is based on four levels of linguistic analysis: lexical, grammatical or morpho-syntactic, semantic, and discourse (for a comprehensive overview, see Leech & Short, 2007).

10Quantitative analysis involved keyword analysis using corpus tools to determine lexical patterns and data analysis with Python for the purpose of analyzing morphological patterns. We looked at both positive keywords (words whose frequency is exceptionally high) and negative keywords (those with exceptionally low frequency) in comparison with a reference corpus. Qualitative analysis relied on concordance data and observations during close reading. Following Baker (2000), our quantitative analyses were focused on the target text. However, these were accompanied by qualitative analyses using concordance data and close reading of parallel texts. In line with Saldanha’s (2011) definition of translator style, our translator corpus consisted of 100 of Nihal Yeğinobalı’s literary translations to establish stylistic characteristics that constitute observable patterns across translations produced by the same translator. To demonstrate that the observed patterns are distinct from general Turkish linguistic trends, we compared our translator corpus with a reference corpus consisting of literary translations representative of the same period as the translator corpus. One limitation of our present study is that we focus on a single translator. In future research, we intend to include further translators in order to examine whether our observed stylistic patterns distinguish one translator’s (i.e., Nihal Yeğinobalı) work from that of other translators.

  • 4 The training set is used to train the model, while testing set is used to evaluate the model’s ge (...)

11By employing quantitative and qualitative corpus analyses, the present study attempted to i) determine the distinctive measurable characteristics of translator style and ii) examine to what extent a customized MT system can reflect the same translator’s style when the system is trained on previous translations by that translator. To address the first question, a corpus of English-Turkish translations by NY was compared with a reference corpus representative of general Turkish linguistic trends. This juxtaposition yielded stylistic traits unique to NY’s language use. To address the second question, the corpus of NY was first used to train and test a set of machine translation models.4 Then, the stylistic features observed in the NY corpus were used to compare machine translation models. The same set of features was also used to identify translators in an authorship attribution analysis.

2. Method

2.1. Translator

  • 5 A pseudotranslation is a text written as if it had been translated from a foreign language, even (...)

12Nihal Yeğinobalı (1927-2008) was born in Manisa, the Aegean part of Turkey, then moved to Istanbul in 1935 and graduated from the American High School for Girls. She started working as a literary translator of American literature before she finished high school. After working as a translator for about a decade between 1945-51, she married an American diplomat and moved to New York, where she started to study at New York University but dropped out to take care of her first child. She returned to Turkey in the 1960s where she continued to work as a translator. Her portfolio encompasses a diverse range of literary works, spanning from Nobel laureates to Latin American authors, as well as popular novels that have been adapted into cinematic productions. Throughout her career timeframe, NY produced more than 120 translations, 2 pseudo-translations,5 and 5 novels in Turkish (Yağcı, 2019).

2.2. Materials

13The present stylistic investigation is based on three corpora. The translator (i.e., NY) corpus contains the entire body of works by NY. The NY corpus has been digitalized with the informed consent of her heirs in compliance with the pertinent copyright laws. The digitalization process involved procuring physical copies of the texts for scanning, revising the optically recognized digital versions, and manually aligning the target texts with their respective source texts to train the machine translation system. Due to the practical inaccessibility of certain texts, the digitalized corpus consists of 100 optically recognized texts and 54 manually aligned texts. The manually aligned texts were used to fine-tune a pre-trained machine translation model, specifically the Helsinki-NLP English-Turkish Transformer model, which was part of the OPUS-MT initiative (Tiedemann & Thottingal, 2020). The resulting output from this fine-tuned model was compared with the NY corpus to evaluate the extent to which the model was successful in reproducing the style. The present investigation also incorporates a reference corpus, comprising 512 texts that are reflective of the linguistic trends observed in Turkish literary translations during NY’s active period from 1946 to 2015. The reference corpus served as a benchmark to validate the idiosyncrasies of stylistic traits identified in the NY corpus.

2.3. Data analysis

  • 6 Normalized frequency refers to the frequency of an input relative to the total length of the corp (...)

14This study integrates two distinct yet interdependent methodologies: qualitative analysis via close-reading and quantitative analysis via distant-reading. Previous studies on stylistics have often viewed close-reading and distant-reading as mutually exclusive approaches. While some scholars have prioritized quantifiable data (Baker, 2000; Saldanha, 2011, 2014), others consider close-reading as the backbone of stylistic analysis (Leech & Short, 2007; Boase-Beier, 2011). Drawing upon Youdale’s (2019) hybrid methodology, this study acknowledges the limitations of both approaches, and synthesizes close- and distant-reading methods. The primary objective of close-reading is to counterbalance the decontextualization and depersonalization of style that may result from computer-aided analyses of a large volume of texts. Likewise, literary texts often contain rhetorical elements, such as figures of speech, that are predominantly non-quantifiable in nature. Close-reading draws on the checklist of style markers compiled by Leech and Short (2007) mentioned in Section 1.4. However, in isolation, close-reading may induce research bias and selective attention to features of personal interest. Therefore, the study integrates qualitative observations with quantitative-driven techniques. Distant-reading involves quantitative analysis of lexical and morphological stylistic features, including comparison of the NY corpus with the reference corpus to identify keywords and key clusters at the lexical level, and analysis of average morphemes per sentence/word, including characteristic inflectional morphemes, at the morphological level. Quantitative stylistic features are computed by means of average normalized frequency6 to ensure the comparability of results across texts of varying lengths. These traits are then contrasted with reference values to validate idiosyncrasies.

2.4. Authorship attribution

  • 7 These algorithms are widely used methods to train models that classify new data based on previous (...)
  • 8 A statistical hypothesis test used to determine the independence of two variables in a contingenc (...)

15Authorship attribution is the science of detecting the author of a given text (Juola, 2008). In the context of our work, we use authorship attribution in the sense of determining the translator of a target text, based on the features applied in the data analysis step. Previous works on authorship attribution for translators—sometimes under the name translatorship attribution—include Caballero et. al. (2021), where the authors focused on features such as unigrams (individual words), punctuation, parts of speech (grammatical category of a word) and used machine learning algorithms such as support vector machine, naïve bayes, decision tree and logistic regression classifiers.7 Their method included a χ2 (chi-squared) test8 to choose 25 most important features.

16Our authorship attribution efforts are based on our quantitative analysis of style. We have incorporated computationally feasible features from Leech and Short’s (2007) proposed methodology, added some traditional natural language processing (NLP) metrics, and included morphological analysis, due to its particular importance in Turkish. Unlike previous works in authorship attribution, we do not consider punctuation as a style marker, mainly due to technical constrains of our pipeline, but also due to other factors such as the preferences of the publishing houses or their technical limitations in its variance being the likely reason.

2.5. Machine translation

17The neural machine translation approach is the dominant one currently used in the machine translation field. In this study, the Transformer neural network architecture (Vaswani et al.,, 2017), which is currently the state-of-the-art model in neural machine translation, was employed to train NMT models aimed at capturing the stylistic features of NY. Transformers are neural models with high complexity and a high number of parameters that need to be adjusted, leading to the necessity of a large corpus to train the model from scratch for it to perform well on unseen (test) data. Since the necessary amount of literary parallel data is unavailable and extremely expensive to gather and align, we chose not to train from scratch, but rather to use an already pre-trained model, which we would fine-tune using the 54 manually aligned books—all translated by NY—which include around 300,000 parallel sentences. Pre-training is achieved by training a model from scratch using a high amount of general-domain data. Fine-tuning a pre-trained model with small, in-domain data further tunes and adjusts the model’s parameters, adapting the model to a new domain. This way, the model is expected to generate good translations due to the large amount of parallel data used during pre-training, and conform to a new, specific domain (in our case the literary domain with NY’s style) due to the small parallel data used during fine-tuning.

  • 9 The pre-trained model can be found at: https://huggingface.co/Helsinki-NLP/opus-mt-tc-big-en-tr

18As stated, we chose to fine-tune a pre-trained model trained on a much larger English-Turkish parallel corpus, a process which amounts to adapting a general model to the works of the translator. The chosen MT model had been pre-trained in the scope of the Tatoeba challenge (Tiedemann, 2020) and the OPUS-MT project (Tiedemann & Thottingal, 2020) on the OPUS (Tiedemann, 2012) and Tatoeba corpora that contain around 108 million sentences. Uploaded to a machine learning library called the Huggingface platform (Wolf et al.,, 2020) with the model name Helsinki-NLP/opus-mt-tc-big-en-tr,9 the English-Turkish pre-trained Transformer model was used in this study for fine-tuning on the manually aligned NY corpus.

1948 of the manually aligned books (referred to as Manual-large corpus, with 283,809 sentences) were used for training (fine-tuning the pre-trained MT system) and validation (adjusting hyperparameters of the system), and 3 manually aligned books (5,550 sentences) were used for testing. The sentences in the test set were not shuffled but left as three whole books to aid in the process of qualitative stylistic analysis.

20Six parallel corpora with varying sizes were constructed from the Manual-large corpus to observe the effect of training set size on machine translation quality (measured by the BLEU score) and the success of capturing the translator’s style. The number of sentences within each training and validation set is provided in Table 2. The 50K, 100K, 150K, 200K, and 250K portions of the Manual-large corpus were sampled randomly.

Table 1. Number of sentences of training and validation corpora.

Model Name

Training (# sentences)

Validation (# sentences)

50K

50,000

2,500

100K

100,000

5,000

150K

150,000

7,500

200K

200,000

10,000

250K

250,000

12,500

Manual-large

269,618

14,191

  • 10 Tokenization is the process of breaking a text down into tokens (words and sub-words) for the pur (...)

The six parallel corpora were pre-processed before training which corresponds to standardization of punctuation marks and tokenization10 by using the pre-trained model’s built-in SentencePiece (Kudo & Richardson, 2018) tokenizer. After the pre-processing of the corpora, six models were fine-tuned by using each of the training and validation corpora shown in the table for 5 epochs and then evaluated on the test set.

3. Results

3.1. Stylistic analysis of the NY corpus

21Combining close- and distant-reading methods, we identified a multitude of idiosyncratic lexical features that exhibit higher incidence rates than the reference values. Noteworthy among these traits are the orthographic variant gene (or again) for the adverb yine (also again), the conjunction ki (which can be translated as that), and the conjunction cluster gelgel+, which comprises gelgelelim (i.e., having said that) and gelgeldim (also having said that). These features were identified through keyword lists generated using the reference corpus as a benchmark. Through close-reading sessions, we also observed frequent use of reduplications (i.e., ikileme). In turn, these qualitative observations inspired a heuristic approach to quantify reduplications and compare them against established reference values. The NY corpus exhibits a notable frequency of reduplications, albeit only marginally above established reference values. In addition to more formal terminology, the keyword list also contained colloquial expressions, including the term of endearment kuzum (i.e., my dear/lamb). Additionally, the subordinator diye (similar to say in reported speech in English) displayed a prominent keyword value, providing insight into how NY forms indirect sentences. Another compelling characteristic of the lexicon is the reduced incidence of the conjunction ve (i.e., and) compared to the benchmark value. This observation provides a partial explanation for the elevated frequency of alternative conjunctions in the NY corpus, signifying a tendency to eschew the use of ve. Negative keywords such as ve and yine shed light on the lexical attributes that NY intentionally avoided, thereby enhancing our understanding of her style.

  • 11 Density distribution plots for the rest of our observed stylistic features are available in a Git (...)
  • 12 Morphological parsing (morphological analysis) is the process of dividing words into morphologica (...)

22At the morpho-syntactic level, the NY corpus features fewer morphemes per sentence than is typical of the reference values. Figure 1 shows average morphemes per sentence in both corpora on the left and a more fine-grained analysis of density distribution within each corpus on the right.11 We used the Morphological Parser and Disambiguator developed by Sak et al., (2008) to extract morphological data. The tool initially identifies all feasible morphological parses12 of words, followed by selecting the appropriate parse that aligns with the contextual information. It should be acknowledged that morphological parsers are not entirely error-free when processing Turkish language data, although the margin of error is acceptable for obtaining general numerical data instead of fine-grained suffixes.

Figure 1. Average morphemes per sentence.

Figure 1. Average morphemes per sentence.

Corpus average values on the left, density distribution within corpus on the right. Lines denote kernel density estimation.

The results obtained on morphemes per sentence are further substantiated by our observations of a lower average of words per sentence in the NY corpus than the reference corpus (see Figure 2).

Figure 2. Average words per sentence.

Figure 2. Average words per sentence.

Corpus average values on the left, density distribution within corpus on the right. Lines denote kernel density estimation.

Integrating the type-token ratio with average morphemes and words per sentence can enable a more holistic comprehension of the complexity of the lexicon within the NY corpus. While this graph (see Figure 3) may be difficult to interpret in isolation, it offers intriguing insights when considered alongside other variables. Taken alongside findings of specific lexical items that are frequently used by NY, the lower average type-token ratio in the NY corpus than the reference corpus suggests that this translator prefers to repeat the same vocabulary forms across translations rather than opting for variations.

Figure 3. Type-token ratio.

Figure 3. Type-token ratio.

Corpus average values on the left, density distribution within corpus on the right. Lines denote kernel density estimation.

  • 13 The uppercase letter within a suffix indicates that the sound is phonologically conditioned. D st (...)

Another recurring feature is the predominance of particular morpheme combinations in both the NY and reference corpora. In Turkish computational linguistics, morphemes are often represented in a generalized form when there are allomorphs, as is the case with the verbal adjective suffix (-dik, -dık, -duk, -dük, -tik, -tık, -tuk, -tük, -diğ, -dığ, -duğ, etc.) that is commonly represented as +DHk.13

The 5 most common morpheme combinations in the reference corpus are as follows, in the given order:

1. +(s)H+nA (genitive + dative case, e.g., ev+i+ne or towards his/her home)
2. +
(s)H+nDA (genitive + locative case e.g., ev+i+nde or at his/her home)
3. +
(s)H+nH (genitive + ablative case, e.g., ev+i+ni or his/her home)
4. +
Hyor+DH (past progressive tense, e.g., gel+iyor+du or he/she was coming)
5. +
DHk+(s)H (verbal adjective, e.g., gel+diğ+i or that he/she came)

The NY corpus, on the other hand, features the following frequently occurring morpheme combinations:

1. +(s)H+nDA (genitive + locative case, e.g., ev+i+nde or at his/her home)
2. + (s)H+nA (genitive + dative case e.g., ev+i+ne or towards his/her home)
3. +(s)H+nH (genitive + ablative case, e.g., ev+i+ni or his/her home)
4. +Hyor+DH (past progressive tense, e.g., gel+iyor+du or he/she was coming)
5. +lar+(s)H (plural + genitive case, e.g., ev+ler+i or their homes)

A notable observation is that the most frequent morpheme combination in the NY corpus is the +(s)H+nDA combination followed by the +(s)H+nA combination, and the reverse pattern is observed in the reference corpus. The +(s)H+nDA morpheme cluster combines a genitive and a locative morpheme (as in evinde or at his/her home), whereas +(s)H+nA is a combination of the genitive and dative morphemes (as in evine or towards his/her home).

3.2. Translation Quality

  • 14 An n-gram is a sequence of “n” consecutive elements (words, syllables, letters, etc.) in a text.

23The MT models fine-tuned on different portions of the NY corpus in the English-Turkish direction were used to make predictions on the test set that contains three translations by NY (5,550 sentences). The predictions were compared against NY’s original translation and evaluated with the BLEU score to measure the quality of the translation. The BLEU metric (Papineni et al.,, 2002) is regarded as a standard evaluation metric in the machine translation field. It is an automatic evaluation metric that relies on the number of n-gram14 matches of words between reference translation and machine translation and depicts how much the synthetic translation overlaps with the original. Being used in most of MT research (not specific to literary texts) due to its correlation with human evaluation and ease of use, this score has been used as an indicator of machine translation quality in this research. The results are presented in Table 2.

Table 2. BLEU scores of the machine translation models on the test set.

Training Set

BLEU

Pre-trained

5.81

50K

8.82

100K

8.88

150K

8.97

200K

8.91

250K

8.93

Manual-large

9.04

The result of the pre-trained model (5.81 BLEU) is taken as the baseline score used to compare the performance of the other models. As stated in Section 2.5, this model had been pre-trained on more than 108 million sentences as a mixture of different domains, such as news, scientific articles, subtitles, books, etc. When the results are examined, we observe an increase of 3.01 BLEU when 50,000 sentences from the literary domain are used for fine-tuning. This points out the importance of domain compatibility between the training set and the test set in question. As the number of training sentences increases, the translation quality almost consistently improves, finally reaching 9.04 BLEU when the training set size goes up to 269K sentences.

3.3. Authorship attribution analysis

24In this analysis, we aim to determine whether the translator of a book is NY (i.e., the book is from the NY corpus) or another translator (i.e., the book is from the reference corpus) by using machine learning algorithms based on features. Based on the analyses explained in Section 2.3. and computational feasibility (i.e. extractable through computational methods), we chose to use all of following features: average morphemes per sentence, median morphemes per sentence, average morphemes per word, median morphemes per word, TTR (type-token ratio), number of unique words, number of unique words that occur at least 10 times, mean word length, standard deviation of word lengths, reduplications, ellipsis, questions, exclamations, mean sentence length, standard deviation of sentence lengths, median of sentence lengths, mode of sentence lengths, and normalized frequencies of the unigrams gelgelelim (having said that), gelgeldim (having said that), maamafih (an archaic form of however), gene (again), ki (that), ve (and), pek (much), hem (both), derken (I mean/meaning), acaba (I wonder), sahiden (really), doğallıkla (naturally).

  • 15 To do this, we adjust the values so that they all fall between 0 and 1. Specifically, each value (...)

25Each book in the reference and NY corpora is represented by a set of features that capture its characteristics. These features are stored as numerical values, for example if the type-token ratio of a book is 17 and its number of unique words is 19000, they are represented as a computational vector (also called an array) like [(...), 17, 1900, (....)]. For each book in the reference and the NY corpora, we first create a book feature vector v. Before comparing books using these vectors, it’s important to ensure that each feature is equally weighted.15

  • 16 Ten-fold cross validation refers to executing the algorithm n times by using different portions o (...)
  • 17 F1 score is the harmonic mean of precision and recall, where precision is the proportion of corre (...)

26As for the machine learning algorithms, we applied K-Nearest Neighbor (KNN, with 3 neighbors), Support Vector Machine (SVM, with a linear kernel), Gaussian Naïve Bayes (GNB), Decision Tree (DT), Random Forest (RF), and Gradient Boosting (GB) classifiers (Alpaydın, 2020) to an 80-20 % split of the dataset (80 % for training, 20 % for test), where we have 90 NY and 521 reference books. Ten-fold cross validation16 is applied to the dataset to ensure the reliability of the classification results. Since our classes are unevenly distributed, but both labels are equally important for us, we are focusing on macro average of F1 score as the indicator of classification performance.17 Table 3 shows the results.

Table 3. Classifier performance results.

Classifier

Macro Average F1 score with 29 Features

K-Nearest Neighbor (K=3)

86.9 %

Support Vector Machine

91.0 %

Gaussian Naïve Bayes

88.0 %

Decision Tree

87.6 %

Random Forest

94.5 %

Gradient Boosting

93.1 %

In this binary classification of books as authored by NY or any other translator from the reference corpus, the best performing approach is the Random Forest classifier with 94.5 % macro average F1 score, while the lowest score is 86.9 % with KNN (3 neighbors) classification, which is still very accurate.

3.4. Stylistic analysis of MT output

27Our machine translation outputs are evaluated through the authorship attribution classifiers trained on the NY and reference corpora. Each of the six classifiers trained on 90 NY and 521 reference books is given the test set of 3 books we used for testing in Section 2.5. Table 4 shows the results. The rows denote the models with which the books in the test set are translated, where the last row corresponds to the original translations of NY. Each entry in the table indicates whether the classifier identified the translations as those of NY or the reference corpus.

Table 4. Bold denotes incorrect and regular typeface shows correct predictions from the classifiers.

Training Set/Classifier

KNN

SVM

GNB

DT

RF

GB

Pre-Trained

Ref 

Ref

Ref 

Ref 

Ref

Ref 

50K

Ref 

Ref

NY 

NY 

NY 

NY 

100K

Ref 

NY

NY 

NY 

NY 

NY

150K

NY 

NY

NY 

NY 

NY

NY 

200K

NY 

NY

NY 

NY 

NY 

NY

250K

NY 

NY

NY 

NY 

NY 

NY 

Manual-large

NY 

NY

NY 

NY 

NY 

NY 

Original

Ref 

Ref

NY 

NY 

NY 

NY 

As seen in Table 4, KNN and SVM results classify the original test set, which consists of NY’s original translations, as belonging to the reference class. This overshadows the interesting results of those classifiers where fine-tuned models are considered as a part of reference until 100K and 150K thresholds. Further enhancement of the features used in this setup may prevent these false negatives.

  • 18 The full lists of keywords are available in a Github repository: https://github.com/Enahodzik/Lit (...)

28To determine whether the quantitatively measurable stylistic features we identified have been reproduced in the fine-tuned MT output, we also conducted a comparative investigation of the keyword lists18 of the human translator corpus and the MT version. These texts were compared against the OPUS-MT pre-trained output to generate keyword lists. The use of generative MT models to generate keyword lists allows for a one-to-one comparison of source text fragments and target sentences, unaffected by any preconceived ideology that influences human translations (Mikhailov, 2021). Theoretically, the pre-trained MT model delivers a literal translation of the source text and thus can be used as an objective baseline for generating keywords and detecting potential transfer of stylistic features on lexical level. Given that the objective is to examine the lexicon, any potential grammatical or semantic errors in the pre-trained machine translation output are unlikely to compromise the accuracy of the derived keyword list. The results indicate a considerable degree of correspondence, with 28 out of 66 keywords overlapping. This 42.4 % similarity suggests that the lexicon in the fine-tuned MT output is very similar to the translation by NY, while simultaneously highlighting a striking deviation from benchmark values. Another equally intriguing finding is that both the translation by NY and the fine-tuned MT output exhibit a substantially high degree of similarity in negative keywords, with a 64.9 % correspondence rate. The significant overlap of 37 out of 57 negative keywords between the translation by NY and the fine-tuned MT output indicates that in both corpora, particular lexicon is avoided, and similar alternatives are used instead. Overall, the analysis of positive and negative keywords strongly indicates the possibility of transfer, at least on lexical level.

29The presence of transfer of stylistic features can be further examined through a qualitative analysis of selected excerpts by comparing the original NY translation, translation by the pre-trained model, and translation by the model fine-tuned on Manual-large (see Table 5). In both Example 1 and Example 2, the Manual-large model exhibits salient stylistic features, including the adverbial conjunction gene de, the conjunctions da, gelgelelim and ki, and the noun ihtiyar (old), all of which are recognized keywords in both the NY test corpus and the fine-tuned MT output. Certain conjunctions, such as ancak (however) and ve (and) are significantly scarce in both the NY test corpus and the fine-tuned MT output. These particular conjunctions are present in the pre-trained output and are classified as negative keywords in both the NY test set and Manual-large outputs. Unlike the pre-trained model, both the NY test corpus and the fine-tuned model exhibit fewer words per sentence in both examples. Similar to what is observed in the general NY corpus, source sentences are divided into multiple fragments in Manual-large outputs through the idiosyncratic use of specific conjunctions, such as gene de and gelgelelim. In the case of Example 2, one source sentence is fragmented into two segments in the human translation, and into three fragments in the Manual-large output, illustrating the extent of fragmentation in both translation approaches. In a broader context, the fine-tuned MT model produces translations that align more closely with the NY test set compared to the pre-trained model. This synchronization is evident not only in the incorporation of keywords but also in the simplification of sentence structures, achieved through the distinctive use of conjunctions (also established as keywords) for sentence demarcation.

30The higher BLEU scores from the fine-tuned MT model, compared with the pre-trained model, indicate a more acceptable translation quality for an intended Turkish audience, since a higher BLEU metric demonstrates more phrase matches between the MT output and the reference translation. It is important to note that the BLEU scores reported for individual sentences are based on matching n-grams between the predicted and the original Turkish sentence, making it an inadequate metric for stylistic evaluation.

Table 5. Two examples of MT output with source text and three versions of the translations produced by: i) Nihal Yeğinobalı ii) the pre-trained model, and iii) the fine-tuned model (Manual-large).

Example 1

But the good old woman appeared again, and when she learnt the cause of her grief, she said, “Be of good cheer, my child. Go into the thicket and lie down and sleep; I will soon do thy work.

From Grimms’ Fairy Tales by Brothers Grimm (1812), trans. Margaret Hunt (1884)

Nihal Yeğinobalı

Pre-trained model

BLEU: 2.40

Manual-large

BLEU: 9.49

Example 2

The room was very dark, too dark to be observed with any accuracy, though Scrooge glanced round it in obedience to a secret impulse, anxious to know what kind of room it was.

From A Christmas Carol by Charles Dickens (1843)

Nihal Yeğinobalı

Pre-trained model

BLEU: 9.77

Manual-large

BLEU: 9.04

4. Discussion

31Although variables like average morphemes, words per sentence and type-token ratio do not necessarily indicate simplicity in the lexicon, combined with other stylistic features, they suggest that NY forms shorter and simpler sentences than the benchmark values. The NY corpus demonstrates relatively shorter sentence structures, with specific adverbial conjunctions (e.g., gene de or nevertheless) and conjunction clusters (e.g., gelgel+, which is the root of however) serving as effective sentence dividers, thereby facilitating fluidity and readability. On the other hand, the lower incidence rate of the conjunction ve (and) suggests that sentences are segmented into multiple units through the idiosyncratic use of the aforementioned lexical features. Given the presence of highly colloquial keywords (e.g., kuzum or my dear/lamb), the NY corpus features vernacular style and shorter and simpler sentence structures. Several derivational and inflectional morphemes, including the compound verb form tezlik fiili (i.e., +Hver), which denotes immediacy, and the diminutive morphemes +cHk and +cAğHz, are used distinctively. Together with conjunctions that serve as sentence dividers, these derivational morphemes contribute to a more informal style. More importantly, these morphemes do not have English language counterparts, communicating NY’s distinct perspectives on the narrative without any connection to the source linguistic features.

32The NY corpus demonstrates marked differences compared to the reference corpus in terms of morpheme combinations. While the genitive + locative morpheme combination (i.e., +(s)H+nDA) was observed most frequently across texts in our NY corpus, the genitive + dative combination (i.e., +(s)H+nA) was the most frequent in our reference corpus. Syntactic implications that arise from morpheme combinations can have a profound impact on the process of meaning-making. Our morpheme combination data consistently supports our translator’s preference to speak about a location (as in evinde or at his/her home) rather than direction or movements towards a location (as in evine or towards his/her home). The use of dative presupposes a simpler syntactic structure comprised of fewer words (e.g., compare Evindeyim or I am at his/her home with Evine gidiyorum or I am going towards his/her home). The preference for simpler and shorter sentence structures by our translator can be confirmed by our findings of lower average number of words and morphemes per sentence in the NY than the reference corpus.

33As such, a heuristic exchange between the morphological patterns and lexical observations yields a nuanced understanding of the distinctive discourse produced by NY. On top of these findings, our analyses of authorship attribution show that there is at least some degree of stylistic difference between NY and other translators in our reference corpus, and we are computationally capturing some of these through our features. Whether there are more features that help our classifiers get a better performance or help us pinpoint the exact occurrences within the natural flow of reading is an open question.

34When it comes to our analysis of machine translated texts, our authorship attribution analyses with classifiers trained on feature vectors perform well in attributing the pre-trained model to reference, and all fine-tuned models, as well as the original set, to NY. Since these classifiers have high accuracies on the task to classify the original translations of NY and the translations in reference corpus, it is a strong indication for transfer of our quantitatively measurable stylistic features from our human translator into a machine translation generated by a system trained on the same translator’s corpus. What is more, our keyword analysis of MT output demonstrates an important degree of similarity in lexical choice between the fine-tuned model and NY, both with respect to lexical items with exceptionally high and those with exceptionally low frequency.

Conclusions

35Our stylistic analysis reveals features on lexical and morphological levels that distinguish the NY corpus from a general reference corpus representative of Turkish literary translations. In addition, our authorship attribution analyses suggest that our observed stylistic features can be used to distinguish between MT models fine-tuned on our translator’s stylistic features from a pre-trained MT model. Since our pre-trained model does not belong to the literary domain, it is difficult to say whether the observed differences between MT models are due to a particular translator’s style or a characteristic of style in literary translation, more generally. However, our keyword analysis strongly indicates transfer of stylistic features on lexical level from our translator’s corpus to the fine-tuned MT model output. To be able to draw more valid conclusions, our future work will include stylistic analysis of a second translator’s corpus in order to determine stylistic features that can be attributed to a particular translator (but not to others). Some criteria that we will need to consider include the time period in which the second translator’s corpus was created as well as the genre their work belongs to, which would need to be matched to those of NY. Moreover, a more-fine grained analysis of stylistic features based on time periods will be carried out to determine whether our observed features are particular to a translator or representative of other factors, such as the time periods in which the translations were produced. Another step in this research is to expand our stylistic analysis to include further features on syntactic level, such as word order patterns, resulting in a more comprehensive stylistic analysis. In the near future, some publishing houses or public institutions may opt to adopt such a methodology to reproduce cultural artefacts such as literary translations with a certain style. Such an attempt would require careful consideration of copyright issues. Yet, this method is still in its infancy, at least for the English-Turkish language pair, for literary texts. We witness similar works around the globe such as the MUTUALIST (Daems, 2020-2022) for other language pairs, but the results have not been shared yet. Building larger models and datasets in the literary domain is likely to increase translation accuracy as well as to reflect the style of a translator on the MT output better. These issues remain to be investigated by scholars from Translation Studies and Computer Engineering fields.

Haut de page

Bibliographie

Aksan, Yeşim, Aksan, Mustafa, Koltuksuz, Ahmet, Sezer, Taner, Mersinli, Ümit, Demirhan, Umut Ufut, Yilmazer, Hakan, Kurtoğlu, Özlem, Atasoy, Gülsüm, Öz, Seda, and Yildiz, Ipek, “Construction of the Turkish National Corpus (TNC),” LREC, 2012, pp. 3223-3227.

Alkım, Barış E., “Re-creating H.P. Lovecraft’s Distinctive Style in Turkish” [Unpublished master’s thesis], Boğaziçi University, 2002.

Alpaydın, Ethem, Introduction to Machine Learning, United Kingdom, MIT Press, 2020.

Baker, Mona, “Towards a Methodology for Investigating the Style of a Literary Translator,” Target. International Journal of Translation Studies, Volume 12, Issue 2, 2000, pp. 241-266.

Besacier, Laurent, “Machine Translation for Literature: A Pilot Study” (Traduction automatisée d’une œuvre littéraire: une étude pilote), in Proceedings of TALN 2014 (Volume 2: Short Papers), Marseille: Association pour le Traitement Automatique des Langues, 2014, pp. 389-394.

Besacier, Laurent, Schwartz Lane, “Automated Translation of a Literary Work: a Pilot Study,” Fourth Workshop on Computational Linguistics for Literature-co-located with NAACL 2015, 2015.

Boase-Beier, Jean, A Critical Introduction to Translation Studies, Bloomsbury Publishing, 2011.

Bosseaux, Charlotte, “A Study of the Translator’s Voice and Style in the French Translations of Virginia Woolf’s The Waves,” CTIS occasional papers 1, 2001, pp. 55-75.

Caballero, Christian, Calvo, Hiram, and Batyrshin, Ildar, “On Explainable Features for Translatorship Attribution: Unveiling the Translator’s Style with Causality,” IEEE Access, Volume 9, 2021, pp. 93195-93208.

Daems, Joke, MUTUALIST. Machine Translation with User-Specific Training and User-Specific Adaptation for Literary Texts. Impact on Individual Style in Translation. Postdoctoral research project funded by the Special Research Fund (BOF, Bijzonder Onderzoeksfonds) of Ghent University, 2020-2022.

Dinçel, Burç, “From Motherless Brooklyn to Öksüz Brooklyn: Translating the Style of Jonathan Lethem,” İstanbul Üniversitesi Çeviribilim Dergisi, Volume 1, Issue 1, 2010, pp. 51-65.

Durmuş, Hilal E., “Reframing an Author’s Image Through the Style of Translation: The Case of Latife Tekin’s Swords of Ice,” Babel, Volume 67, Issue 6, 2021, pp. 683-706.

Frankenberg-Garcia, Ana, “Can a Corpus-Driven Lexical Analysis of Human and Machine Translation Unveil Discourse Features that Set Them Apart?,” Target, Volume 34, Issue 2, 2022, pp. 278-308.

Guerberof-Arenas, Ana, Toral, Antonio, “The Impact of Post-Editing and Machine Translation on Creativity and Reading Experience,” Translation Spaces, Volume 9, Issue 2, 2020, pp. 255-282.

Hansen, Damien, Esperança-Rodier, Emanuelle, Blanchon, Hervé, and Bada, Valérie, “La Traduction littéraire automatique: Adapter la machine à la traduction humaine individualisée”, Journal of Data Mining & Digital Humanities, 2022.

Juola, Patrick, Authorship Attribution, Netherlands, Now Publishers, 2008.

Kenny, Dorothy, Winters, Marion, “Machine Translation, Ethics and the Literary Translator’s Voice,” Translation Spaces, Volume 9, Issue 1, 2020, pp. 123-149.

Kuzman, Taja, Vintar, Špela, and Arcan, Mihael, “Neural Machine Translation of Literary Texts from English to Slovene,” Proceedings of the Qualities of Literary Machine Translation, August 2019, pp. 1-9.

Kudo, Taku, Richardson, John, “SentencePiece: A Simple and Language Independent Subword Tokenizer and Detokenizer for Neural Text Processing,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Brussels, Belgium, Association for Computational Linguistics, 2018, pp. 66-71.

Leech, Geoffrey N., Short, Mick, Style in Fiction: A Linguistic Introduction to English Fictional Prose (No. 13), Pearson Education, 2007.

Li, Defeng, Chunling, Zhang, Kanglong, Liu, “Translation Style and Ideology: A Corpus-Assisted Analysis of Two English Translations of Hongloumeng,” Literary and Linguistic Computing, Volume 26, Issue 2, 2011, pp. 153-166.

Malmkjær, Kirsten, “What Happened to God and the Angels: An Exercise in Translational sSylistics,” Target. International Journal of Translation Studies, Volume 15, Issue 1, 2003, pp. 37-58.

Manning, Christopher D., Raghavan, Prabhakar, Schütze, Hinrich, Introduction to Information Retrieval, Cambridge, UK, Cambridge University Press, 2009.

Mastropierro, Lorenzo, “Key cCusters as Indicators of Translator Style,” Target, Volume 30, Issue 2, 2018, pp. 240-259.

Matusov, Evgeny, “The Challenges of Using Neural Machine Translation for Literature,” Proceedings of the Qualities of Literary Machine Translation, 2019, pp. 10-19.

Michel, Paul, Neubig, Graham, “Extreme Adaptation for Personalized Neural Machine Translation”, in Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Melbourne, Australia, Association for Computational Linguistics, 2018, pp. 312-318.

Mikhailov, Mikhail, “Corpus-Based Analysis of Russian Translations of Animal Farm by George Orwell,” Corpus Exploration of Lexis and Discourse in Translation, Ji, Meng and Oakes, Michael, Routledge, 2021, pp. 56-82.

Munday, Jeremy, “The Relations of Style and Ideology in Translation: A case study of Harriet de Onís,” Actas del III Congreso Internacional de la Asociación Ibérica de Estudios de Traducción e Interpretación. La traducción del futuro: mediación lingüística y cultural en el siglo XXI. Barcelona. 2008.

Olohan, Maeve, Introducing Corpora in Translation Studies, Routledge, 2004.

Papineni, Kishore, Roukos, Salim, Ward, Todd and Zhu, Wei-Jing, “Bleu: A Method for Automatic Evaluation of Machine Translation”, in Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, Association for Computational Linguistics, July 2002, pp. 311-318.

Şahin, Mehmet, Nilgun, Dungan. “Translation Testing and Avaluation: A Study on Methods and Needs,” The International Journal for Translation & Interpreting Research, Volume 6, Issue 2, 2014, pp. 67-90.

Şahin, Mehmet, Gürses, Sabri, “Would MT Kill Creativity in Literary Retranslation?” Proceedings of the Qualities of Literary Machine Translation, August 2019, pp. 26-34.

Sak, Haşim, Güngör, Tunga, and Saraçlar, Murat, “Turkish Language Resources: Morphological Parser, Morphological Disambiguator and Web Corpus,” Advances in Natural Language Processing: 6th International Conference, GoTAL 2008 Proceedings Gothenburg, Sweden, August 25-27, Berlin/Heidelberg, Springer, 2008, pp. 417-427.

Saldanha, Gabriela, “Translator Style: Methodological Considerations,” The Translator, Volume 17, Issue 1, 2011, pp. 25-50.

—, “Style in, and of, Translation,” A Companion to Translation Studies, Sandra Bermann and Catherine Porter (eds.), 2014, pp. 95-106.

Sluyter-Gäthje, Henny, Zinsmeister, Heike, Barteld, Fabian, “Neural Machine Translation for Literary Texts,” 40th Annual Conference of the German Linguistic Society (Poster), 2018.

Tiedemann, Jörg, “Parallel Data, Tools and Interfaces in OPUS”, in Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey, European Language Resources Association (ELRA), 2012, pp. 2214-2218.

—, “The Tatoeba Translation Challenge – Realistic Data Sets for Low Resource and Multi-Lingual MT,” in Proceedings of the Fifth Conference on Machine Translation, Online, Association for Computational Linguistics, November 2020, pp. 1174-1182.

Tiedemann, Jörg, Thottingal, Santhosh, “OPUS-MT – Building Open tTranslation Services for the World,” in Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, Lisboa, Portugal, European Association for Machine Translation, 2020, pp. 479-480.

Toral, Antonio, Way, Andy. “Machine-Assisted Translation of Literary Text: A Case Study,” Translation Spaces Volume 4, Issue 2, 2015, pp. 240-267.

—, “What Level of Quality Can Neural Machine Translation Attain on Literary Text?, Translation Quality Assessment: From Principles to Practice, Joss Moorkens, Sheila Castilho, Federico Gaspari and Stephen Doherty (eds.), Springer International Publishing, 2018, pp. 263-287.

Vajn, Dominik, Two-Dimensional Theory of Style in Translations: An Investigation Into the Style of Literary Translations, Diss, University of Birmingham, 2009.

Vaswani, Ashish, Shazeer, Noam, Parmar, Niki, Uszkoreit, Jakob, Jones, Llion, Gomez, Aidan N., Kaiser, Łukasz and Polosukhin, Illia, “Attention is All You Need,” in Advances in Neural Information Processing Systems, 2017, pp. 5998-6008.

Wang, Yue, Hoang, Cuong, Federico, Marcello, “Towards Modeling the Style of Translators in Neural Machine Translation,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, pp. 1193-1199.

Wolf, Thomas, Debut, Lysandre, Sanh, Victor, Chaumond, Julien, Delangue, Clement, Moi, Anthony, Cistac, Pierric, Rault, Tim, Louf, Remi, Funtowicz, Morgan, Davison, Joe, Shleifer, Sam, von Platen, Patrick, Ma, Clara, Jernite, Yacine, Plu, Julien, Xu, Canwen, Le Scao, Teven, Gugger, Sylvain, et al.,, “Transformers: State-of-the-Art Natural Language Processing,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Online, Association for Computational Linguistics, 2020, pp. 38-45.

Yağcı, Ahu Selin Erkul, “Hepsatan kitapların çevirmeni Nihal Yeğinobalı,” Kelimelerin Kıyısında, Şehnaz Tahir Gürçağlar (ed.), İthaki, 2019, pp. 216-245.

Youdale, Roy, Using Computers in the Translation of Literary Style: Challenges and Opportunities, London/ New York, Routledge, 2019.

Haut de page

Notes

1 This research is funded by the Scientific and Technological Research Council of Türkiye (TÜBİTAK) under Grant No: 121K221. The research project is titled “Literary Machine Translation to Produce Translations that Reflect Translators’ Style and Generate Retranslations” and Mehmet Şahin is the principal investigator.

2 This constitutes the ratio of different words (types) to the total number of words (tokens) in a corpus. In corpus-based studies, it gives researchers an idea of vocabulary richness, i.e., unique lexical items, in a corpus.

3 The possibility of using MT to reproduce a translator’s style.

4 The training set is used to train the model, while testing set is used to evaluate the model’s generalization performance.

5 A pseudotranslation is a text written as if it had been translated from a foreign language, even though no foreign language original exists. These were not included in the Materials of the present study.

6 Normalized frequency refers to the frequency of an input relative to the total length of the corpus in question. We prefer average normalized frequency over average frequency to reduce the impact of outliers in our corpus (i.e., very short or very long books). Average frequency is disrupted by length, as longer books tend to have more occurrences of an input and vice versa. However, more occurrences may not be as impactful proportion-wise.

7 These algorithms are widely used methods to train models that classify new data based on previous data points. In the context of authorship attribution, it means hypothesizing the author of a book that is previously unseen by the model.

8 A statistical hypothesis test used to determine the independence of two variables in a contingency table.

9 The pre-trained model can be found at: https://huggingface.co/Helsinki-NLP/opus-mt-tc-big-en-tr

10 Tokenization is the process of breaking a text down into tokens (words and sub-words) for the purpose of natural language processing.

11 Density distribution plots for the rest of our observed stylistic features are available in a Github repository: https://github.com/Enahodzik/Literary-Machine-Translation-to-Produce-Translations-that-Reflect-Translators-Style.git

12 Morphological parsing (morphological analysis) is the process of dividing words into morphological units (i.e., root forms and morphemes).

13 The uppercase letter within a suffix indicates that the sound is phonologically conditioned. D stands for d or t; H stands for ı, i, u, or ü. A parenthesis within a suffix indicates that the sound inside the parenthesis is optional.

14 An n-gram is a sequence of “n” consecutive elements (words, syllables, letters, etc.) in a text.

15 To do this, we adjust the values so that they all fall between 0 and 1. Specifically, each value is recalibrated according to the lowest and highest values found for that feature across all books using min-max scaling:

Image 10000201000006F6000002E86AC83D1F7303D1B6.png

where i ¡Ê [1, n] is the index of the feature, vi is the value of feature i in book vector v, mini is the minimum value of feature i among all books, and maxi is the maximum value of feature i among all books.

16 Ten-fold cross validation refers to executing the algorithm n times by using different portions of the dataset as training data and test data, and then taking the average of the n results.

17 F1 score is the harmonic mean of precision and recall, where precision is the proportion of correct positive predictions among all positive predictions and recall is the proportion of correct positive predictions among all actual positive instances (Manning et. al., 2009). Macro averaging ensures the performance in predicting each class (NY or reference in our case) is equally important when calculating the overall score.

18 The full lists of keywords are available in a Github repository: https://github.com/Enahodzik/Literary-Machine-Translation-to-Produce-Translations-that-Reflect-Translators-Style.git

Haut de page

Table des illustrations

Titre Figure 1. Average morphemes per sentence.
Légende Corpus average values on the left, density distribution within corpus on the right. Lines denote kernel density estimation.
URL http://journals.openedition.org/palimpsestes/docannexe/image/8696/img-1.png
Fichier image/png, 49k
Titre Figure 2. Average words per sentence.
Légende Corpus average values on the left, density distribution within corpus on the right. Lines denote kernel density estimation.
URL http://journals.openedition.org/palimpsestes/docannexe/image/8696/img-2.png
Fichier image/png, 47k
Titre Figure 3. Type-token ratio.
Légende Corpus average values on the left, density distribution within corpus on the right. Lines denote kernel density estimation.
URL http://journals.openedition.org/palimpsestes/docannexe/image/8696/img-3.png
Fichier image/png, 44k
URL http://journals.openedition.org/palimpsestes/docannexe/image/8696/img-4.png
Fichier image/png, 176k
URL http://journals.openedition.org/palimpsestes/docannexe/image/8696/img-5.png
Fichier image/png, 111k
URL http://journals.openedition.org/palimpsestes/docannexe/image/8696/img-6.png
Fichier image/png, 118k
URL http://journals.openedition.org/palimpsestes/docannexe/image/8696/img-7.png
Fichier image/png, 122k
URL http://journals.openedition.org/palimpsestes/docannexe/image/8696/img-8.png
Fichier image/png, 112k
URL http://journals.openedition.org/palimpsestes/docannexe/image/8696/img-9.png
Fichier image/png, 119k
Haut de page

Pour citer cet article

Référence électronique

Harun Dallı, Olgun Dursun, Tunga Güngör, Sabri Gürses, Ena Hodzik, Mehmet Şahin et Zeynep Yirmibeşoğlu, « Giving a translator’s touch to the machine: Reproducing translator style in literary machine translation »Palimpsestes [En ligne], 38 | 2024, mis en ligne le 30 novembre 2024, consulté le 17 mars 2025. URL : http://journals.openedition.org/palimpsestes/8696 ; DOI : https://doi.org/10.4000/12sp6

Haut de page

Auteurs

Harun Dallı

Harun Dallı est doctorant au département de linguistique appliquée, de traduction et d’interprétation de l’université d’Anvers. Il est titulaire d’un master en études de traduction de l’université Boğaziçi et d’une licence en langue et littérature anglaises de l’université Hacettepe. Ses recherches portent sur la traduction audiovisuelle, l’accessibilité des médias et la stylistique. Parallèlement à ses recherches, il travaille comme sous-titreur professionnel.

Olgun Dursun

Olgun Dursun est titulaire d’un master du programme de sciences cognitives de l’université de Boğaziçi, avec une spécialisation en informatique et en linguistique, et d’une licence en études de traduction de l’université de Boğaziçi. Ses domaines de recherche comprennent la compréhension du langage naturel, la traduction automatique, les méthodes de tokenisation des sous-mots, le traitement morphologique et la reconnaissance vocale.

Tunga Güngör

Tunga Güngör est professeur au département d’ingénierie informatique de l’université Boğaziçi. Il a obtenu sa maîtrise et son doctorat dans ce même département. Ses recherches portent sur l’intelligence artificielle, le traitement du langage naturel, la traduction automatique et l’apprentissage automatique. Il est membre du laboratoire d’intelligence artificielle et du laboratoire d’analyse de texte et de bio-informatique du département. Il donne des cours de premier et de deuxième cycle sur le traitement du langage naturel, la traduction automatique et l’analyse des algorithmes.

Sabri Gürses

Sabri Gürses est un chercheur indépendant. Il est diplômé du département de langue et de littérature russes de l’université d’Istanbul et a obtenu un master en traductologie dans le même établissement. Il a obtenu son doctorat en langue et littérature russes à l’université d’Erciyes. Ses domaines de recherche comprennent la traduction littéraire, le plagiat en traduction, l’histoire et la sémiotique de la traduction, les technologies de la traduction. Il a lancé une revue de traductologie, « Çeviribilim », en 2005, et une maison d’édition du même nom. Il est aujourd’hui président de l’Association de traduction et d’interprétation de Turquie (Çeviri Derneği). En 2009, il a été récompensé par l’Institut Pouchkine pour sa traduction de Glossolalie d’Andreï Bely ; en 2010, sa traduction d’Oblomov d’Ivan Gontcharov a été récompensée comme l’une des meilleures traductions de l’année par Dünya Kitap ; en 2018, sa traduction d’Eugène Onéguine a été présélectionnée pour le prix Read Russia (Читай Россию) de l’Institut Perevoda, et en 2020, il a reçu le prix pour ses traductions de Tolstoï.

Ena Hodzik

Ena Hodzik est Associate Professor au département des études de traduction et interprétation de l’Université Boğaziçi. Elle a complété son doctorat en anglais et linguistique appliquée à l’Université de Cambridge. Ses domaines de recherche portent sur les processus d’interprétation et la traductologie de corpus. Elle est actuellement chercheuse invitée à l’université de Gand et mène un projet de recherche sur les processus prédictifs dans la traduction à vue anglais-turc et anglais-néerlandais. Elle travaille également sur un projet de recherche qui compare le style du traducteur dans la traduction littéraire humaine à celui de la traduction automatique.

Mehmet Şahin

Mehmet Şahin est professeur au département de traduction et d’interprétation de l’université Boğaziçi d’Istanbul. Il a terminé ses études de premier cycle en traduction et interprétation à l’université Bilkent d’Ankara, en Turquie, où il a également obtenu un master en formation des enseignants. Il est titulaire d’un doctorat en programmes d’enseignement et enseignement (Curriculum and instruction) avec un focus sur la linguistique appliquée/TESOL de l’université d’État de l’Iowa. Ses recherches doctorales ont porté sur l’apprentissage des langues assisté par ordinateur et les technologies de l’éducation. Ses recherches actuelles portent sur la traductologie, les technologies de la traduction et de l’interprétation, la traduction automatique et la formation des traducteurs et des interprètes. Mehmet Şahin a été chercheur principal dans le cadre de plusieurs projets de recherche sur la traduction et l’interprétation, financés par le Conseil de la recherche scientifique et technologique de Turquie (TÜBİTAK), dont le projet intitulé “Literary Machine Translation to Produce Translations that Reflect Translators’ Style and Generate Retranslations” sur lequel est basé le présent article.

Zeynep Yirmibeşoğlu

Zeynep Yirmibeşoğlu est assistante et doctorante au département d’ingénierie informatique de l’université de Boğaziçi. Elle a obtenu sa licence en ingénierie informatique à l’université technique d’Istanbul et son master en ingénierie informatique à l’université de Boğaziçi. Elle mène des recherches sur le traitement du langage naturel, l’apprentissage profond et la traduction automatique, et est membre du laboratoire d’intelligence artificielle et du laboratoire d’analyse de texte et de bio-informatique du département.

Haut de page

Droits d’auteur

Le texte et les autres éléments (illustrations, fichiers annexes importés), sont « Tous droits réservés », sauf mention contraire.

Haut de page
Rechercher dans OpenEdition Search

Vous allez être redirigé vers OpenEdition Search