Skip to navigation – Site map

HomeAnglophonia30The Potential Role of Surprisal i...

The Potential Role of Surprisal in Predicting Pronunciation Problems in L2 English Phonology

Monika Pukli


This paper investigates whether phonological properties of articulation complexity coupled with familiarity and frequency of learner usage can explain pronunciation mistakes in L2 English. In an analogy of the (native) phonologization model by Hume and Mailhot (2013), it was expected that one would encounter more mistakes in high surprisal words (i.e. uncommon words whose articulation is complex) than more familiar, average surprisal words, and that low surprisal words (i.e. the most frequent and simple words) would be likewise more often mispronounced than the average. The analysis of pronunciation mistakes in French speakers of English does not indicate a significant correlation between word surprisal categories (low/mid/high) and the presence of errors. Results show that speakers do make mistakes in very high surprisal words but not systematically, while very low surprisal words are rarely mispronounced. The main finding, however, is the irregular relationship between the CEFR levels of lexical forms and the number of errors: unsurprisingly, the category with the most problems is C2 (encompassing archaic and unfamiliar words), but as for the other five categories, the number of mistakes decreases as we move from the easiest A1 level words to the more difficult C1 level words.

Top of page

Full text

1. Introduction

  • 1 Phonologization was originally defined (Hyman 1976, 2008) to refer to a change in a language sys (...)

1 The phonological system of a non-native second language has often been described in terms of L1 transfer in both perception and production (Trubetzkoy 1939, Lado 1957, Kuhl & Iverson 1995, Best & Tyler 2007, for example), and more particularly in terms of universals and markedness (Eckman 1981 and 2008, or Dressler 1996, for instance). As has been pointed out, L1 transfer can play a positive role as well, and perceptual closeness between L1 and target phonemes sometimes hinder, and at other times facilitate the construction of intelligible L2 phonology (Flege 1987, Major & Kim 1999, McAllister, Flege & Piske 2002). The aim of this paper, however, is to characterise L2 phonology on its own, to verify whether system internal features leading to phonologization1 in L1 might be working in the same way in the emerging L2 systems in which they lead to learner difficulties and failure of acquisition.

  • 2 I would like to thank Nicolas Ballier for his comments on an earlier version of this paper, as w (...)

2 Instead of comparing two systems, I posit that the same cognitive processes are at work in L1 and L2 phonologies and I seek to establish similar patterns across native and non-native English pronunciation by using the entropy and surprisal model of Hume and Mailhot (2013) that links instability in a communication system to probable causes and outcomes of language change through three combined sources: perceptual distinctiveness, frequency/familiarity of use and articulatory complexity. The model, as I will argue, can be extended to L2 phonology, bearing in mind some important differences such as the relative stability of English phonology in native speakers vs. the less stable phonology of learners of English, and the level of attainment that is perfect in L1 English but often incomplete and variable in L2 English. English L2 productions in young adults will be used to gain insight into the viability of the entropy/surprisal model and its predictions in L2.2

2. Theoretical framework: the entropy model

3 We have all wondered to what extent the anticipation of learner difficulties is possible in English pronunciation skills. This problem becomes all the more interesting as we surpass the beginner level and accompany university students preparing their MA degree in English studies. While there are many aspects to be taken into account in the phonological acquisition of L2 English from global teaching strategies to specialised phonology training as well as individual variation and learning strategies, this paper focuses on a very specific issue: can the combination of frequency/familiarity of use and articulatory complexity of English words predict learner difficulties? Are unexpected, rare or archaic words easier or harder to produce correctly? Does the fact that they are articulatorily complex contribute to the difficulty? These questions will be investigated through a reading task to see if word surprisal can be related to the number and nature of reading mistakes.

4 Thus, as mentioned earlier, the immediate interest in this study will not be the comparison of French and English, but rather the overall appraisal of a reading task in English and verifying whether global tendencies that are not based on French vs. English phonological differences can be observed in a corpus of 12 speakers. The reason why the entropy/surprisal model of Hume and Mailhot (2013) is particularly interesting is that, as they claim, it can make powerful predictions as to the “likely targets of change as well as the nature of the resultant change” (2013: 30). The purpose of this paper is to report on the viability of the entropy/surprisal model originally formulated for native phonological variation in the acquisition of L2 English phonology. The following sections explain the conceptual details of the model, and the analogy I intend to draw between predictions about phonologization and predictions about problems and learning paths in L2 phonology.

2.1 The original model: strengths and weaknesses

  • 3 Hume and Mailhot’s (2013) model is based on the information-theoretic concepts of entropy and su (...)

5 The information-theoretic model3 of language change proposed by Hume and Mailhot (2013) may be summed up briefly as follows: predictions can be made about phonologization based on the level of surprisal of a unit or pattern. More specifically, we can predict what element or pattern will change (e.g. the reduction of highly frequent function words in English) and also the type of change that will occur (either towards an existing similar pattern/element, or towards a new pattern/category). Surprisal is the degree of unexpectedness, unusualness (or surprise) of, usually a word, or any other element (phoneme, syllable structure, prosodic patterning). It is sometimes defined as the information content of an entity within a system since one can quantify the information conveyed by a linguistic unit (phoneme, morpheme, phrase, utterance) in its context. Surprisal is directly related to contextual probability: the more probable the element, the less surprising it is. In other words, a likely event has low surprisal while an unlikely event has high surprisal.

  • 4 Shannon entropy is a mathematical measure that quantifies the amount of information in a given u (...)

6 Entropy4 is related to surprisal: it refers to the degree of unpredictability of the information content within a system and is determined by the average level of surprisal. When entropy is small, the outcome of the system is almost certain (The dog is barking), but when all the candidates have equal probability to be the outcome, then entropy is highest (The dog is sniffing/hiding/sleep-barking/scratching/howling, or The dog is teaching/cooking/driving). In Hume and Mailhot’s words: “entropy models a cognitive state of the language user associated with the amount of uncertainty regarding the outcome of identifying or producing some linguistic event” (2013: 30).

Figure 1. Surprisal in Hume and Mailhot’s (2013) model: predicting what changes

Figure 1. Surprisal in Hume and Mailhot’s (2013) model: predicting what changes

7 Figure 1 above is a visual representation of the concept of surprisal in the context of the model predicting phonologization to occur to certain phonetic forms but not to others as phonological elements at the two extremes will be affected by change. Instability at the low surprisal end can be illustrated by the deletion of alveolar plosives that are more frequent in just and and compared to other instances of alveolar plosives in the same environment (Hume and Mailhot 2013: 42 and further references there). At the high surprisal end, the authors refer to, for example, metathesis occurring more frequently in consonant clusters that are more unfamiliar and difficult to pronounce (Hume and Mailhot 2013: 45). The figure also shows that while surprisal is a continuous measure that ranges from zero to infinity, it is not a given element’s surprisal value that determines whether the former is likely to change, but rather whether the value is in the relative centre of the system or at one of the extremes. An extreme degree of surprisal (low or high) leads to small entropy which in turn leads to unstability. A pattern or element situated in the centre is not expected to change, as opposed to one close to 0 or with a very high value: that is where phonologization, a significant transition in phonetic variation, will take place.

8 Figure 2 below shows that according to Hume and Mailhot’s model the nature of the change predicted depends on the value of surprisal. A low measure of surprisal is characteristic of changes that do not seek similar patterns or elements but create new ones. A high surprisal value, on the other hand, is typical of changes that assimilate to existing patterns or units that have a lower surprisal value. This assumption is based on indirect examples such as processing speed that is more rapid in high frequency words and words containing high frequency sounds, the duration of words that is longer in similar but infrequent items, speaker errors in syllables with less common rimes as opposed to more common rimes, and on direct observations of phonologization such as the role of frequency in analogical change, as in the deletion of /h/ in Old English that first applied to low frequency words, or the preference of high frequency clustering of /lt/ compared to /tl/ in chipolte vs. the original order in chipotle /tʃɪˈpoʊtleɪ/ (Hume and Mailhot 2013: 44-45).

Figure 2. Surprisal in Hume and Mailhot’s (2013) model: predicting the type of change

Figure 2. Surprisal in Hume and Mailhot’s (2013) model: predicting the type of change

9 Finally, the model is also attractive as it purports to combine all potentially effective factors that bring about language change instead of simply testing which variable is the strongest predictor of a particular event or trying to oppose one factor against another. In fact, the more factors one tries to combine, the better predictions are made. The principal weakness of the conceptual system proposed by Hume and Mailhot (2013), however, is the missing link between concrete contextualised measurements of surprisal of sounds, patterns, words or other units and the general model and its predictions.

  • 5 “Information is defined probabilistically in terms of the amount of uncertainty (Entropy) associ (...)

10 Before proceeding to the extension of the model to L2 phonology, I would like to stress that I will not present another information theoretically based model with mathematical derivations. For further reference on the original concepts of Information Theory, one of the best and shortest summaries of Shannon’s entropy can be found in Shaw et al (2018).5

2.2 The entropy/surprisal model extended to L2 phonology

11 Hume and Mailhot (2013) discuss three aspects that bear on surprisal and therefore on phonologization and eventually language change: lexical frequency, perceptual distinctiveness and articulation complexity. Low surprisal is associated with high frequency, weak perceptual distinctiveness and simple articulation (2013: 41), while high surprisal is characterized by very low frequency, very strong perceptual distinctiveness and complex articulation (2013: 42). Familiarity, although less well defined than frequency of usage, also appears among the conditioning factors. High surprisal is associated with unfamiliar words whereas low surprisal words are very familiar (2013: 46). Elements with either very high or very low surprisal are unstable and likely to change, to re-categorize, to reduce.

12 Language change is complex and has no direct connection to the phonological system that a learner of English constructs and moulds during the acquisition process. Nevertheless, the same principles might apply as learner difficulties arise and are overcome in L2 acquisition. It is possible, for instance, that a speaker pays more attention to words that stand out from the basic inventory one uses every day (such as rime, ancient, mariner that occur in the text of the poem used in this study), and that the higher level of attention results in fewer mistakes. Extremely frequent and simple words on the other hand may be victims of mispronunciation because of the absence of attention and routine usage of ill-acquired forms. In the same way, it seems reasonable to expect that complex articulatory gestures are more difficult to perform and hence that words requiring such gestures are more liable to mistakes than those requiring simpler ones.

13 Thus, our first intuition could be that frequent words are among the first to be mastered and university students majoring in English would have no difficulty in producing items like that, because, or thanks. The expectation of this paper, which runs opposite to this instinctive view, is based on the combined effects of familiarity, difficulty and frequency of a word, and more particularly on the counter-intuition that the presence of rare and difficult items may lead to a more careful, and hence up to a point, better pronunciation. The contention is perhaps more controversial concerning common words, but exactly because of the omnipresence of low surprisal items, it is an aspect that deserves to be examined since a simple and very frequent word, if acquired with an incongruous pronunciation, may become fossilised and reproduced in this form even at a much later stage.

  • 6 I would like to hereby thank all the anonymous participants in MiReLeCo.

14 Although the study was conducted at university, the hypothesis and the observations are not restricted to the immediate context of tertiary education; students who participated in the corpus used for the purposes of this paper6 have very different linguistic backgrounds and communications skills. Simple reading mistakes may also occur in the assignment that was chosen to test the model, as literacy does not automatically come with equally good reading skills. Therefore, the assessment of the reading task cannot entirely rule out factors non-related to L2 phonology.

15 The extension of the original model to L2 phonology in sum is not motivated by similarities between the two fields of language change and second language acquisition, but by an impulse to find out if surprisal could be a valid descriptor of learner difficulties that do not come from cross-linguistic influences but from the internal structure of the target language itself. We are interested in what can be predicted as difficult by applying Hume & Mailhot’s model to the analysis of mistakes collected from a reading task, assuming that production errors are necessarily related to difficulties in L2 phonology such as inappropriate articulation, confusion between phonemes, absence of natural transition and linking within and across words or any other segmental factor that contributes to problematic pronunciation, including the consequences of erroneous suprasegmental processes (stress and intonation, i.e. typically problems with the proper use of reduced and full forms).

3. Methodology

16 The present section describes the conditions of the study in detail: the corpus, the selection of words, and the criteria according to which the predictions were made.

3.1 The MiReLeCo corpus

17 The Miscellaneous Readings Learner Corpus started in 2019 and currently contains two texts, Aesop’s The North Wind and the Sun and a short passage from The Rime of the Ancient Mariner by S. T. Coleridge, read by 47 and 12 university students, respectively. No background information is collected on the participants; each text is identified by a number and the university level at which the reader is enrolled. This policy intends to favour a wider range of linguistic data collection at the expense of the availability of personal background information (and hence the possibility to use the corpus for traditional sociolinguistic variable testing).

3.1.1 Participants

18 Twelve university students preparing their MA degree in English studies at a French university read the passage from the first part of The Rime of the Ancient Mariner by Samuel Taylor Coleridge, written in 1797-98. The text of 1834 was presented in modern spelling, and only the beginning of the poem was read. No visual cues accompanied the task, which is worthy of mention in so far as it has been shown that expectation bias and surprisal in general can be modified in the non-linguistic context of delivering a real-life task (Ankener et al 2018).

19 The recordings were conducted by the participants themselves at home with the digital equipment and software at their disposal. No instructions were given. Recordings that used the pause function were eliminated in order to maintain a corpus that is also available for prosodic analysis. All participants gave their written consent, participation was voluntary and no one received payment for their contribution. It has to be noted that native reading models had been available and students were free to decide whether or not to listen to other readings of the same text, or to practise with a given model to enhance the quality of their reading skills for the task.

  • 7 I am grateful to one of the reviewers for having addressed this question.

20 Corpus tasks are often an artificial way of recreating continuous speech in a controllable manner for research purposes. It is, for instance, clear from the setup that this is not a reading experiment that tests reading time as a function of unexpected occurrences in a text. There is an additional conceptual layer in surprisal here that allows for strategies that are natural in second language acquisition especially for a formally performed unspontaneous task, strategies such as looking up a word in a dictionary, listening to someone else reading the poem, or reading the text themselves several times before recording it. But a word can be unexpected in its context only once. In a second reading of the poem, or after several rehearsals, the original concept of contextualised surprisal values becomes uninterpretable.7 The analysis of the results must therefore take into account that surprisal is not variable in this study but rather defined as an internal property of a word. This point will be further discussed in 3.2.

3.1.2 Selection of words

21 The passage from The Rime of the Ancient Mariner was selected originally so that the reading task should last approximately two minutes. No further selection was made within the text, all 231 words were analysed, yielding 2772 occurrences in all. The passage brings together high frequency, everyday words such as open, hear, eye with archaic word forms like eftsoons, quoth, spake. Among the 134 different word forms (there are 97 repetitions in the text) the shortest contains one phoneme (eye, I), while the longest has ten phonemes (minstrelsy). Some of the rarest words are: eftsoons, stoppeth, may'st, unhand, minstrelsy, quoth, bassoon, spake, wherefore, loon, mariner, and some of the most frequent ones are at, but, are, we, he, on, for, is, it, I, a, of, and, the. The text itself is presented in the Appendices, in spelling and in transcription.

3.2 Measuring surprisal

  • 8 I am grateful to Stefan Frank (PC) who helped me disambiguate the question whether or not corpus (...)
  • 9 Cloze probability is used in neurolinguistics and refers to “the predictability of the sentence- (...)

22 Measuring surprisal, the extent to which the appearance of a given item is unexpected in a given context, is a complex task: “surprisal of a word is equivalent to the difference between the probability distributions of possible utterances before and after encountering that word (Kullback-Leibler divergence), quantifying the amount of information conveyed by that word” (Zarcone et al 2016: 3). Furthermore, surprisal can be modelled at different levels: phonemes, phoneme combinations, words, phrases, discourse. Word surprisal is used in psycholinguistics especially in experiments on reading times where its role seems to be significant (see references in Delogu et al 2017, for example). According to Monsalve et al (2012), surprisal estimates can be obtained through statistical language models such as n-gram language models, Markov chains, and phrase structure grammars (also, see further references in Zarcone et al 2016). It is important to keep in mind, however, that estimates are valid for the corpus that was used to compute them.8 In other words, surprisal is always situated in discourse, in a set of sentences designed for the experiment, or in a larger corpus. An alternative method is using cloze probabilities9 that also allow to quantify the information content of a word (the reader can refer to Ankener et al (2018) and Hale (2016) for a more comprehensive overview of the different metrics). Furthermore, as Stefan Frank (PC) suggested, there is a mismatch between the contemporary readers’ knowledge of the language and the surprisal estimates that a language model trained on a Romantic poetry corpus would yield.

23 Among the different ways of obtaining surprisal estimates, the most appropriate for the present experiment would be the one based on a generic language model (arguably modern International English rather than the poetic language set in the Romantic period) even if the results of a generic classifier of probabilities necessarily involve a certain loss in accuracy. Stehouwer and Zaanen (2009) discuss the efficacy of using corpora of international news texts to identify and correct context-specific spelling errors, for instance, and it seems that at least for their task there is a balance between the aforementioned disadvantage of some degree of inaccuracy and the advantage of being able to use a robust model trained on a large corpus. The most important difference between the two methods of computing is that a model trained in situ would be able to assign specific weight to a given combination of word sequences in context that may be different from the typical collocation frequency that a generic model would ascribe. Thus, mariner introduced early on in the poem, as it recurs several times, would have its surprisal in context diminish each time. A specific language model can follow this increase in contextual probability whereas a large, generic model cannot, and the surprisal estimate of mariner would remain the same from the beginning of the reading task to the end. Unfortunately, there are no measurements of lexical probabilities from such a large, generic language corpus available, be that for American, British or International English.

3.2.1 Representing surprisal as a categorical variable

24 Due to the complexity and the contextual nature of the computation of surprisal values and in the absence of surprisal estimates from a robust, generic language model, I decided to establish three categories of surprisal measured by the combined value of the articulatory effort and the CEFR level of each word for the reading task under study.

25 The two components of this metric estimating surprisal for word forms in the corpus are themselves symbolic numbers representing reader familiarity with the text (CEFR level), and the number and difficulty of producing the required articulatory gestures when pronouncing a given word (articulatory complexity). The Common European Framework of Reference (CEFR) levels were determined by the English Vocabulary Profile based on the Cambridge Learner Corpus (a yearly updated corpus containing several hundred thousand examination scripts from around the world) as well as the Cambridge English Corpus of spoken and written English from several native varieties. Importantly, the ranking is corpus-driven, i.e. it reflects actual usage and not external decisions as to what should be mastered at a given level. Occurrences of archaic usage such as the third-person singular simple present indicative form of stop (stoppeth) were treated as C2 level words.

26 It is important to mention that the log10 frequency of lexical items from the COCA corpus (the largest database available today) that was meant to be initially part of the surprisal value was eventually removed since the CEFR category is already based on the frequency counts of learner usage from the Cambridge corpora. The other advantage of not integrating lexical frequency into the aggregate score is that it can be retained as a separate check-up variable. Surprisal is certainly linked to lexical frequency, but there seemed to be a risk of turning the variable into a slightly modified frequency indicator. Also, the Zipfian power-law distribution of frequency data (whether word form or lemma frequency (Corral et al (2015)) might fit ill with the other indicators that have a very reduced range: the CEFR levels classify words according to the frequency with which learners encounter words in six categories from A1 to C2.

27 Articulatory effort was calculated on the word level. I first considered estimating gestural effort based on the gestural approach of articulatory phonology (Browman and Goldstein 1992) by counting how many times articulators move along the different tract variables: lip aperture and protrusion, tongue tip constriction degree and place, tongue body constriction degree and place, velic aperture, and glottal aperture. To illustrate this with Hall (2018), during the production of the word Tom [thɑɑ̃m], we can follow and count each change on each gestural plane rather than in a sequential manner: 1) LIPS: the closing of the lips leading to /m/, 2) TONGUE TIP: the aperture of the closure between the alveolar ridge and the tip of the tongue after /t/, 3) TONGUE BODY: the change in the position of the tongue body after the vowel sound, 4) VELUM: the opening of the velum for the nasal, and 5) GLOTTIS: the start of the vibration of the vocal folds after the aspirated transition towards the vowel from the plosive. This count, however, would not be very different from the number of phonemes in the word as at least one feature changes in passing from one sound to another, and counting phonemes is considerably quicker than counting changes across gestural planes. Also, tracking the changes in articulatory gestures does not weigh the articulatory effort that distinguishes a fully pronounced vowel such as // from a reduced schwa /ǝ/, for example. Thus, in addition to changes, the internal structure of each gesture should have to be studied.

28 Therefore, I decided to calculate articulatory effort for each word based on the established complexity measures in first language acquisition and speech therapy. While there are slightly different ways in which articulatory complexity can be measured, the average learning curve in L1 English is a reliable indicator of increasing difficulty levels in individual sounds and in sound clusters. As Kuruvilla-Dugdale et al (2018) write, the earliest sounds, such as the bilabial /b/ require very simple motor skills, or motor adjustments compared to sounds that are acquired later on and that need finer adjustments and the coordination of different articulators. Based on their classification (Kuruvilla et al 2018: 2208, Table 2) and the original propositions of R. D. Kent (1992: 72-77), the following scoring system was adopted: five levels from 1 to 5 for vowel sounds and six levels from 3 to 7 for consonants. Complexity levels were then summed and provided a score for each word. Table 1 below provides an overview of the scoring system.

29Table 1. Classification of vowels and consonants based on articulatory motor complexity adapted from Kuruvilla et al (2018) and Kent (1992)



Motor skills and adjustments


STRUT, commA

Slightly raised tongue position as it moves essentially from front to back; mastery of low central vowels (1y)



Three maximally dissimilar ‘corner’ vowels form a triangle for vowel production (2y)



Articulatory space shifts: the triangle becomes a quadrilateral; tongue movement precision for gliding of diphthongs (3y)



Coordination of articulatory gestures for front and non-peripheral vowels (4y)



Retroflexion of the tongue


/p m n w h/

High (/p m n/) and low velocity movements (/w h/); Velopharyngeal valving (oral vs. nasal stops)


/b d k g j f/

Additional rapid (/b d k g/) and steady (/j/) movements;

Control of fine force for frication (/f/)


/t ŋ r l/

Additional stops (/t, ŋ/); mastery of tongue positions for /r, l/


/s z ʃ v θ ð tʃ dʒ/

Dental, alveolar, and palatal fricatives; affricates


Any consonant in a cluster within the same prosodic unit (onset or coda)

Fine motor skills to move rapidly from one articulatory position to another

  • 10 The term rhoticity refers to whether the letter ‘r’ is sounded before a consonant and at the end (...)
  • 11 Eight of the twelve speakers in the study used General American and four of them near-RP to the (...)

30 Rhoticity10 affects complexity scores as retroflexion of /ɚ/ and /ɝː/ (e.g. in her and heard) is the most complex vocalic articulation (level 5) and the pronunciation of the consonant /r/ itself (e.g. in hear) has the score of 5 as well. The corpus is not controlled for the native model of pronunciation (typically non-regional American or near-RP systems are targeted by French learners), and importantly, participants in the reading task have different degrees of mastery. This constitutes a general drawback in corpus work on L2 phonology and even though one cannot be reasonably expected to avoid it, it may be important to keep in mind when discussing the results of the study.11 As a decision had to be made for measuring articulatory complexity, the American model was chosen because of the more easily accessible results of acquisition difficulties in the literature. For this reading task, twenty-two words were concerned: eight instances of /ɚ/, two words with /ɝː/ and twelve occurrences of /r/ before a consonant or a pause (22 in all for 134 word forms). Further differences concern other vocalic articulations as the RP model differs from General American, the lexical sets in parentheses (in Table 1 above) help disambiguate the calculation as the same method was used for the text irrespective of the reader, their accent and the individual pronunciations.

  • 12 The aggregate score of surprisal was obtained by allotting equal weight to articulatory complexi (...)
  • 13 First quartile (Q1) = 20.1, the 50th percentile (median) = 34.56, third quartile (Q3) = 53.43; m (...)

31 Based on these two features, the surprisal score of each word form was established by calculating the weighted average12 of CEFR levels and articulation complexity expressed in a score ranging from 9 to 98. Also, as our goal is to test whether extreme values of surprisal show a particular behaviour with regard to reading mistakes and their type, it had to be determined what counts as very low and very high: the 25th and the 75th percentiles of the actual value distribution observed in the corpus mark the division between the three categories of high, mid and low surprisal.13 High level of surprisal was assigned to words having a score of 53.43 or above, low level of surprisal to words with a score of 20.1 or below, the mid-range was in between the two values.

3.2.2 Separate control variables

32 As a tentative estimation procedure was necessary to establish surprisal values, additional measures were also collected to assess their potential influence on learner mistakes: word length and word frequency. Real surprisal estimates take into account frequency values, collocation frequencies, and other computations of the degree to which a given item is expected to occur in a given context. Given the fact that this value was replaced by a three-way distinction based on an aggregate score in this study, it was necessary to keep a safeguard in the form of separate variables in order to verify the effects of other factors. Neither variable, however, is entirely independent of the surprisal score: word frequency is linked to the CEFR level and word length is proportionate to some extent to word complexity.

33 The frequency counts were taken from the Corpus of Contemporary American English (COCA), the largest English corpus to date with over one billion words from different registers (spoken, fiction, popular magazines, newspapers, academic texts, TV and film subtitles, blogs, and other web pages) (Davies 2020). A corpus of International English would have been perhaps more adequate for an investigation on L2 English, but the decision to use COCA was made owing to the balanced composition across discourse types, as well as its size. Relative frequency of use might well vary across varieties to some extent, within and across English-speaking countries, but the corpus had to be selected for the whole group of participants in any case and could not be tailor-made to different speaker subsets.

34 Word length was counted in phonemes. Just like articulatory complexity, the choice of accent affects phoneme constituents within words. Working with General American implied having longer words as the consonant phoneme /r/ is pronounced in all positions adding to the length of words such as hear and doors (12 items in 134 altogether). Vowel length was ignored and all vowel sounds (short, long, diphthong) accounted for one unit.

3.2.3 Intelligibility and articulatory precision

35 Learner mistakes can be situated in different phonological domains as segmental, suprasegmental and prosodic levels overlap. Although the foremost aim of the study was to verify if a given word surprisal category (low/mid/high) predicted the number of mistakes on that word in an L2 English reading task, a complementary qualitative approach was adopted here to note the nature of the mistakes for each occurrence and each speaker.

  • 14 In fact, a separate paper could be devoted to the question of what constitutes a mistake; I am c (...)

36 Precision judgements were carried out by one person with over twenty years of teaching experience (the author). Articulatory and co-articulatory imprecision, grapho-phonemic choice, intrusive sounds ([ˈhevri] for /ˈevri/) and the non-pronunciation of phonemes ([ʧaɪl] for /ʧaɪld/) were rated in the context of L2 English. All attested native variants were accepted in terms of regularly occurring elision and epenthesis. As for articulation, coarticulation, linking and lexical distribution of phonemes, all authentic sounding pronunciations within the framework of the chosen accent were accepted, i.e. errors denote non-English phonological features such as misplaced aspiration, inappropriate palatalization, and also inconsistencies between two native systems (e.g. the tapping of <tt> in glittering in a near-RP accent). Such inconsistent pronunciations are not really mistakes inasmuch as mixing accents rarely leads to communication breakdown. On the other hand, if L2 phonology describes the acquisition of the ability to successfully communicate in a second language and if that entails the use of phonemes that are similar enough to the target phonemes, distributed in the appropriate words, linked together with neighbouring sounds in a natural fashion, stressed, unstressed, reduced, connected to surrounding words in accordance with the target L1 phonology, then instances of a phonological feature such as tapping or rhoticity that are absent or have a different patterning in a given native model can logically be defined as a mistake.14

37 Effects of prosody, namely lexical and phrasal stress were considered with regard to the rhythm of the verses as well as the expected regular reduction of function words (as in the reading of a non-poetic text; see the transcription of the poem in the Appendices for more detail). The consequences of prosodic difficulties (the absence or presence of vocalic reduction, for example) were retained as mistakes in the analysis but not the prosodic domain per se. Also, all degrees of reduction (when appropriate) were accepted in function words, not only the forms that appear in the transcription; for example [ˈwʌn əv ˈθɹiː], [ˈwʌn ə ˈθɹiː], [ˈwʌn əf ˈθɹiː] for one of three. Finally, when several attested pronunciations are available for a content word, all alternatives are taken into consideration (e.g. /ˈmerɪnɚ/ and /ˈmærɪnɚ/, /ˈlɔːŋ/ and /ˈlɑːŋ/, etc.).

38 The phonetic analysis indicated which phonemes were misarticulated, which co-articulatory or linking processes were missing or inappropriate, and also the type of (co-)articulation error. The qualitative analysis provided a binary variable (correct/incorrect) per word per speaker that served in the quantitative analysis of error prediction.

3.2.4 Summary of descriptors

39 So far, I have established and described reader-independent factors that might predict pronunciation mistakes. Table 2 below provides an overview of the different variables: the phonological property of articulation complexity expressed at the word level, the lexical property of learner difficulty level captured by the word’s CEFR classification representing its frequency and familiarity, and the additional lexical properties of frequency, and length. It is expected that the combination of low articulation complexity and an A1 level (low surprisal) will just as frequently lead to learner mistakes as the combination of high articulatory complexity and a C2 level (high surprisal).

40Table 2. Variables used in the analysis of the occurrences of mistakes in the reading task



Articulation Complexity

The total of articulatory effort estimated for each phoneme in the word

CEFR level

The frequency and familiarity of learner usage of a word

Surprisal Score

The aggregate score of the two previous values

Surprisal Category

The division of the above scores into three categories of surprisal

Log10 Frequency

The logarithmic value of the lexical frequency of a word form in COCA


The number of phonemes in a word

4. Findings and discussion

41 This section discusses the main findings of the study, first presenting general characteristics and then assessing the performance of the model and its viability in L2 English. As Table 3 below indicates, individual competences (general English speaking skills and reading skills) distribute the number of mistakes unevenly across speakers. The overall rate of making mistakes in reading was fairly low: 158 words in 1608.

42Table 3. Number of words mispronounced per speaker















Number of words mispronounced




























4.1 General tendencies

43 Mistakes occur in different words, and as these occurrences are assigned a surprisal category from low to high (as discussed in 3.2.1) we can see their distribution in Figure 3. Most of the mistakes come from the central category, i.e. non-extreme surprisal (neither low, nor high). Mistakes are also frequently made on the ‘difficult’ words with high surprisal. However, since the mid-range has the most items in it, the distribution of the errors needs to be seen in more detail.

Figure 3. Number of mistakes per speaker according to surprisal categories (1 standing for low, 2 for mid and 3 for high values of surprisal)

Figure 3. Number of mistakes per speaker according to surprisal categories (1 standing for low, 2 for mid and 3 for high values of surprisal)
  • 15 In lexical words; elision in function words is considered normal.

44 Errors concerned consonants almost as often as vowel sounds (articulation, wrong phoneme choice and stress-related (non-)reduction). Further study of the chief categories of error types (Figure 4 below) shows that the majority of articulation problems concerned vowels, for example /ɪ/ in kin, din, /ʊ/ in stood or // in hold, stone, but some of them were related to phoneme choice (e.g. thou pronounced /ðoʊ/ or ancient pronounced /ˈenʃənt/), not the trajectory of articulatory gestures proper. The most frequent problems when elision15 occurred involved word-final consonants clusters (e.g. child, opened, guests). As for intrusive sounds, rare on the whole, they appeared slightly more often at the end of the word than in a word-initial position (e.g. holds off, heftsoons).

Figure 4. Proportion of mistakes within the three types of error: articulation (blue), elision (green) and epenthesis (orange)

Figure 4. Proportion of mistakes within the three types of error: articulation (blue), elision (green) and epenthesis (orange)

45 Finally, word forms that were mispronounced by the most speakers are child, opened and unhand (58% of the speakers), followed by ancient, for, glittering (50%), then hand, on and stood (41.7%), and din, eftsoons, stoppeth, stopp’st, thou, upon (33.3%). The top fifteen mispronounced words are followed by those occurrences that were misread by only two or three readers: beard, doors, guests, hath, hold, kin, loud, shone, Wedding-guest and breast, harbour, mast, ship, still, stone, wherefore, respectively. It can be noted that among the words that are most often mispronounced, some are not particularly rare (opened, stood, upon), and a few of them are even very common (for, on, hand, child).

4.2 The performance test of the model

46 Let us now turn to the predictions made based on surprisal categories. We have put forward a tight relationship between the surprisal category of a given word and the presence of a mistake and expect to encounter reader difficulties on low and high surprisal words proportionately more often than on mid-level surprisal words. The full dataset that contained reader mistakes had each word form from the text as its entry point without repetition, characterised by the absence or presence of a pronunciation mistake, the nature of the mistake when present, the surprisal category, plus lexical frequency and word length.

  • 16 For details of the model: see Appendices.

47 Generalised linear mixed models (GLMM) were fitted on the data set with word and speaker as random effects using the lme4, car and afex packages (Bates et al. 2015, Fox and Weisberg 2019, Singmann et al. 2020) in R (R Core Team 2020). The relationship between surprisal category and the presence of mistakes is weak (p=0.192), and the latter seems to be more correlated with the length of the word (p=0.045).16 Very slight changes appear in the significance estimates when the three categories of surprisal are transformed into a binary value: categories 1 (low) and 3 (high) against category 2 (mid) (p=0.097 for surprisal, and p=0.049 for length). Similarly, if integers are handled as factors in R (other than the numeric values of Log10 frequency and articulatory complexity), proportionately the same tightness appears between the variables (p=0.404 for surprisal and p=0.189 for length) and the number of mistakes.

48 The absence of tight correlation can be interpreted as a sign either of the weakness of the metric used, or of the irrelevance of the original idea of surprisal being useful in predicting learner mistakes. Overall, the number of reading mistakes was unexpectedly small for most of the speakers. One cannot help thinking that more speakers with weaker reading skills (producing more numerous mistakes), or the reading of a different, more difficult text would allow for a more solid approach and surer conclusions. The misgivings are especially due to the fact that the metric used to obtain the surprisal categories has not been tested on other datasets. It is therefore not certain whether the absence of a stronger link between surprisal and L2 reading mistakes results from the misconception of the metric or from a real absence of relationship between the two variables. Moreover, the results from the metric were transformed into a categorical variable; its use can be debatable as it replaces numerical values that real surprisal estimates operate on.

49 To improve on the efficiency of the surprisal estimates in the absence of contextual measures, additional features could be part of the aggregate score: presence of shared features across phonemes (place, manner, sonority for consonant clusters, for example), phoneme frequency, distributional probability in specific phonemic environments, effects of phoneme and word activation, collocation frequencies and above all, perceptual distinctiveness. Since perceptual distinctiveness values are not established for the whole inventory of English phonemes in the literature, this potentially important aspect was ignored in the measures of this study from the start. One can conjecture, for example for English, that tense vowels and vowels articulated in the peripheral zones of the vowel quadrilateral are potentially more distinctive than lax or reduced vowels and vowels produced at the centre of the articulatory space. It is also true that many vowels overlap in several varieties of English, hence perceptual distinctiveness varies across accents. As for consonants, the value seems to be more controversial to establish. Crucially, this feature might also be strongly influenced by the perceptual overlap between L1 and L2.

4.3 Further findings

50 It is incontestably a combination of factors that are at play. Nevertheless, when we look at mispronounced words and their CEFR category in isolation, there seems to be a tendency according to which the number of mistakes decreases as the words get more ‘difficult’ except for the most difficult words (see Figure 5 below). A simple Pearson's Chi-squared test (X-squared = 25.277, df = 5, p-value = 0.0001232) confirms that the correlation is significant. So, archaic words are difficult for the students, as are words at a C2 level in general, but when we consider easier words the tendency is reversed: A1 level words are more often mispronounced than C1 level words.

Figure 5. The number of mispronounced words according to the CEFR level (1=A1, 2=A2, 3=B1, 4=B2, 5=C1, 6=C2)

Figure 5. The number of mispronounced words according to the CEFR level (1=A1, 2=A2, 3=B1, 4=B2, 5=C1, 6=C2)

51 In opposition, when we compare words whose articulation is simple and words with more complex articulation, no clear tendency can be seen (Figure 6 below). So it seems that articulation alone is not likely to determine pronunciation difficulties. This aspect, however, is not very meaningful in isolation: it could gain in importance when related to perceptual distinctiveness as mentioned previously. The weakness of the metric used in the study might well be found in the application of articulation scores from L1 acquisition.

Figure 6. The number of mispronounced words (in black) compared to the total number of words (in green) according to the complexity of articulation (from the highest complexity at the top to the lowest at the bottom)

Figure 6. The number of mispronounced words (in black) compared to the total number of words (in green) according to the complexity of articulation (from the highest complexity at the top to the lowest at the bottom)

4.4 Surprisal values and the type of mistake

52 As described in 4.1, readers often made mistakes of different types. The analysis of low surprisal words in the reading corpus (such as a, day, came, hear, and, man, goes, for example) shows that mistakes are very rare but when they occur, most of the time they concern articulation and phoneme choice in vowels (e.g. /ʌ/ one, // came, /ɪ/ it, // or /əʊ/ below) and in consonants (e.g. /r/ here, are, hear, /t/ met). Two instances of the total of thirteen in low surprisal words were related to an intrusive /h/ (the words [hænd] and, [hɪz] is read by two different speakers). Mistakes in high surprisal word forms (such as minstrelsy, wherefore, thou, or quoth), on the other hand, were slightly more numerous and error types were more varied: not only articulation or phoneme choice (as in mast, feast, din, stopp'st or hath, still, stoppeth), but often elision (e.g. eftsoons, cleared, Wedding-guest), and sometimes insertion (breasts, heftsoons).

53 Initially, insights to be gained from the model consisted of expecting two different types of mistake at the two ends of the continuum: if low surprisal words are read with an error then the latter represents a new pattern or category rather than a shift towards an existing pattern which, in turn, is characteristic of high surprisal words. The interpretation of the results is problematic: the corpus is too small to point to any tendency since the proportion of mistakes is low in the whole corpus, and very few occurrences have an extreme value of surprisal. Ten times as many errors may provide enough material permitting to verify tendencies.

54 In addition to the technical difficulties of efficient measurements and their consequences commented on in the previous section, it may be just as important to control the composition of the corpus: to accompany, for example, the reading task with a different activity involving spontaneous speaking in which a rapid diagnostic assessment of general English skills would allow to work on more homogeneous speaker groups. Some of the mistakes (intrusive /h/, the noncompliant use of -s and -ed) are typically made by learners with weak speaking skills and/or low proficiency. Interpreting such occurrences on different word types in the context of pronunciation difficulties and surprisal could be performed more efficiently on a less heterogeneous group of speakers.

55 Also, prosodic rhythm and poetic rhythm may interfere and thereby directly affect the occurrences of mistakes. While both types of renderings were acceptable (function words and some weak syllables are concerned, as highlighted in the appendix), the immediate influence of the use of one or the other type of rhythm on the number of mistakes was not taken into consideration. The battery of musical rhythm if applied by the reader may play in favour of a good reading, but this supposition was not verified, and the two styles were often mixed within one reading. If we link this issue back to the construction of the metric, either including the interplay of poetic rhythm in the metric or separating this effect from the surprisal effect complicate the original task even further. On the other hand, the prose reading section in the MiReLeCo corpus, and being able to compare occurrences of the same word forms in both contexts could shed light on this question.

56 Finally, it feels artificial to categorize mistakes as structure preserving (approximating an existing pattern, vowel, or consonant) as opposed to mistakes introducing a new pattern or category. If all articulatory errors are considered as the former type (approximating a different but existing pattern) whereas elision and epenthesis are the latter type (introducing a novel form), then the predicted tendency is absent: low surprisal words do not systematically display mistakes introducing a new form and high surprisal words show mistakes of different types.

5. Conclusion

57 In this paper, I have attempted to draw an analogy between phonetic variation/change in L1 English and difficulties in L2 English pronunciation. I have postulated that words that are characterised by very high surprisal (unfamiliar words with very low frequency and complex articulation) or by very low surprisal (familiar words with high frequency and simple articulation) will be more frequently mispronounced by non-native speakers. The results from French learners of L2 English have not confirmed my hypothesis. There still might be a link between descriptors of L1 phonologization and those of L2 pronunciation difficulties. However, the investigation protocol and the robustness of the metric in particular have to be improved in order to confirm it. Learners make mistakes on words that are distinguished by their relative unfamiliarity/frequency combined with the use of complex articulatory gestures (high surprisal) but not systematically, whereas errors on low surprisal words are very rare. It was also found that some of the words that were the most frequently mispronounced are neither rare nor very difficult. Finally, as a general tendency, we could see that except for the highest CEFR level, the number of mistakes falls as we move from A1 (‘easiest’ words containing more numerous pronunciation problems) to C1 (more ‘difficult’ words having fewer errors).

6. Appendices

Appendix 1: Orthographic and transcribed versions of the text

58The Rime of the Ancient Mariner (text of 1834)
By Samuel Taylor Coleridge


It is an ancient Mariner,
And he stoppeth one of three.
'By thy long grey beard and glittering eye,
Now wherefore stopp'st thou me?

The Bridegroom's doors are opened wide,

And I am next of kin;
The guests are met, the feast is set:
May'st hear the merry din.'

He holds him with his skinny hand,

'There was a ship,' quoth he.
'Hold off! unhand me, grey-beard loon!'
Eftsoons his hand dropt he.

He holds him with his glittering eye—

The Wedding-Guest stood still,
And listens like a three years' child:
The Mariner hath his will.

The Wedding-Guest sat on a stone:

He cannot choose but hear;
And thus spake on that ancient man,
The bright-eyed Mariner.

'The ship was cheered, the harbour cleared,

Merrily did we drop
Below the kirk, below the hill,
Below the lighthouse top.

The Sun came up upon the left,

Out of the sea came he!
And he shone bright, and on the right
Went down into the sea.

Higher and higher every day,

Till over the mast at noon—'
The Wedding-Guest here beat his breast,
For he heard the loud bassoon.

The bride hath paced into the hall,

Red as a rose is she;
Nodding their heads before her goes
The merry minstrelsy.

The Wedding-Guest he beat his breast,

Yet he cannot choose but hear;
And thus spake on that ancient man,
The bright-eyed Mariner.


59 The transcription (in GA) marks stressed syllables with the diacritic /ˈ/ as expected in a regular reading task. An additional highlight set in bold marks the stressed syllable in each iambic foot that characterizes the ballad. Although a classical reading of the poem was not required, readers often followed the poetic rhythm instinctively. The metric pattern within each stanza of four lines of verse is based on four iambic feet (weak/strong syllables) in lines one and three, and three iambic feet in lines two and four. It is to be noted that the rhythmic pattern has a direct effect on pronunciation, namely in the non-reduction of function words and the final stress in mariner; these alternatives have been taken into account in the error score, and the transcription below is only one of several possible renderings.

ɪt ɪz ən ˈeɪnʃənt ˈmerɪnɚ
ən hi ˈstɑːpəθ ˈwʌn əv ˈθriː
baɪ ðaɪ ˈlɔːŋ greɪ ˈbɪrd ən ˈɡlɪtərɪŋ ˈ
naʊ ˈwerfɔːr ˈstɑːpst ðaʊ ˈmiː
ðə ˈbraɪdɡruːmz ˈdɔːrz ɚ ˈpənd ˈwaɪd
ənd ˈ əm ˈnekst əv ˈkɪn
ðə ˈɡests ɚ ˈmet ðə ˈfiːst ɪz ˈset
meɪst ˈhɪr ðə ˈmeri ˈdɪn
hi ˈhoʊldz ɪm wɪθ ɪz ˈskɪni ˈhænd
ðɚ wǝz ə ˈʃɪp ˈkwoʊθ ˈhiː
ˌhoʊld ˈɔːf ʌnˈhænd mi ˈgreɪbɪrd ˈluːn
eftˈsuːnz hɪz ˈhænd ˈdrɑːpt ˈhiː
hi ˈhoʊldz ɪm wɪθ ɪz ˈɡlɪtərɪŋ ˈ
ðə ˈwedɪŋ ˈgest ˈstʊd ˈstɪl
ən ˈlɪsn̩z ˈlaɪk ə ˈθriː jɪrz ˈtʃaɪld
ðə ˈmerɪnɚ ˈhæθ hɪz ˈwɪl
ðə ˈwedɪŋ ˈgest ˈsæt ɑːn ə ˈstoʊn
hi ˈkænɑːt ˈtʃuːz bət ˈhɪr
ən ˈðʌs ˌspeɪk ˈɑːn ðæt ˈeɪnʃənt ˈmæn
ðə ˈbraɪtaɪd ˈmerɪnɚ
ðə ˈʃɪp wəz ˈtʃɪrd ðə ˈhɑːrbɚ ˈklɪrd
ˈmerəli ˈdɪd wi ˈdrɑːp
bəˈloʊ ðə ˈkɝːk bəˈloʊ ðə ˈhɪl
bəˈloʊ ðə ˈlaɪthaʊs ˈtɑːp
ðə ˈsʌn ˌkeɪm ˈʌp əˈpɑːn ðə ˈleft
ˈoʊt əv ðə ˈsiː ˈkeɪm ˈhiː
ən hi ˈʃoʊn ˈbraɪt ənd ɑːn ðə ˈraɪt
ˌwent ˈdaʊn ɪnˈtuː ðə ˈsiː
ˈhaɪɚ ənd ˈhaɪɚ ˈevri ˈdeɪ
tɪl ˈvr̩ ðə ˈmæst ət ˈnuːn
ðə ˈwedɪŋ ˈgest hɪr ˈbiːt ɪz ˈbrest
fɚ hi ˈhɝːd ðə ˈlaʊd bəˈsuːn
ðə ˈbraɪd hæθ ˈpeɪst ɪnˈtuː ðə ˈhɑːl
ˈred əz ə ˈroʊz ɪz ˈʃiː
ˈnɑːdɪŋ ðer ˈhedz bɪˈfɔːr hɚ ˈɡoʊz
ðə ˈmeri ˈmɪnstrəlsi
ðə ˈwedɪŋ ˈgest hi ˈbiːt ɪz ˈbrest
jet hi ˈkænɑːt ˈtʃuːz bət ˈhɪr
ən ˈðʌs ˌspeɪk ˈɑːn ðæt ˈeɪnʃənt ˈmæn
ðə ˈbraɪtaɪd ˈmerɪnɚ

Appendix 2: GLMM results

60Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) [glmerMod]
Generalized linear mixed model fit by maximum
likelihood (Laplace Approximation) [glmerMod]
Family: binomial ( logit )
Mistake ~ SurprisalCAT + Length + FrequencyLog10 + (1 | Speaker) +
(1 | Word)

  AIC       BIC     logLik     deviance     df.resid

906.6      938.9     -447.3     894.6        1602

Scaled residuals:

  Min      1Q      Median      3Q       Max
-1.2915  -0.2935  -0.1983     -0.1296    5.5934

Random effects:

Groups         Name       Variance      Std.Dev.
Word          (Intercept)    1.5537       1.246
Speaker        (Intercept)    0.5141       0.717
Number of obs: 1608, groups: Word, 134; Speaker, 12

Fixed effects:

                        Estimate         Std.         Error z value            Pr(>|z|)    
(Intercept)            -5.034535        1.504227        -3.347            0.000817 ***
SurprisalCAT         0.486236        0.326249        1.490             0.136123
Length                0.258038        0.135667        1.902             0.057172
FrequencyLog10      0.005322        0.160776        0.033             0.973595
Signif. codes:
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation of Fixed Effects:

                  (Intr)      SrpCAT      Length
SurprislCAT    -0.671
Length          -0.553        -0.147
FreqncyLg10    -0.921        0.526        0.520

A (afex package)
Mixed Model Anova Table (Type 3 tests, KR-method)

Model: Mistake ~ SurprisalCAT + Length + FrequencyLog10 + (1 | Speaker) +

Model: (1 | Word)
          Effect          df          F          p.value
1      SurprisalCAT      1, 130          1.72           .192
2      Length          1,130        4.09*       .045
3      FrequencyLog10   1, 130          0.08           .774
Signif. codes:
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘+’ 0.1 ‘ ’ 1

B (car package)
Analysis of Deviance Table (Type II Wald chisquare tests)

Response: Mistake

                    Chisq       Df        Pr(>Chisq)
SurprisalCAT      2.2212   1          0.13612
Length            3.6176    1          0.05717
FrequencyLog10   0.0011   1          0.97360
Signif. codes:
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Top of page


Ankener Christine S., Sekicki Mirjana and Maria Staudte. “The Influence of Visual Uncertainty on Word Surprisal and Processing Effort.” Frontiers in Psychology 9 (2018): 2387 doi: 10.3389/fpsyg.2018.02387.

Block, Cady K. and Carryl L. Baldwin. “Cloze Probability and Completion Norms for 498 Sentences: Behavioral and Neural Validation Using Event-Related Potentials.” Behavior Research Methods 42(3) (2010): 665-670.

Davies, Mark. The Corpus of Contemporary American English (COCA): One billion words, 1990-2019. 2020,

Bates, Douglas, Martin Maechler, Ben Bolker and Steve Walker. “Fitting Linear Mixed-Effects Models Using lme4.” Journal of Statistical Software 67(1) (2015): 1-48. doi:10.18637/jss.v067.i01.

Best, Catherine T. and Michael Douglas Tyler. “Nonnative and Second-Language Speech Perception.” Language Experience in Second Language Speech Learning. Eds. Ocke-Schwen Bohn & Murry J. Munro. Amsterdam: John Benjamins Publishing Company, 2007, 13-34.

Browman, Catherine P. and Louis Goldstein. “Articulatory Phonology: An Overview.” Phonetica 49 (1992): 155-180.

Cambridge University Press. “Compiling the EVP.” English Profile: The CEFR for English. 2015,

Corral, Álvaro, Gemma Boleda and Ramon Ferrer-i-Cancho. “Zipf’s Law for Word Frequencies: Word Forms versus Lemmas in Long Texts.” PloS one 10 (2015): 1-23. doi:10.1371/journal.pone.0129031

Delogu, Francesca, Matthew W. Crocker and Heiner Drenhaus. “Teasing apart Coercion and Surprisal: Evidence from Eye-Movements and ERPs.” Cognition 161 (2017): 46-59, doi: 10.1016/j.cognition.2016.12.017.

Dressler, Wolfgang. U. “Principles of Naturalness in Phonology and across Components.” Natural Phonology: The State of the Art. Eds. Bernhard Hurch & Richard A. Rhodes. Berlin: Mouton de Gruyter, 1996, 41-52.

Eckman, Fred R. “On Predicting Phonological Difficulty in Second Language Acquisition.” Studies in Second Language Acquisition 4 (1981): 18-30.

Eckman, Fred R. “Typological Markedness and Second Language Phonology.” Phonology and Second Language Acquisition. Eds. Jette Hansen Edwards & Mary L. Zampini. Amsterdam: John Benjamins Publishing Company, 2008, 95-116.

Flege, James Emil. “The Production of New and Similar Phones in a Foreign Language: Evidence for the Effect of Equivalence Classification.” Journal of Phonetics 15 (1987): 47-65.

Fox, John and Sanford Weisberg. An {R} Companion to Applied Regression, Third Edition. Thousand Oaks CA: Sage, 2019.

Hale, John. “Information-Theoretical Complexity Metrics.” Language and Linguistics Compass 10/9 (2016): 397-412.

Hall, Nancy. “Articulatory Phonology.” The Routledge Handbook of Phonological Theory ed. S. J. Hannahs and Anna R. K. Bosch. London: Routledge/Taylor & Francis Group, 2018, 530-552.

Hume, Elizabeth and Frédéric Mailhot. “The Role of Entropy and Surprisal in Phonologization and Language Change.” Origins of Sound Patterns: Approaches to Phonologization. Ed. Alan C. L. Yu. Oxford: OUP, 2013, 29-47.

Hyman, Larry M. “Phonologization.” Linguistic Studies Presented to Joseph H. Greenberg. Ed. Alphonse Julliand. Saratoga: Anima Libri, 1976, 407-418.

Hyman, Larry M. “Enlarging the Scope of Phonologization.” Proceedings of the Symposium on Phonologization. University of Chicago, 2008, 382-409.

Kent, Ray D. “The Biology of Phonological Development.” Phonological Development: Models, Research, Implications. Eds. Charles A. Ferguson et al. Timonium, MD: York Press, 1992, 65-90.

Kuhl, Patricia K. and Paul Iverson. “Linguistic Experience and the “Perceptual Magnet Effect”.” Cross-language Speech Perception. Ed. Winifred Strange. Baltimore: York Press, 1995, 121-154.

Kuruvilla-Dugdale, Mili, Claire Custer, Lindsey Heidrick, Richard Barohn and Raghav Govindaraj. “A Phonetic Complexity-Based Approach for Intelligibility and Articulatory Precision Testing: A Preliminary Study on Talkers with Amyotrophic Lateral Sclerosis.” Journal of Speech, Language, and Hearing Research 61 (2018): 2205–2214. doi: 10.1044/2018_JSLHR-S-17-0462.

Kutas, Màrta and Kara D. Federmeier. “Thirty Years and Counting: Finding Meaning in the N400 Component of the Event-Related Brain Potential (ERP).” Annual Review of Psychology 62(1) (2011): 621-647.

Lado, Robert. Linguistics across Cultures. Anne Arbor: Arizona State University, 1957.

Major, Roy C. and Kim Eunyi. “The Similarity Differential Rate Hypothesis.” Language Learning 49 (1999): 151-183.

McAllister, Robert, Flege, James Emil and T. Piske. “The Influence of L1 on the Acquisition of Swedish Quantity by Native Speakers of Spanish, English and Estonian.” Journal of Phonetics 30(2) (2002): 229-258.

Monsalve Irene Fernandez, Frank, Stefan L. and Gabriella Vigliocco. “Lexical Surprisal as a General Predictor of Reading Time.” Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics (2012): 398-408. Web. Accessed 25 August 2020.

O’Rourke, Bernadette, and Joan Pujolar. “From Native Speakers to ‘New Speakers’-Problematizing Nativeness in Language Revitalization Contexts. Histoire Épistémologie Langage 35 (2013): 47-67

O’Rourke, Bernadette, and Joan Pujolar. “New Speakers and Processes of New Speakerness Across Time and Space”. Applied Linguistics Review 6.2 (2015): 145-150. doi: 10.1515/applirev-2015-0007.

R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2020.

Shaw, Jason A. and Shigeto Kawahara. “Predictability and Phonology: Past, Present & Future.” Linguistics Vanguard 4 (2018): 1-11. doi: 10.1515/lingvan-2018-0042.

Singmann, Henrik, Bolker, Ben, Westfall, Jake, Aust Frederik and Mattan S. Ben-Shachar. afex: Analysis of Factorial Experiments. R package version 0.27-2, 2020.

Staub, Adrian, Margaret Grant, Lori Astheimer and Andrew Cohen. “The Influence of Cloze Probability and Item Constraint on Cloze Task Response Time.” Journal of Memory and Language 82 (2015): 1-17. doi: 10.1016/j.jml.2015.02.004.

Stehouwer Herman, and Menno van Zaanen. “Language Models for Contextual Error Detection and Correction.” Proceedings of the EACL 2009 Workshop on Computational Linguistic Aspects of Grammatical Inference (2009): 41-48.

Trubetzkoy, Nicolai. Principles of Phonology. Berkeley, CA: University of California Press, 1939/1969.

Zarcone Alessandra, Marten van Schijndel, Jorrig Vogels and Vera Demberg. “Salience and Attention in Surprisal-Based Accounts of Language Processing.” Front. Psychol.7:844. (2016): 1-17. doi: 10.3389/fpsyg.2016.00844.

Top of page


1 Phonologization was originally defined (Hyman 1976, 2008) to refer to a change in a language system in which a phonetic feature becomes phonologically pertinent, for example when a particular set of phonetic variations leads to a change in a phonological unit or pattern. The concept implies the existence of a surface level of speech forms involving a considerable amount of inter- and intraspeaker variation, and an abstract level of speaker-internal phonological systems that operate with cognitive units either devoid of variation, or emerging as a result of variation.

2 I would like to thank Nicolas Ballier for his comments on an earlier version of this paper, as well as two anonymous reviewers whose remarks have greatly improved the text.

3 Hume and Mailhot’s (2013) model is based on the information-theoretic concepts of entropy and surprisal; the reader can refer to endnotes iv and v for the definitions and mathematical formulae, and will forgive me for turning key notions from information theory into broadly accessible general cognitive concepts that will allow anyone to grasp the claims made in the different models under discussion.

4 Shannon entropy is a mathematical measure that quantifies the amount of information in a given unit; it informs about the degree of randomness of each unit in a message based on the number of different values it could take. This means that a word with a high contextual entropy is not expected, whereas a word with a very low entropy is entirely predictable from the context.

5 “Information is defined probabilistically in terms of the amount of uncertainty (Entropy) associated with a message, encoded as sequences of symbols. A random variable, x, in a set of N elements has an Entropy, which is a function of the number of members in N and their probabilities. By giving a definite value to x, we remove Entropy and communicate information. Entropy is defined as average predictability within a given context, predictability itself being defined as Surprisal, which is the negative log of contextual probability.
Image 1006DC640000296F00000AC449BDBE812300A911.emfImage 100002010000019100000068307B7553156FFDCC.pngShaw et al (2018: 2).

6 I would like to hereby thank all the anonymous participants in MiReLeCo.

7 I am grateful to one of the reviewers for having addressed this question.

8 I am grateful to Stefan Frank (PC) who helped me disambiguate the question whether or not corpus-independent surprisal values can be obtained.

9 Cloze probability is used in neurolinguistics and refers to “the predictability of the sentence-final word on the basis of the prior context of the sentence” (Block & Baldwin 2010: 666). Expressed in percentage it ranges from 0% to 100% (representing the “percentage of individuals who would continue a sentence fragment with [the given] word frame” (Kutas & Federmeier 2011)): a high cloze value means that a high proportion of the participants provided a certain response to fill a gap in a sentence or passage. For an example in cloze task response time, see Staub et al (2015), for a brief history of sentence completion tasks, see Block & Baldwin (2010).

10 The term rhoticity refers to whether the letter ‘r’ is sounded before a consonant and at the end of the word before a pause (non-prevocalic environment). Non-rhotic varieties do not pronounce the letter in these positions, whereas rhotic accents do.

11 Eight of the twelve speakers in the study used General American and four of them near-RP to the exclusion of other native accents.

12 The aggregate score of surprisal was obtained by allotting equal weight to articulatory complexity (ranging from 1 to 51) and familiarity/frequency of usage (ranging from 1 to 6):
Surprisal Score = ((CEFR Level x 51) + (Articulatory Complexity x 6)) / 6.12

13 First quartile (Q1) = 20.1, the 50th percentile (median) = 34.56, third quartile (Q3) = 53.43; minimum value = 9.31, maximum value = 98.04, and the average is situated at 38.39.

14 In fact, a separate paper could be devoted to the question of what constitutes a mistake; I am certainly not a language purist and this is not an ideological paper, I do not think that it is wrong to mix up different accents. Who is to judge, in general, and along what criteria what a mistake is in new speakers of English (O’Rourke & Pujolar 2013, 2015)? The assessment protocol was defined according to a native reading model and deviations from that model were marked: for (near-)RP, not tapping an intervocalic /t/ was considered part of the system to the same extent as aspirating a foot initial /t/.

15 In lexical words; elision in function words is considered normal.

16 For details of the model: see Appendices.

Top of page

List of illustrations

Title Figure 1. Surprisal in Hume and Mailhot’s (2013) model: predicting what changes
File image/png, 41k
Title Figure 2. Surprisal in Hume and Mailhot’s (2013) model: predicting the type of change
File image/png, 38k
Title Figure 3. Number of mistakes per speaker according to surprisal categories (1 standing for low, 2 for mid and 3 for high values of surprisal)
File image/png, 12k
Title Figure 4. Proportion of mistakes within the three types of error: articulation (blue), elision (green) and epenthesis (orange)
File image/png, 45k
Title Figure 5. The number of mispronounced words according to the CEFR level (1=A1, 2=A2, 3=B1, 4=B2, 5=C1, 6=C2)
File image/png, 8.8k
Title Figure 6. The number of mispronounced words (in black) compared to the total number of words (in green) according to the complexity of articulation (from the highest complexity at the top to the lowest at the bottom)
File image/png, 14k
Top of page


Electronic reference

Monika Pukli, The Potential Role of Surprisal in Predicting Pronunciation Problems in L2 English PhonologyAnglophonia [Online], 30 | 2020, Online since 20 December 2020, connection on 27 January 2022. URL:; DOI:

Top of page

About the author

Monika Pukli

Université de Strabourg, LiLPa - Linguistique, Langues, Parole (UR 1339)

Top of page


Licence Creative Commons
Anglophonia – French Journal of English Linguistics est mis à disposition selon les termes de la licence Creative Commons Attribution - Pas d'Utilisation Commerciale - Pas de Modification 4.0 International.

Top of page
Search OpenEdition Search

You will be redirected to OpenEdition Search