Skip to navigation – Site map

HomeAnglophonia29The Lexicon of Argumentation for ...

The Lexicon of Argumentation for Argument Mining: methodological considerations

Patrick Saint Dizier

Abstracts

In this article, in contrast with numerous statistical approaches which show little interest for linguistic analysis, we propose methodological considerations aiming at identifying several categories of linguistic cues which are typical of argumentation and can be used for automatic argument mining within the framework of computational linguistics. These are elaborated from a corpus of arguments that was compiled for this study. An argument being a relation between a controversial statement (claim) and a support or an attack, we deal with the marks specific to each class and those related to the expression of the relations which hold between them. To conclude we propose an experimental model to evaluate the strength of each of these components. Some elements of an implementation and an evaluation are provided.

Top of page

Author's notes

I wish to thank Dr. Marie Garnier for her comments which greatly helped improving this contribution. I also thank two anonymous reviewers whose comments greatly contributed to improve this article.

Full text

1. Introduction 

1.1 The Structure of Argumentation

1An argument is a pair which includes a claim and one or more justifications, where some form of support is provided. Arguments are necessarily supported by one or more justifications even if some of them are implicit. Such implicit justifications are called enthymemes. Let us first outline the main theoretical characteristics of an argumentation and then focus on how argumentation is structured in areas covered by argument mining. It should be noted that an opinion is not necessarily supported by one or more arguments; it is just a personal view on an event or any state of affairs. An opinion is therefore not an argument.

2An argument is composed of a claim associated with one or more justifications. An argumentation pattern is a structure composed of one or more claims, frequently hierarchically structured, with related justifications (supports) and attacks. These can be associated with several claims. An argumentation pattern is therefore a complex lattice of claims, attacks and supports. The notions of attack and support are not necessarily very well defined and clear-cut; they are not easy to identify for readers and annotators alike. As a consequence, the construction of an argumentation lattice is quite complex. The difficulties encountered by guideline elaboration illustrate this problem, as described in (Janier and Saint-Dizier 2019).

3An argument is a complex structure composed of a claim and a set of propositions that support or attack that claim. Supports or attacks may concern other propositions which are then considered as secondary claims. An argumentation pattern is then gradually elaborated. This view was initially developed in modern terms in (Toulmin, 1958, 2001). The aim of such a pattern is to reinforce the strength of the propositions related to the claim or to cancel out their effect, for example via the attack of a proposition that supports or attacks the main claim. Supports and attacks define the polarity (i.e. for or against) of a proposition with respect to a claim. They suggest a bi-polar analysis of arguments since only attacks or supports are considered, even if some forms of concession or contrasts could also be included in an argumentative schema.

4Weak or strong forms of attacks or supports can also be introduced. Arguments are also a major component of rhetoric, in particular from the revival of this area, as developed in (Perelmann et al. 1969).

5Argumentation is often contrasted with explanation. Indeed, when one explains something to a listener, the aim is to provide some form of new knowledge to that listener or to help them modify their initial beliefs which may turn out to be incomplete or incorrect. This new knowledge is hypothesized to be true and non-controversial, unless otherwise stated. Argumentation does not bring any new knowledge to the listener: it is aimed at persuading them of the validity of a certain claim. However, in an argumentative discussion, it is frequent to have a combination of arguments and explanation. The difference between these two notions in discourse is highly pragmatic: it depends on the knowledge and the beliefs of the speaker and the listener.

6An argumentation can take different forms. For example, it can be a monologue, where propositions for and against a claim are developed, for example in news editorials. In that case, the author attempts to present an overall picture of the different positions for or against that claim. However, authors may have their own opinions and may provide a certain orientation to their texts, while attempting to be as objective as possible. They may also wish to anticipate attacks from readers by adding additional information in their text. An argumentation can also occur in a conversation between two or more speakers who express different points of view concerning a claim or a standpoint. (Moeschler 1985) develops an in-depth analysis and theoretical perspective of argumentation in conversation. TV on-line debates, deliberations (written or oral) or litigation situations (Janier and Reed 2015) are examples of this form of argumentation. Other forms of arguments and argumentations can be found on forums or on customer satisfaction web pages.

7Argumentation can be oral or based on written elements produced on the basis of various types of media. Argumentation may have two distinct goals: (1) convincing someone or a group of people of a certain point of view, as in a political campaign, or (2) coming to a reasonable agreement between two or more parties about a disagreement (e.g. in mediation and deliberations). In the latter case argumentation rules or protocols must be developed to guarantee that the outcome of the process can be accepted by all the parties at stake.

8When one argues for a given standpoint, they are the proponent of that standpoint. The individuals that disagree and argue against it are called the opponents, whatever the level of disagreement. Arguing is not demonstrating: a proponent presents good reasons to support a claim; they do not logically demonstrate that the claim is true. They simply give good reasons that justify the claim. A demonstration, on the contrary, is based on facts which can be verified, inference rules and axioms that can be described in a formal language. Argumentation is based on facts and schemes which can be questioned. Schemes are less formal than inference rules; they reflect common-sense forms of reasoning (Walton et al. 2008), (Walton 2015). It follows the well-known statement uttered by a judge: I need proofs, not arguments!

9Supports and attacks of a given claim are crucial components of argumentation which is based on the recognition of a difference of opinion between parties: these parties express doubts about the other party's standpoint. A preliminary step is to identify differences of opinion and then the basis on which they can be resolved. Doubts may bear on a unique or on multiple topics when the claim is complex or has multiple facets. Analyzing the structure of arguments and then evaluating them with respect to a given claim, within a certain context is the ultimate goal of argumentation. This perspective is described in detail in e.g. (Peldszus and Stede 2013).

10From a linguistic point of view, claims can be almost any kind of proposition (or statement). This difficulty and related effects are developed for example in (Amossy 2010), following principles stated in (Grice 1975). However, a claim has the form of an evaluative expression which, indeed can be debated. Claims include forms such as: theses, judgments, opinions, evaluations, rhetorical questions, etc. with the goal of putting forward a debatable topic. Claims can be introduced through epistemic expressions such as I think, it seems, or by performative verbs such as claim, recommend and their nominalizations, which indicate that a personal position is being uttered. A claim can be stated with a strong conviction or in a much weaker way as a possibility or a suggestion. Various operators express different levels of certainty, such as I am certain, I feel, it seems that, and these are crucial to identify the strength of a justification or an attack. The interpretation of these terms in an argumentative context remains, however, largely fuzzy: it may vary considerably depending on the type of debate being considered, or depending on the news editorial styles and aims, just to cite two examples.

11Finally, we consider, from a theoretical point of view, that argumentation is a cognitive and discursive action or process which is supported by a linguistic activity and the production of speech acts. The reverse perspective is sometimes developed in linguistic circles, for example by (Anscombre and Ducrot 1976), but we feel it does not make much sense in our approach, where the goal is to identify the conceptual structure of an argumentation, which may or may not be explicitly linguistically marked.

1.2 Computational Linguistics and Argument Mining

12Coming to the aim of this article, argument mining is an emerging research area which introduces new challenges both in linguistics, natural language processing (NLP) and in Artificial Intelligence. The objective is to mine propositions for or against a given claim, at various degrees, or controversial issue in various types of documents. Argument mining must identify the structure and orientation (support or attack) of these arguments. A second task, which is even more challenging, is to organize the supports and attacks that have been mined in relation with a controversial claim to produce a set of elements that is readable and usable for an end-user.

13For at least twenty years, the dominant trend in Computational Linguistics has been the use of various types of statistical models, viewed as black boxes, whose aim was to account for the structure of a linguistic phenomenon by using a sufficiently large set of annotated texts. Several parsers have been developed on that basis with some success compared to equivalent parsers solely based on linguistic analysis. Depending on the task, the accuracy of such systems ranges from 60 to 90%. Going beyond these values is then extremely costly, even with the use of hybrid systems or architectures, where statistics are combined with linguistic analysis. However, for complex tasks, such as argument mining, which involves a considerable amount of semantic and pragmatic considerations, such statistical approaches are not as accurate. In this article, besides a theoretical analysis of what argumentation is from a language processing point of view, we show that several features of an automatic analysis of argumentation can be successfully carried out from lexical considerations. These are relatively stable over domains but are not proper to argumentation, such as causal connectors. It is however too early to compare the statistical-based and linguistic-based approaches since evaluation methods dedicated to argument mining remain to be defined. There are indeed several components which are difficult to evaluate, starting, for example, with the problem of segmentation (e.g. where in a text a claim exactly starts and ends), which of course applies when we attempt to evaluate the accuracy of a given argument mining system.

14This article further illustrates and confirms the analysis provided in (Janier and Saint-Dizier 2019), and some of the trends of (Stede and Schneider 2019) which is a textbook presenting the various techniques for argument analysis and mining. The main focus in (Janier and Saint-Dizier 2019) is argumentation theory and its application to annotation methodologies, where an accurate state of the art in annotation techniques for argument mining is given. This topic is not developed here. Our aim is rather to show the feasibility of using well-categorized lexical resources for argument mining. This article is therefore both a methodological analysis and the presentation of a specific approach which illustrated this analysis.

15From a methodological point of view, the categorizations and the data presented below have been elaborated and compiled from corpus observations and annotations. This is a complex task which requires the development of stable guidelines over large sets of texts. Guidelines are also useful when such an investigation involves several annotators. Due to the sparsity of data in general, observations are completed by data found in dictionaries. For example, some causal connectors have a low occurrence frequency (for example therefore, resulting in), nevertheless, it is important to include them as possible cues. Dictionary data can be used efficiently for that purpose to enrich corpus observations by including for example synonyms or derived terms.

16Inter-annotator agreement tests, which we do not develop here, are used to measure if and how the different participants in an annotation task converge in their analysis or not. Due to the fact that annotation tasks are very time consuming, agreement is measured in general on the basis of three annotators doing the same task, with the same guidelines. Given that argument analysis is a notoriously complex task, even on simple texts, the agreement level for argument annotation remains in general quite low, around 60 to 70%.

17For the analysis presented below, we have considered several types of claims and related texts where supports or attacks can be found. We have eliminated the numerous duplicates, since commentators frequently copy one another. Our corpus has been validated regarding language and topic diversity (simple language versus more technical language for example), linguistic complexity and context: authors have a higher control of their arguments in some contexts whereas some others (blogs) can be excessive in their attacks.

2. Challenges of Argument Mining

18In an argumentative text or debate, propositions which act as justifications may take various forms. Because of the pragmatic nature of arguments, linguistic cues associated with these claims and their associated justifications are very diverse, sometimes ambiguous and, in fact, quite frequently implicit, in particular in English. This is developed in sections 4 and 5 below. A well known set of ambiguities are temporal marks where a temporal sequence of two events does not mean a causal link and therefore an argumentative dimension. Furthermore, justifications may not be adjacent to their related claim, therefore analyzing if and how a justification or an attack is related to a given claim requires complex identification processes and contextual knowledge since, for example, extensive lexical variations may be observed.

19Therefore, argument mining, which consists in identifying arguments in texts with a program, is a very challenging task that involves complex language analysis and parsing processes as well as taking into account reasoning and pragmatic processes. Identifying most of these linguistic features is necessary to reach an accurate automatic argument and argumentation analysis and to produce argumentation diagrams and various forms of synthesis. This is obviously a task that needs to be achieved step by step. To make a comparison, argument mining is much more complex than information retrieval. Indeed, argument mining is not just looking for precise data (a date, a name, etc. which is one the main goal of information retrieval) but it entails looking for at least two propositions and a justification or attack relation between them.

20The most crucial elements which can be expected from an argument mining system are, from the simplest to the most complex ones:
- the identification of claims in a text, also called viewpoints or stances, when these are not explicitly identified (contrary to a question-answering system where the question is given). This task includes evaluating whether a statement can be debated, via the evaluative elements it contains (see below), and identifying where the statement starts and ends. It can be a sentence, a shorter fragment or several sentences, in particular when restrictions are included. To simplify our presentation, in this article we consider that a claim is a sentence. In a natural language text, claims as well as sub-claims can be found. Identifying this hierarchy is complex for humans and even more for a machine;
- the identification of statements related to these claims, potential supports or attacks. This task is very complex when there is no explicit connection with a claim. In general contextual knowledge and specific forms of reasoning are required;
- the analysis of whether these statements can act as justifications to form arguments, and their orientation (attack or support); this includes the delimitation of the text span which represents the statement, similarly to claims;
the identification of the strength inherent to each attack and support.

21More complex research issues include:
the identification of the argumentation scheme that has been used for each claim-support or claim-attack pair. This is an important issue to estimate the validity of an argument. An argumentation scheme binds a claim with a justification or an attack in a way that is not purely based on logical principles. Argument schemes (Walton et al 2008) are based, for example, on expert opinions, doxa, causes, or similarity between situations,
for a given claim, the construction of a synthesis that helps readers to have an overall understanding of the pros and cons related to that claim, with their strength and occurrence frequencies. Argumentation graphs may be useful to represent small sets of supports or attacks. Many other types of information may be useful to have a real-life and concrete analysis of argumentation which can be useful to users, for example to help them make decisions.

22In the following sections, we illustrate the different facets of argument mining, in particular from the lexical point of view, as it is probably the most stable and the simplest at this stage, although linguistic cues may not be as clear-cut as expected and may in fact be largely ambiguous with other constructions (for example the causal construction which is not argumentative a priori). It is obviously not possible to cover all the lexical aspects in this article: we focus on a few illustrative ones so that the reader might have a better grasp of the dimensions of the problem.

3. Corpus Elaboration: methodological considerations

23As indicated above, lexical data useful for argument mining can be identified in at least two ways which can be combined to a certain extent: (1) through manual identification, using dictionaries and our own knowledge of argument structure or (2) via the annotation of argumentative texts by trained annotators. The first option requires a relatively modest corpus aimed at providing examples from which lexical data is induced. It focuses on the main linguistic structures and lexical data which are typical of argumentation. This is illustrated in the next sections. The second option requires a very large corpus since the hypothesis is that only the lexical data found in the corpus is considered and kept in a parser. Furthermore the annotation task is very time consuming, and subject to relatively frequent disagreements between annotators. We have been involved in annotation guideline preparation and revisions for the annotation of the Moral Maze (BBC radio debates, transcribed version) with the University of Dundee in aifDB format (http://corpora.aifdb.org/​). The reader will notice the large amount of data and the difficulty of providing reliable and homogeneous annotations for such a corpus. Annotations have been revised at least three times to reach a consensus. This took about three years of work for several, highly-trained annotators. Any change in the analysis or any adjunction of a feature entail a costly revision of the annotated corpora.

24It is clear that the second option is more valuable from a scientific point of view, since the data that is collected result from real observations. However, results are lists of lexical terms, including full expressions, with no internal organization. In our analysis of a the Moral Maze annotations, more than 70% of these terms are very specific and contextualized and appear less than three times, therefore their accuracy and relevance may be questioned. The first option, although more subjective and more limited, produces results which are more structured and relevant from a linguistic point of view than statistical-based approaches since this approach is based on a priori existing lexical categories that are gradually both refined and enriched manually.

25Our approach, due to limited human means, is based on the first option, but with the use of a limited annotated corpus, as a means of providing a more empirical basis to our analysis, and to have an estimate of the coverage of the phenomena which have been observed at the lexical level. The detail of our corpus is given below. This approach, which is a feasibility analysis, provides an estimate of the lexical coverage of our analysis. In this corpus, rhetorical effects, emphasis or forms of trickery, frequent in argumentation, have been reduced as much as possible: our aim is to focus on the fundamentals of lexical expression in argumentation.

26In spite of these methodological limitations, we consider that this work is a valid investigation, probably preliminary, whose main aim is (1) to gradually develop ready-to-use lexical resources for argument mining, even limited ones, (2) to offer more relevant and sound annotation guidelines, as it is the case for most argument annotation projects, (3) to evaluate the impact of knowledge in argument mining processes, which turns out to be as crucial as the linguistic data and finally (4) to suggest new directions for investigation.

27This lexical data can then be used by different, more or less complex types of parsers, which identify the different classes of propositions playing a role in argumentation. At the moment, there are very few parsers or statistical-based systems for argument mining, since research is still in an early development stage; furthermore, evaluation methods for such parsers remain to be defined.

28For this first experiment, we considered the four following issues, which involve very different types of arguments, from concrete to abstract, and diverse language realizations. These claims are:
(1) Ebola vaccination is necessary,
(2) Women's conditions have improved in India,
(3) The development of nuclear plants is necessary,
(4) Organic agriculture is the future.

29To these issues, we added some data from the Moral Maze, annotated by students of the Dundee group. These issues have been selected because they involve a large diversity of expression modes and supports or attacks. For most of them, they are large public topics which do not require any specific knowledge to be understood.

30Texts were collected on the web, using the above issues as web queries. The text fragments which were selected are extracts from various sources where these issues are discussed, in particular newspaper articles and blogs from associations. These documents are publicly available; they can therefore be understood by a standard reader. Sources are, for example, for issue (1): The Lancet (a British medical journal), The Howard Hughes Medical Institute, and a web site named "Ebola vaccines, therapies, and diagnosis."

31A large number of texts (over 30 texts, of an average size of 300 words) were collected for each issue. For each text, the next task was to manually identify arguments, since texts may be quite verbose, may include factual data of no argumentative interest, and have pictures or data that cannot be taken into account in this investigation.

32As a result, from the four issues above, a set of 624 arguments was collected, with 162 different arguments and an overlap rate of about 4. The overlap rate measures the number of arguments which address the same topic, but with different terms, or forms of rephrasing, which may have a specific rhetorical effect. The number of different arguments is not very large for each issue: on average 25 arguments for the four first claims and 62 arguments for the Moral Maze corpus, which covers several topics. This is not surprising since debates are in general quite focused and arguments on a given topic are not so diverse. Two elements are worth noticing: the linguistic variations observed (leading to an overlap in arguments) and the occurrence frequency of the lexical data that is used. Depending on the issue, we observe an overlap rate which ranges between 3 and 5. However if the corpus were larger, this rate would certainly be higher. This is particularly useful to us since the identification of linguistic variations for a given argument is a good indicator of the lexical variation which can be expected. Those variations occur in different texts, where authors tend to copy-paste others and add nuances to statements they think are, for example, too strong or too weak.

33This overlap rate is somewhat subjective since it depends on each annotator's analysis and on corpus size. In spite of this subjectivity, this rate gives an interesting rough redundancy level. This rate is likely to increase with the size of the corpus, while the number of new arguments would gradually decrease. A more detailed analysis of those repetitions would be of much interest from a rhetorical and sociological perspective.

34The last step in our corpus analysis consists in tagging the discourse structures found in those sentences identified as arguments so that the lexical data can be identified and categorized. For that purpose, the <TextCoop> platform (Saint-Dizier 2012) is used to identify discourse structures, with an accuracy of about 90%, computed from a comparison with a manual annotation, since the structure of arguments is in general relatively simple. Discourse structures which are identified in our corpora are those usually found associated with arguments: conditions, circumstances, causes, goal and purpose expressions, contrasts and concessions, as illustrated below.

35The goal is to automatically identify the kernel of the argument (tagged <main arg>), which is in general the main proposition of the sentence, and its sentential modifiers, analyzed as discourse structures by <TextCoop>. In addition, the discourse structures may give useful indications on the argumentation strategy that is used. Here is an example of the tagging in XML at this stage (there are many other possible representations, see e.g. (Peldszus and Stede 2013):

<argument>
<concession> Even if the vaccine seems 100% efficient and without any side effects on the tested population, </concession>
<main arg> it is necessary to wait for more conclusive data before making large vaccination campaigns. </main arg>
<elaboration> The national authority of Guinea has approved the continuation of the tests on targeted populations.</elaboration>
</argument>.

36Then, these representations are controlled manually. After this stage we have a validated corpus of arguments which can be used to identify the lexical data presented above, which will be used for an automatic recognition of arguments, their orientation and strength. This corpus is the development corpus, i.e. the corpus from which the system is developed, while a second, independent corpus, needs to be developed to serve as the test corpus, i.e. a corpus which is aimed at evaluating how relevant and accurate the implemented system is. This latter corpus is discussed in section 7. The categorization task associated with this identification of lexical data is described in the next section.

4. The Lexical Patterns for Claims: a gradual categorization approach

37The identification of the lexicon of argumentation is based on two factors:
(1) the analysis of the impact of usual semantic and syntactic classes of terms such as, for example, psychological verbs or claim verbs,
(2) the analysis of the structure and the content of a statement and different parameters such as its strength and its argumentative orientation (for or against an issue). For example in ‘
The Ebola vaccine is dangerous because its adjuvant is toxic’ the claim (the Ebola vaccine is dangerous) is supported by a fact with the same orientation (negative) and a high strength since ‘toxic’ is an adjective with a strong negative orientation.

38The definition of the lexicon of argumentation requires an iterative and intensive categorization activity where the usual categories may be gradually refined or reshaped for a more accurate understanding of arguments. From a methodological point of view these categories and subcategories progressively emerge through corpus analysis and intuitive generalizations based on linguistic knowledge. These intuitions can then be evaluated against a test corpus. Let us now further investigate the two points advocated above before we present our main results.

39As we said it earlier, a claim must introduce a debate: it must therefore be based on some form of evaluation, a thesis, a belief, a position, a rhetorical question or an assumption. A claim states something that is plausible but which may be questioned, possibly in a number of ways, in particular if the claim has several facets. This specific character of a claim can be induced by listeners from pragmatic factors. In a number of circumstances, this specific character is linguistically marked, as illustrated below. For example, in nuclear energy must be banished, must and banished both induce some form of debate. These marks may be ambiguous because of an overlapping with other forms, in particular explanation that we have mentioned above. In that case, only the co-text or pragmatic factors may be of any use. Causal forms may also be ambiguous as they may be mistaken for argumentative constructions.

40Although a local form of syntactic and semantic analysis is necessary to identify the debatable character of a statement, for example to identify a comparative form or the expression of a doubt, it turns out that identifying claims on the basis of specific lexical items is relatively efficient and rather simple. Let us review here some of the main classes of lexical items which are frequently encountered in any type of text or dialogue, elaborated from or confirmed by our corpora. The presence of one of these items is in general sufficient to assume that the statement at stake can be a potential claim. It becomes a real claim and therefore the basis for an argument when it can be bound to a support or an attack, arguments being relational objects. If no attack or support can be found, then the statement is considered at this stage as an opinion.

41Let us now describe the main lexical patterns or categories which are typical of claims. These are:
- adjectives (ex.
expensive, cold, illegal, unfair, pleasant), either positively or negatively oriented. They evaluate an event or the property of an object that is central to the statement which can become the main controversial element of an argument,
- adverbs and adverbial locutions which focus on a notion of necessity or possibility also induce claims (ex.
necessarily, presumably),
- adverbs related to a temporal dimension may also be at the basis for a claim since the temporal dimension is fuzzy or imprecise and therefore can be questioned (ex.
frequently, occasionally, never, usually),
- finally, modality adverbs directly induce a debatable character to the statement they modify (ex.
surely, probably, obviously).

42For example, statements such as the following can be analyzed as potential claims because of their evaluative dimension(s) (the relevant elements are underlined, they are frequently copular constructions):

Vaccination campaigns will become most probably unnecessary in the near future.
The vegan diet is
certainly the most ecological diet.

43If we now consider predicative forms, let us mention:
- verbs with a positive or negative orientation such as
improve, disturb, or verbs expressing causality such as entail or provoke confer a debatable character to a statement when the causal link is not straightforward. These are frequently encountered in arguments;
- modals (ex.
could) and modal expressions (ex. it is possible that) may also result in forms of debates through the uncertainty they convey;
- expressions with a frequent ironic connotation or interpretation (e.g.
strangely enough), which convey forms of irony may also induce claims, since irony is a way to challenge a listener;
- finally, verbs or verb compounds which express propositional attitudes (ex. 
believe P), psychological aspects (ex. feel P) or epistemic constructs (ex. recommend P) introduce a claim P. They describe a personal point of view which can be discussed.

44Note that verb structures can be modified by the types of adverbs mentioned above or by a negation.

45For example, the following propositions are candidates for claims because of their verbal structure:

I feel nuclear energy should be banished.
Women’s safety has
largely deteriorated in new Delhi over the last three years.

46From a methodological point of view, at this stage of our experiment, it is not clear whether these relatively well-known semantic classes need to be further decomposed for the purpose of improving argument mining analysis accuracy. However, some internal organization is necessary as will be seen below to deal with the evaluation of strength.

5. Identifying Supports and Attacks

47While a claim can be identified on the basis of the typical lexical items it contains, the situation is more challenging concerning the identification of supports and attacks. There are several reasons for this situation. First, these may not be adjacent to the claim: it is frequent, given a claim, to find supports or attacks in remote parts of a document or in different documents such as those found on the web (a claim is viewed as a query from which supports or attacks can be mined in different documents). Similarly, in a discussion, the different protagonists discuss a point, debate a subtopic and then may come back to the initial one. Detecting such argumentative paths is tedious for an audience and obviously even more difficult for a machine with limited contextual knowledge and inferential capabilities.

48The second reason for these difficulties is that the conceptual elements which contribute to establishing a relation between a claim and a support or an attack, called relatedness, may be quite indirect and may require various forms of knowledge and inference, including lexical inference. If in some cases identical terms are used in the two parts of the argument, it is nevertheless frequent to have no lexical intersection. For example, the relation between the claim:

Vaccination is dangerous

and a support such as:

the adjuvant is toxic

requires to know that a vaccine’s active principles are diluted into adjuvants which are also injected into humans, hence the risk of toxicity. Such forms of knowledge and reasoning are relatively simple for humans, but they are really challenging for machines. No measures of semantic similarity, to the best of our knowledge, would be able to carry out this task accurately. Concerning this problem, our goal in this article is to outline such difficulties, which are investigated further in (Saint-Dizier 2015) and (Janier and Saint-Dizier 2019).

49If we now consider the situation where a claim is adjacent or close to its supports or attacks in a text, the problem of relatedness is less crucial. Connectors between these propositions are frequently used and may be taken into account when they are not implicit. Indeed, in a large number of cases, such propositions are simply juxtaposed, using punctuation. The comma is frequently used instead of explicit connectors in texts which have a style that is direct and which are easy to read. In more elaborate documents, such as juridical deliberations, news editorials or technical requirements, connectors are explicit in order to reduce ambiguities and make the argumentative articulations clearer. In such documents, the problem of relatedness is less crucial since one can assume that the focus of the discussion has not evolved. A support or an attack can be in the same sentence as the claim or in close sentences in the same paragraph or dialogue turn.

50A support can be introduced by various types of connectors, in particular:
- causal connectors, for example:
because, since, resulting in,
- connectors describing a result or forms of examples which may be challenged, such as:
therefore, such as.

51Other types of connectors, with a causal orientation, can introduce supports as well as attacks; they include connectors or expressions describing a result, such as: thus, hence, resulting in, leading to, etc. The distinction between supports and attacks is frequently made on the basis of the notion of contradiction characterized by the presence of a negation in the attack or an element which is negatively oriented if the claim is positively oriented or vice-versa. This polarity change is not necessarily straightforward: lexical data must be structured according to the antonymy relation, and its variants, to detect orientation changes.

52Finally, attacks as well as supports, since they are related to a claim, must also have an evaluative dimension, which can, in turn, be challenged by other statements which are then interpreted as supports or attacks. These evaluative constructions use some of the linguistic categories presented above for claims, in particular:
- scalar adjectives,
- modal adverbs,
- verbs with a positive or negative polarity,
- report verbs, epistemic constructions and adverbials,
- propositional attitude verbs and verbs denoting forms of acceptance or rejection such as:
agree, deny, argue, refute, acknowledge, reckon, disagree, accept, reject, possibly combined with modals or negation. For example, the statements above are potential supports or attacks:

I reckon the positive aspects of the ecological dimension of a vegan diet.
I
disagree: nuclear plants are clean and well overlooked by competent staff.

53In our analysis, the formal bi-polar dimension introduced by supports and attacks needs some nuances. We frequently observed a kind of continuum between these two poles, with, for example, weak attacks or weak supports. Furthermore, attacks or supports may concern only a certain facet of the claim: they are then partial. In the above example:

Vaccination is not recommended because the adjuvant is toxic

54The facet ‘adjuvant’ of the vaccine is considered, but not, for example, its costs for poor countries or the difficulties to preserve it in good conditions.

55It may be of interest to consider these weak forms as concessions or contrasts, where a concession is a weak support and a contrast is a weak attack, with variable degrees. To support our observations, specific connectors are used to express the notion of concession and contrast. Although they can also accompany strong attacks or supports, they are, in particular:
- for concessions: although, though, even if, despite, in spite of, however, nonetheless, nevertheless and
- for contrasts: while, whereas, but, in contrast to, yet, instead of, rather, unlike for contrasts.

56Some connectors may introduce concessions as well as contrasts; in that case identifying the orientation of the proposition is a matter of pragmatic evaluation. Modals such as would can also contribute to weaken a support or an attack.

6. The Challenge of Strength Evaluation

57 It is of much interest to evaluate the strength of a claim and the strength of a support or an attack separately and then to bind them. Strong claims expect or induce in general stronger forms of supports or attacks than claims with a moderate strength. Therefore, if the strength of a claim or of a support or an attack can be evaluated separately, then the evaluation of a support or of an attack must be tuned with respect to the a priori strength of the claim to determine the overall strength of an argument. Furthermore, for example in debates, because of politeness, attacks may be rather weak and expressed with great care even if they address a major disagreement concerning crucial issues. Strength is then a parameter where different contextual and rhetorical considerations must be taken into account. Persuasion is another dimension to consider, but which is more contextual: the profile of the protagonists must be taken into account in order to evaluate the impact of such as function. It is not considered here. In our approach, measuring the strength of an argument is therefore relatively empirical and results from the combination of several factors, mainly semantic and pragmatic ones, which are not easy to measure. It must be contrasted with the notion of illocutionary force, as introduced by J. Searle (Searle 1962) and a few others, which is essentially associated with the notion of speech act, with its pragmatic dimensions (Van Vallin and La Polla 1997). In particular, illocutionary force is measured between dialogue turns via the possible dialogical transitions which bind these turns.

58 Strength can be measured from a logical and pragmatic perspective or it can be measured from a linguistic point of view. Both approaches are not necessarily coherent but they must be combined to produce a relatively accurate measure of strength. We address here the second option, which is the least complex of the two in terms of feasibility and reusability. Let us note for example (Grasso et al. 2011) that presents an in-depth analysis of persuasion and which investigates the content of persuasive messages. The sensitivity to argument strength of various populations is developed in e.g. (Corella et al. 2004). The impact of strength in conjunction with rhetorical questions is also a crucial factor. A number of these linguistic factors are analyzed in e.g. (Anscombre and Ducrot 1976) and later (van Eemeren et al. 1996, 2001).

59 A number of language resources have been developed to evaluate strength, in general out of context. Let us note for example the Stanford Sentiment Treebank (e.g. https://nlp.stanford.edu/​~socherr/​EMNLP2013_RTNT.pdf) which is a well developed and stable resource used for opinion analysis to measure the strength of an opinion. However, to the best of our knowledge little has been done to characterize argument strength from a linguistic point of view with its contextual environment, which can include a paragraph or more. (Janier and Saint-Dizier 2018) describe the main resources which are available for that purpose and how they can be used.

60 From a linguistic point of view, strength is characterized by the types of terms which are used in the claims on the one hand and in the supports and attacks on the other hand. We can again consider the lexical items presented above in sections 3 and 4, since they also convey some form of strength. We have for example positively oriented verbs (ex. benefit, guarantee, consolidate) and negatively oriented ones (ex. alter, ruin, damage). The same remark holds for adjectives (ex. useful, consistent vs. polluted, harmful), nouns (ex. risk, reward) and for some classes of adverbs (ex. probably, usually, surely). Finally control constructions such as believe, assume, certify, which subcategorize for a proposition which may be an attack or a support, also convey some strength in their affirmative character or in the nuances they introduce.

61 It is now important to elaborate a formalization which can capture in a flexible and realistic way the a priori strength of such lexical items. It is not realistic to assign, out of context, a strength value to lexical items such as those given above. This would be too subjective and error prone since it does not take the context into account. A relatively feasible and simple way to achieve a form of evaluation is to develop scales along with precise dimensions, for specific classes of lexical items such as verbs or scalar adjectives, where their relative strength is described. Such scales do not allow for the association of precise strength values to a statement, but they nevertheless allow us to compare various propositions and to identify among them those which are the strongest and those which are the weakest. This is a crucial result for argumentation graphs where the challenge is to measure the interactions between supports and attacks and how they reinforce or cancel out the effects of others.

62 Let us introduce here a few examples of scales, as presented in (Cruse 1986), where these are called non-branching proportional series, meaning that they are linear, with a partial order, and that they do not develop trees. These scales are based on some precise property which is gradable. These are found in ontologies or terminologies. Properties may be concrete, such as temperature, or more abstract, as those developed below. When these are abstract, scales are more subjective. In such a scale the most negative terms appear to the left, then, gradually, less negative ones are mentioned until a neutral term or point is reached, then positive terms are given in increasing order. The scale is thus symmetrical and has a priori no values attached to terms to quantify their negative or positive character. When terms cannot be distinguished by the criteria at stake, they just appear together. This approach defines a partial order on the terms classified by the scale.

63 For instance, an interesting and large scale is the scale related to the informal property ‘damage actions’ about an entity. The terms it is associated with frequently occur in arguments. For example, the negatively and positively oriented verbs which can be included into this scale can be structured as follows, where opening and closing brackets are used to delimit subsets of terms:

[[ruin] - [break, demolish] - [hurt, alter, lessen, undermine, damage] - [affect] - Neutral –
[preserve, guarantee] - [benefit] - [improve, consolidate, strengthen] - [optimize]].

64Note that there is a priori no neutral term in this scale because there is no lexical item available, so the meta-term ‘Neutral’ is used instead to mark it. The lists of terms to the left of the neutral point are negative, the leftmost term being the most negative. Similarly for those terms to the right of the neutral point, the rightmost is the most positive. This scale is somewhat subjective and must be adapted to the context, although the orientation and the strength of some terms are rather stable over domains. Finally, note the partial order: in this scale, the terms:

[hurt, alter, lessen, undermine, damage]

are assumed to have approximately the same impact. Granularity considerations must be taken into account in such modeling: it may not be necessary to go into very refined semantic distinctions between these terms.

65A scale that represents a kind of `safety' dimension can be developed as follows:

[[dangerous, harmful] - [risky, polluted] - [unsafe] –
Neutral –
[useful, beneficial] - [safe] - [certified]].

66Other adjectives could be added in specific contexts where they are used such as ‘tested’. Finally, here is a scale for propositional attitude verbs:

[[deny - refute - reject] - [disagree] –
Neutral –
[believe, think, accept] - [agree, acknowledge, reckon] - [argue]].

67The verbs to the extreme parts of the scale are more crucial in the acceptance or rejection of the claim than those close to the neutral point. There are several such scales which can be defined. Then, given a statement, claim, support or attack, when several such terms are used in a proposition, an equation which combines these different terms in a compositional way can be defined, although it is also quite subjective.

7. Implementation and Preliminary Results

68The lexical resources presented in the previous sections above have been implemented and integrated into the TextCoop platform, a platform designed to perform discourse analysis within a logic and linguistic perspective (Saint-Dizier 2012). Since the system is declarative and modular, the integration of such lexical data is quite straightforward, however, their semantic types must be specified accurately to avoid any conflict or ambiguities with similar terms which could have been specified for other constructions, such as causal expressions.

69For example, for adjectives such as: easy, difficult, toxic, efficient, unsafe, lexical entries such as the following can be defined. These are expressed as predicates with three arguments: their conceptual domain (for example manner or state), their orientation (positive, neutral, negative) and their strength (low, average, high, but more detailed data can be introduced depending on the scale profile):

adjective(manner, positive, average) --> [easy].
adjective(manner, negative, average) --> [difficult].
adjective(state, negative, high) --> [toxic].
adjective(process, positive, average) --> [efficient].
adjective(process, negative, high) --> [unsafe].

where the element to the right of the arrow is the word to be derived from the terminal symbol `adjective', given the property values specified in the body of the predicate.

70Verbs with a positive or negative polarity are described in a similar manner. If we omit the sub-categorization frame and the morphological aspects, of little interest for the present purpose, we have the following examples, where the first argument of `verb' encodes the orientation and the second the strength:

verb(positive, average) --> [protect].
verb(positive, low) --> [prevent].
verb(negative, high) --> [destroy].
verb(negative, average) --> [damage].

71Adverbs express different features such as necessity or possibility. They can also be intensifiers or have a temporal dimension which can be debated. Their polarity is of interest when it exists as well as their strength:

adverb(intensifier, _ , low) --> [possibly].
adverb(necessity, _ , high) --> [necessary].
adverb(intensifier, _ , average) --> [very].

72Lexical resources of adverbials are represented in a similar manner, noted also here for the sake of simplicity ‘adverbs’:

adverb(opinion, _ , strong) --> [most, definitely].
adverb(opinion, _ , weak) --> [usually].
adverb(opinion, _ , average) --> [surely].
adverb(opinion, _ , weak) --> [probably].

73Adverbs as well as verbs or adjectives can be classified in more than one category. More examples can be found in (Janier and Saint-Dizier 2020), where grammatical patterns are also developed and which shows how these resources are used by a parser.

74Claims as well as attacks and supports are recognized on the basis of these lexical entries which are included into patterns or rules in the TextCoop format. To give the reader an idea of those rules, here are two simple examples. The first example describes an adjective phrase (for example, very toxic):

adj(SemType, Orientation, Strength) -->
   adverb(intensifier, _, St1),
   adjective(SemType, Orientation, St2),
   {combine(Strength, St1, St2)}.

75This rule says that an adjective phrase is composed of an adverb of the type “intensifier” followed by a terminal symbol adjective, as defined above. The predicate ‘combine’ is designed to evaluate the overall strength of the adjective phrase from the strength of each of its components (St1 and St2). Then, for example an evaluative expression based on a scalar adjective can be defined as follows with the integration of the auxiliary construction ‘be’ (e.g. could be dangerous):

Evaluative(SemType, Orientation, Strength) -->
   aux(be,St1),
   adj(SemType, Orientation, St2),
   {combine(Strength, St1, St2)}.
aux(be, neutral) --> auxiliary(be).
aux(be, Strength) -->
   modal(Strength),
   auxiliary(be).

76The last two rules define the structure of an auxiliary phrase where, for example, it can be just the verb or auxiliary ‘be’ or a composition of a modal followed by this auxiliary.

77The same type of rule has been developed for verbs and the other lexical categories described above. A total of 31 rules have been defined following this schema, which account for the different constructions presented above. This is not a large number of rules compared to language parsers, which include several hundreds of rule, most of them recursive, and it allows an easy revision of these rules quite reliably. Selective binding rules are then included to establish links between claims on the one hand and supports or attacks on the other hand. For the time being, only adjacent structures are considered since the problem of relatedness remains largely open when these structures are not adjacent or at least not in the same text paragraph.

78Let us now consider the question of evaluation. Evaluation is a difficult task since it is based on manually annotated texts which are compared to the system's performance. Two problems arise: manual annotation is difficult and largely error prone, and vice-versa, the system may recognize arguments where annotators haven’t annotated any. Next, the type of text, how well it is written, its domain, etc. are parameters which have a deep influence on performance. If we consider a well-written text transcribed from the BBC Moral Maze, where connectors are frequently explicit, given the restrictions presented above on adjacency, we found a recognition rate of about 76%, with a low level of noise of about 10% (structures incorrectly recognized as arguments). Very preliminary tests have been carried out on texts related to social aspects: these are much more verbose than the Moral Maze, with a lot of information which is not argumentative. As a consequence, the risk of ambiguity is higher, and the manual annotation is also more challenging. Results should be around 50% for the recognition rate. Finally for texts with a low level language quality, such as blogs or forums, it seems to be necessary to associate the argument mining system with specific algorithms dedicated to the treatment of deteriorated language.

Epilogue and Perspectives

79In this article, from a theoretical analysis of what argumentation is from a language processing point of view, we show that several features of an automatic analysis of argumentation can be successfully carried out from lexical considerations. These are, furthermore, relatively stable over application domains but are not proper to argumentation, such as causal connectors. Our approach is in contrast with the main trends in Computational Linguistics which aim at defining ‘black boxes’ on the basis of statistical methods, where there is little interest for linguistic analysis. Our approach is preliminary and must be viewed as a feasibility study. However, hints are given on the way such a system can be implemented by a classical system of rules. This partial implementation gives interesting partial results and shows the gaps which must be investigated.

80In this article, we have presented the main lexical dimensions which are considered in argument mining annotation tasks and system development. The reader can note that, although some lexical items can really help to automatically recognize and characterize arguments, there is still a long way to go to build accurate and reliable argument mining systems. The main reasons are polysemy and the ambiguity of a number of lexical items, the impact of contextual factors which are difficult to model in any language processing system and the diversity of the language which is used. Nevertheless, it is of much interest to investigate the lexical dimension, which is the simplest and the most stable of these aspects, but of paramount importance in the field of argumentation. Then, the next step would be the introduction of partial syntactic analysis and, most importantly, the introduction of knowledge and reasoning, as humans do when they argue.

81Besides contextual factors proper to a text, domain knowledge as well as general purpose knowledge is frequently necessary to identify and characterize arguments, for example what is exactly supported or attacked, why and how. The strength of an argument also frequently requires considering extra-linguistic knowledge besides the cues presented here. Using knowledge, whatever it is, to identify arguments, requires the development of dedicated inference patterns which simulate how knowledge is used in argument understanding by humans. We feel that without this reasoning dimension, argument mining will have low accuracy scores. However, incorporating such a dimension in a real-life computer system is definitely extremely costly and, obviously, challenging.

82As we have underlined in the previous sections, another challenge of argument mining is evaluation. It is certainly possible to develop test corpora and to compare the results of a system with manual annotation. However, as we have seen, manual annotation is not so reliable, with an accuracy of about 70% to a maximum of 80%. Then, we feel evaluation must show some form of flexibility. For example when identifying a text span which may potentially be a claim, a support or an attack, an accurate evaluation should consider that even if there is a difference of one word in the span, the system results should be marked as positive. Such flexibility is not yet in the tradition of evaluation methods which are too rigid for argumentation mining.

Top of page

Bibliography

Amossy, R., L’argumentation dans le discours, Armand Colin, 2010.

Anscombre, J. C., Ducrot, O., L’argumentation dans la langue, Mardaga, 1976.

Cruse, A., Lexical Semantics, Cambridge University Press, 1986.

Corella, J., Spencer, S., Zannab, M., An Affirmed Self and an Open Mind: sensitivity to argument strength, journal of experimental psychology, vol 40(3), 2004.

van Eemeren, F., Grootensdorf, F., Hoekemens, F., (1996). Fundamentals of Argumentation Theory. Routledge.

Van Eemeren, F., Grootensdorf, F., Hoekemens, F., Argumentation, Analysis, Evaluation, Presentation, Routledge, 2001.

Grasso, F. Ham, J., Matshoff, J., User Models for Motivational Systems – The Affective and the Rational Routes to Persuasion – UMAP Workshops, 2011.

Grice, J. B., Logic and Conversation, Academic Press, 1975.

Janier, M., Reed, C., Towards a theory of Close Analysis for Dispute mediation Discourse, Journal of Argumentation, vol 22, 2015.

Janier, M., Saint-Dizier, P., Evaluating the Strength of Arguments on the Basis of a Linguistic Analysis : A synthesis, 18th workshop of Computational Models of Natural Argumentation CMNA, pp 32-39, London, 2018.

Janier, M. Saint-Dizier, P., Argument Mining: LinguisticFoundations, J. Whiley, 2019.

Moeschler, J., Argumentation et Conversation, Hatier, 1985.

Peldszus, A., Stede, M,. From argument diagrams to argumentation mining in Texts: a survey, International Journal of Cognitive Informtics and Natural Intelligence, vol 7(1), 2013.

Perelman, C., Olbrechts-Tyteca, L., The New Rhetoric: A Treatise on Argumentation. University of Notre-Dame Press, 1969.

Saint-Dizier, P., Processing natural language arguments with the TextCoop platform, journal of Argumentation and Computation, vol 3(1), 2012.

Saint-Dizier, P., Argument Mining: the Bottleneck of Knowledge and Language Resources, in proceedings of LREC 2015, Portoroz, 2015.

Searle, J.R., Speech Acts: An Essay in the Philosophy of Language. Cambridge University Press, 1969.

Stede, M., Schneider, J., Argumentation Mining, Morgan and Claypool, 2019.

Toulmin, S.E., The Uses of Argument, Cambridge: Cambridge University Press, 1958.

Toulmin, S.E., Return to Reason. Harvard University Press, 2001.

Van Valin, R.D., La Polla, R.J., Syntax: structure, meaning, and function Cambridge text books in linguistics, vol. 37, Cambridge University Press, 1997.

Walton, D., Reed, C., Macagno, F., Argumentation Schemes. Cambridge: Cambridge University Press, 2008.

Walton, D. Goal-based Reasoning for Argumentation. Cambridge: Cambridge University Press, 2015.

Top of page

References

Electronic reference

Patrick Saint Dizier, The Lexicon of Argumentation for Argument Mining: methodological considerationsAnglophonia [Online], 29 | 2020, Online since 16 December 2020, connection on 26 January 2021. URL: http://journals.openedition.org/anglophonia/3187; DOI: https://doi.org/10.4000/anglophonia.3187

Top of page

About the author

Patrick Saint Dizier

Directeur de recherche CNRS, Institut de Recherches en Informatique de Toulouse (IRIT)
Patrick.Saint-Dizier@irit.fr

Top of page

Copyright

Licence Creative Commons
Anglophonia – French Journal of English Linguistics est mis à disposition selon les termes de la licence Creative Commons Attribution - Pas d'Utilisation Commerciale - Pas de Modification 4.0 International.

Top of page
Search OpenEdition Search

You will be redirected to OpenEdition Search