Figure 1: The University of Texas (UT) at Austin: Main campus with UT Tower
Source: https://news.utexas.edu/2017/05/04/bridging-barriers/ Introduction
1In the field of testing, four main criteria have been set to guarantee the quality of the results (Benson 1998; McMillan 1999; ALTE 2011; Markle & al. 2014). A good test must be feasible, which means that the material conditions are met. Secondly, it must be valid (Cronbach et Meehl 1955; Bolton 1987; Bachman 1990; 2007; Brown 1996; Benson 1998; Mc Millan 1999; Alderson 2000; McNamara 2000; ALTE 2011; Markle & al. 2014). Validity considers the construct of the assessment in terms of content and structure. It means that the test must focus on the knowledge or skills that are being tested by the evaluation itself. Validity also has to do with interpretation, such as professional judgment. Face validity, i.e. the appearance of the test, is not enough: “Validity refers to the appropriateness of the inferences, uses, and consequences that result from the assessment” (McMillan 1999: 5). A test must also be reliable: “Reliability is concerned with the consistency, stability, and dependability of the scores” (McMillan 1999: 6). This means that a reliable evaluation will yield the same results when repeated.
2Finally, fairness must be considered: a fair assessment is not biased or in favor of one subgroup of test-takers (genre, ethnicity, and so on) over another. In other words, “a fair assessment is one that provides an equal opportunity to all students to demonstrate achievement” (McMillan 1999: 7).
3It may also be useful to distinguish between “low stakes assessments,” those that evaluate the institution and not the test takers per se, or “planning classroom interventions for individual students”, and “high stakes assessments”, those “with consequences for the assessed” (Mehrens 1998: 4). In the following study, high stakes assessments are involved within the context of institutional assessments in a public American university, the University of Texas at Austin (UTA). Although exams are not meant to be scientifically certified in the same way as language tests, it seems that the very concepts of feasibility, validity, reliability, and fairness should not be overlooked when one deals with assessments within the context of higher education which involves considerable investment on behalf of the students, particularly in the United States with its high tuition costs, with huge consequences on their future.
- 1 Such personal considerations may sound inappropriate in a research paper. However, they are meant t (...)
4When I first set foot on UTA’s campus, on January 13, 2019, I must confess that designing a feasible, valid, reliable, and fair assessment of my students for my two classes on British poetry was not particularly on my mind.1 I wanted to immerse myself in the beauty and freedom of this open-air campus, where lofty trees compete with the roofs of neo-classical buildings.
5On that first day, in Calhoun Hall (Figure 2), home to the English department, I received a warm welcome from the Head of the department and staff. A technician guided me around the building in which my two classes were to take place: in each room, a computer was awaiting me with all sorts of peripheral devices, among which a document viewer and every manner of adaptors forming a sort of clawed paw. I wished we had all these conveniences in the French university where I normally work, not to mention the size of the office, to which I had just been given the key. Then I suddenly remembered the students’ tuition fees in the United States which were skyrocketing — about $15,000 per year for a Texan student, and twice as much for an out-of-state student.
6There were free buses, bikes, free repairs, huge trees (not the kind they are planting in France) allowed to grow as they wish, mockingbirds, and squirrels everywhere. And no security guards to whom one had to prove one’s right to be there. I will miss all of this on my return to France, I thought, feeling nostalgic already.
Figure 3: The Harry Ransom Center
Source: https://www.austinchronicle.com
7On the other side of Calhoun Hall, like the Cullinan in the Tower of London, the Harry Ransom Center (Figure 3), a literary jewel set in its green velvet case, which gave you access to all sorts of old and recent manuscripts and works of art from writers and artists from all over the world. A heaven and haven for researchers. La vraie vie! True life no longer existed elsewhere. Only the here and now in this literary sanctuary.
8I would like to mention one more thing that I did not immediately notice on my first day on the campus: there was no smoke in the vicinity. Then I saw it: the sign saying that UTA was a “Tobacco-free campus”.
Figure 4: Tobacco-Free Campus Sign
Source: https://www.facebook.com/TacosNotTobacco/
9You could have added in a tobacco-free city (I can count on two hands the number of times I could smell cigarette smoke during my 6-month stay). “Tobacco-free, but not gun-free” I was told from the very beginning. Here we are. Is it not frightening to think that one of your students might have a gun?
10If any problem occurs, such as a robbery at a near-by shop, one immediately gets a message on one’s smartphone calling for witnesses; about one hour later, another message comes as a relief: a suspect has been arrested. “Thank you to everybody for your help”, says the police officer — the campus has its own police station. For a French person like me, this fact alone was quite surprising since police are usually not welcome on French campuses. In Austin, however, everyone still had the 1966 Tower shooting in mind in which 14 people were killed. In June 2015, after another shooting at a university in Oregon, the Texas Legislature passed a law authorizing ‘concealed carry’ on campus, i.e. people were allowed to carry concealed guns. UTA’s President and Board were firmly against this decision, but they had to comply because UTA depended on state subsidies.
11The city of Austin remains a Democratic oasis in an ocean of Republicans and other NRA supporters.
12That is the situation. It’s best to be in the know.
13Even if the exam issue was not on my mind on that first day on campus, it had been there a while before. I had to send in my class descriptions a year in advance, including detailed requirements (uploading the full schedule and the dates of when all the exams were due only a few days before the beginning of the semester). The online course description was of great help.
14As I was given the opportunity to spend an entire semester teaching in an American university, I thought it would be interesting to compare the way American and French students were assessed, specifically with exams. Starting with the peculiarities of my own university, in which students are expected to hand in at least two papers per class (usually prepared in class, not at home), where attendance is optional, and marking traditional French (using a scale going from 0, the lowest mark, to 20), I wondered about the special features of assessments at UTA. How many papers did the students usually have to hand in, and what types were required (essays, reading reports, quizzes, oral presentations, etc.)? Were attendance and participation compulsory and part of the overall final grade? What percentage of the final grade were the essays worth? To what extent were some procedures implemented, including peer review, revision opportunities, and tier-grading?
15I had not thought of the “safety issue”, with students being allowed to carry guns on campus. I decided to add this dimension to my research questions.
16What are the main features of assessments in the English department at the University of Texas at Austin (UTA)? Are there any factors (such as money, the presence of firearms, or grade expectations for courses in the Humanities) that put pressure on professors and force them to raise grades to avoid confrontational situations?
17This paper will more precisely address issues regarding the organization of exams in terms of the number of papers, requirements, and grading in the English department at UTA for undergraduate students during the 2019 Spring semester.
18Methodologically, I will first analyze the online descriptions of all the English courses offered that semester. This quantitative analysis will highlight the main features of assessments in the context of this university and make it possible to confront them with the issues of reliability and fairness. Then, I will conduct a qualitative analysis of six interviews with professors who explain how they evaluate and grade their students. This second type of data analysis makes it possible to discuss the validity of the construct and, to some extent, the issue of proper grading. To what extent do the interviewees feel forced to raise grades and why? Is the gun issue a relevant factor?
19Finally, I offer a case study which focuses on an upper-division course of English Romantic Poetry given by the present researcher to a class of 26 students, including first-year (freshmen) to fourth-year (senior) undergraduate students majoring in English or other subjects. This will offer some ways to answer the second question, that of formatting a valuable course that results in higher grades.
20UTA has over 51,000 students and 3,000 teaching faculty, with an alumni base of 480,000 people. It is oriented towards the sciences, humanities, and the arts. According to U.S. News & World Report, UTA is one of the top 20 public universities in the country and has top graduate programs in accounting, Latin American history, and petroleum engineering — in addition to more than 15 undergraduate programs and more than 40 graduate programs ranked in the top 10 nationally (see the university website’s ‘facts and figures’ page). The English program is not the most prominent program in terms of the number of students: students enrolled in English classes for the 2019 Spring semester represented by 3,709 seats (early-semester enrolment snapshot), including undergraduate and graduate courses); nor is it the most important in terms of faculty (108 members: 34 professors, 31 associate professors, 4 assistant professors, 1 instructor, 2 senior lecturers, 6 lecturers, and 30 professors emeriti). Still, UTA’s English department has a strong tradition in Literature and Creative writing (including some well-known authors who teach there).
21Here are the results of our quantitative analysis of the online descriptions of all undergraduate English courses for the 2019 Spring semester in terms of requirements and grading. Overall, 84 lower-division and upper-division courses (including creative writing) have been analysed considering four dimensions:
-
Number of essays (papers)
-
Other types of assessments (quizzes, reports, oral presentations, etc.)
-
Participation and attendance
-
Indication of peer review and revision policy
22Our purpose was to describe the overall exam policy within the department and to determine to what extent it contributed to enhancing quality assessments.
23Although written assignments can take different forms, essay-writing remains the paragon of academic assessment.
Figure 5: Number of essays per course
*This figure does not include the professional outcomes class, which focuses on writing CVs and oral presentations, nor the creative writing and rhetoric courses, which require a written assignment for every class.
Source: Claire Tardieu.
24In 41% of the courses, students had to submit three essays. The number of courses with fewer or more than three essays is evenly distributed (29% and 30%). Essay-writing is worth, on average, 64% of the final grade.
Figure 6: Compulsory attendance and participation
Source: Claire Tardieu.
25The importance of attendance and participation was specified in 60% of the course descriptions, while only 24% did not mention it. The remaining 17% of the courses, for which attendance and participation were implicitly crucial, were creative writing and rhetoric courses. On the whole, 76% included this requirement.
26The average percentage of attendance and participation in the final grade amounted to 17.5%.
27Along with essay writing, students had to submit other types of papers: quizzes, short assignments, presentations, annotated bibliography, etc.
Figure 7: Other types of requirements
Source: Claire Tardieu.
28Only 10.5% of the courses did not mention other types of assessments. At least one other type was indicated in 34.5% of courses, two to four in 36%, and five or more in 19%. This shows that essays are not the sole mode of assessment and it demonstrates how professors vary their assignments creatively.
29The average percentage of other types of assessments in the final grade amounts to 17.5%.
Figure 8: Peer-review and revision policy
Source: Claire Tardieu.
30Regarding peer review and revision policy (the possibility given to students to improve their papers and get a higher grade), 51% of the courses did not mention it, 30% explicitly mentioned it, and 19% mentioned it implicitly (especially for creative writing and rhetoric courses in which students are invited to share their productions and continuously revise them).
31In almost 80% of the courses, final grades include attendance and participation. Still, it represents only 17.5% of the final grade.
32Essays (three per semester on average) remain the most important type of assignment in the final grade, worth on average 64% of the final grade.
33Giving other types of assignments is the norm (only 11% do not mention it) but holds only a minor part of the final grade (17.5%), the same percentage as attendance and participation.
34Peer reviews and a revision policy are the norms for nearly the majority of the courses, at 49%, although it is not necessarily mentioned in the grading policy but often appears in the course description. When taking this remark into account, the percentage rises to more than 52%.
35To further understand the ins and outs of the exam policy in the English department at UTA, I conducted six semi-directed interviews of six different professors, yielding a corpus of 3.33 hours of audio recordings.
36The professors were chosen for the originality of their course description available online, and, more specifically, for some peculiarities including:
-
Peer review and revision
-
Take-home final exam
-
Grade tiers
-
Grading.
37The professors had different profiles, as outlined in Table 1.
Table 1: The profiles of the six professors interviewed
Course code and title
|
Professor
(M: Male; F: Female)
|
Position
|
Teaching experience at UTA
|
Interview date and length
|
E329R The Romantic Period
|
Prof.1 M
|
Post-Doctoral lecturer
Finished PhD in the previous year
|
Less than a year
|
11 March 2019
14’37”
|
380F Literature for Writers
|
Prof.2 M
|
Associate professor
|
15 years
|
12 March 2019
45’22”
|
343L, Modernism and Literature
|
Prof.3 F
|
Post-doctoral Lecturer
Finished PhD in the previous year
|
7 years
|
13 March 2019
37’14”
|
338E, British Literature: Victorian through WWII
|
Prof.4 F
|
Post-doctoral lecturer Finished PhD in the previous year
|
7 years
|
14 March 2019
29’07”
|
371K, Modern and Contemporary Poetry
|
Prof.5 M
|
Assistant professor Promoted to Professor
|
9 years
|
8 April 2019
23’38”
|
364T Eng Lang & Its Social Context
|
Prof.6 F
|
Associate professor of English, rhetoric, and writing, semi-retired
|
35 years
|
11 April 2019
25’42”
|
38One may note that all but one had a minimum of 7 years of teaching experience at UTA. Even Prof.1 is not a novice teacher, however, having had three years of prior teaching experience at the University of Pennsylvania.
39Each interviewee was asked the same 8 questions. The first four questions dealt with their personal and professional data, the next four focused on special features in their way of assessing students, and the last one centered on the grading issue without mentioning the 2015 gun law directly.
-
Gender
-
Position/status
-
Length of service at UTA
-
Courses taught
-
Mode of assessment
-
Mode of grading
-
Peer review and revision policy
-
Higher grades in the Humanities
40The interviews yielded four main aspects for analysis: peer review and revision policy, take-home final exam, specification grading, and overall grading.
41Prof.1 explained that he uses peer review before and after handing in papers, especially for his classes with the “writing flag”, which is part of UTA’s Core Curriculum Requirements and involves a lot of writing. He highlighted the importance of providing detailed notes in the margins of the essay as well as final summative comments. The students first get a tentative grade, and then, after revising their paper, a final grade.
42One sustains the impression that Prof.1 is concerned with training his students until he can proceed to a summative assessment and grade. In other words, formative assessment is the whole process, even for exams.
43Prof.2, who teaches creative writing, went even further by saying that failure is part of the assessment: “It is in this context, obviously, a very subjective assessment. And I think I need to leave room for that, and I have some students who are experimenting, and you know, experiments often fail.”
44According to him, peer-review in groups of 3 or 4 before they hand in their paper is what brings the objective into the subjective, especially when it involves reading short critiques prepared by the other students. Still, it is a delicate operation, because writing fiction involves emotions and personal feelings: “And the peer review is that it’s incredibly difficult, particularly when you’re attached to the material in some emotional way, [it’s] incredibly difficult to discern if something is good and something maybe needs to be redirected.” However, by being trained to critique one another, in the end, he says, the students develop self-reflection.
45These two samples put our French system into question, in which continuous assessment (at least 2 papers per class) does not seem to allow peer-review and/or revision, nor the possibility to give tentative grades, except in particular situations.
46Prof.1’ described as follows his assessment procedure:
-
Class preparation and participation: 10%;
-
Weekly discussion posts: 30%;
-
Short paper: 20%;
-
Final Exam (take-home): 40%.
47In my French university, due to different events (student strikes followed by the COVID crisis), exams have been carried out online for the past two years, which has been a negative experience according to the majority of professors. One main reason for this negative opinion is the risk of cheating and plagiarism.
48Contrary to this experience, Prof.1 said that at UTA:
Plagiarism is rare. It has often been students that are struggling in the class and are just trying to sort of find the easiest way of dealing with this problem. I haven’t really found issues of plagiarism. Especially because they are fully aware that when they submit it on Canvas [a CMS equivalent to Moodle], there is often a plagiarism detection software on Canvas.
49This denotes two main attitudes: the first is being aware of the causes of plagiarism; the second is the reassurance that plagiarism is made unlikely thanks to software used on the platform where students submit their written assignments. Another argument for take-home exams, according to Prof.2, is that the type of work required makes it difficult to resort to plagiarism. It seems that professors do not fear getting papers written by people other than their students, which is a serious obstacle according to French professors. This is perhaps because American professors at UTA, who on average have only two classes to teach per semester (who meet twice a week), entertain a closer relationship with their students who can also come and see them easily during office hours and at other times. Trust is not a vain word. The students feel challenged by the assignments and they want to do well. The way the curriculum is organized in France is rather different: students have two to three times more classes to take and they meet with their teachers only once a week with hardly any possibility to see them outside class. Such conditions do not favour the same type of trust relationship as at UTA.
50Prof.3 was interviewed because of the specific type of grading system that she implemented in her course. The online description reads: “This course uses a system of assessment called specifications grading, an alternative to a points-based grading system. Students choose the grade “tier” (A, A-, B+, B.) they plan to work towards at the beginning of the semester, then earn the grade associated with that tier by completing all tier requirements at a satisfactory level or above.” Prof.3 clarifies that this form of “grading agreement” involves students individually choosing a tier and then the group determines the specific criteria within the grading tier.
51Such an approach implies a more detailed breakdown of the different requirements. Prof.3 went on:
The students who are looking for an A have to complete 8 author thinking pieces; one-page single-space papers on different themes and novels and they meet in small groups. Each group sets criteria for preparation. Some decide to bring three written responses, some decide to bring questions for discussion, passages, lots of different activities. They decide the criteria and the sub-readings schedule in each group. Then, I check in with them individually and give them feedback to clarify the expectations for what they produce each day. With other groups, I am asking them questions. They complete the group evaluation at the end.
52This system leaves room for the students’ own choice of tier and then a group decision on criteria. Individually, they decide for which grade they are aiming (A or B, B+) and, in groups, they decide on the work and the number of activities they will have to do to achieve the desired grade. The process involves 4 weeks of “traditional teaching” and then the groups are formed. Therefore, in the same class, one has students whose objectives are to get an A or a B. How is the grade validated? According to Prof.3, to earn the grade they are aiming for, the students have to complete the requirements at a satisfactory level (it does not have to be perfect: “And instead of assigning letter grades for individual assignments, I assign satisfactory or incomplete. […] And the way that they earn a higher-grade tier is if they complete more of those assignments or they complete some pieces or more complex assignments.”) Usually, the A students are required to revise some of their work, and the B students are not. B students have slightly looser attendance requirements, whereas attendance is very strict for A students, as well as participation, preparation, etc. There is more flexibility for B students. No student aims for a C, because, in Prof.3’s class, all major in English. Prof.3 also uses tokens. Students have four tokens that they can use for attendance or exchange for an absence or a revision for a higher grade. Revision is compulsory for the A group. For Prof.3, “the process is more important to me than the, like, kind of finished product.”
53How do the teachers grade the students? Do they give them an A easily? Before we examine the answers, I wish to recall the four models discussed by French scholar Christian Puren (2014). The first is a rising model: at the beginning, everyone is credited with zero and earns points moving upwards, depending on their performance; the second type is the top-down model, very common in France, where everyone starts with 20 out of 20 and gradually loses points for each mistake; the horizontal model, more typical in British or American systems, starts from an average grade, like a C, and allows upwards or downwards movements along the line; and finally, the transversal model applies to grading pedagogical projects when both the outcome and the learning process must be taken into consideration.
54It seems that all six interviewees adopted the most common model in the USA: the horizontal one.
55Prof.1 considered C to be the average. At the beginning of the semester, every student is credited with a C and all the work and assignments contribute to rising above C and earning a B or an A (rather exceptional for Prof.1).
56Prof.6 had a similar approach. C is the average: Final grades include “plus” or “minus” grades. Final class scores may be rounded up or down, according to students’ class participation and performance on minor and ungraded assignments. A C grade will indicate work that meets all the basic course requirements; As and Bs are honors grades, designating work of some distinction. Grades are based only on work assigned to everyone in the class; no extra credit work can be accepted.
57Prof.3 stood as an exception for that matter, as seen above, since the students decided what grade they were aiming for, as well as determining the criteria to earn that grade, provided that they performed the activities at a satisfactory level. Although Prof.3 stood apart, her assessment policy belongs to the same type: the horizontal model with a flexible average.
58As for Prof.2 (creative writing), he acknowledged giving an A to his students if they did the work properly and their writings were satisfactory. It seems that his starting grade was not a C but a B, allowing the students to walk out with an A if the critiques they had been writing were satisfactory. He also relied on their capacity of revising and improving their work: “I will tell them, you know, right now, you know, you’ve got a high B or you’re in a good spot to get an A and, you know, it’ll depend on the revision, you know, um, I’ll try to give them as much as I can.”
59He even experimented giving no grades for one semester but confessed it did not work: “And there are some professors though, the visiting ones who don’t have, you know, are just sort of passing through, who will not assign grades until the very end. And I think I tried that in one semester and it just drove everyone crazy.”
60Different grading systems are compared in Table 2.
Table 2: Table of equivalence between grading systems
US GPA
(grade-point average)
|
US letter grade
|
US percentage grade
|
French scale
|
4
|
A+
|
97-100
|
18-20
|
4
|
A
|
93-96
|
17-17.9
|
3.7
|
A-
|
90-92
|
15-16.9
|
3.3
|
B+
|
87-89
|
13-14.9
|
3
|
B
|
83-86
|
11-12.9
|
2.7
|
B-
|
80-82
|
10.5-10.9
|
2.3
|
C+
|
77-79
|
10.1-10.4
|
2
|
C
|
73-76
|
10
|
1.7
|
C-
|
70-72
|
9-9.9
|
1
|
D
|
65-69
|
8-8.9
|
0
|
E/F
|
Below 65
|
0-7.9
|
Source: Adapted from OECD and GPA Tables of conversion.
61At the high school level, only F counts as a Fail. But, in higher education, a D is not a passing grade and cannot be transferred. At UTA, any grade below a C counted as a Fail.
62If we consider the table above, the French and the American system share a similar vision of success and failure, even though the former tends to use the top-down model.
63The last question which needed to be addressed concerned the pressure that teachers might feel when grading the students, due to several factors, such as the 2015 law issue on firearms.
64My final question did not directly address the gun issue. This is how I formulated my question to Prof.3, for instance:
So, I think I may have one more question, which is a little silly, but one of my colleagues told me here, you know, nowadays they all get an A grade because, in fact, humanities are different and if they were doing medical studies or sciences, they would get a C or a B, but because we are dealing with the humanities, we tend to give them As all the time for fear they would go elsewhere, you know, we need students in the humanities. And they say that when they were themselves students, it wasn’t the same at all. But today there is this tendency to give higher grades to students in humanities. What do you think of this? To what extent do you agree?
65Prof.3 began replying thus:
Um maybe a little bit, but I think it’s kind of a pessimistic…, I think it’s a more negative take on them than I would, I think, I guess, let me think about that. One of the things that I like about the grade tier system that I use with the, like the kind of different requirements is that they do a lot of work and, and because I’m interested in the process rather than, like kind of, the finished thing, right? It’s like if they do all of these steps, if I do my job in terms of differentiating the teaching and providing feedback and allowing them to do these revisions and stuff, they’re going to learn. And I think that they do learn—
Yeah, they have, they have worked for it and they’ve earned it. And so, and what I like about that system is I think that it’s kind of, um, maybe it ends up with a lot of As at the end.
66According to Prof.3, her teaching and assessment methods certainly favour higher grades (B+ or A), but this does not mean they are not deserved. In fact, the whole process requires a lot of work and learning on behalf of the students. Is that not the goal of higher education? Prof.3 adopted a method of assessing her students that circumvents the obstacle: if you want to avoid pressure, just have your students work in such a way that they can only get satisfactory grades. Her position is in line with construct validity and the notion of fair assessment.
67Prof.1 was inclined to disagree with the statement contained in my question above:
Most of my colleagues do not [give As to anyone]. As are typically earned, not given. Err…, but I will concede that I think because of pressure by students, who tend to reinforce this notion that the humanities are subjective, easier and, err, less objective in terms of exams and tests, that they are easier courses, I think that is a misconception that sometimes is reinforced by faculty who then give those grades expecting students to do that. But I would disagree. I think for the most part my colleagues, at least that I know, don’t just hand As out. I would not say that.
68Prof.3 mentioned pressure from the students but not the kind of pressure that is related to being allowed to carry concealed guns on campus. Her main point was that professors should resist the pressure and remain as fair and as objective as possible (fairness).
69Prof.2 was asked to position himself on the issue of humanities delivering higher grades than other subjects. He totally disagreed with the idea. Interestingly, speaking as a fiction writer, he considered the necessity of raising the level of his teaching:
It’s been two years now, maybe, since the most recent election and, and just the unrest that we see throughout the world and the way, uh, I think it is eventually affecting the humanities. And, just how are we, how are we to live in this world? —[N]ow I feel this tremendous responsibility — that they need to walk outta here, having understood the world from a lot of different perspectives, some that they may not agree with, some that will shock them, some that may affirm what they were already feeling. It may reflect some of their own lives. I don’t, I don’t know that it’s, it’s changed the way I assess them and maybe it’s changed the way I assessed the curriculum.
70Prof.4 replied as follows to my question:
I do think that, well I don’t know I haven’t taken a science class in a long time so I can’t really speak to how rigorous they are. I guess what I would say is that I don’t really like grades. And so, err, I don’t want to think about them any more than they have to. I like designing assessments that I think produce good artifacts of my students’ learning. But I don’t like haggling over what score that is and I… I find it displeasurable and so I… if, if there’s a higher curve in my class I don’t think it’s because I want to seduce them into staying in the humanities.
71In other words, much like Prof.3, Prof.4 explained that she would rather focus on the quality of the learning process and her assessment devices being relevant to her teaching (construct validity).
72In a semi-directed interview, the interviewer (INT) is allowed to ask other questions to enrich the exchange. So, I continued with my train of thought, eliciting the following exchange with Prof.4:
INT: But in general, I mean, did you notice such a tendency, you know, because you’ve been teaching for seven years so you can maybe see that there is a tendency to give higher grades nowadays compared to seven years back or — maybe not.
Prof.4: Maybe. I mean, I think, err… maybe yeah, maybe. I mean I just think there are other factors too. Like, we have we have campus carry on UT campus which makes myself and a lot of my colleagues sort of afraid of haggling over grades.
INT: What did you say?
Prof.4: Our students are allowed to have firearms in class.
INT: Oh yes, yes.
Prof.4: And that that I do think does introduce a level of…
INT: Tension.
Prof.4: … Unease and tension. And so, I do think that there’s more than the humanities that are being under threat… err… I think that I’ve heard multiple colleagues talking about it not being worth arguing over grades because you hear about professors getting shot over it.”
73Then, I asked Prof.4 if she had noticed any of her students carrying a gun. Prof.4 reported a recent incident: a good friend of hers just had a student have a seizure in class and when he fell, everyone could see he was carrying a gun. The other students became nervous about it and the incident ruined the class dynamic.
74As a matter of fact, one of my own students witnessed the incident and missed classes for almost two weeks. His final results were affected in the end.
75Prof.4 brought up the subject of the gun issue on her own and associated it with “other factors” leading to more careful grading. She did not mention the money factor at all, contrary to Prof.5.
76Prof.5 compared his experience at UTA with his former position at Cornell University:
I did my graduate study at Cornell University and the students I taught there, actually, it was just sort of just a different student body in their own right. Like incredibly wealthy students for the most part. And there, I did feel like there was a kind of sense of entitlement that they felt like they had paid a certain amount of money and they were entitled to the to an A grade that they expected. The students here I don’t think so.
77Here, he noted the fees issue but considered that UTA was not concerned with the kind of pressure linked with wealth and money.
78Still, during my stay at UTA, a major college admissions scandal broke out nationally involving athletic coaches all over the country who were accused of “accepting millions of dollars to help admit undeserving students” (Medina 2019). Famous film or television stars, fashion designers, and other wealthy parents were involved in this general fraud in which SAT or ACT scores were falsified. One UTA professor was fired and tried.
79Prof.5 continued:
I think the state legislature and a lot of the administrative apparatus around the university create that sense of, like, the students are customers and I think that attitude is definitely in the air but I don’t think that’s coming from, in my experience at least, from the students at all. That’s not how they think of it. A lot of them are… err… I have lots of first-generation college students who are really grateful to be here.
I think they are less, in a refreshing way, less focused on their grades than at other institutions I’ve taught at. They seem more sincerely interested in learning.
80Prof.5 referred to a sociological factor to support his point. He did not mention the gun issue, contrary to Prof 4, and also Prof.6 for whom it is a factor not to be underestimated.
81According to Prof.6, the gun issue is a major issue, but she personally did not feel threatened. She considered that the students in the humanities shared the same attitude towards guns as teachers (who stick pictures of a kitten on the door of their office with the caption: “This is my gun policy” or “no guns allowed in my office”):
The gun thing isn’t exaggerated, unfortunately. I mean, I don’t feel it, the threat. Because I, you know, since I’m in the humanities, we run into students who have more our perspective than maybe other students in other fields would have. You know, they’re kind of drawn to the humanities because of that. Yeah, I do think that there has been pressure to raise grades.
82Although she does not personally feel in danger, perhaps a sign that the widespread presence of guns in Texas blunts the perceived threat of gun violence over time, she admitted that the gun issue represents a factor for raising grades, among others, like a change in her own views on assessment.
83In what follows, she explained how she started as a “tough” grader, or “grade deflator” according to the statistics produced by the main office; being in linguistics could explain the low grades, however, since, as she puts it, “it was kind of easy because I taught with linguistic-type content, and so, there are a lot of the, the exams were objective and so some students just couldn’t do the material you know. It was like math to them.” Her assessment policy changed to better take the students into account (feasibility and fairness):
Now, I have a very different philosophy, so, and especially a different philosophy for classes like this one. This one is, my view of it is, its aim is, to make people understand about non-mainstream dialects before they go into a class and start marking people up with a red pen. So, I now give take-home exams. So, the midterm and the final [exams] are take home and I started thinking that my philosophy was wrong. I used to give really hard in-class exams. And sometimes people would even be in the hallway trying to finish [their exam] because I would get them out before the next class came in [to the classroom]. But I would let them finish. But I felt like in a way what I was doing was trying to catch them unprepared and what my real goal should be is to just get them somehow to absorb the material and really learn the material. So, I give, in a way, the same kind of exam, but I let them take it home. I let them work on it for a whole week. And what I say to them is “I’m, you know, I’ll give as many As people earn or as many Bs as people earn.” But I don’t have sympathy for people who haven’t really read the material when they’re answering these essay questions.
84Among the other factors she mentioned which she believes contribute to raising grades is the awards policy:
And once you know, they always tell people that evaluations are not positive evaluations, are not correlated with high grades but it’s sort of patently obvious to anyone who looks around at what teachers get, teaching awards and so on. That is not that and I’m not saying they’re not good teachers, I mean I’ve got no teaching award myself, but you know it’s, it’s probably not the hardest, toughest grader. Like, I have a colleague who’s just very well known as being really tough, a really hard grader and, and you know, typically when she comes up for evaluation people are always worrying about her student evaluations. So yeah, I mean it’s definitely, it’s definitely there and people do talk about it explicitly.
85In other words, colleagues and the whole institution tend to prefer “gentle grading”.
86Prof.6 also mentioned the money issue: “I sort of feel there have always been students who feel that way you know I’m paying for this, I’m signing up for your class and you know I’d say it’s a contract you work for me, you know?”
87And she saw a difference with foreign students:
Most students don’t behave that way but there are always a few that seem to behave that way and I really notice a difference when I work with foreign students. So, the graduate courses that I’ve taught in my career have a higher attraction rate than I think most courses do for foreign students. So, you know I’ve had students from the French department or German, Italian, you know et cetera, other departments and they have a much different way of behaving in front of professors I feel like.
88Prof.6 was the only one to mention that student expectations could also vary according to whether they were US nationals or foreign students, although she did not fully clarify in what way these expectations differed with US students.
89Overall, two teachers out of six (Prof.4 and Prof.6) admitted that the 2015 law was an issue, but only one (Prof.4) considered it a real threat.
90The two professors who mentioned the firearms issue are female. The gender factor cannot be excluded. Still, our limited corpus is not relevant for quantitative analysis. Two teachers (Prof.5 and Prof.6) considered the money factor as an issue, but only one of them blamed the fact on the students (Prof.5).
91Four of them (Prof.1, Prof.2, Prof.3, Prof.4) tend to free themselves from such pressure and do not feel forced to raise the grades because they have adopted a way of teaching and assessing in which the students work and learn enough to actually deserve higher grades.
92Interestingly, these two factors were raised by the same professor, Prof.6, who happens to have had the longest career (35 years). She was perhaps more able to stand back from the present time and take a long-range view of the situation.
93In the final section of this article, I will present my own experience regarding assessments at UTA and discuss the possibility of adapting some of the features to my French university.
94I had two classes which met for two hours twice a week. The first one, E371K, was on English contemporary poetry with a group of 10 students (1 male and 9 female). The other class, E329R, on English romanticism, gathered 26 students, including 16 majoring in English and the others in other subjects including history, biology, business, accounting, health and society, philosophy, theatre, music, radio & TV, and government. I will focus my attention on the latter course because the number of students is more significant.
95Although E329R was an upper-division course, it was open to lower-division students (freshmen and sophomores), specialists, and non-specialists.
96I developed the course building on the task-based approach (Nunan 1989; Widdowson 2002; Ellis 2003) and the concept of collaborative learning (Baudrit 2007), notions that require clarification.
97There are many definitions of a task (Breen 1989; Long 1985; Richards, Platt & Weber 1985; Crookes 1986; Prabhu 1987; Nunan 1989; Skehan; 1996; Lee 2000; Bygate, Skehan & Swain 2001; Widdowson 2002; Ellis 2003; Dörnyei 2009). I chose to refer to Skehan’s definition which is simple and corresponds to our purpose here: A task is “an activity in which: meaning is primary; there is some sort of relationship to the real world; task completion has some priority; and the assessment of task performance is in terms of outcome” (Skehan 1996, qtd by Ellis 2003: 4).
98The general framework to design a task, as suggested by Rod Ellis, is also appropriate here:
-
Input, i.e. the nature of the input provided in the task;
-
Conditions, i.e. the way in which the information is presented to the learners and the way in which it is to be used;
-
Processes, i.e. the nature of the cognitive operations and the discourse the task requires;
-
Outcomes. i.e. the nature of the product that results from performing the task (Ellis 2003: 217).
99As for collaborative or cooperative learning, it generally involves group work on a common project. Still, the two modes present some differences. Cooperative learning was developed by prominent figures from American social psychology, among whom John Dewey in the 1920s; collaborative learning relies on the constructivist (Piaget 1976, 1977) and socio-constructivist paradigms (Vygotsky 1985 ([1934]). According to Baudrit (2007), cooperation is often opposed to competition and tends to minimize conflicts between the members of a group. Conversely, collaboration induces the co-construction of knowledge and is liable to generate socio-cognitive conflict. Cooperation usually refers to contributing to the achievement of the same task by the group. A collaborative project may involve a macro-task divided between students or small groups of students, each one performing a consistent part of the overall assignment. In the field of foreign language didactics, collaboration is generally considered more challenging, with complex tasks to perform, such as “opinion-gap activities” (Prabhu 1987, qtd by Ellis 2003: 213), or “creative projects” (Willis 1996, qtd by Ellis 2003: 212) that require the participants to integrate their partners’ ideas and contributions. Our project was mostly of a collaborative type.
100The overall goal was to build an online collaborative anthology made of all the students’ papers. Each student had to prepare three papers:
-
the first was a preface to the romantic period,
-
the second a presentation of a poet and a poem with artistic illustrations, and
-
the third a criticism of the full anthology and three samples of their choice which relied on individual oral presentations and open access to the online anthology.
101Grading involved some criteria inspired by the online, generic description of all courses and was agreed upon in advance by the students:
-
1 short paper (20%);
-
1 short research paper (20%);
-
1 final exam (given on the last class day) (30%);
-
Class participation and preparation (30%).
102Also added to this description was a short paragraph:
The reading for the class happens outside class. But all the work of the class happens in class. Group work will also be used. Therefore, class attendance is mandatory. To receive a grade of C or higher, you may not have more than two unexcused absences.
103In fact, the whole approach required the students’ presence, and they played along.
104The final product was a 188-page anthology composed as follows:
-
The 7-page preface was based on our readings and made from excerpts of all the students’ first papers put together (short paper, 20%).
-
The 132-page first part consisted of the collection of all second papers (1 short research paper, 20%).
-
The 30-page second part contained all their third papers (1 final exam, 30%): a collaborative general review followed by each students’ critiques of a selection of three presentations (in part 1).
-
The anthology ended with the list of “editors” in which each student collaborated in drafting a short self-presentation (3 pages).
105Table 3 presents the overview of the project.
Table 3: The project’s design features according to Ellis’s model
Design Features
(Ellis 2003: 217)
|
Paper 1
|
Paper 2
|
Paper 3
|
Input
The nature of the input provided in the task
|
The course on romanticism
The Longman Anthology of British Literature – The Romantics and their Contemporaries
|
The course on romanticism
The Longman Anthology of British Literature – The Romantics and their Contemporaries
2 films:
-
Bright Star (Jane Campion, 2009)
-
Mary Shelley (Haïfa Al Mansour, 2018)
Original manuscripts from the Harry Ransom Centre
|
The anthology minus the review and critiques
|
Conditions
The way in which the information is presented to the learners and the way in which it is to be used
|
Class and home readings;
academic writing
|
Class and home readings;
documentary research, both academic and artistic;
academic writing
|
Oral presentations and personal readings
critique writing
|
Processes
The nature of the cognitive operations and the discourse the task requires
|
Synthetical skills: summarizing
|
Analytical skills + emotional and artistic intelligence:
Presentation of a poet and a poem with three illustrations
|
Evaluating:
appreciating
|
Outcomes
The nature of the product that results from performing the task.
|
Preface
|
Contributions to a literary anthology
|
Reviews and critiques
|
106For papers 1 and 2, the input was provided by the course on romanticism based on the Longman Anthology of British Literature – The Romantics and their Contemporaries (5th Edition); for paper 2, two films were added: Bright star by Jane Campion (2009), and Mary Shelley by Haïfaa Al Mansour (2018), as well as original manuscripts, after our visit to the Harry Ransom Centre. But for paper 3, the input was the collaborative anthology itself (papers 1 and 2 had already been collected), minus the outcome of paper 3, i.e. the general reviews and the critiques.
107With regard to the conditions, one may notice an evolution from the first to the third paper, especially in the degree of autonomy required from the students.
108As for the processes, they involved summarizing, expressing oneself both academically and emotionally or artistically, and evaluating.
109Finally, in line with Skehan’s definition of a task (see supra), the outcomes were termed “in some sort of relationship to the real world”: “a preface”, “contributions to a literary anthology”, and “reviews and critiques”, which is typical of a task-based approach as opposed to a purely pedagogical one.
110Regarding assessments, I adopted the transversal model more appropriate for collaborative projects. I was also curious enough to try the American approach and included two features in the initial schedule: peer review and a revision policy, as well as an adaptation of take-home exams.
111Before uploading their paper, the students had the opportunity, in the previous class, for about an hour, to bring their work in progress and discuss it with their peers. I would supervise the session and make sure each student was happy with the feedback they received and that all would be able to improve their paper. Being free to choose their partners and interact privately minimized “face-saving” issues (Goffman 1955; Brown & Levinson 1987) that may arise in peer assessment practices.
112Once graded, the students who did not receive an A were given the opportunity to modify their production following the teacher’s feedback. Simply correcting mistakes was not enough; a substantive improvement of the paper was expected.
113I thought take-home exams would fit the purpose of the third paper, but I adapted it to a mix: the students would finish their critiques in class but work on them at home. The last two sessions before the final exam consisted of oral presentations of the first part of the anthology (the students’ second papers). In parallel, the content with the illustrations (videos, songs, pictures, other poems, etc.) was made available online (Figure 9). Consequently, the students had to actively participate in the last sessions and read the anthology before they could produce an overall assessment (see Appendix 1) and select the three contributions they wanted to review for the final paper.
Figure 9: Illustrations from student contributions
Source: UTA students, Spring 2019.
114As for the assessment model, as mentioned earlier, I chose the transversal one which allows the teacher to grade on two dimensions: literary and pragmatic. The literary dimension concerns the quality of writing and the ability to develop a personal reflection on poetry; the pragmatic dimension was the ability to comply with the task and cooperate with others in order to reach the final goal. This second criterium was linked to attendance and class participation (30%).
115In the end, all but two of the students earned an A (which corresponds to a GPA score of 4). One student obtained a B+ because he missed some classes and handed in papers that did not meet all the requirements (eg. his second paper contained only one illustration instead of three).
116Another student received a D+ (the grade recommended to me by the Head of the department instead of an F (Fail), because she dropped out of university altogether and never handed in the third paper, despite my messages urging her to do so.
117I do not really know what would have been the average grade if I had not chosen the task-based approach (the collaborative anthology as the final task, which really appealed to the students as shown in Appendix 2), nor the peculiar assessment features described above.
118It is difficult to compare with more general figures. The average GPA score for the 515 students majoring in English at UTA in 2019 was 3.28, which is almost a B+ (3.3). As for the graduate students, the department has about 100 of them, and grades less than an A are very rare.
119To what extent is a 4.0 GPA obtained in E329R due to the teaching approach, the assessment method, the lack of experience of a French professor grading American students, or simply the skill and competency of the students enrolled?
120Our research questions read: What are the main features of assessments in the English department at the University of Texas at Austin? Are there any factors (such as money, the presence of firearms, or grade expectations for courses in the Humanities) that put pressure on professors and force them to raise grades to avoid confrontational situations?
121Regarding the first issue, this paper described the main traits and suggests that some could be borrowed profitably in France, including:
-
A peer review and revision policy;
-
Take-home exams;
-
Specifications grading (tiers).
122Still, how can we adapt them to the humanities in the French context? I believe peer review and revision policies are easy to implement if attendance and class participation are part and parcel of the final grade. They are of interest because they require collaboration between students and foster mutual trust, as well as a spirit of solidarity.
123Revision policy is also relevant in the French context with the same conditions. Indeed, those two features are based on reciprocity. In other words: “if you attend my class and participate in all the activities, you will benefit from the support of your peers and the feedback from your teacher so that you can get better grades”. This is part of what in France is called “a learning agreement” (“contrat pédagogique”).
124As for take-home exams, I suppose teachers are likely to be less suspicious of malpractice from students with whom they meet twice a week and can more easily establish a high-quality relationship. The possibility of a mix between a take-home exam and a class exam could be a solution, especially if the approach enhances a personal reflection on a topic relevant to the class, as in the E329R final exam.
125Specifications grading seems to be an attractive approach, especially for France where the Baccalauréat is both a secondary school leaving examination and a post-secondary school diploma giving automatic access to higher education, a general principle marred by the sky-high drop-out rate for first-year students who are often ill prepared to study in French public universities which have limited means to welcome and nurture incoming students. A specifications grading would enable students to design a flexible work pattern, depending on their abilities and goals, and without fear of negative judgement. This would take them even further along the path of a learning agreement.
126It could be implemented at universities in France (some have already adopted the system) since it keeps with the philosophy of the new nationally-mandated “plan licence” which aims for each age group to reach a 50% success rate for undergraduate degrees, as it enables students to take more time to graduate without “failing” their exams and eventually dropping out of university without a degree.
127As for the second issue concerning the pressure that can be felt within the humanities regarding grading, even if the majority of the professors interviewed at UTA denied such a problem existed, at least some of them noticed tensions related to: student fees (if they are too high, it may become a real problem); the 2015 law allowing students to carry concealed guns on campus; the deterioration of the overall situation in the country and the world. These tensions do not carry over to France where tuition costs have remained largely the same over the past two decades, unlike in the UK, where students can now be charged upwards of £9,000 per year, producing a client mentality among some of them. In French public universities, students cannot pretend to be customers (unlike in some “Grandes Écoles” or private business schools). With respect to guns, in France, guns are prohibited and the police are usually not allowed on campus. But to return to the provocative title of this paper, I must concede that all but one of the professors I interviewed did not really feel threatened by “concealed carry” and that their freedom of action in grading the work of their students remains generally unimpaired. More surprisingly, perhaps, is that their relative indifference precludes the possibility of any open discussion about who, in class, is ‘hiding’ a gun which could perpetrate a massacre against his/her fellow classmates.
128An abiding difference with the grading system in both countries is that, in France, we can still give low, even humiliating grades with impunity. Fair enough. But why not experiment with a New Deal with the students based on trust, participation, and real training, leading to a more valid, reliable, and fair assessment?