On My Way Out – Advice to Young Scholars VII: Taking Exams Seriously (Part 1)

I have, as is increasingly evident, reached the final phases of my academic and professional career, and as I look back I want to offer, for what it is worth, some dos and don’ts on different topics for scholars in the early phases of theirs. This is the seventh instalment, and it is dedicated to that central feature of teaching – exams.

I take exams seriously because I take teaching very seriously. My vocation as a scholar comes second to my vocation as an educator and teacher. Though in certain jurisdictions and certain universities some attention is given to the training of young academics as teachers (as if the old geezers are perfect and could not well do with a refresher here and there), and though in certain jurisdictions and certain universities attention is given (often no more than lip service) to the quality of teaching in the progress of an academic career, I am unaware (and would be pleased to be corrected) of any serious and systematic attention to exams.

As a result, one of the most stable, if not the most stable, university institution is the exam. In many cases, I am sure there are exceptions, the kind of exam one had as a student, assuming one remains in the same system, is the kind of exam one will administer to students. If one moves, as many do today, from one system to another, one is simply told: this is how we do it here and one falls into line.

There is huge variation in the manner in which exams are conceived and administered at different universities. You might adopt a Darwinian approach – natural selection in different environments has resulted in the best possible form for any given environment. Do not kid yourself! It is the victory of inertia over reflection.

The form and format of exams are typically not the result of serious reflection, collective or individual. You may put an awful lot of effort and creativity, year to year, course to course, into the questions you will include in your exams, that yes. But the framework – the form, the format (the two are not the same), the underlying concept and philosophy of the exam – tends to remain the same and is frequently unarticulated.  The questions might change, as the law changes, but it is the same persons, just wearing different clothes. How else, other than inertia, might one explain the attachment of, say, my Italian colleagues to their 20-minute oral exam, one of the most deficient forms of examination – a charade merged with farce where arbitrariness of result  combines with unfairness (I speak from experience).

And yet, I am always struck by the fact that, despite this victory of inertia over reflection, my interlocutors over the years, when attempting to question university practice of exams, become fiercely – fiercely – locale patriotic. A matter of constitutional identity: ‘Change our exams?’ … imperialism, neo-colonialism, changing civilization as we know it today.

My purpose in this reflection is not to offer a blueprint for the ‘best’ form of exam – though I will not hide my preferences. Instead, I will walk through some of the choices that have to be made in reaching a reasoned result. Thus, not ‘what is the best form and format of an exam’, but ‘how to think about this’ – indeed, taking exams seriously. I will start with some conceptual issues and in further instalments move to the practical.

The ‘Philosophy’ of Exams

The most fundamental point I want to make – more important than the list of choices available – goes to an issue which I think is so obvious that it is often forgotten. You may call it ‘the underlying philosophy of exams’.

Thinking seriously about exam design must, should, force us to think seriously about course design. Yes, I want to teach constitutional law or international law, etc. But what are the educational objectives I want to impart to my students in the course of teaching them these subjects? Which skill sets? What type of understandings of the subject matter, especially given the obvious constraint that in a course of, say, 44 classroom hours I can hardly make them proficient in all doctrinal aspects of the subject? So, what are these educational objectives in a very concrete way? Surely there are more than one.

It is only if I articulate these objectives to myself and design my course accordingly that I can begin to think seriously about the exam design, since, as day follows night (or from a student point of view, as night follows day) the exam should test the extent to which the students have mastered the different facets of the skill set and knowledge that constitute my educational objectives.

I will now illustrate this by reference to my choices as regards educational objectives and how these translate into the format of an exam – with the caveat mentioned above that there can be different choices, but I do insist on a nexus between the educational objectives and skill set and the exam.

Here then are my choices for course design and the consequences for exam design:

1. Doctrinal coverage – knowledge of the positive law. This of course begs, as you all know, two questions. The first question: What is the correct balance between breadth and depth, between widening and deepening? The more I try to cover, the more superficial will their knowledge be. We all are habituated in making these choices, my own preference is depth at the expense of breadth. The second question is trickier, and I can explain it in two ways: a student can learn and understand the textbook, the manual, perfectly, but that is like giving fish without teaching them how to fish. What skill set did the author of the textbook have to have in order to look at the raw materials of the law (legislation, cases, etc) in order to synthesize it into positive law. And/or how does it help me and my students if I teach them, as I must, the law as it stands at the time of teaching (say, second semester of first year) if three years later when they graduate, it has, as always happens, changed significantly?
2. Teaching students, then, ‘how to fish’ – how to read analytically and synthetically the raw materials of the law and translate such into positive doctrinal law. I regard this skill as important – and possibly even more important – than the first objective of doctrinal coverage.
3. Hermeneutics – interpretation is at the heart of legal discourse as a consequence of the inbuilt indeterminacy of large swathes of the law. Since most of my students will be practising lawyers, and not law professors, my approach to hermeneutics is heavily dressed with large doses of legal realism – structures of argumentation, the art of persuasion relevant both in litigation as well as negotiation.
4. All three dimensions mentioned so far come to a head together in the fourth objective – serious experience (if not mastery) in applying the law to complex factual situations. Such situations invite the students to come up with equally complex and creative analyses as well as sorting out from their doctrinal toolkit the relevant and meaningful parts of ‘the law’.
5. A systemic, conceptual, and normative understanding of the entire subject matter – the equivalent in medical school to anatomy and the public health aspects of medicine. We are, after all, at a university – not a bar exam course. And I will mention here something that is often forgotten in our law faculties – that justice is the underlying telos of the law. So how does one weave justice into the material we are learning?
6. Finally, oral and written articulateness – law, after all, to a much greater degree than, say, mathematics, is a communicative discipline.

This is my list – other lists are obviously possible. The main point is that whatever the list, the exam should test all these aspects of the course; in other words, there should be a consonance between the course design and objectives and the exam design.

Finally, here is another important truism that is oft forgotten: the exam is also an exam of us as teachers. If a large number of students perform poorly in relation to one or more of these elements, it is a wake-up call for me that it was my failure as a teacher and that I need to introduce corrections in the design and execution of my course next time I teach it. 

So how does one translate these elements into the exam design? How do they reflect on the choice of form – e.g., oral or written, in class or take home, open or closed books, and so on?

To be continued.