exams – Double Aspect

F-Words

Some words and arguments to avoid in law school exams

This post is the first in what I plan on making a short series dealing with some things that bother me while marking public law exams. I once wrote a post along these lines, but happily at Reading its very basic advice is not as necessary as it was where I used to teach. Do refer to it for 101-level stuff. And please also refer to Mark Elliott’s excellent and helpful post, over at Public Law for Everyone, on the importance of making an argument in an essay question (or indeed a longer-format essay). That would be the 201 module.

This, by contrast, is going to be “issues in” course. What this means, really, is a course in the professor’s pet peeves. Of course, different people have different ones, and these might only be mine. But I do hope that they are of some use, both to colleagues and to students, and I hope that they will make for some entertainment if nothing else. After all, the first lesson concerns f-words! No, no that f-word. There are others, including four-letter ones. The one I’ll mostly focus on is “fair”. But first, let me say something about “floodgates”.

Floodgates arguments generally suggest that A’s claim against B should not be entertained by a court, because if it is, other similar claims will be brought ― the floodgates of litigation will open ― and the courts will be deluged with more cases than they can handle. I don’t think that floodgates arguments are often successful in real life. If A’s claim against B is without legal basis or factual merit, it can be rejected, and the rejection ought to serve as a deterrent to analogous future claims. To invoke the floodgates argument is to implicitly concede that, at the least, there may be something to A’s claim. But having made that concession, an advocate and especially a court will find it awkward to peremptorily refuse doing justice for no other reason than to economise resources. If many analogous claims ― all possibly meritorious ― are out there, the injustice of refusing to consider them is only compounded.

As a result, floodgates arguments are rarely persuasive in student work either. Truth be told, they are often the mark of a weak script. It is sometimes difficult not to suspect that the student could think of nothing better, simply remembered this catchy name, and went with a floodgates argument for lack of an alternative. Perhaps even a weak argument, at least if it is used accurately, is better than none at all, though one might want to consider whether making an inherently feeble argument does not harm one’s cause by exposing the defects of one’s position. (This is certainly true of the “kitchen sink approach”; not only do many bad arguments not add up to a good one, but they make it clear that one hasn’t understood which arguments are worth making.) Anyway, if you are choosing among a floodgates argument and a different one, always go for the other idea, whatever it might be. I would suggest making it a rule to simply banish this particular f-word from your vocabulary. It is a crutch, and not relying on it will only help you by forcing you to think a bit harder and more creatively.

My beef with the other f-word, fair (or its derivative fairness), is much the same: it is a crutch makes students think they’ve made a satisfactory case when they haven’t. But the explanation is perhaps a little more complicated in this case, or at least harder to believe. Unlike the floodgates of litigation, fairness is not a fancy-sounding technical concept, but one that we appeal to all the time. Unfortunately, that is part of the problem. Nobody wants to be against fairness, of course. But we should all be wary when someone ― including, I am afraid, a student in an exam answer ― seeks to persuade us by making opposition emotionally difficult rather than logically impossible. We should also be wary of making such arguments ― ideally, out of respect for our readers but, failing that, out of a self-interested concern not to arouse their suspicion that we might be trying to trick them.

More substantively though, fairness ― despite its intuitive appeal ― is also an elusive notion. Just what it means in any given context is often unclear. Now, sometimes ― and in our daily life, often enough ― we have a good, and, importantly, shared, sense of what fairness requires. If you insisted on choosing where to go for dinner with your friend last time, it is fair to let the friend choose now. If you were late to the pub, it is fair to buy your friends a drink. And so on. The trouble is that shared understandings of what is fair run out quickly in the kind of situations that law school exams, and indeed a great deal of law ― perhaps especially, though by no means only ― public law in the real world deal with.

Does fairness mean that people should be subject to human rights constraints or allowed freedom from them? Does it require government to seek parliamentary approval for a given course of action? Does it mean officials need to comply with rash, perhaps untenable promises to members of the public? Students ― and not only students, to be, ahem, fair ― may think that there are answers to such questions. But there are usually people on both sides of them. If you find one in an exam paper, you can be very confident indeed that there are serious arguments on both sides. And people on both sides probably think that their answer is fair. This suggests that no real concept of fairness is doing the work of compelling an answer one way or another. At best, people rely on intuitions about what is fair. At worst, they are actively covering up their true motivations under the specious rhetoric of fairness. (To be clear, I don’t suppose students do this often, if at all.)

Of course, these questions must have answers, if only provisional ones, and there are reasons why the answers are or ought to be one way rather than another. But fairness is not such a reason. There are other considerations involved. Some have to do with specific constitutional principles such as individual liberty, government accountability, the Rule of Law, the sovereignty of Parliament, or what have you. Others are policy arguments (including the dreaded floodgates, though to repeat it is a particularly weak one). Usually, more than one reason bears on a given answer. Relevant considerations sometimes complement one another, and sometimes pull in different directions. But it is their summing up, untidy and unsatisfactory as it sometimes is, that actually answers difficult questions, rather than appeals to fair play.

As with “floodgates”, I think that students should banish the other f-word from their exam-writing vocabulary. If you feel the itch to use it ― and, given its ubiquity, that is understandable ― you should ask yourself why you think that this course of action, or this approach to the problem, or this rule, is or would be fair. And then, give that explanation, in as much detail as you have room for, instead of speaking of fairness. Again, this will force you to think harder ― but it will also make for better results, because you will be discussing actual principles and policy arguments instead of hoping that the marker shares your intuitions, or at least understands them ― and neither is a given.

Whatever their differences in detail, marking grids at every law school I have known as either a student or a lecturer reward, first, understanding of the subject and then, to get really high marks, critical thinking and creativity. Clichés and stock arguments add little to a demonstration of competence, and actively get in the way of showchasing originality. The less you rely on them, the better off you are likely to be.

The Cruel Ritual of the Ontario Bar Exam

Every June, lawyers-to-be in the province of Ontario make the pilgrimage to one of a few cities to undergo the ritualistic exercise of writing the bar exam. For many, the bar exam—otherwise known as the lawyer licensing exam—represents a large part of the process to become a lawyer in the province of Ontario. Students migrate into convention halls with their little Ziploc baggies of highlighters and granola bars, and carry in large bags their heavily indexed binders of material. But the Law Society of Ontario’s (LSO) bar exam is not an exercise in actually demonstrating competence. Instead, the regulator has imposed a search-and-destroy style exam on students, in which the charges must find the right answer buried in their indexed binders. If the goal of the bar exam process is to ensure that students have a minimal level of competence in the profession, it is hard to see how the bar exam achieves that goal.

More to the point, it appears that the only good justification for a bar exam of this sort is to teach students how to take a timed examination under strict conditions. But what purpose does this justification have in terms of competency for a lawyer? Indeed, other that imposing needless anxiety and pressure on already debt-burdened students, how does this ritual do anything for competency? Maybe it could be justified as a tradition that all lawyers should have to bear—but this seems like a thin reed on which to charge students for the pleasure of it.

Instead of taking the bar exam as a given, perhaps Ontario should start from first principles. What is the goal of the licensing process? Starting from the highest level of abstraction, the Law Society Act states that a goal of the LSO is to ensure that “all persons who practise law in Ontario or provide legal services in Ontario meet standards of learning, professional competence and professional conduct that are appropriate for the legal services they provide.” In its “Options for Lawyer Licensing: A Consultation Paper,” the LSO noted the following:

Lawyer candidates are required to demonstrate proficiency in respect of competencies that reflect the minimum requirements of both barristers and solicitors entering the profession in the seven areas of law that are most frequently practised. The current barrister and solicitor examinations provide a means of testing candidates’ abilities in core knowledge, application and critical thinking competencies, irrespective of their educational background.

With this connection to the competency requirement in mind, the LSO also defended its licensing process, citing only one article from American scholars:

The LSO’s licensing examinations are internationally-recognized as high-quality, psychometrically-defensible professional qualification assessments.

That relatively unsupported assertion aside, the LSO to its credit recognized in its consultation paper that the licensing process needed some amendment. In fact, the LSO outlined four options for the licensing process. Two of those options would add a “skills-based” examination to the existing bar exams. Indeed, the option ultimately chosen by the LSO would involve ”[c]onsideration of some form of skills testing in the licensing process.” But there did not appear to be a serious appraisal of whether the bar exam—from a common-sense, regulatory perspective—really advances the goals of competency. In other words, while some form of skills-testing is probably tied closer to a regulatory goal of ensuring competency, I fail to see how the mode of the existing licensing examinations does anything to even basically ensure that competency.

The Ontario mode is not the only one that could be followed. In Manitoba, for example, the focus of the process is almost entirely skills-based rather than examination-based. Instead, through the CPLED program, students are exposed to different tests in oral advocacy, writing, and other areas that are more directly related to lawyer competency. One can at least make the regulatory business case for this mode of testing as connected to testing lawyer competency.

I cannot say the same for Ontario’s whack-a-mole-style licensing examinations. While the LSO tries to suggest that its licensing exam is acclaimed internationally (simply on the basis of one US article), I truly would like to know, at the level of regulatory policy, the justification for a bar exam of Ontario’s sort. From where I am standing, it appears as nothing more than a cruel joke to already-burdened students.

Passing Observations

Some thoughts on writing exams, from a guy who just graded 240 of them

As I’ve mentioned in previous posts, I recently graded (or, as we say in New Zealand, marked) more than 240 exam papers (or scripts). So I thought I’d volunteer some observations, in case any students who might be reading this are looking for tips. Of course, much of what follows will feel intuitive to many, and perhaps to most. The art of answering exam questions is not especially difficult to master. But there are, I can now tell, more than a few people who really could use some advice before they sit another final. (Whether they read my blog is a different question, admittedly.)

By way of introduction, let me say something of which students don’t think (I know: it’s not very long ago that I was a student myself!). A student writes only four or five exams, at most, at the end of a semester, but an instructor has many dozen, and possibly (as in my case) several hundred of them to read. This means that I only have a few minutes to devote to each script. (Ever complained about the marking taking too long? I know I have. But if it had to be done faster, that would mean even less time to look at your answers!) If I don’t know what you are saying ― whether that’s because your answers are poorly structured or even because your handwriting is atrocious ― I’m not going to spend a lot of time figuring it out. If you want me to understand you, it’s your job to make sure I do.

And beyond that, it’s in your self-interest to make sure that I… how to put this nicely… don’t get too worked up while reading your answer. Sorry as I am to say this, when reading the answer to the same question two hundred times over, it is unfortunately easy to find small things aggravating. I know one should not get aggravated, and I try not to. But still, don’t give me reasons to become annoyed. Try to spell correctly ― especially when you are writing my name on the exam booklet. (Seriously. I’ve seen my name spelled a couple dozen different ways, though the best one was the student who wrote my last name as Sinatra.) Try to punctuate sensibly ― instead of just randomly strewing periods all over your answers, or at the end of each line. Try to use proper syntax. In particular, ensure that your sentences have subjects and conjugated verbs, and that they are not just subordinate clauses floating around without anything to attach them to. (Of all the annoying things I’ve seen, this one is perhaps the most bizarre.) If your writing tone is formal, don’t be pretentious; if it is conversational, don’t be familiar. Oh, and please, don’t make unfunny jokes. Keep in mind that if you feel the need explain your joke, it’s probably not funny. And when in doubt about whether a joke you want to make is funny, abstain.

This all goes to the form of your answers. Let’s now turn to the content. The single most important thing is also the simplest one: answer the question you are asked! I will at least try to overlook those annoying periods all over the place, ignore ignorance of apostrophes, and put those free-floating subordinate clauses down to the stress of the exam room; but I can’t pretend that you are answering the question when you are not. In particular, if the question is a descriptive one, asking you what the law on a certain point is, don’t answer it as if it were a normative one, asking you what the law ought to be. And if the question asks you for a prediction about the consequences of a development in the law, don’t answer by explaining why this development ought not to, or will not, happen. That’s just not what I want to see, and as a result, your grade for that question will not be one that you want to see.

Another general point is that you won’t get very far by simply spewing the notes you took in class, and a fortiori the notes that I provided, right back at me. For the most part, doing this just shows that you have no idea what you are talking about and are throwing the proverbial kitchen sink at me. The same goes, of course, for keywords from my Powerpoint slides inserted into answers regardless of relevance. A related point is that if the exam is wholly or partly open book, you shouldn’t just print out your entire notes for the semester. Prepare an aide-mémoire that synthesizes what you’ve learned ― it will help you study, and finding things during the exam will be much easier than rummaging through a semester’s worth of notes. The one I used for the first year contracts exam, for a full-year class, was all of seven pages long, in size 12 font. It’s perfectly doable if you put in the effort. And of course, “putting in the effort” means actually understanding the material, enabling you to show the instructor that you have understood ― which is precisely what he or she wants to see.

Some more specific issues now. Perhaps the most important one is that you need to distinguish what is and what ought to be. This is one of the most important things in legal education, and it’s a safe bet that most instructors try to get you to do this, and want to see you do it on an exam. So don’t assume that things are necessarily right the way they are, and don’t assume that things were necessarily wrong in the past, when they were not as today. Don’t assume that judges always act as they are supposed to ― they are only human beings, prone to error and susceptible to the corrupting effects of power, especially to the desire to increase the power of courts at the expense of other institutions. But don’t assume that Parliaments and governments are always looking out for the public good, either. Don’t assume that they are all always wrong, or corrupt, or evil, of course. Judge each case on its own merits, and don’t forget that there is a decent chance that, if you are being asked a question, the answer to it is not altogether clear-cut or obvious. Pay attention to the context of your answer, perhaps especially on problem questions: if you are asked to write a memo for a client, it is probably not helpful to launch into philosophical disquisitions, or discussions of Roman law. Whatever the question, however, avoid making pompous general statements, which are invariably untrue and almost as invariably irrelevant (these include, for example, declarations that something has been done “throughout history” or needs to be done “in every country”). Last but not least, know your stuff! Don’t confuse Governor-General and Attorney-General. Don’t represent a concurring or a dissenting judgment as that of the court (even if I focused on that particular judgment in class). And don’t bring up a case to illustrate the application of a common law rule developed or a statute enacted years after that case was decided (in other words, know when the cases we studied were decided).

Contrary to what some students think, it’s actually a lot more fun for an instructor to give good grades than bad ones. It’s certainly more fun for me. But that doesn’t mean I’ll do it without good reason. I’m happy to interpret borderline cases favourably to you ― but not to pretend that your work is better than it really is. Do it well, and we’ll both be happy. Good luck!