The value of exams as a signal

Exams have been in the news this week for all the wrong reasons. However, last week Michael Lee (University of Auckland) wrote an article in the New Zealand Herald on the real value of exams as a teaching tool rather than just an assessment:

We use exams as an encouragement tool to compel greater engagement with the material. That is actually where the real value of an exam is. When students feel the stakes are high and are unsure of what exactly will be asked, they are incentivised to take a look at everything seriously.

That's why teachers should never tell students exactly what will be examined, because 99 per cent of students will then focus only on that material, which defeats the true purpose of the exam.

In exams and in the real world, the first step to topic mastery is a general overview of key concepts and facts with as much detail as one can remember. Clearly, exams reward students that can do these things in a relevant way to answer a specific question.

A more advanced stage of mastery is the ability to creatively apply, integrate, and challenge the knowledge you have been taught. But it is difficult to get to that level if you haven't got enough base material to work with.

I have a slightly different take on the value of tests and exams. I agree with Lee that they are useful as learning exercises, especially if organised well. I disagree that we shouldn't tell students what will be examined (although I will admit that when asked what will be examined my usual answer is "everything we have covered", which is true!). However, I see tests and exams as having another important function for students, as an important signal that students can give to future teachers and employers. This relates to solving the employers' adverse selection problem that I have written about before:

Students are engaging in a sophisticated array of signals, on multiple levels. It's not possible to avoid signalling in this case, since trying not to provide a signal is itself a signal. The problem that this signalling is trying to avoid stems from private information about the quality of the student - students know whether they are high quality (intelligent, hard working, etc.), but employers don't. Employers want to hire high-quality applicants, but they can't easily tell them apart from the low-quality applicants. This presents a problem for the high-quality applicants too, since they want to distinguish themselves from the low-quality applicants, to ensure that they get the job. In theory, this could lead the market to fail, but in reality the market has developed ways for this private information to be revealed.

One way this problem has been overcome is through job applicants credibly revealing their quality to prospective employers - that is, by job applicants providing a signal of their quality. In order for a signal to be effective, it must be costly (otherwise everyone, even those who are lower quality applicants, would provide the signal), and it must be costly in a way that makes it unattractive for the lower quality applicants to do so (such as being more costly for them to engage in).

Qualifications are an effective signal (they are costly, and they are more costly for lower quality students, who face having to expend more time and effort to complete the qualification). Exams are also an effective signal for exactly the same reason (though not at the same level as the whole qualification). Because exam performance is an effective signal, high quality students can use their performance in exams to separate themselves from lower quality students, because it is very difficult for lower quality students to pass themselves off as higher quality students in the exam format. The quality of the signal is much lower for other types of assessment such as take-home tests, assignments, or group projects, where lower quality students can easily get additional help (often from the high quality students!) to boost their grades.

To me, that is one of the key reasons why we shouldn't eliminate high-stakes tests and exams from student assessment. Take-home or open-book tests, online tests, group projects, and the myriad of other assessment types that are used all have their place, and can all be valuable as learning exercises if used well. But they'll never be able to provide the same quality of signal of student quality as a test or exam.

Read more:

• Why all students need to understand adverse selection and signalling

[HT: David, one of my ECON100 tutors]