Friday, 14 April 2017

What to do about students buying essays?

I recently read this 2015 paper by Dan Rigby (University of Manchester), Michael Burton (University of Western Australia), Kelvin Balcombe (University of Reading), Ian Bateman (University of East Anglia), and Abay Mulatu (London Metropolitan Business School), published in the Journal of Economic Behavior & Organization (ungated here). I thought this was an interesting paper because it applied non-market valuation techniques to a good that is actually sold in markets - essays.

The authors use a specific non-market valuation technique that is called discrete choice modelling (which my colleague Riccardo Scarpa is a world-leading expert in, and which I have also been involved in for a couple of projects, including this one). Discrete choice modelling involves presenting the survey participants (in this case, 90 humanities and science students from three UK universities) with a number of hypothetical choices. Each choice involves a number of goods with different attributes (in this case, the attributes included the price of the essay, the quality of the essay in terms of the grade it would receive, the risk of being caught, and the penalty if caught), and often there is also the choice to buy nothing at all. The participants make several choices, which allows us to determine the implicit weighting the participants place on the different values of the attributes.

In analysing the data, Rigby et al. use a latent class model. I won't go into the detail underlying this, but essentially it determines how many different types of decision-makers there are, with each type placing different weight on the attributes of the good (in this case, essays). They found that there were two types of students, corresponding to students who were very reluctant to buy essays, and those who were more willing to do so. They also found that:
...half of our subjects indicate a willingness to buy one or more essays in the hypothetical essay choice experiment. Students’ stated willingness to participate in the essay market, and their implicit valuation of purchased essays, vary with the characteristics of student and institutional environment. Risk preferring students, those for whom English as an additional language, and those expecting a lower grade are willing to pay more. Purchase likelihoods and essay valuations decline as the probability of cheats being detected, and the penalties if caught, increase.
There's probably nothing too surprising there. However, why is cheating through buying essays a problem? Because it reduces the signalling value of education. As I wrote in this 2014 post:
One of the key characteristics of a degree or diploma is the signal that it provides to prospective employers about the quality of the applicant for positions they have available. Employers don't know up front whether any particular applicant is good (intelligent, hard working, etc.) or not - there is asymmetric information, since each applicant knows their own quality. One way to overcome this problem is for the applicant to credibly reveal their quality to the prospective employer - that is, to provide a signal of their quality. In order for a signal to be effective, it must be costly (otherwise everyone, even those who are lower quality applicants, would provide the signal), and it must be more costly for the lower quality applicants. Qualifications (degrees, diplomas, etc.) provide an effective signal (costly, and more costly for lower quality applicants who may have to sit papers multiple times in order to pass, or work much harder in order to pass).
In the same way that qualifications are a signal, the grade students receive is also a signal of their quality, because it is harder (more costly, in terms of effort) to get an A grade than a C grade. However, if some students are cheating, then high grades are no longer as effective a signal to employers of students' quality. This is because it is no longer more costly for low-quality students to get an A grade, because any student can do so by buying an essay. This reduces the value of education for everyone.

The overall conclusion by Rigby et al. was, unsurprisingly, that if penalties are high enough, students will avoid buying essays. So, universities should be vigilant, and heavily penalise students who are caught cheating. That reduces the expected net benefit of cheating, and reduces the incentive to buy essays. Gary Becker would be proud.

However, an alternative is to make asymmetric information work for you. Most of the online markets where students buy essays simply link up a willing buyer with a willing essay-writer. The sites themselves mostly don't employ people to write essays directly (and those that do are pretty low quality). So, universities could combat this by flooding the online markets with low-quality rubbish essays. How would this work to reduce cheating?

Students can't be sure about the quality of any essay they buy. Essay writers can't easily prove to students that they will write a high-quality essay. So, there is asymmetric information, but is there adverse selection? I argue yes, since sellers of low-quality essays can take advantage of students' inability to distinguish between low- and high-quality essays.

Will the market fail? Students' willingness-to-pay for an essay is affected by the quality of the essay they expect (as per Rigby et al.'s results above). So, if universities flood the market with lots of low-quality rubbish essays, then students will start to expect lower quality essays and adjust their willingness-to-pay downwards, and may drop out of the market entirely (why bother trying to buy an essay if the quality is highly likely to be rubbish). Sites selling essays will make less money and begin to shut down, since they can't easily prove to students that their essays are high-quality. It probably won't make the market fail completely, [*] but it would reduce the problem and be extremely funny (and no doubt distressing for cheating students).

*****

[*] Readers of a certain vintage will remember the music sharing service Napster. In the last months before Napster was shut down, the music companies (I assume - who else would do this?) started flooding the service with fake MP3 files. This didn't work in terms of shutting Napster down, but it was pretty frustrating for users.

Thursday, 13 April 2017

Strong evidence that laptop use in lectures is bad for learning

A couple of years ago, I wrote a post about how laptop use in lectures was bad for student learning. That post was based on this article (ungated version here). Here's what I said then:
Mueller and Oppenheimer conducted three studies with students in experimental settings...
In terms of results, participants who took notes by hand wrote significantly less than those using laptops, and wrote fewer verbatim notes. There was no statistically significant difference in terms of factual-recall question performance, but students who took notes by hand did significantly better than laptop users on conceptual-application questions...
So, perhaps students would be better off without their laptops in class, even if they are using them diligently for note-taking rather than watching the NBA finals (as I saw one student doing in class last semester).
Now, a new paper by Richard Patterson (US Military Academy) and Robert Patterson (Westminster College), and published in the journal Economics of Education Review (sorry I don't see an ungated version anywhere), provides even stronger evidence that laptop use is bad for student learning. The authors use data from 5571 students from a private liberal arts college over the period 2013-2015.

What really sets this study apart is the care with which Patterson and Patterson design the study in order to identify the causal (rather than correlational) impacts of laptop use on student academic performance. The college they draw their sample from allows lecturers to decide whether to make laptops required in class, make them prohibited in class, or make them optional (neither required nor prohibited). Importantly, a university policy that requires all students to own a laptop. So there is no question that students choose their courses based on whether laptops are required or not (and the authors confirm this with a student survey, where only 4 percent mentioned that this was a factor in their course selection decision).

This is where the study gets very smart. The combination of the laptop policies mean that students in classes where laptops are optional will be more likely to have them (and use them) if they have classes on the same day where they have one or more classes where laptops are required. Patterson and Patterson exploit this and look at how a student's performance in one class is affected by the laptop policies in that student's other classes with lectures on the same day.

Laptop use in Class A where laptops are required shouldn't affect the student's performance in Class B where laptop use is optional, except through the fact that students will be more likely to have (and use) their laptop. Similarly, laptop non-use in Class C where laptops are prohibited shouldn't affect the student's performance in Class B, except through the fact that students will be less likely to have (and use) their laptop.

This allows Patterson and Patterson to make pretty strong claims of causality in their results by looking at the students' results in classes where laptop use is optional and whether they have laptop-required or laptop-prohibited classes on the same day. They also conduct a bunch of robustness checks in their analysis that make their findings quite compelling.

In their main results, they find that:
...having a laptop-required class on the same day increased the probability that a student used a laptop in class by 20.6% or 14.2 percentage points (significant at the 1% level) and having a class that prohibited laptop use on the same day decreased the probability of using a laptop by 48.9% or 36.7 percentage points (significant at the 5% level)...
Our results suggest that computer use has a significant negative impact on course performance, on the scale of 0.14–0.37 grade points or 0.17–0.46 standard deviations... Additionally, we find evidence that computers have the most negative impact on male and low-performing students and in quantitative and major courses.
So, they found that laptop use makes students significantly worse off, and the effects are much larger for male students and for low-performing students, and in quantitative subjects (presumably including economics). So, perhaps we really should be banning laptops from lectures?

Read more:


Tuesday, 11 April 2017

Right-leaning politicians are better looking, but we know less about right-leaning scholars

Two recently published research papers caught my attention. Both compared the relative attractiveness of people on the right and left of the political spectrum. One of the research papers was good, and one was decidedly less so.

Let's start with the good paper, which was written by Niclas Berggren and Henrik Jordahl (both of the Research Institute of Industrial Economics in Sweden) and Panu Poutvaara (LMU Munich), and published in the Journal of Public Economics (ungated earlier version here). In the paper, they compare the relative attractiveness of politicians on the right and on the left. Why would anyone care about this? The authors explain:
If one side of the political spectrum has a beauty advantage, it can expect greater electoral success and to have political decisions tilted in its favor. We put forward the hypothesis that politicians on the right look better, and that voters on the right value beauty more in a low-information setting. This is based on the observation that beautiful people earn more... and that people with higher expected lifetime income are relatively more opposed to redistribution...
This is a very nice study. The authors first demonstrate that politicians on the right are indeed more attractive than politicians on the left, using data from Australia, the European Union, Finland, and the United States. Then they:
...study beauty premia in municipal and parliamentary elections. The former can be regarded as low-information and the latter as high-information elections, where voters know little and reasonably much, respectively, about candidates. We show that in municipal elections, a beauty increase of one standard deviation attracts about 20% more votes for the average non-incumbent candidate on the right and about 8% more votes for the average non-incumbent candidate on the left. In the parliamentary election, the corresponding figure is about 14% for non-incumbent candidates on the left and right alike.
They argue with a nice theoretical model that the reason for these differences is based on two things: (1) attractiveness is itself valuable, and voters are more likely to vote for attractive candidates; and (2) attractiveness signals that politicians have views that are further to the right. So, this explains why the attractiveness premium is greater for politicians on the right in low-information settings (where both effects work in the same direction) compared to politicians on the left (where the effects work in opposite directions, since left-preferring voters are more likely to see an attractive left candidate as being to the right of their views).

Finally, the authors confirm their results with an experiment:
Experimental election results confirm the observational findings from real elections. When matching candidates of similar age, the same gender and the opposite ideology in a random manner and asking respondents whom they would vote for solely on the basis of facial photographs (i.e., with low information), we find that candidates on the right win more often because they look better on average. Candidates on the right get a higher vote share, both from voters on the right and voters on the left, but with larger success among the former.
One of the cool things the study does is that it uses a different survey group to assess the attractiveness of the politicians from those who provided an assessment of whether the politicians were from the left or right, and different from those participating in the experiment. This ensures there is no cross-contamination across the study (they also got Europeans to rate the American politicians, and vice versa). The conclusions, that politicians on the right are better looking, that attractiveness is a cue for voters as to a candidate's conservatism (in a low-information election), and that attractiveness confers an extra benefit for a right-leaning politician in a low-information setting, are all fairly robust.

On to the second, not-so-good paper, by Jan-Erik Lönnqvist (University of Helsinki), and published in the journal Personality and Individual Differences (sorry I don't see an ungated version anywhere). This follow-up study (Lönnqvist cites an earlier version of the Berggren et al. paper):
...sought to investigate whether the attractiveness advantage of the political Right is specific to politicians. To investigate this, the attractiveness of Right-leaning scholars (referring to people who professionally engage in mental labor, such as academics or writers) was compared to that of Left-leaning scholars.
The data collection was ok, although both the subjective assessments of the attractiveness of the scholars and of their political orientation were both provided by the same respondents (being five research assistants, compared with the hundreds in the Berggren et al. study). However, that isn't the main problem with the study, which is this:
We first regressed the ideological tone of the magazine on ratings of attractiveness and perceived political orientation.
So, in all of Lönnqvist's regression models, actual political orientation is the dependent variable (the variable he is trying to explain), and subjective perception of political orientation is one of the explanatory variables. This creates two big problems that I can immediately see. First is related to interpretation. I'll try to keep this from getting too pointy-headed, but if you have the actual variable on the LHS of your econometric model, and a subjective measure of the same variable on the RHS, you think you have this model:

[Actual Political Orientation] = f{[Perceived Political Orientation], other stuff}

What that equation says is that actual political orientation is a function of perceived political orientation and some other stuff (which includes attractiveness). But actually what that equation really is, is a rearranged version of this:

[Actual Political Orientation - Perceived Political Orientation] = f{other stuff}

So, really this is a model of deviations between actual and perceived political orientation (being a function of other stuff, including attractiveness). Keep that in mind when you read the results:
...perceived political orientation accurately predicted actual magazine-related political orientation. However, physical attractiveness did not.
Then Lönnqvist splits the attractiveness variable into attractiveness and grooming ("the extent to which the target person appeared to have prepared physically for the photograph") and finds:
Perceived political orientation still predicted magazine-related political orientation, but now also physical attractiveness and target grooming predicted magazine-related political orientation.
In other words, there was a negative correlation between attractiveness and right political ideology, and a positive correlation between grooming and right political ideology.  Lönnqvist argues this means that left-leaning scholars are more attractive, while right-leaning scholars are better groomed.

However, go back to the equations above. What these results really mean is that the difference between actual and perceived political orientation is more negative for attractive scholars. This may be because attractive scholars are more left-leaning, or because the research assistants who did the rating thought that the attractive scholars were more to the right than they actually were! Notice that this result could easily just be in line with the Berggren et al. results above - people believe that better-looking people have a more right-leaning political orientation.

Similarly, the results demonstrate that the difference between actual and perceived political orientation is more positive for well-groomed scholars. This may be because well-groomed scholars are more right-leaning, or because the research assistants who did the rating though that the well-groomed scholars were more to the left than they actually were.

So, the results of the second paper tell us very little about the attractiveness of scholars and their political ideology. At the very least, we'd need to know about the results, excluding the subjective rating of political orientation from the models.

[HT: Weird Science on the NZ Herald, but I had seen the Berggren et al. paper earlier but I forget the source]

Monday, 10 April 2017

Teacher proposal tries to increase market power

As reported in the New Zealand Herald last week, the Education Council (the professional organisation for teachers) are floating a change that would require all prospective teachers to have a post-graduate qualification. Here's what was reported:
People wanting to become teachers would need to get a post-graduate qualification, under a change floated by the Education Council in a bid to raise the status of teaching.
The change would cover all teachers - early childhood, primary and secondary - and has the backing of universities.
First, it's not at all surprising that this proposal would have the backing of universities. If teachers require post-graduate qualifications, that means more people studying post-graduate qualifications (albeit probably not a whole lot more - see below), which means more fees for universities.

Second, it's really hard not to be cynical and see this as a simple way for existing teachers to increase their market power. If you raise the threshold for new graduates to become teachers, this creates a barrier to entry for graduates into the teaching market. [*] Essentially this restricts the supply of new teachers, because it would cost more to become a teacher (an additional year of student loans and foregone earnings, made even worse by student allowances not being available for post-graduate study). This would make alternative occupations (that are cheaper to get into), just a little bit more attractive. So, while this wouldn't stop everyone considering teaching as a career from following that path, at the margin it will be enough to dissuade some. Which is essentially what the PPTA says in the article:
However, the PPTA is unconvinced, saying teacher supply problems could worsen as a result.
Requiring all future teachers to get a post-graduate qualification would raise the bar for entry into the profession, and would likely be most keenly felt at the primary and ECE level.
Restricting competition makes sellers (in this case, sellers in the labour market for teachers) better off. It raises the bargaining power of the existing teachers, and when schools are looking for new teachers there are fewer options available, so they have to offer slightly higher salaries to attract applicants. Voila! Existing teachers are made better off, whether they are staying at the same school or moving between schools, at the expense of people who would have become teachers but for the higher barrier to entry.

The Education Council argues that this proposal would "raise the status of teaching". However, it's hard to see how it does that, other than by increasing teacher salaries at the margin. It may increase the quality of new teachers, to the extent that you believe a post-graduate qualification adds to their quality (dubious, when compared with the alternative of giving a graduate a year of actual teaching experience).

I'd argue that we probably need to go in precisely the opposite direction, especially for high school teachers of STEM subjects. When my father was at university, he taught chemistry at a high school in Auckland while studying his Bachelor of Science. That gave him a little bit of free cash flow, and benefited the school by providing a cost-effective way of teaching a subject that was difficult to recruit quality teachers for (and it certainly hasn't gotten any better since then). Why not open up teaching in STEM subjects to final-year graduates (probably economics too), given the teacher shortages in those subjects? Who knows? Some of them may actually enjoy the experience enough to stay on and become teachers (which they wouldn't have done otherwise). Raising the bar for people to become teachers is ridiculous, when you can't fill the vacancies for quality teachers in STEM subjects (and other subjects) now.

*****

[*] Ok, there is already a barrier to entry because teachers require a qualification. However, this proposal raises the barrier to entry even higher.