Back in October, I posted about gender differences in multiple choice answering, which was based on research that used data from the PISA survey of high school children. The results demonstrated that male students perform better than female students, which is a common feature of multiple-choice tests (see the links at the bottom of this post for more). The research also provided some suggestive evidence that confidence in answering was important, along with stereotype threat, as I noted in this 2019 post. Related to confidence, part of the difference in performance between male and female students depends on differences in the propensity to leave some questions blank. Students that are less confident (who are relatively more likely to be female students than male students) are less likely to leave an answer blank.
Now, PISA is a 'low-stakes' test, in the sense that the results don't matter for the students at all. So, there is no negative consequence to leaving a question unanswered. That isn't the case in a high stakes examination, where guessing may come with a positive net payoff (if there is no penalty for a wrong answer), or may offer no net advantage (if there is a penalty). In that case, students may differ in whether they leave questions blank, depending on their confidence and their degree of risk aversion.
The propensity of male and female students to leave questions blank is investigated in this recent article by Perihan Saygin and Ann Atwater (both University of Florida), published in the journal Economics of Education Review (sorry, I don't see an ungated version online). They use data from the Turkish OSS, which is the main college admissions examination. This examination has an interesting feature that it has several sections, and those sections have different weights for different students. As Saygin and Atwater explain:
The Turkish high school curriculum features high school students being split into tracks of their choosing in their second year. The tracks are Science-Mathematics (Quantitative), Literature-Mathematics (Equally Weighted), Social Science-Literature (Qualitative), Foreign Languages, and the Arts. These fields line up with the topics covered on the ¨OSS. It has a total of four core sections: social science (which covers history, geography, and philosophy), science (biology, chemistry, and physics), mathematics, and literature... For each subject, the exam has a lesser difficulty section and a higher difficulty one. Thus, there are 8 sections in total on the ¨OSS...
The section weights then depend on which track the students were in during high school. So, a student in the quantitative track will have a higher weighting on the maths sections, but a lower weight on the social science section, whereas a student in the social science-literature track has the opposite. This allows Saygin and Atwater to investigate how differences in the stakes associated with particular sections affects the difference in propensity to leave questions blank between male and female students. Based on a sample of 1792 randomly selected OSS participants taking the OSS examination for the first time, they find that:
...not only is the gender difference in tendency to leave questions blank largest on sections that cover mathematics, but that this difference is only significant for the test takers on the track that places the most emphasis on these sections... We also provide evidence that this gap is larger on more difficult sections of a given subject despite these sections being weighted equally to the lower difficulty sections in score calculations.
Saygin and Atwater argue in a number of places in their paper that their results demonstrate that risk aversion isn't playing a role. However, when you see the results summarised as they are above, it is hard to draw any other conclusion. If a student is worried about the risk associated with guessing, then that risk is highest on the sections of the examination where the stakes are highest. Unfortunately, Saygin and Atwater don't have any measure of risk aversion. They do have measures of confidence, and looking at those they find that:
...a positive self-assessment on a given subject is related to skipping behavior on that subject and explains part of the gender differences in tendency to leave questions blank. In addition to this, we provide evidence for gender differences in self-assessment in a given subject conditional on the performance on the corresponding test section. Male test-takers are more likely to report a positive self-assessment than their test performance would suggest in math, science, and social science while this gender difference is inverted in literature. This variation in reported self-assessment across subjects matches the pattern of the observed gender differences in tendency to leave questions blank across subjects.
So, overconfidence explains at least some of the difference in leaving questions blank. The inability to convincingly eliminate risk aversion, or the interaction between risk aversion and confidence, leaves the question of the mechanisms underlying this behaviour still open.
Also, the sample that Saygin and Atwater use is somewhat idiosyncratic. The 1792 final sample is drawn from a larger random sample of nearly 10,000 OSS students. As far as I can tell, the smaller sample arises when they eliminate students that are attempting the OSS examination for the second or subsequent time. That suggests that around 80 percent of students in the OSS make multiple attempts before they get an examination ranking that they are satisfied will gain them entry into a good Turkish university. Clearly, there is a risk of selection bias in the sample in this research that is not accounted for.
Anyway, this paper modestly contributes to the idea that confidence contributes to the gender difference in multiple choice examination performance. However, we still need more research to better understand this topic.
Read more:
No comments:
Post a Comment