Monday, 14 October 2019

Why study economics? Improve your analytical skills edition...

Consider the following three questions:

  1. A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost?
  2. If it takes 5 machines 5 minutes to make 5 widgets, how long would it take 100 machines to make 100 widgets?
  3. In a lake, there is a patch of lily pads. Every day, the patch doubles in size. If it takes 48 days for the patch to cover the entire lake, how long would it take for the patch to cover half of the lake?
Those three questions make up the Cognitive Reflection Test (CRT), an increasingly used measure of analytical skills and problem-solving ability (and in case you are wondering, the correct answers are at the end of this post). Measuring analytical skills is difficult but important. It allows us to evaluate wild claims like: "Taking economics courses will improve your analytical skills more than taking other courses".

Interestingly, that's exactly the claim that this new article by Seife Dendir (Radford University), Alexei Orlov (US Securities and Exchange Commission), and John Roufagalas (Troy University), published last month in the Journal of Economic Behavior and Organization, does (sorry, I don't see an ungated version). They use data from the beginning and end of a semester at "a comprehensive, medium-sized university", where the CRT was completed by students at the beginning of the semester (the pre-test), and a functionally equivalent three-question test was completed by students at the end of the semester (the post-test). They then compare the differences in the before-and-after semester results between students who were, and were not, taking at least one economics class (this is what is referred to as a difference-in-differences research design).

Using data from 620 students (including 529 pre-tests, and 381 post-tests), they first note that:
While the majority of students did not answer any questions correctly on either the pre-test or the post-test (as indicated by the medians of 0), there was an overall improvement in student scores between the pre-test and the post-test (as reflected in the means of 0.54 and 0.90, respectively).
You read that right. More than half of everyone got zero out of three correct, both in the pre-test and in the post-test. You're not feeling so confident about your answers to the questions at the top of the post now, are you? The important results compare economics and non-economics students, and find that:
...the typical student in our sample scores 0.225 points better in the post-test as compared to the pre-test, almost a 42% improvement [ = 0.225/0.54]. The DiD coefficient equals 0.226, which means that the typical student enrolled in an economics class improves his or her post-test score by an additional 0.226 points (over and above the improvement made by non-economics students). This improvement is about 42% of the average pre-test score [ = 0.226/0.54], or 25% of the average post-test score [ = 0.226/0.90], or 7.5% of the maximum possible score of three [ = 0.226/3].
The students taking economics start from a lower pre-test score (on average) than non-economics students, and then improve their score by significantly more over the semester than non-economics students. The results are robust to the inclusion of various control variables, and robust to alternative specifications of the econometric model. Based on these results, economics improves students' analytical skills.

But wait! What about students enrolled in other courses that might plausibly improve analytical skills, like STEM? It turns out that the results are similar for STEM students, and for mathematics students, but not for statistics students, or business students, social science students, or arts students. That makes sense.

It seems that the improvement in analytical skills from economics derives entirely from economics principles classes, because Dendir et al. find that the results for other economics classes are statistically insignificant. There are two interpretations to this result. First, economics principles is where students learn the basic building blocks of 'thinking like an economist', which plausibly provide the greatest gains in analytical skills. Alternatively, perhaps there is something unusual about the way that economics principles is taught at this one university, such as a focus on the types of analytical skills that are picked up in the CRT?

This is an important first result demonstrating the impact of economics on analytical skills in a plausibly causal research design. However, some effort at replication of this result is clearly warranted. In the meantime though, there is support for the claim that "Taking economics courses will improve your analytical skills more than taking other courses (except for mathematics)".

Oh, and the intuitive (but wrong) and correct answers to the CRT questions:

  1. Intuitive answer: 10 cents. Correct answer: 5 cents.
  2. Intuitive answer: 100 minutes. Correct answer: 5 minutes.
  3. Intuitive answer: 24 days. Correct answer: 47 days.
Read more:

Posted by at
Labels: ,

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)