The persistence of economic misconceptions

Once people have made their mind up about something, they are generally unwilling to change their minds easily. We can link this to the idea of loss aversion from behavioural economics - we feel greater pain from losing something than we receive from gaining that same thing, and that applies to opinions just as much as it does to physical objects.

That unwillingness to change their minds leads people to hold a number of economic misconceptions - beliefs that are contradicted not only by economic theory, but are at odds with the beliefs of the vast majority of economists. Two examples of misconceptions are that trade is zero-sum (i.e. that there are not gains from trade for both parties), and that rent controls have generally beneficial (rather than negative) outcomes. [*] We might hope that economists could counter these misconceptions, either by teaching people some economics (but that doesn't appear to work) or by writing books (such as Economic Facts and Fallacies, which I reviewed earlier this week). However, misconceptions appear to be stubbornly persistent.

The futility of trying to address economic misconceptions is neatly on display in this new article by Jordi Brandts (Instituto de Análisis Económico), Isabel Busom (Universitat Autonoma de Barcelona), Cristina Lopez-Mayan (Universitat de Barcelona), and Judith Panadés (Universitat Autonoma de Barcelona), forthcoming in the Journal of Economic Psychology (ungated earlier version here). Busom, Lopez-Mayan and Panadés were all co-authors on earlier research on whether economics teaching could reduce misconceptions (which I discussed here), and this appears to be a follow-up to that earlier work.

In this article, Brandts et al. report on two studies that attempt to reduce misconceptions about the negative effects of rent controls. Both studies make use of a technique called a 'refutation text' (RT), which Brandts et al. explain as:

...a communication tool designed to help people revise their false beliefs through slow, analytical processing of information... Essentially, the RT must first explicitly state the belief and assert it is a misconception. It then should emphasize the negative consequences of the belief and refute it explaining the arguments and evidence obtained through scientific research. In this way the RT intends to connect this new information to the incorrect information pre-existing in a person’s memory. In addition, the RT should acknowledge the motivation for the misconceived belief.

The first study that they report on was a laboratory experiment, where research participants were allocated to one of three conditions: (1) RT; (2) non-refutational text (NRT); or (3) control. Each participant, either individually or as part of a team, answered some questions (which included their views on rent controls), then read the RT (or NRT, or neither, depending on which condition they were assigned to), and then were asked the rent control questions (along with a bunch of other questions) again, both immediately after the experiment, and several weeks later.

The second study was conducted in an economics class, across three cohorts (2015, 2017, and 2019), where:

The first cohort is exposed to a standard lecture on price controls and to a standard practice session where problems about supply, demand and price controls are solved; the second cohort is exposed to the standard lecture and to a practice session with the RT; and the third cohort is exposed to the standard lecture and to a practice session with the NRT.

The students completed questionnaires at the start and the end of their semester, which include questions about their views on rent controls.

Brandts et al. extract the effect of the RT on misconceptions by comparing the change in views between research participants (or students) in the RT group with those in the control group. They also compare RT with NRT, and NRT with control, as well as comparing those who completed individual tasks with those completing group tasks, as well as some other comparisons.

Brandts et al. present and discuss their results, but the results and their interpretations are not always in unison, and the results are not concordant across the two studies. I think they best summarise their results in their conclusion to the article:

What we learn is, first, that providing scientific information, be it through the RT or the NRT, about a salient issue such as the case of rent controls does change participants’ opinions in the direction of scientific consensus. Second, we learn that the way that the information is presented (RT vs NRT) does not make a difference... Third, a large proportion of participants still sticks to the misconception.

There is some weak evidence in the paper that the RT might have a small effect, when more reflective people discuss it in teams rather than individually. The main problem with this study was the small sample size, which did not have sufficient statistical power to detect small effects. Study 1 had 180 participants only, and while Study 2 had over 1200 students, it is based on cohort differences and there were differences in rates of attrition between the three cohorts, which may have contributed to the lack of statistical significance of the results.

The key takeaway is that this is the sort of research that is begging for additional replication studies, not just in economics but across many fields. Understanding how we can best counter misconceptions is important not just for economists, but also for public health researchers (to take a very salient current example), scientists, and researchers more generally.

*****

[*] Note that this doesn't apply to cases where there is still reasonable debate among economists, such as the disemployment effects of the minimum wage (although, see my most recent post on that topic). It also doesn't apply to misconceptions driven by underlying behavioural biases and heuristics, such as the sunk cost fallacy, or where the economic theory is more difficult to understand.

Read more:

• Does studying economics overcome misperceptions?

Book review: Economic Facts and Fallacies

Over the last few years, a number of people (including some students) have recommended that I read books by Thomas Sowell. I have a few of his books, but until recently I had never read any of them. The one that I chose to begin with was the 2011 second edition of Economic Facts and Fallacies, which I just finished reading.

To be honest, I found it a bit uneven. Sowell is an excellent writer, and the book is easy to read. However, in this book he clearly had some pet hates that he wanted to air. The book sets out a grand purpose:

The purpose of all this is not simply a debunking, in order to conduct a sort of demolition derby of ideas, but to reveal fallacies that have had harmful effects on the well-being of millions of people in countries around the world. Economic policies based on fallacies can be - and have been - devastating in their impacts.

That charge against economic policies seems fair, for example there are those policies that were highlighted in Joseph Stiglitz's book Globalization and Its Discontents (which I reviewed recently here). Sowell begins the book by outlining some specific categories of fallacies, including the 'zero-sum fallacy' (that what is gained by one person must have been lost by someone else), the fallacy of composition (that what is true of a part must be true of the whole), the post hoc fallacy (that because one thing happened after another, the second event must have been caused by the first), the chess-pieces fallacy (that policy can be devised to change people's behaviour in much the same way that a chess player moves pieces on a chess board), and the open-ended fallacy (that there is no scarcity, and no matter how much is done, more could be).

Those fallacies are well explained (much better than I could do in a single sentence). However, for the most part, the first chapter is the only place where those fallacies are encountered. After the first chapter, Sowell launches into what he considers various fallacies, generally without referencing any of them back to the types he outlined in the opening chapter. That left me wondering why the need to outline those fallacies in the first place. To be fair, they are in there, especially the fallacy of composition. It is just that Sowell doesn't refer to them much, if at all.

Each chapter after the first outlines a collection of fallacies within a topic area, and outlines various facts and research that Sowell uses to argue against the fallacy. He begins with urban fallacies, and then moves onto gender, academia, income, race, and development. Sowell clearly has a lot of bugbears that he wants to counter. However, he often overstates his case, and in some instances I believe he creates his own fallacious strawman arguments in order to do so. For instance, in relation to the declining middle class:

One of the simplest statistical illusions has been created by defining the middle class by some fixed interval of income - such as between $40,000 and $60,000 - and then counting how many people are in that interval over the years... the number of middle class people declines when there is a fixed definition of "middle class" in a country with rising levels of income.

The second part of the quote is correct, but is generated by the original claim that the middle class is based on a fixed level of income. I'm unaware of anyone serious who has made that claim, and Sowell doesn't attribute that claim to anyone (despite providing a number of references throughout the book). He sets up a strawman argument, and then sets it on fire. There are several other examples where he does the same, such as his claim that:

Here are encapsulated the crucial elements in most critiques of "income distribution" to this day... In reality, most income is not distributed, so the fashionable metaphor of "income distribution" is misleading.

People refer to the 'income distribution' because it is a statistical distribution, not because they believe that income is distributed (for example, by the government). Sowell walks back that claim a couple of pages later, but I find it highly ironic that he accuses advocates of redistribution of engaging in "verbal sleight of hand". To me, a significant portion of this book is a master class in verbal sleight of hand.

One further example of this should suffice. In considering audit studies (such as the one I described here), where researchers send out job applications that differ only in the race of the applicant in order to identify the extent of racial discrimination in decisions to invite applicants to a job interview, Sowell writes:

The fallacy in this approach comes from ignoring the high cost of knowledge and the high costs of making wrong decisions. Neither objective job qualifications nor income tell the whole story for anyone of any race. Other sorting devices may be resorted to where acquiring more specific information is costly, such as seeking more detailed information from previous employers - which many former employers are reluctant to provide, given the legal risks they face when providing adverse information - or hiring private detectives to look into the private lives of job applicants, housing applicants, or applicants for loans.

If the samples in audit studies are matched across all criteria except race, then none of these other considerations matter. Sowell sells an eloquent story here, but he is really just engaging in obfuscation.

The book is not all bad, though, just uneven. On the positive side, I found myself agreeing wholeheartedly when Sowell writes:

Concerns over poverty is often confused with concern over differences in income, as if the wealth of the wealthy derives from the poverty of the poor.

This is a point that I have made before (e.g. see here or here). There are many other examples of fallacious arguments that Sowell adeptly dismantles, especially those that are used by decision-makers (like bureaucrats, or university administrators) to impose costs on others. Clearly, imposing costs on others where a decision-maker faces no consequences themselves is one of Sowell's pet hates.

Overall, the book is easy to read, and may go some way towards helping readers to recognise that people make fallacious arguments to support their preferred policy prescriptions. However, as I'll note in my next post, it is unlikely that simply exposing those fallacies and arguing against them will actually change anyone's mind.

Financial education may be effective in raising financial literacy, but its effect on financial behaviours is more complicated

I have previously conducted research on financial literacy among teenagers, and continue to explore economic literacy among university students. As my previous research has shown (see here; with ungated earlier version here), financial literacy is low. It isn't just low among the teenagers we studied; financial literacy is low across society more generally (e.g. see here).

An obvious solution would seem to be to build up financial education. For example, making financial education compulsory in schools might increase financial literacy. So might making accessible adult financial literacy courses more available. In my reading of the previous research on financial education, it seemed to me that the evidence of the effectiveness of financial education is weak. However, I have been known to be wrong on occasion, and this might be one such occasion.

The evidence on the impact of financial education on financial literacy and financial behaviour is reviewed in this 2017 meta-analysis article by Tim Kaiser (University of Kiel) and Lukas Menkhoff (German Institute for Economic Research), published in the journal World Bank Economic Review (ungated earlier version here). Kaiser and Menkhoff's meta-analysis is based on 126 impact evaluation studies, and they summarise their findings as (emphasis is theirs):

...(i) increasing financial literacy helps. Financial education has a strong positive impact on financial literacy with an effect size of 0.26 (i.e., above the threshold value of 0.20 that characterizes “small” statistical effect sizes...). Moreover, effects on financial literacy are positively correlated with effects on financial behavior; (ii) financial education has a positive, measurable impact on financial behavior with an effect size of 0.09. An effect size of 0.08 is still found under rigorous randomized experiments (RCTs); (iii) effects of financial education depend on the target group. First, teaching low-income participants (relative to the country mean) and target groups in low- and lower-middle–income economies has less impact, which is an obvious challenge for policymakers targeting the poor. Second, it appears to be challenging to impact financial behavior as country incomes and mean years of schooling increase, probably because high baseline levels of general education and financial literacy cause diminishing marginal returns to additional financial education; (iv) success of financial education depends on the type of financial behavior targeted. We provide evidence that borrowing behavior may be more difficult to impact than saving behavior by conventional financial education; (v) increasing intensity supports the effect of financial education; and (vi) the characteristics of financial education can make a difference. Making financial education mandatory is associated with deflated effect sizes. By contrast, a positive effect is associated with providing financial education at a “teachable moment” (i.e., when teaching is directly linked to decisions of immediate relevance to the target group...).

I think there is a lot of good news that we can take away from that meta-analysis. However, notice that mandatory education doesn't make much difference, and targeting financial education at the right groups and at the right times ('teachable moments') is likely to be most effective.

The effect on financial behaviours might be the most questionable though. Kaiser and Menkhoff merge a huge variety of behavioural effects together in their meta-analysis, everything from reducing informal borrowings (e.g. from a moneylender), to having a bank account or insurance, to having a financial plan, to measures of net wealth. So, it's difficult to interpret what behaviours are actually improved by financial education. Were all financial behaviours improved? That seems unlikely. Which behaviours improved, and which did not? Did the characteristics of the target group and the type of financial education matter for which behaviours were affected? These questions are left unanswered. Also unanswered is the important question of what the mechanism for changes in financial behaviours is. We clearly need more studies linking financial education, financial literacy, and financial behaviours, but where the particular behaviours are pinned down.

That brings me to this recent article by Kenneth De Beckker, Kristof De Witte, and Geert Van Campenhout (all KU Leuven), published in the Journal of Economic Behavior and Organization (ungated earlier version here). They ran a randomised controlled trial among Flemish school students (average age 13) from 20 schools. In the trial, they gave treated students access to an online financial literacy course. De Beckker et al. explain:

The course deals with budgetary choices in everyday life. Afterwards, students are expected to be familiar with concepts like interest and inflation, have insight in different saving and investment products, understand the benefits of saving for long-term goals or unanticipated expenses, and grasp the risks of credit. The learning path consists of five modules with multiple exercises, information sheets and a formative test. The exercises contain videos, interactive learning games, and case studies adapted to the living environment of students from the eighth and ninth grade.

De Beckker et al. then measured students' financial literacy, as well as their financial behaviour, comparing students who were given access to the course with control students who were not. The 'financial behaviour' is measured using a discrete choice experiment, where students were presented with various hypothetical choices about the purchase of a new smartphone at various prices, and where some of the options required direct cash payments, whereas others had a payment plan. This research design allows them to look at how the financial education course affects the students' preferences for the smartphone purchase. That is, they can look at whether the financial education makes them more price sensitive, or more likely to avoid purchasing on credit.

Turning to their results, De Beckker et al. found that the course was effective in raising financial literacy among the students:

The results provide evidence that the financial education course is effective. Controlling for all observed heterogeneity in terms of school and student characteristics, the financial education course increases students’ financial literacy scores by 0.46 standard deviations on average...

However, in terms of financial behaviour:

Overall, we observe that the treatment does not affect how attributes like price, credit availability, information on the quality of the product and promotions are valued. This suggests that the financial education course increased students’ level of financial literacy but this did not trickle down further: students did not change their buying behavior. 

That is disappointing. The results of a single study of young Flemish teenagers is not enough to overturn the meta-analysis by Kaiser and Menkhoff. However, I think we need to consider further the mechanisms through which financial education, and financial literacy, lead to financial behaviour. Perhaps these students' hypothetical decisions were not affected, but the result might be different for financial education delivered at 'teachable moments'. Exploring the mechanisms further in future research would help policy makers and financial educators to better design financial education programmes (whether mandatory in schools, or delivered at 'teachable moments' like before a student takes out a student loan, or a first home buyer takes out a mortgage), so that financial behaviour is better aligned at improving people's long-term financial wellbeing.

Live streamed video lectures and student achievement

As I have noted many times on this blog (see the links at the bottom of this post), I strongly believe that online learning has heterogeneous effects on student learning and achievement. As I said in my most recent post on this topic:

Students who are motivated and engaged and/or have a high level of 'self-regulation' perform at least as well in online learning as they do in a traditional face-to-face setting, and sometime perform better. Students who lack motivation, are disengaged, and/or have a low level of self-regulation flounder in online learning, and perform much worse.

The problem with a lot of the research that tries to establish the effects of online or blended or hybrid learning on student achievement is that it doesn't distinguish between the effects on students at the top of the ability distribution and the effects on students at the bottom of the ability distribution. So, it doesn't really tell us a lot about how a change in teaching practice will affect the whole distribution of student achievement - it might only tell us what will happen at the middle of the distribution, which often isn't very helpful.

One recent exception is this recent article by Paula Cacault (EPFL), Christian Hildebrand (University of St. Gallen), Jeremy Laurent-Lucchetti, and Michele Pellizzari (both University of Geneva), published in the Journal of the European Economic Association (open access, but just in case there is an ungated earlier version here). Cacault et al. investigated the effect of live streamed video lectures on student attendance behaviour and their achievement across eight compulsory management, statistics, and economics courses at the University of Geneva. Students in each class (nearly 1500 students in total) were given access to a live stream of lectures in some weeks, but not others. As they explain:

Based on the enrolment lists of each course from the e-learning platform, we first randomly assigned students to three groups. A first group of students (15% of all students) never had access to the streaming service and we label this group the Never-access. Another 15% of the students were given access to the service in all the weeks of the term and we label this group the Always-access. The remaining 70% of students were given access to streaming only some weeks at random and we label this group the Sometimes-access. Every week, a varying share of students in this group was given access. In the Spring semester 2017, we randomly assigned weekly access to 50% of the sometimes-access group. In the Fall semester 2017, we decided to vary this share between 20% and 80%...

Physical attendance in the classroom was always possible and students could freely decide to go to class in person even in the weeks when they had access to streaming...

Cacault et al. then look at student performance in the final examinations for questions drawn from lectures where they had live streaming access against those from lectures where they didn't. First, they find that the take-up of live streaming is low:

Using information from the server of the streaming platform, we can identify the students who actually accessed the service. On average, only about 5% of the students (i.e., including those who had no access) used the service at least once in each week...

Combining information on assignment and usage we also construct measures of take-up, that is, the share of students with access who logged into the platform. This share is on average around 10%–11%, ranging across weeks from a minimum of 6.7% to a maximum of 13%.

So, very few students made use of the live streaming. Cacault et al. find that there are no differences in take-up between students at different 'ability levels' (defined by their academic performance in high school). They also find only a small effect on classroom attendance (which was measured from photos taken of the classroom):

...for every 100 students who are offered lectures via live streaming about 8 of them do not show up in class.

So, the effect on learning should be pretty small, given that few students make use of the live streaming option. Indeed, Cacault et al. find that:

Results indicate that on average there is no detectable effect of the experimental assignment, nor of actual usage of the streaming platform. However, once we look at effects by ability groups, we uncover large heterogeneity, with a sizeable negative ITT [intent-to-treat] effect on the low-ability students and a positive effect on the high-ability students...

The magnitudes of these estimates are sizeable. For students in the bottom 20% of the ability distribution, having access to the streaming platform (regardless of whether one uses it or not) lowers the share of correct answers by approximately 2 percentage points over an average of about 55%. The positive effect at the top of the ability distribution is even larger and in the order of about 2.5 percentage points. The ATT [average effect of treatment on the treated] estimates are very large: around -18 percentage points for the low-ability and about +25 percentage points for the high-ability students.

The ATT results are the effects on the students who actually participated in the live streaming, rather than just being given the option to do so, which explains the much larger effect.

Now, these results relate to live streaming, so we should be cautious about over-interpreting them as applying to all online learning. However, overall these results accord with those I have discussed on the blog earlier - online learning options tend to make more motivated, higher ability students better off, and less motivated, lower ability students worse off. Cacault et al. suggest a possible mechanism that explains this difference in effects:

Consider a situation in which the streaming technology is not available. Assume that in normal times most students attend lectures in person, but when the cost of going to class is too high, the good students tend to stay at home and study on their own. This happens because very good students can read the material in the book and understand most of it easily, even without the professors presenting and explaining it, whereas students of lower ability would have a harder time learning in autonomy and prefer attending.

Introducing the streaming technology in this context would allow all students to use it when the cost of going to class happens to be high. The good students replace own study with streaming, which improves learning and leads to the observed positive effect on exam performance. The low-ability students watch the streamed lectures instead of going to class, which is a more effective (but also more costly) mode of learning, and eventually perform worse. The students in the middle range of the ability distribution, substitute streaming to attendance for small shocks and streaming to no-attendance for larger shocks. Hence, average effect on grades tend to be close to zero, as we see in our data...

It's possible that is the mechanism at play in their context, where live streaming is available but the lectures are not recording, but I'm unsure how that translates to other contexts, particularly where the lectures are recorded. I do think we need more research on this area, but in particular, we need research on how to mitigate the negative impacts of online learning on students at the bottom of the ability and motivation distribution. If universities are serious about an enduring shift to online learning, this is a problem that needs urgent attention.

Read more:

Read more:

• Online vs. blended vs. traditional classes
• Flipped classrooms work well for top students in economics
• Flipped classrooms are still best only for the top students
• Online classes lower student grades and completion
• Online classes may also make better students worse off
• Meta-analytic results support some positive effects of using videos on student learning
• New review evidence on recorded lectures in mathematics and student achievement
• Meta-analytic results may provide some support for flipping the classroom
• The value of an in-person university education
• The pandemic may have revealed all we need to know about online learning