Generative AI, cognitive offloading, and escaping the 'illusion of competence'

Last week, for maybe the first time, I found myself telling a student not to use AI. That might sound extraordinary. After all, generative AI hit the big time with the release of ChatGPT in November 2022, and increasing numbers of students have been using it ever since. Many lecturers immediately freaked out, and many institutions initially reacted by banning or restricting AI use, before realising that they were fighting a losing battle against the incoming tide of generative AI and trying to impose 'guardrails'.

I've never asked my students not to use generative AI. In fact, I've encouraged it. I even have custom AI tutors set up for each of the papers I teach, that use a knowledge base of materials from the paper to give students targeted assistance. I have little to fear from generative AI, because the vast majority of assessment in my papers is in-person and invigilated (that's one of the beautiful things about teaching first-year papers - I can argue that basic concepts and applications can be authentically assessed in an exam environment).

Anyway, back to the story. The student was using our class AI tutor during class, to give them a solution to a problem we were working on during class. I pointed out that it defeated the purpose of doing the problem in class, if they used Jane (our ECONS102 AI tutor is named after Jane Marcet, the author of the 19th-Century popular economics book, Conversations on Political Economy) to solve it for them. The problem wasn't the use of Jane per se (after all, I encourage them to use her). It was that by using Jane to solve the problem for them, they were missing out on a key learning opportunity.

Probably, I was a little hard on the student. After all, they were using the tools available to them, and engaging in cognitive offloading - reducing the demand or mental load that they face by offloading a task onto generative AI. And increasingly, students are engaging in this cognitive offloading, sometimes in helpful ways, but often in ways that are detrimental. That is one of the conclusions from this 2026 report (with non-technical summary on The Conversation) by Jason Lodge and Leslie Loble (both University of Technology Sydney).

The report has a lot of quotable quotes. For instance, they note that:

It is not possible to engage in critical thinking when one has nothing to think critically about. A person does not simply think critically in a vacuum. A scientist thinks critically about a flawed methodology by drawing on a vast store of knowledge about experimental design. A historian thinks critically about a primary source by drawing on their knowledge of the document’s social, political, and historical context...

This, to me, highlights the key challenge that education faces with generative AI. In order for students to be well prepared for engaging with generative AI, they need to be able to evaluate AI output. And without a thorough grounding in disciplinary knowledge, their evaluations would at best be superficial. And that is why I hold the line on having assessment in my papers that explicitly excludes the use of generative AI. My papers build the foundation on which students' later use of generative AI, and their evaluation of AI outputs, can build.

Lodge and Loble note that:

Every task or learning activity is essentially now a group activity. It just so happens that the other member or members of the group are machines that have practically all human knowledge at their fingertips (in their databases/algorithmic weights). Like any other group activity, students can benefit from that collaboration or get the smart kid to do all the work for them.

That is absolutely what is happening. Students' learning activities are now mostly group activities, even when they are the only human in their group. In group work, how the work is shared is important, and that is where cognitive offloading comes in. Lodge and Loble distinguish between two forms of offloading:

• Beneficial offloading occurs when AI is used to manage extraneous cognitive load (e.g., checking grammar), freeing a learner’s limited working memory to focus on essential, intrinsic tasks.

• Detrimental offloading (outsourcing) occurs when a learner uses AI to bypass this intrinsic cognitive effort (the desirable difficulties) required to build long-term knowledge schemas. This offloading also seems to extend to vital metacognitive and self-regulated learning capabilities, compounding the negative impact of outsourcing on learning.

Importantly, Lodge and Loble note that students typically don't understand metacognition. They haven't intentionally engaged in thinking about their own thinking and understanding how they learn or managing that process. In my experience, many students tend to have been passive recipients of learning approaches, without really engaging with the process themselves. And even those that do engage usually haven't thought deeply about how they learn. And so, when they use a tool that gives them ready answers, it may seem to students that they are learning more efficiently. Lodge and Loble label this an 'illusion of competence', noting that:

Research has long shown that fluent learning materials, such as high-quality videos, can lead people to greatly overestimate how much they have learned by mistaking the ease of processing (fluency) for the depth of learning...

Lodge and Loble's report doesn't stop at the point of diagnosing the problem though. They present three main solutions, that involve:

• shifting generative AI use towards beneficial offloading, where students free up cognitive resources to focus on intrinsic learning. Lodge and Loble offer the example that "AI can be used to provide scaffolding, structured practice, and feedback, all aimed at managing the cognitive burden on the learner and enabling progressive independence...";
• deliberately designing AI interactions to include metacognitive responsibilities, so that students must pause, reflect, and assess their own understanding; or
• shifting the fundamental role of AI from being an 'answer oracle' to a tool that provokes intrinsic load. Lodge and Loble offer examples such as asking students to teach the AI (which plays the role of a confused student), setting up AI as a Socratic tutor, or asking students to independently verify AI outputs.

Those solutions have implications for how I design and use AI tutors in my papers. In order to limit students from engaging in detrimental cognitive offloading, the AI tutor shouldn't simply act as an answer machine. Instead, they should encourage students to attempt problems themselves, offer hints or scaffolding when they get stuck, and ask them to explain or justify their reasoning. And, importantly, the AI tutor could also prompt students to reflect on what they understand, what they don't understand, and whether they could solve the problem on their own. The master prompt for my AI tutors does instruct them to take a Socratic approach, but they don't adhere to it strictly. I'll certainly be putting more thought into the master prompt to see if I can dissuade them from being answer machines and to incorporate more of the metacognitive elements in the future.

The solutions provided by Lodge and Loble are useful, and hopefully they prompt other lecturers to think intentionally about students' (and possibly their own) engagement with generative AI. I especially like the second option, because I believe that we all (and not just students) can benefit from better understanding our learning process, and recognising when we are engaged in genuine and effortful learning. This is not the first time I've encountered concerns about cognitive offloading in the context of generative AI and education (see here). And it is interesting that the same, or a similar, set of solutions keep being presented. In particular, integrating technology as a complement, rather than a substitute, for thinking is important. Generative AI has dramatically lowered the cost of getting answers. It hasn't lowered the cost of learning. A better understanding of metacognition is therefore important too.

Perhaps, then, the lesson from my interaction with the student isn't that they shouldn't have been using generative AI in class. It's that they need to understand when using generative AI in class supports their learning, and when it substitutes for the thinking that learning requires.

I think most universities are now considering explicitly including generative AI in the curriculum, in order to better prepare students for future careers that will no doubt involve substantial interactions with generative AI. Perhaps we should also be considering explicitly including metacognition in the curriculum?

Read more:

• Neuroscientific insights into learning and pedagogy, especially in the age of generative AI

The impact of using the CORE textbook in Uruguay

We introduced the CORE textbook The Economy at the University of Waikato when we recoded the compulsory economics paper in our management degree from ECON100 to ECONS101 (see here). We were early adopters, as the CORE textbook was only released in 2017. It was a big change, and largely a positive one. The CORE textbook was free, substantially lowering the cost for students to access an important learning resource. Because the textbook was online, it could be constantly updated. And I really liked the way that it turned the traditional approach to the teaching of microeconomics on its head. Instead of starting with perfect competition and the supply and demand model, and then teaching imperfect competition as an exception, the CORE textbook started with monopolistic competition (where firms sell products that are differentiated from those of their competitors) and teaches perfect competition as an exception. Since many firms operate in monopolistically competitive markets, the approach that CORE adopted seems more attuned to the real world that students see.

I've often wondered whether the CORE textbook improved students' learning though. So, I was interested to read this recent article by Federico Araya (Universidad de la República, Uruguay) and co-authors, published in the journal Economica (sorry, I don't see an ungated version online [*]). They evaluate the impact of adopting the CORE textbook for the introductory economics course at the Faculty of Economic Sciences and Administration (FCEA) at the Universidad de la República, the largest university in Uruguay.

Although the CORE textbook was introduced at FCEA in 2020, Araya et al. start their analysis from 2021, to avoid the impacts on online teaching during the pandemic. FCEA offered two introductory microeconomics courses, one of which used CORE and the other continued to use their traditional textbook. Students were randomly assigned to either course based on the last number of their identification document, with 30 percent of students assigned to the course that used CORE.

However, the textbook was not the only difference between the two courses. As Araya et al. explain:

Although attendance is optional in both courses, in 2022 and 2023, the CORE course introduced a modification to its evaluation system, assigning 10% of the total grade to group activities conducted during class sessions. This change may have created an incentive for higher attendance.

So, their evaluation is not a clean comparison of the same course taught with two different textbooks, but will compare two different pedagogies, one which uses the CORE textbook and in-class group activities that are worth grade points, and one that uses a traditional textbook without the in-class group activities.

Araya et al. then compare the two groups in terms of whether students passed the introductory microeconomics course, as well as whether students passed an introductory calculus course and whether they passed the intermediate microeconomics course that follows on from the introductory course, while controlling for a range of demographic and socioeconomic variables for each student. They find:

...no statistically significant differences in pass rates between CORE and the conventional course, with the exception of the 2022 cohort.

I was initially surprised that they decided to evaluate each cohort separately, rather than pooling them. However, the 2021 cohort is different because that year the CORE course didn't have the in-class group activities, whereas it did for the 2022 and 2023 cohorts. Combining the 2022 and 2023 results would give us a better sense of the overall effect (of the combined CORE textbook plus group activities intervention). Instead, we are shown a statistically significant positive effect in 2022, but no statistically significant effect in 2023. That doesn't tell us whether the effects in those two years were actually different from each other, or whether the combined intervention had a positive overall effect across the two cohorts. One further problem here that muddies the comparison is that the CORE and traditional courses didn't use the same assessment, and so passing one course may be different from passing the other. And that might also explain the different cohort-specific results (if the 2022 traditional course had more difficult assessments than the CORE course, for example).

That won't be a problem for comparisons in terms of student performance in introductory calculus and intermediate microeconomics. For those courses, Araya et al. also find no statistically significant effects on passing.

So, at least there is no evidence from this study that the CORE textbook (with or without in-class group activities) made students worse off. Although equally, there is no evidence that it made them better off either. That allows me to raise an issue that is general to much of the similar research on educational interventions (including my own research on the impact of AI tutors in ECONS101). We might expect to see no significant effect on student performance even from a successful intervention. That's because a successful intervention may make studying easier for students, freeing up time that they can then devote to other activities. That might be studying for their other courses (although notice that in this case any reallocation of study effort doesn't appear to have affected the probability of passing introductory calculus), or something entirely different (maybe working more, or having more leisure time). So, I'm not surprised to see no effect of CORE on student performance in this study.

We continue to use the CORE textbook in my ECONS101 class (although last year we moved to the new edition, The Economy 2.0). We don't follow the text very closely, at least not in the microeconomics section of the paper that I teach. Nevertheless, it continues to provide the base material for a lot of what we teach. And it's good to know that at least one study says that there is not evidence that continuing to use a 'non-traditional' text is doing harm to students.

*****

[*] It's kind of ironic that a paper evaluating the impact of an open-access teaching resource is not itself published open-access.

Read more:

• Goodbye to ECON100, hello to ECONS101
• Differing perspectives on introductory economics textbooks
• Does the CORE textbook not go far enough?

How important is the apprenticeship model to entering a research career?

Those of us working in research careers can invariably share stories about working as a research assistant, cleaning datasets, coding, running models, doing literature reviews, and many of the other less-glamorous tasks that make up the research process. That was a key aspect of our apprenticeship into the world of research, and a gateway into a research career. But how important really is research assistance as an entry point into a research career?

That is the question addressed in this 2025 NBER Working Paper (ungated version here) by Ina Ganguli (University of Massachusetts, Amherst) and Raviv Murciano-Goroff (Boston University). They look at the impact of working in a university lab (an important subset of research assistance work) on subsequently pursuing a scientific career. Interestingly, Ganguli and Murciano-Goroff use changes in the local minimum wage as an exogenous source of variation in lab employment, so this paper also indirectly contributes to the literature on the employment effects of the minimum wage, in a context (research labs) that is not often the focus of that literature.

Their data comes from UMETRICS, which collates data on research grants across universities, and their dataset covers 32 universities over the period from 2000 to 2019 (although the dataset has coverage up to 2022, including those additional years would mean having to account for the COVID-19 pandemic).

First, using a dataset collated at the lab level, Ganguli and Murciano-Goroff use a staggered difference-in-differences approach to look at the impact of minimum wage changes on employment of undergraduates in the labs. This analysis essentially compares the change in undergraduate employment between the time before and the time after an increase in the minimum wage, between university labs that were affected by the minimum wage increase and those that were not. In that analysis, they find that:

...following minimum wage increases, labs decrease the employment of undergraduates by 7.4% on average...

So, not dissimilar to the literature on minimum wage effects on employment, when focused on young people in exposed occupations. Ganguli and Murciano-Goroff then use the minimum wage change as an instrument for students' exposure to laboratory research as an undergraduate. The key assumption is that minimum wages while an undergraduate affect later scientific careers only through their effect on lab employment opportunities.

Using a dataset of over 28,000 undergraduates and their subsequent career paths, Ganguli and Murciano-Goroff find that:

...decreased exposure to scientific work translates into significantly lower rates of undergraduate research assistants pursuing doctoral degrees or working in the life sciences sector after graduation. We find that working one fewer quarter in a lab during an undergrad student’s college years translates into between a 7.0 and 10.3 percentage point decrease in the rate of enrolling in a doctoral-level program. Given our sample of 28,283 students, this implies that if all students had experienced a minimum wage increase, roughly 500 fewer undergraduates in our sample would have pursued these advanced degrees.

So, the results imply that one fewer quarter of lab experience as an undergraduate reduces enrolment in a doctoral programme by 7.0 to 10.3 percentage points. That is a fairly large effect and should make graduate research programmes take notice of the importance of undergraduate research assistance opportunities for the pipeline into graduate research.

So, working in a lab as an undergraduate is a key pathway towards a career in the life sciences, and when those opportunities are restricted, students are less likely to embark on such a career. These results won't be too surprising to those of us who have been through an apprenticeship as a researcher. It is likely that I would be doing something very different right now if I hadn't been pulled into a lot of research projects towards the end of my undergraduate studies. I almost certainly wouldn't have pursued a PhD, or become an academic. Research assistant jobs are more than just jobs. They provide mentoring, information, skills, networks, references, and a chance to try out being a researcher in a safe setting.

These results also highlight two other things for me. First, they suggest that minimum wage increases may have a negative impact on the career pipeline into science. I'm unsure that this negative impact of the minimum wage has been identified before. However, we should be cautious because Ganguli and Murciano-Goroff are primarily interested in using minimum wage changes to identify the effect of undergraduate research experience on later careers, rather than estimating the overall long-run consequences of minimum wage increases for the scientific workforce. Nevertheless, their results suggest that this may be an important unintended consequence of higher minimum wages, and one that would be worth further research.

Second, given that research assistants do a lot of the 'drudge work' in research, if generative AI is also able to do a lot of that work, that further suggests a negative impact on the career pipeline into science through the rise of generative AI. I know others have written on this before (see here), so the challenge here is not a new idea.

On the plus side, that suggests another method that could be used to further test for the impacts of undergraduate (and graduate) research assistance on future academic careers, beyond focusing on life sciences (as Ganguli and Murciano-Goroff do). By comparing academic fields that are more (or less) exposed to generative AI (especially in the early days of generative AI), we might be able to tease out how important research assistance is to future academic careers across many fields.

Research apprenticeship is important, and these results give us a clear indication of how important it can be. Cleaning data, running models, searching the literature, and doing all the other seemingly mundane tasks of a research assistant are not just cheap ways for senior researchers to get research done. They are also how the next generation of researchers learns what research is, discovers whether they enjoy doing it, and gets started on a research career. If these opportunities are reduced, whether through higher minimum wages or through substitution by generative AI, we may save on some of the drudge work today, but at the cost of having fewer researchers tomorrow.

[HT: Marginal Revolution]

This week in research #138

Here's what caught my eye in research over the past week:

• Akter et al. argue that New Zealand’s low-risk drinking advice is outdated and now overdue for review
• Jones (open access) outlines two scenarios for the impact of artificial intelligence on the economy, providing some guidance about the potential future consequences of AI
• Lee and Porter (open access) provide a simple introduction to the 'weak instruments' problem, including some practical do’s and don’ts implied by the findings of the weak instruments research literature
• Poterba and Werning (open access) summarise the work of last year's John Bates Clark medal winner, Stefanie Stantcheva