I've spent a substantial proportion of the last couple of weeks on a new project, testing out and training a new AI tutor for my ECONS101 class. And now she's ready to debut. I released her for the human ECONS101 tutors to try out over the weekend, and to my ECONS101 class in their first lecture today.
You can find the AI tutor here. Her name is Harriet, and currently she is trained on material from Topics 1 to 3 from ECONS101. That covers some of the basic economic concepts like opportunity cost, incentives, and economic rent, a simple production model (with iso-cost lines, but not isoquants), constrained optimisation models (with budget constraints and indifference curves), and game theory. There will be separate instances of Harriet for later groupings of three topics.
The idea for having an ECONS101 AI tutor has been percolating for a while. However, the impetus for development was set off by two seminars that I briefly mentioned in this post last month - this seminar by Justin Wolfers, and this seminar by Kevin Bryan (and especially his description of the AI teaching assistants at alldayta.com), as well as this 2024 paper describing the effectiveness of an AI tutor in Harvard engineering papers (which I will blog about later this week). I was convinced that we could develop something that would work for our students.
I should also acknowledge the parallel work on AI tutors in ECONS101 being undertaken by Michael Ryan at our Tauranga campus. Michael started development in advance of me, so I was able to free ride a little bit on his learnings (although we have slightly different use cases for our AI tutors). Michael was able to point out some important blind spots that AI tutors in economics seem to suffer from. I hope that he has learned something from me along the way as well.
In testing, it has become clear that Harriet is very good at some things, and less good at others. With basic economic concepts, Harriet is excellent, albeit a bit more verbose than I would like. I'd rather that she prompts students to ask more questions, but it seems like she prefers to give a more fulsome answer herself to any question she is presented with. I know that we could train her out of this, but it seems that the only effective way to do so is to try and anticipate exactly (or nearly exactly) the questions that students may ask her. In the Harvard paper I referred to above, they turned this into a key feature of their AI tutor, which was trained to walk students through a particular tutorial, step-by-step. Instead, I wanted students to be able to ask any question they like, and get a helpful answer.
Harriet is great at interpreting economics diagrams, and solving game theory problems, provided the diagram or problem is uploaded as a screenshot when the student asks the question. She is rubbish at drawing her own diagrams, and at creating her own game theory examples and solving them. Clearly, there is still substantial scope for human tutors in ECONS101, at least until we can train an AI to draw more than a very simple supply and demand diagram (and even then, Harriet has trouble lining up the price and quantity with the equilibrium point).
Why name our AI tutor Harriet? Harriet Martineau was a 19th Century social theorist, author, and populariser of economics ideas. She wrote a famous book in 1832, titled Illustrations of Political Economy, which drew on the work of Adam Smith, and applied a tutorial style to help readers understand Smith's ideas. Her later books included similar treatments of economic ideas by David Ricardo, James Mill, and Jeremy Bentham. Harriet Martineau is a perfect model of what we wanted our ECONS101 AI tutor to be - someone who could explain economic concepts in an accessible way. So, we named our AI tutor after Martineau.
I can see from the server logs on Moodle that Harriet has already been accessed dozens of times by students, and we're only one day into the new trimester. No doubt most of those students are simply curious about what the AI tutor is. Hopefully, they will find her useful as we move through the trimester. And, if they identify explanations that Harriet gets wrong, we are even offering students a bounty (a small amount of extra credit) for letting us know, so that we can improve her training.
I'm looking forward to seeing how things progress. Harriet is one of several new initiatives we are trying in ECONS101 this year, to get students more engaged and improve their grades and their learning of business economics. My ECONS102 class can expect to have their own separate AI tutor in the B Trimester later this year, based on similar principles.
No comments:
Post a Comment