What we learn from Freelancer.com about labour market signalling in the age of generative AI

In yesterday's post, I outlined my case for why generative AI reduces the quality of signalling in education. That is, how good education (qualification, or grades) is as a signal to employers of an applicant's ability. There is evidence to support this case, from two recent papers.

The first paper is this pre-print by Jingyi Cui, Gabriel Dias, and Justin Ye (all Yale University), which looks at the signalling benefit in cover letters. Specifically, they study:

...the introduction of a generative AI cover letter writing tool on Freelancer.com, one of the world’s largest online labor platforms. Freelancer connects international workers and employers to collaborate on short-term, skilled, and mostly remote jobs. On April 19, 2023, Freelancer introduced the “AI Bid Writer,” a tool that automatically generates cover letters tailored to employers’ job descriptions that workers can use or edit. The tool was available to a large subset of workers depending on their membership plans.

Cui et al. use eight months of data on two skill categories (PHP, and Internet Marketing), which covers over five million cover letters submitted to over 100,000 job opportunities. They observe who had access to the tool, as well as who used the tool to generate a cover letter, and how much time they spent refining the AI-generated cover letter.

Cui et al. look at the impact of the availability of the generative AI tool on callback rates, using a difference-in-differences research design. This effectively involves comparing differences in callback rates between applicants with and without access to the tool, before and after the tool was made available. Cui et al. find that:

...access to the generative AI writing tool increased cover-letter tailoring by 0.16 standard deviations, while actual usage raised tailoring by 1.36 standard deviations. Applying the same design to callbacks as the outcome, we find that access to the generative AI tool increased the probability of receiving a callback by 0.43 percentage points, and usage raised it by 3.56 percentage points. The latter represents a 51% increase relative to the pre-rollout average callback rate of 7.02%.

All good so far. Job applicants are made significantly better off (in terms of receiving a callback) by using the tool. However:

Our second finding is that AI substitutes for, rather than complements, workers’ pre-AI cover letter tailoring skills... We find that workers who previously wrote more tailored cover letters experienced smaller gains in cover letter tailoring—indeed, the best writers... experienced 27% smaller gains than the weakest ones. By enabling less skilled writers to produce more tailored cover letters, AI narrows the gap between workers with different initial abilities.

In other words, employers are now less able to distinguish the quality of the worker by using the quality of the writing in the cover letter. The consequence of this is that:

The correlation between cover-letter tailoring and receiving a callback fell by 51% after the launch of the AI tool, and the correlation with receiving an offer fell by 79%. Instead, employers shifted toward other signals less susceptible to AI influence, such as workers’ past work experience. The correlation between callbacks and workers’ review scores—the platform’s proprietary metric summarizing past work experiences on the platform and determining the default ranking of applications—rose by 5%. These patterns suggest that as AI adoption increases, employers substitute away from easily manipulated signals like cover letters toward harder-to-fake indicators of quality.

The total number of interviews and job offers were unchanged during this period. Cui et al. don't directly report whether the number of callbacks changed, but if we infer that from there being no aggregate change in the number of interviews, then this is consistent with the idea that the key difference is in the distribution of who received the jobs (and callbacks). Workers with a strong alternative signal (other than a well-written cover letter) received more callbacks, meaning that workers who lack an alternative signal received fewer callbacks. That has an important distributional consequence. New workers typically lack past review scores, so as employers lean more heavily on reviews, workers who are new to Freelancer.com will be disadvantaged and will find it more difficult to get a callback. Overall, in this case, the impact of the generative AI tool on the quality of signalling is negative.

The second paper is this job market paper by Anaïs Galdin (Dartmouth College) and Jesse Silbert (Princeton), who also use data from Freelancer.com. The difference is that they carefully evaluate employers' willingness-to-pay for workers, using the bid data. They also look at customisation of the text of the whole proposal, not just the cover letter. Another difference is that Galdin and Silbert look at a different job type, coding. Their data covers 2.7 million applications to 61,000 job openings, by 212,000 job applicants. Although Galdin and Silbert's paper is far more technical than the Cui et al. paper, Galdin and Silbert's results are somewhat similar (in terms of what they tell us about signalling):

First, we show that before the mass adoption of LLMs, employers had a significantly higher willingness to pay for workers who sent more customized proposals. Estimating a reduced-form multinomial logit model of employer demand using our measure of signal, we find that, all else equal, workers with a one standard deviation higher signal have the same increased chance of being hired as workers with a $26 lower bid... Second, we provide evidence that before the adoption of LLMs, employers valued workers’ signals because signals were predictive of workers’ effort, which in turn predicted workers’ ability to complete the posted job successfully. Third, we find, however, that after the mass adoption of LLMs, these patterns weaken significantly or disappear completely: employer willingness to pay for workers sending higher signals falls sharply, proposals written with the platform’s native AI-writing tool exhibit a negative correlation between effort and signal, and signals no longer predict successful job completion conditional on being hired.

This is strong evidence that, in this context at least, the introduction of the generative AI tool substantially reduces the quality of the job application signal. Galdin and Silbert then build an economic model calibrated based on their empirical results, and using that model they find that:

Compared to the status quo pre-LLM equilibrium with signaling, our no-signaling counterfactual equilibrium is far less meritocratic. Workers in the bottom quintile of the ability distribution are hired 14% more often, while workers in the top quintile are hired 19% less often.

This suggests an even worse outcome than what Cui et al. find. Galdin and Silbert's results suggest that the distributional changes in who gets offered work make high-quality workers worse off, and low-quality workers better off. That is what we would expect when the quality of signalling is reduced. Galdin and Silbert go on to say that:

These effects are driven by three mechanisms. First, employers previously relied on signals to make hiring decisions, so losing access to them impinges on their ability to discern worker ability. Second, more indirectly, the significant positive correlation between a worker’s ability and cost implies that, when employers lose access to signals and workers are forced to compete more intensely on wages, the prevailing workers with lower bids tend to have lower abilities. Third, since workers’ observable characteristics are poor predictors of their ability, employers have little to no information to distinguish between high and low-ability workers.

These changes to hiring patterns lead to a 5% reduction in average wages, a 1.5% reduction in overall hiring rate per posted job, a 4% reduction in worker surplus, and a small, less than 1%, increase in employer surplus.

The overall takeaway from both papers is that generative AI reduces the quality of signals to employers. They don't speak directly to the quality of education signalling, but we can infer that if the quality of other signals of worker quality are reduced by generative AI, then the quality of the education signal likely is as well. That's because proposals and cover letters on Freelancer.com play much the same signalling role as degrees and grades. In both cases, employers can’t observe ability directly, so they rely on an observable, costly signal. On Freelancer.com, that is the proposal or cover letter, and for education, that is the degree or grade. Generative AI makes it much easier for almost anyone to produce a polished proposal or assessment, so the observable output becomes less tightly linked to ability, weakening the value of both kinds of signal.

Read more:

• The impact of large language models on job interviews, from both sides
• David Deming on generative AI and commitment to learning, and the impact of generative AI on signalling in education

David Deming on generative AI and commitment to learning, and the impact of generative AI on signalling in education

When I was writing yesterday's post on generative AI and the economics major, I really wished I had read this post by David Deming on generative AI and learning, and then I could have linked the two together. Instead, I'll use this post to draw on Deming's ideas and flesh out why I think that generative AI makes signalling in education harder, and why that is a problem (in contrast with Matthew Kahn, who as noted in yesterday's post thinks that generative AI reduces problems of information asymmetry).

First, Deming writes about the tension in education between students' desire to learn, and their desire to make life easier (the 'divided self', drawing on the example of Odysseus:

A vivid illustration of our divided self comes from a famous behavioral economics paper called “Tying Odysseus to the Mast: Evidence from a Commitment Savings Product in the Philippines”. They found that customers flocked to and greatly benefited from a bank product that prevented them from accessing their own savings in the future. Just like when Odysseus tied himself to the mast of his ship so that he would not be tempted by the alluring song of the Sirens...

The Sirens offer Odysseus the promise of unlimited knowledge and wisdom without effort. He survives not by resisting his curiosity, but by restricting its scope and constraining his own ability to operate. The Sirens possess all the knowledge that Odysseus seeks, but he realizes he must earn it. There are no shortcuts. This is the perfect metaphor for learning in the age of superintelligence.

The analogy to generative AI is obvious. Generative AI is a tool that offers unlimited knowledge without effort, but using that tool means that the effort necessary for genuine learning is not expended. As Deming concludes:

Learning is hard work. And there is now lots of evidence that people will offload it if given the chance, even if it isn’t in their long-run interest. After nearly two decades of teaching, I’ve realized that my classroom is more than just a place where knowledge is transmitted. It’s also a community where we tie ourselves to the mast together to overcome the suffering of learning hard things.

How does this relate to the quality of signalling? It is worth reviewing the role of signalling in education, as I discussed in this post:

On the other hand, education provides a signal to employers about the quality of the job applicant. Signalling is necessary because there is an adverse selection problem in the labour market. Job applicants know whether they are high quality or not, but employers do not know. The 'quality' of a job applicant is private information. High-quality (intelligent, hard-working, etc.) job applicants want to reveal to employers that they are hard-working. To do this, they need a signal - a way of credibly revealing their quality to prospective employers.

In order for a signal to be effective, it must be costly (otherwise everyone, even those who are lower quality job applicants, would provide the signal), and it must be costly in a way that makes it unattractive for the lower quality job applicants to attempt (such as being more costly for them to engage in).

Qualifications (degrees, diplomas, etc.) provide an effective signal (they are costly, and more costly for lower quality applicants who may have to attempt papers multiple times in order to pass, or work much harder in order to pass). So by engaging in university-level study, students are providing a signal of their quality to future employers. The qualification signals to the employer that the student is high quality, since a low-quality applicant wouldn't have put in the hard work required to get the qualification.

What does generative AI like ChatGPT do to this signalling? When students can outsource much of the effort required to complete assessments, then not-so-good students no longer need to spend more time or effort to complete their qualification than do good students. Take-home assignments, essays, or written reports might be completed to a passing standard with little effort from the student at all. Completing a qualification is no longer costly in a way that makes it unattractive for lower quality job applicants to attempt. That means that employers would no longer be able to infer a job applicant's quality from whether they completed a qualification or not.

A solution suggested by Deming's post is for students to find some way of committing themselves to not using generative AI in assessment. For this to solve the signalling problem, the commitment has to be credible (believable), such as being verifiable by potential employers later. While students could commit themselves to not using generative AI, and maintaining effortful learning, it is difficult to see how students who do so could credibly reveal that they have done so. They require some way of ensuring that potential employers could verify that the student didn't use generative AI. This is where universities could step in. If universities can certify that particular qualifications were 'AI-resistant', such as where assessment includes substantial supervised, in-person components (for example, tests or examinations), then that would help maintain the quality of the education signal. There are other options of course, including oral examinations, group or individual presentations, or supervised practice assessments that make learning harder to fake. However, anything that falls short of being AI-resistant in the eyes of employers is unlikely to work. However, limiting assessment styles in order to certify effortful learning doesn't come without a trade-off. AI-resistant assessment is likely to be less accessible, less flexible, less authentic, and potentially more likely to promote anxiety in students.

Kahn suggested in his post that "AI-proctored assessments and virtual tutors suddenly make effort and mastery visible in real time". That could work. However, AI proctoring by itself is not a solution. In order to retain its status as a signal of quality for students, assessments need to require more effort to complete well for not-so-good students than for good students. Having an assessment where an AI proctors while a student uses a generative AI avatar to make an AI-generated presentation is not going to work. I'm sure that's not what Kahn was envisaging. Proctoring of online assessment (either by humans or by AI) is not as easy as it sounds. Last year I was part of a group tasked with evaluating online proctoring tools, to be rolled out for our new graduate medical school, and I was left thoroughly underwhelmed. All of the tools that we evaluated seemed to have simple workarounds that moderately tech-savvy students could easily employ. The solution that was offered (when the demonstrators could even offer a solution) was to have students complete assessments on-site, which more or less defeats the purpose of online proctoring.

Anyway, the point is that generative AI reduces the signalling value of education. There are solutions where that signalling value can be retained, but that requires students to commit to effortful learning, and universities to certify that effort in a way that students who don’t expend it cannot mimic.

[HT: Marginal Revolution]

Read more:

• Some notes on generative AI in higher education
• Some notes on generative AI and assessment (in higher education)
• More notes on generative AI and teaching (in higher education)
• Universities' (and teachers') cheap talk on generative AI and assessment
• Simas Kucinskas on AI, university education, and the 'mushy middle'
• The wicked problem of generative AI and assessment
• Matthew Kahn on generative AI and the economics major

Principal-agent and adverse selection problems among Senegalese taxi drivers

A principal-agent problem (or agency problem) may arise in interaction that involve a person or group (the agent) being given the power to decide how to use resources that ‘belong’ to someone else (the principal). The problem that may arise is when the principal wants the agent to use the resources in a particular way that is contrary to the interests of the agent (who has their own goals and motivations), but the agent uses the resources in some other way that is detrimental to the principal. This principal-agent problem is an example of moral hazard. The market failure here is that the principal becomes unwilling to engage an agent, if they can't be sure that the agent will act in the principal's best interests.

An interesting example is described in this post from the Development Impact blog late last year, which was based on the job market paper of Deivy Houeix (MIT). They describe the problem as follows:

The Senegalese taxi industry exemplifies common principal-agent challenges faced by small firms in lower-income countries. The typical arrangement involves a car owner (employer) and a single driver (employee) linked by a relational contract. The driver keeps any revenue exceeding a rental fee paid weekly and sometimes receives an upfront payment from the owner. Due to limited liability, drivers can default on the rent by claiming low earnings for the week. Importantly, the owner has no way to observe whether this is due to bad luck, lack of effort from the driver, or whether the driver misreports revenue. This creates scope for moral hazard in both driver's effort and reported output and allows drivers to capture informational rents, contributing to inefficiencies common in informal arrangements. Default may lead to the (costly) termination of the relationship to mitigate moral hazard.

Notice that this has all the features described above. The car owners are the principals. The drivers are the agents. The resources belonging to the principal, that the agent has the power to decide how to use, are the taxis. The car owner wants the taxi drivers to pay the full rent for the car. However, the drivers have an incentive to claim low earnings and default on paying the rent. The market could fail, because the car owners choose to terminate the agreement with the drivers.

Houeix's post was about how to solve the principal-agent problem, and describes the results of two field experiments that were conducted to test whether adopting digital payment technology could reduce the principal-agent problem. Since closer monitoring of the agent by the principal is one solution to principal-agent problems, using digital payment technology to facilitate this monitoring seems like it could be beneficial.

I recommend reading the entire post (and following up with the job market paper itself, for further detail), but in short the results show limited success. On the one hand, the digital payment technology:

...significantly cuts drivers' cash-related costs (e.g., small-change shortages) by half and serves as effective monitoring tools even with partial digitalization of transactions (about 13% of revenue).

The digital payment technology made it easier for drivers to accept payment, and easier for car owners to monitor what drivers were doing. However, drivers had the option not to adopt the digital payment technology, by simply not providing the car owner's details:

I find that observability is an important barrier to technology adoption, especially for the worst-performing and poorest workers. Initially, 50% of drivers did not want to adopt the technology, citing various privacy concerns for not sharing owners' information.

This should be no surprise. The lowest-productivity workers are likely to be the ones hurt the most by closer monitoring, so of course they wouldn't want to adopt the digital payment technology.

However, there may be an upside. Since car owners don't know who the high-productivity drivers are (productivity is private information), there is also an adverse selection problem here. That problem is that, because the car owners can't tell high-productivity and low-productivity drivers apart, they must assume that all drivers are low-productivity drivers, and will treat them accordingly. This is a pooling equilibrium (because all drivers are pooled together and treated the same). The high-productivity drivers don't want to be treated as if they are low-productivity, so they will be more likely to leave the market. If this continues, eventually only low-productivity drivers are left. The market fails.

This adverse selection problem could be solved by the car owners asking drivers to adopt the digital payment technology (this is a type of screening, since it involves the uninformed party (the car owners) attempting to credibly reveal the private information). The answer to the question about whether they will agree to install the digital payment technology reveals the productivity of the drivers. The drivers who don't agree are likely to be the low-productivity drivers, and so the car owners should choose not to rent their car to those drivers. Instead, the drivers who accept the digital payment technology will be the high-productivity drivers, and the car owners should be happy with them. That creates a separating equilibrium (since the low-productivity and high-productivity drivers can now be separated), and solves the adverse selection problem for the car owners.

Houeix didn't note the potential solution to the adverse selection problem in the post, nor in their job market paper. That was a missed opportunity, and may be the real benefit of the digital payment technology that they investigated.

The AI arms race in hiring may cause the market for high-quality job applicants to fail

Two years ago, which is a lifetime or more ago in generative AI years, I wrote a post about the impact of large language models on job interviews. I concluded that post with:

So, perhaps there are offsetting benefits of AI on the job interview process, if the AI is being used by the employer. If AI leads to job interviews that are more effectively able to screen for high-quality job candidates, by reducing bias in the interview process, then that can be a good thing. The AI might also be better able to ask the searching questions that distinguish high-quality and low-quality job candidates.

Of course, that assumes that the AI is interviewing a human job candidate. What happens when a job candidate, being interviewed by an AI job interviewer, is being given real-time prompts on how to answer by an AI chatbot? Or, in a Zoom job interview, the job candidate simply replaces themselves with an avatar or a 'deep fake' video of themselves generated in real time, and using an AI-scripted voiceover. If it hasn't happened already, it is going to be happening soon. Will that be the death of the job interview as a screening tool? Time will tell.

Time has told, indeed. The Financial Times reported back in May (paywalled):

You can almost hear the howls of frustration from HR departments. Jobseekers have discovered artificial intelligence and they’re not afraid to use it. Employers have become snowed under by people using the new tools to churn out impersonal applications. Some applicants are using AI to bluff their way through online assessments, too...

So it should be no surprise that jobseekers have turned to new generative AI tools such as ChatGPT to speed up or “game” a process that already felt dehumanised. Videos have even appeared on TikTok in which people demonstrate how to use ChatGPT to provide answers to questions in asynchronous video interviews, which the applicant then simply reads out.

The problem here is that the online tools that employers have been using in recent years as a tool for screening job applicants, have suddenly become less effective due to generative AI. The issue is one of adverse selection, which my ECONS102 class covered last week.

Job applicants know their quality as a worker (how hard-working, conscientious, smart, etc. they are), but employers do not. Quality is private information. Faced with a lack of information about the quality of each job applicant, employers would assume that every job applicant is low quality. This is referred to as a pooling equilibrium (because all job applicants are pooled together and treated the same). Employers would offer low wages to each new hire, because they assume that the new hire is low quality. High quality job applicants don't want to be treated as if they are low quality and receive low wages, so they would reject any low wage offer, and stop applying for jobs. Only low-quality job applicants would be left in the market. The market would fail for high-quality job applicants.

Of course, employers have found ways to deal with this situation. When the uninformed party (the party that doesn't know the private information, which in this case is the employers) try to reveal the private information (about the quality of job applicants), economists call that screening. Screening in a job application setting will be effective if it accurately reveals the quality of the job applicant. In-person job interviews used to be used as a screening tool. More recently, as noted in the Financial Times article, employers have moved to more technological solutions, such as 'asynchronous video interviews'.

With the rise of generative AI, those new screening tools have suddenly become ineffective, because they can no longer distinguish the high-quality job applicants from low-quality job applicants who are using generative AI. Generative AI has broken the current approach to hiring. As the Financial Times article notes:

Some online assessments are, for now, less vulnerable to AI use, such as those that involve playing short games. But I wouldn’t be surprised to see the return of mass in-person test centres for technical skills assessments.

I wouldn't be surprised to see the return of in-person job interviews as well. Of course, smart glasses combined with real-time generative AI assistance may render even in-person methods of assessment vulnerable, if not now then certainly in the near future. The unfortunate outcome of all of this is that the market for high-quality job applicants will soon be failing, and no one (not employers, not high-quality job applicants, and not even low-quality job applicants) will benefit from that.

Read more:

• The impact of large language models on job interviews, from both sides

Conflicting signals from attractiveness and education may reduce the chance of getting a job

There is fairly robust evidence of a beauty premium in the labour market (see the links at the end of this post for some examples). More attractive people earn more than less attractive people, ceteris paribus (holding everything else equal). One of the arguments for why this correlation may be causal (that is, attractiveness causes better labour market outcomes) is that attractiveness is a signal that conveys that the person will perform better in a particular job. [*] So, employers might employ more attractive people, thinking that they will be better workers. Another signal of good job performance is education. Employers employ more highly educated people, thinking that they will be better workers. But what happens if these two signal conflict?

That is the question that is addressed in this new article by Christopher Marquis (University of Cambridge), András Tilcsik (University of Toronto), and Ying Zhang (Singularity Academy), published in the American Journal of Sociology (ungated earlier version here). They hypothesise that:

HYPOTHESIS 1.—For higher-status positions, employers will favor applicants with both physically attractive looks and elite educational credentials over applicants who lack either or both of those valued status dimensions.

HYPOTHESIS 2.—For lower-status positions, employers will favor applicants without physically attractive looks and elite educational credentials over applicants who possess either or both of those valued status dimensions.

And for those whose education and attractiveness do not send the same signal, Marquis et al. note that:

...perhaps most interesting is the situation of applicants with discrepant status characteristics—those who are more attractive but from a less elite institution and those who possess elite educational credentials but are less attractive. In contrast to expectations from prior research, our theoretical framework implies that status-inconsistent applicants elicit ambiguous expectations and are thus less likely to be seen as a clear fit for either higher-status or lower-status positions.

Marquis et al. test their hypotheses using two studies: (1) a CV audit study in China, where they sent fictitious applications for various jobs and noted the callback rate for fictitious applicants with different characteristics (physical attractiveness, and education); and (2) a survey experiment in the US, where they asked research participants whether a fictitious applicant should be recommended for an interview for a particular job.

In the China study, they applied for jobs in the "sales" category across six cities on the job website Zhaopin.com. They applied once for each job, randomly varying the picture on the CV (since Chinese CVs typically include a headshot photo), and the education of the applicant (distinguishing between local universities ranked 10-39 on Project 985, and those ranked 40-116). Focusing on callback rates, Marquis et al. find that:

Overall, status-consistent combinations led to more callbacks than status-inconsistent combinations at a rate of approximately 1.7 to 1 (11.7% vs. 6.8%). This difference is significant statistically (P < .001) and substantively.

When looking at jobs of different status levels, Marquis et al. then find that:

...when restricting the sample to jobs with above-average salaries... the high-high combination of university status and attractiveness was more likely to lead to an interview invitation than the low-low combination (P < .10), the high-low combination (P < .05), and the low-high combination (P < .01). These differences were substantively significant as well; for example, the callback rate for the high-high combination in this subsample (24.4%) was about 1.7 times as high as for the low-low combination (14.1%)...

...for jobs with below-average salaries, the low-low combination led to more callbacks than any of the other combinations and that callback rates for the other three combinations were not significantly different from one another. The low-low combination had a substantial advantage in this subsample, generating 1.8 times as many callbacks (22.8%) as the high-high combination (12.9%).

This was consistent with their hypotheses. However, is this peculiar to the Chinese context? To test that, Marquis et al. then conducted the survey experiment in the US. They use a sample of 2020 people 'with hiring experience', recruited through the online panel Prolific. They showed each research participant a profile photo (as would be found on a professional social media site like LinkedIn), and the CV, and distinguished education based on three levels:

Using the 2022 National University Rankings of U.S. News and World Report, we selected universities from California, New York state, North Carolina, and Pennsylvania. From each state, we included (a) one institution ranked among the top 25 national universities, (b) one ranked in the 25–75 range, and (c) one not ranked among the top 75 national universities.

Marquis et al. then:

...asked participants to indicate the likelihood that they would interview the candidate for the position and the likelihood that they would give the candidate an offer, both on 1 (low) to 7 (high) Likert-type scales. We averaged these two items into an overall rating of the candidate for the job in question...

Similar to the Chinese results, Marquis et al. find that:

...for the higher-status position, the mean rating for the combination of elite university status and high physical attractiveness was significantly higher than the average of the five other combinations (5.92 vs. 5.53, P < .0003) and significantly higher than the mean of the second most favored combination (5.92 vs. 5.67, P < .04).

And for the lower-status position:

...the overall ratings for the two combinations of nonelite university status and lower physical attractiveness were significantly higher than the average of the other four combinations (6.04 vs. 5.72, P < .001; 6.01 vs. 5.72, P < .003). The ratings of the two combinations of nonelite university status and lower physical attractiveness were also higher than the mean of the next highest combination (6.04 vs. 5.77, P < .014; 6.01 vs. 5.77, P < .027).

Marquis et al. then use the US sample to look into the mechanism driving the results. Unsurprisingly:

...candidates with status-inconsistent combinations might fall between two stools: they are not perceived to be the most qualified candidates when they target higher-status jobs, and they are seen as lacking organizational commitment when they apply for lower-status jobs. Moreover, for both types of positions, these candidates generate greater evaluative uncertainty than other applicants.

When the two signals conflict, evaluators have less certainty about the applicant and rate them lower. This should be a cautionary tale for job applicants: make sure that all of your signals are conveying the same information about your quality as a job applicant, or you may find your application falling to the bottom of the pile.

*****

[*] I'm not sure that I find this argument very convincing, but Marquis et al. rely on a citation to this 1983 article (ungated version here) from a sociology journal to support it. In economics, for a signal to be effective, it must be costly, and costly in such a way that those with lower-quality attributes wouldn't want to attempt the signal. Being attractive is costly (or, at least, personal grooming is). But is it costly in such a way that people with low job performance wouldn't be able (or willing) to send the signal? I have my doubts. Anyway, I've put this argument aside for the moment, because the more interesting point in this article is what happens when two signals are conflicting.

Read more:

• The beauty premium in undergraduate study is small, and more attractive women major in economics
• New results questioning the beauty premium should be treated with caution
• The beauty premium in the LPGA
• The beauty and height premiums in the labour market
• Blonds have more fun (or rather, they get paid more)
• The beauty premium for economists
• More evidence on the blond wage premium
• The beauty premium at the intersection of race and gender
• Narrowing down on the source of the beauty premium
• The beauty premium and student grades for in-person and online education
• Is there a premium for very unattractive workers?

Are restaurant awards a negative signal?

I've been thinking about this for some time. If you are at a restaurant, and you spy a bunch of award certificates on the wall, should this give you confidence that this is a good restaurant? Many people would say 'yes, of course'. It is an award-winning restaurant, so of course it is a good restaurant. I'm not so sure, and in fact I think that in many cases a bunch of restaurant awards on the wall might be a negative signal, or no indicator at all. Let me explain why.

First, let's consider why restaurants need to signal at all. Restaurant quality is what economists call private information. The restaurant operators know whether their restaurant is high quality or not, but the patrons do not know (at least, not until they have eaten there). Restaurant meals are an experience good - the quality of the restaurant (an experience characteristic) is only known to the patron after they have experienced the restaurant meal. This information asymmetry between the restaurant operators and restaurant patrons creates an adverse selection problem. If high-quality restaurants can't distinguish themselves from low-quality restaurants, then the restaurant patron will assume that every restaurant is a low-quality restaurant (this is a pooling equilibrium). If this was to happen, then high-quality restaurants would probably start to close down (because they won't be able to get enough additional patrons to pay for their higher costs), and we'd be left with only low-quality restaurants left.

Of course, that scenario doesn't play out in most cities. Restaurants have found ways to make their quality known to patrons. When the informed party (the party that doesn't know the private information) tries to credibly reveal the private information, this is what economists call signalling. [*] In order for a signal to be effective, it needs to meet two conditions. First, it must be costly. And second, it must be costly in such a way that the low-quality restaurants wouldn't want to attempt the signal.

Now, let's consider the signalling value of award certificates. Is an award certificate costly? Yes, getting an award certificate is costly for a restaurant. The restaurant operators would have to work hard on quality of food and ingredients, and the quality of service, in order to win a certificate. [**] Is an award certificate costly in such a way that the low-quality restaurants wouldn't want to attempt the signal? A low-quality restaurant is low quality because of poorer quality food and service (and whatever other criteria are considered in the awarding of the certificate). If the low-quality restaurant operators were willing to pay the costs of higher quality food and ingredients and higher quality service, then they would do so (and they would be a high-quality restaurant, not a low-quality restaurant). So yes, the low-quality restaurant operators wouldn't attempt the signal, and so it does appear that award certificates are a good signal of quality.

However, not every high-quality restaurant can win an award certificate every year. So, what if the certificates on the wall are five or ten years old? Is that still a good signal of quality? I look at those ancient certificates (more than five years old), and I see a restaurant that is past its peak. It's on a downward slide into oblivion. It's not signalling high quality to me today, it's signalling that the restaurant used to be high quality, but not any longer.

There are two reasons why a restaurant would keep ancient award certificates on the wall. First, they are genuinely on the downward slide, but they want patrons to think that they are still high quality. But, if they were genuinely still high quality, they would still be winning award certificates. Given that they are not winning, then they are really signalling to patrons that they haven't maintained their previous quality.

Second, maybe the award certificates are given out in such a way that multiple winners are excluded. Call this the 'give everyone else a chance' effect. In that case, the award certificates are not such a great signal of quality, since they are not awarded to the highest quality restaurants, but to the highest quality restaurants that haven't been excluded for winning too many times. The quality of the signal is lessened, and patrons shouldn't pay as much attention to the award certificates.

So, a real high-quality restaurant, that hasn't won an award for several years, would probably be better off taking down their ancient award certificates, if they don't have any recent ones, then leaving those ancient certificates on the wall. Ancient restaurant award certificates are a negative signal of quality.

Changing tack now, I have a bunch of ancient award certificates myself - my teaching awards (six of them in total). I've been thinking about hanging them on my office wall. However, I haven't won an ward since 2020. Now that I've thought about restaurant signalling, I'm reconsidering whether I should hang those teaching award certificates on my office wall. Would they just signal that I'm on a downward slide? On the other hand, I continue to be nominated every year, and by multiple students every year. Maybe my lack of award winning is just the 'give everyone else a chance' effect at play?

[HT: Inspired by this Tyler Cowen post from December last year]

*****

[*] The patrons (the uninformed party) can also try to reveal the private information about restaurant quality. When the uninformed party (the party that doesn't know the private information) tries to credibly reveal the private information, this is what economists call screening. 

[**] If that wasn't the case, then the certificate provides no signalling value at all.

Information asymmetry, adverse selection, and large language models

In an article in The Conversation earlier this week, Nicholas Davis (University of Technology Sydney) wrote about information asymmetry as it relates to large language models (like ChatGPT):

A lack of skills and experience among decision-makers is undoubtedly part of the problem. But the rapid pace of innovation in AI is supercharging another challenge: information asymmetry.

Information asymmetry is a simple, Nobel prize-winning economic concept with serious implications for everyone. And it’s a particularly pernicious challenge when it comes to AI...

AI creates information asymmetries in spades. AI models are technical and complex, they are often embedded and hidden inside other systems, and they are increasingly being used to make important choices.

Balancing out these asymmetries should deeply concern all of us. Boards, executives and shareholders want AI investments to pay off. Consumers want systems that work in their interests. And we all want to enjoy the benefits of economic expansion while avoiding the very real harms AI systems can inflict if they fail, or if they are used maliciously or deployed inappropriately.

Davis is generally talking about information in relation to AI safety, but I want to take this post in a slightly different direction first, about AI quality. I will return to AI safety at the end.

When we consider AI quality, information asymmetry will cause a problem of adverse selection, ultimately causing the market for high-quality large language models to fail. So, let's consider how this market fails. There is information asymmetry in relation to the quality of large language models. AI firms know whether their large language model (LLM) is high quality or not, but users do not know. The 'quality' of a large language model is private information. Because of this information asymmetry, a LLM user should assume that any LLM is low quality. This creates a pooling equilibrium, where the user is only willing to pay however much they value access to a low-quality LLM. AI firms with high-quality LLMs, which are more expensive to train and run, will not be willing to accept users who are only willing to pay a small amount. These AI firms with high-quality LLMs drop out of the market. The only AI firms that are left will be those with low-quality LLMs. The market for high-quality LLMs fails.

A key part of the problem here is that LLMs may be experience goods. Experience goods are goods where the consumer doesn't know the quality (or some other important attribute) until after they have consumed it (they are called experience goods, because consumers don't find out what they have really bought until they have experienced it). The quality of a LLM isn't revealed to the user until after they have used it, and seen what the LLM outputs in response to a given prompt. However, users are not experts, and they might not even know the quality of the output after they receive it. That would make LLMs not experience goods, but credence goods (goods where some of the characteristics, the credence characteristics, are not known even after the good has been consumed). It seems likely that for most non-expert LLM users, LLMs will be a credence good.

So, how can the adverse selection problem be solved for LLMs? In other words, how can we move from the pooling equilibrium to a separating equilibrium (where LLM users can separate the high-quality LLMs from the low-quality LLMs)? There are two options: (1) screening; and (2) signalling. Screening involves the uninformed party (the LLM user) trying to reveal the private information (the quality of the LLM). This is unlikely to work well, because the LLM user typically isn't an expert. The only way that they can reveal the quality of the LLM themselves is to try it out, but if LLMs are actually a credence good, then even trying them out won't reveal their quality.

That leaves signalling, which involves the informed party (the AI firm) credibly revealing the private information. Only an AI firm with a high-quality LLM would want to attempt signalling, to try and reveal that their LLM is high quality (and AI firm with a low-quality LLM is unlikely to want to reveal that their LLM is low quality). In order for a signal to be effective, it needs to meet two conditions. First, it must be costly. And second, it must be costly in such a way that the AI firms with low-quality LLMs wouldn't want to attempt the signal.

What might be good signals for an AI firm with a high-quality LLM? Firms that sell experience goods have come up with a number of options, including free trials, testimonials from other customers, branding or advertising. In this context, a free trial might not work. If LLMs are a credence good, then a free trial won't reveal the private information (and we would expect to see AI firms with low-quality LLMs also offering free trials). So, while free trials are costly (they come with an opportunity cost of foregone income for the AI firm), they don't appear to be costly in a way that AI firms with low-quality LLMs wouldn't want to attempt them.

Testimonials from other customers might be an option, but only if the LLM user can trust those other customers. So, testimonials from acknowledged experts in AI might be worthwhile, but testimonials from other non-expert users might not. Are testimonials costly to the firm? Probably not greatly, but they will be more costly for an AI firm with a low-quality LLM. They might have to pay for a testimonial, whereas an AI firm with a high-quality LLM might receive praise from an expert without payment. The problem here, though, is that a non-expert user may not be able to tell the 'experts' giving testimonials apart, and so may not trust any testimonial.

What about branding or advertising? These are costly to a firm, but it's unlikely that they are costly in a way that would prevent AI firms with low-quality LLMs from attempting them.

All of this is bad news for the LLM user. They can't tell high-quality LLMs and low-quality LLMs apart, they probably can't use screening to help tell them apart, and the AI firms with high-quality LLMs probably can't use signalling to reveal that their LLM is high quality. We're left in a situation where the LLM user just has to assume that any given LLM is low quality (the pooling equilibrium).

So, what's left? Davis suggests that:

Well-designed guardrails will improve technology and make us all better off. On this front, the government should accelerate law reform efforts to clarify existing rules and improve both transparency and accountability in the market.

Of course, this is intended as a solution to information asymmetry about AI safety, not a solution to the adverse selection problem related to AI quality. However, now that we've considered the market failing in relation to AI quality because quality is a credence characteristic, it might be worth considering: is AI safety also a credence characteristic?

If AI safety is a credence characteristic, then safety isn't revealed even after the LLM has been used. Then we end up in a similar situation for AI safety as we do for AI quality, in that there is little that users can do to separate the safe LLMs from the unsafe LLMs. We would have a AI safety pooling equilibrium, where all LLM users would have to treat LLMs as potentially unsafe. Since LLM users would not be willing to pay more for LLMs that are safe (since they couldn't be sure about safety), and because safety presumably costs AI firms more, the AI firms with safe LLMs will start to drop out of the market. That would leave only the unsafe (but regulated) LLMs available, which seems like a suboptimal outcome. This isn't an argument against AI safety regulation, but hopefully provides some food for thought about its possible consequences.

Have we reached peak online dating?

On the surface, the persistence of online dating is somewhat of a mystery. As I explained in this post, online dating should have serious adverse selection problems:

Adverse selection arises when one of the parties to an agreement (the informed party) has private information that is relevant to the agreement, and they use that private information to their own advantage at the expense of the uninformed party. In the case of online dating, the private information is the quality of each potential date. They know if they are high quality or not, but no one else knows. That could lead to a pooling equilibrium, where every online dating subscriber assumes that everyone they are matched with is low quality (because they can't tell the high quality and low quality dates apart, and assuming that every match is high quality is a recipe for disaster). High quality dates don't want to be treated as if they are low quality, so they drop out of the online dating market. Eventually, the online dating market only has low quality dates. The 'market' for high quality dates fails.

The online dating market deals with this through screening (an activity that allows the uninformed party to reveal the private information). Screening is achieved by allowing subscribers to chat with the people they are matched with, which they can use to work out who is high quality and who is low quality. However, screening is imperfect, and costly. And the costs tend to be higher when the average quality is low. So, unless the online dating services can keep the quality of their users high, eventually things will turn out bad for them. And, as I noted earlier, that may be increasingly difficult.

So, it was perhaps inevitable that we would get to a point where even screening is not enough, as reported by the Financial Times this week:

“Trying to engage young women is the biggest struggle for dating apps,” said Rebecca McGrath, associate director for media and technology at Mintel. “Significant gender skew means it is harder for men to find matches and, subsequently, women often become bombarded, making the experience worse for all.”...

The number of paid subscribers on Tinder fell to under 10mn in the three months to March, a sixth consecutive quarterly decline. Monthly active users, the majority of whom use the app’s free services, have fallen steadily since 2021, according to figures from Sensor Tower. Bumble also showed a first-quarter drop in the number of active users, data from the app-tracking service showed, even though paid subscribers remained steady.

The declines in users come as reports increase of so-called dating app fatigue. A survey by Bumble, for example, found that 70 per cent of women using the app had experienced “burnout”...

Match Group’s [chief executive Bernard] Kim said Tinder’s rebrand in 2023 was expected “to have some positive impact on users, particularly women and Gen Z”. He noted however that paid subscriber growth would come primarily from “product innovation”, which includes improving profile quality, moderation and the accuracy of its algorithmic matchmaking.

'Burnout' is one consequence of an increasing cost of screening in online dating. It's becoming more difficult for women to find high-quality matches, and they are starting to switch off. The FT article discusses various ways that the apps are trying to keep women engaged. However, to solve the problem, they need to reduce the cost of screening. The option that may come closest is this one:

Tinder said in February it was expanding its own identity verification programme, which compares a video selfie with a passport or driving licence, as well as the images on a user’s profile. Bumble, which already offered a similar feature, said it had enhanced its “computer vision model for likeness comparison” in the first quarter in order to improve verification.

At the least, that will reduce the impact of fake accounts, bots, and catfishing. However, it still doesn't address the challenge that large language models pose for online dating. I'm still expecting a non-trivial amount of the traffic on online dating apps to become ChatGPT (or Claude or whatever) talking to ChatGPT.

Read more:

• Who loses status, and who gains status, with ChatGPT
• AI may be the killer app, as well as the killer of (dating) apps
• Shareholders of dating apps should be freaking out right now

Consumer preferences may explain the pricing puzzle that is diamond engagement rings

On the Marginal Revolution blog last month, Alex Tabarrok presented a bit of a puzzle:

DeBeers also produces lab-grown diamonds and they have a very strange pricing strategy:

De Beers started selling its own lab-grown diamonds in 2018 at a steep discount to the going price, in an attempt to differentiate between the two categories. The company expects lab-grown prices to continue to tumble, in what it sees as a tsunami of more supply coming on to the market, Rowley said. That should create an even bigger delta in prices between natural diamonds and lab grown, helping differentiate the two products, he said.

What? Ordinarily, the bigger the price between a competitor and its substitute the greater pressure on the competitor to lower prices! Yet DeBeers is gambling that the bigger the difference in price between natural and lab grown diamonds the bigger the demand for natural diamonds! Strange. The only way I see this working is if the fiancée knows the price of the ring, which maybe they do! In that case, the buyer still has to spend 10k and doesn’t care whether it’s 10k on synthetic diamonds or 10k on natural grown diamonds. But 10k on synthetic diamonds will get you more carats so we need an equilibrium in which a smaller diamond signals more expensive. But that runs against hundreds of years of expectations! And remember natural and lab grown diamonds are indistinguishable by the naked eye. It’s one thing for the fiancée to know the price of the diamond but surely her friends judge by what they can see, namely the size of the ring. Which signal is the most important to send?

The quote embedded above is from this Bloomberg article. In producing its own lab-grown diamonds, De Beers is competing with itself (as I noted in this post in 2018). In that post, I noted that De Beers was engaged in rent-seeking behaviour:

Synthetic diamonds are a close substitute for natural diamonds, and natural diamonds represent a highly profitable business for De Beers. If some of De Beers' competitors (like Chatham or Diamond Foundry) start producing high-quality synthetic diamonds and selling them relatively cheaply, then some diamond consumers will be induced to switch from natural diamonds to synthetic diamonds, and De Beers will lose profits. By selling synthetic diamonds itself at a very low price, clearly De Beers' goal is to make it unprofitable for the synthetic diamond producers to operate, by seriously undercutting their prices - a tactic known as predatory pricing. The synthetic diamond competitors won't be able to compete with De Beers for long at the low prices, and will soon go out of business (at least, that is what De Beers is hoping). At that point, De Beers can quietly exit the synthetic diamond industry and resume claiming high profits from natural diamonds (having taken a bit of a hit to its profits from both synthetic and natural diamonds in the meantime).

I noted that the problem with that strategy for De Beers is that there needs to be some barrier to entry to keep competitors out once De Beers had exited the synthetic diamond industry and wanted to resume high profits in the natural diamond industry. Tabarrok presents a different problem, and potential puzzle: if consumers can't tell a lab-grown (synthetic) diamond and a natural diamond apart, why would lab-grown diamonds sell at a discount?

I thought that the solution related to signalling, as Tabarrok suggested. That is, until I started writing this post, and then ended up back at the puzzle. But I think I have a solution. So, let's work it through. As I noted in this 2015 post, spending on engagement rings (and weddings, as I noted here) is a signal. In fact, there are two potential signals. But first, let's take a step back and think about why signalling is necessary.

Signalling is a solution to an adverse selection problem. Adverse selection arises when one of the parties to an agreement has private information that is relevant to the agreement, and they use that information to their own advantage and to the disadvantage of the other party. Adverse selection is a problem of 'pre-contractual opportunism', and can cause markets to fail. Signalling is one way that markets have adapted to deal with adverse selection problems. With signalling, the informed party finds a way to credibly reveal the private information to the uninformed party (a signal). There are two important conditions for a signal to be effective: (1) it needs to be costly; and (2) it needs to be costly in such a way that those with lower quality attributes would not want to attempt the signal (and one way this criteria is fulfilled is if it is more costly for those with lower quality attributes).

In the case of engagement rings, there are two adverse selection problems that the engagement ring signal might solve. First, the person proposing marriage has private information about their quality as a future partner. The person receiving the proposal may not know this information. If the proposer is a low-quality partner, they have an incentive to hide this information until it is too late and they are already married! The engagement ring provides an effective signal, that reveals the quality of the proposer. An engagement ring is an effective signal because it is costly, and because it is costly in a way that a low-quality partner would not want to attempt. That's because the proposer would lose their investment in the ring if they are low quality and the relationship breaks up. Going one step further, the more valuable the ring (relative to the income of the proposer), the higher the quality of the signal.

The second adverse selection problem arises because the couple have private information about their quality as a couple (as I noted in this post about the cost of weddings). The people attending their wedding likely do not know this information. The couple receives gifts at their wedding, and wedding attendees are more likely to want to provide gifts to couples who will be successful. So, low-quality couples have an incentive to hide from wedding attendees that they are low quality. Is an engagement ring a good signal here? I'm not so sure. Yes, the engagement ring is costly. But is it costly in such a way that a low-quality couple wouldn't want to attempt? A low-quality couple could buy a wedding ring, receive the wedding gifts, then divorce and sell the engagement ring to re-coup their initial outlay. It seems to me that this signal is not nearly as effective as the overall cost of a wedding (which is not recoverable).

Ok, so I think we have established that an engagement ring is an effective signal from one partner to another, but not necessarily from a couple to others. A more valuable ring provides a higher quality signal. Does whether the diamond is natural or lab-grown matter? In theory, it shouldn't - they are indistinguishable to the untrained eye. However, in practice they may be very different. Natural diamonds are scarce. Lab-grown diamonds are less scarce. Perhaps, in the minds of consumers, a natural diamond is a more desirable product, even if they are indistinguishable? After all, a motivated consumer can find out (through an expert appraisal) whether a diamond is natural or lab-grown. In that case, the perceived quality of the diamond really matters - a smaller natural diamond would provide the same quality of signal as a larger lab-grown diamond. And for two diamonds of the same size, a natural diamond would be priced higher than a lab-grown diamond. In other words, consumer preferences (for natural rather than lab-grown diamonds) would explain this pricing puzzle.

Certification as a barrier to entry for doggy daycare centres

This week, my ECONS102 class covered monopolies. Monopolies arise because of barriers to entry - there is something that stops other firms from getting into the market and competing with the monopoly. Not all markets with barriers to entry result in a single firm operating (a 'pure' monopoly). However, all markets with barriers to entry convey some market power on the firms - the firms can set a price that is above their marginal cost, and make a profit.

One way that barriers to entry arise is when the government grants an exclusive right to produce and/or sell some good or service, or where being a seller requires a license. Patents are an example of an exclusive right granted by the government, while occupational licensing (like the licenses required to practice medicine, or to be a teacher or taxi driver) is an example of the latter.

A slightly weaker (but similar) form of barrier to entry to occupational licensing is created by a certification regime. Certification doesn't keep sellers out of the market, but it does convey some information to consumers. It is a form of signal of the quality of the seller. For signals to be effective, they must be costly (and a certification involves a cost to the seller who wants to be certified), and costly in a way that low-quality sellers would not want to attempt. Certification may impose restrictions on the practices of sellers, like requiring them to follow a code of practice, having a third party audit the operations of certified sellers, and/or having a public register of certified sellers, as well as a record of sellers who have lost their certification. Any of these characteristics of the certification regime would make becoming certified costly for low-quality sellers in such a way that they wouldn't want to become certified in the first place. And because the certification limits the number of willing sellers, it creates a barrier to entry.

Which brings me to this article from the New Zealand Herald from February:

New Zealand’s one and only SPCA-certified doggy daycare in Whangārei is urging other daycare centres to sign up so owners can start to “expect the best” for their pets.

The Grooming Lounge & Daycare is the first doggy daycare to join the SPCA-certified programme, which lets dog owners know of businesses willing to be independently audited to maintain high animal welfare standards.

Owner Rebekah Thompson said she signed up after a futile attempt to find daycare standards and guidelines when starting her business in 2020.

“There’s no guidelines or certification for doggy daycares, it’s completely unregulated as an industry...”

Currently in New Zealand, anyone can set up a doggy daycare facility with no minimum experience or qualifications.

SPCA developed a set of voluntary standards so businesses could raise the bar and help pet owners choose centres that put animal welfare first.

It costs $800 a year to get certified and businesses get audited twice a year. One of the visits is scheduled and the other is unscheduled.

“I went with it to show my customers I have nothing to hide, and that a third party is validating what we’re doing,” Thompson said.

Notice that the certification, as described in the article, corresponds almost exactly to the case I laid out at the start of the post. It provides a signal that "lets dog owners know of businesses willing to be independently audited to maintain high animal welfare standards". The signal is costly ($800 a year), and the twice-yearly audits will make the certification unattractive for low-quality doggy daycare centres, since it would reveal the real quality of the centres to dog owners.

What happens next will be interesting though. This could be just the start of a creeping regime of regulatory capture. The SPCA and certified doggy daycare operators could start lobbying government for mandatory regulation of the sector - note the scare quote in the article: “There’s no guidelines or certification for doggy daycares, it’s completely unregulated as an industry...” The government might introduce licenses for doggy daycare centres. That would make the barrier to entry into the doggy daycare market a bit higher. Who better to manage the licensing regime than the SPCA? Then, the requirements for getting a licence can be gradually raised, in order to "put animal welfare first". Again, the barrier to entry gets a bit higher.

Will we end up with high barriers to entry into the doggy daycare sector? If it was up to the SPCA and the already-certified operators, then yes. Watch this space.

Shareholders of dating apps should be freaking out right now

My ECONS102 class has been covered the economics of information this week, including asymmetric information and signalling. So, it is timely to revisit the topic of online dating (which I have discussed earlier this year, here and here), particularly given the rise of artificial intelligence. As the Financial Times reported last month (paywalled):

Terrible news from the world of online dating. As if a parade of dubious romantic prospects and dead-end chats wasn’t bad enough, artificial intelligence has dipped its toe into the dating pool.

Eleven years ago, Tinder helped to turn dating into a series of quick-fire interactions on the internet. But for some jaded users, even writing “Hi” to a romantic prospect is now too much effort. Tech start-ups such as Rizz and YourMove AI are gaining a foothold in the sector by offering AI assistance in creating witty opening lines and appealing profiles.

Meeting strangers on the internet is by nature a random affair. Artificial chat is at least a less sinister way to help that process along than asking users to swab the insides of their mouth, as DNA-dating app Pheramor once did. But a proliferation of AI-assisted conversations suggests that eventually dating apps will simply be full of computers trying to woo other computers.

One start-up even offers the chance to watch this exact premise unfold. Teaser AI asks users questions about themselves and their personalities and then crafts AI-generated chat that is designed to mimic them. When individuals match, they can sit back and watch as their chatbots try to chat each other up.

I'm sure I wasn't the first to point out that we would eventually find ourselves in a world where one instance of ChatGPT was chatting up another instance of ChatGPT, with no humans involved. On top of being a somewhat amusing situation, there is a downside in this development. It will simply make it more difficult for someone genuinely looking for love to find a match online.

To see why, we need to discuss adverse selection. Adverse selection arises when one of the parties to an agreement (the informed party) has private information that is relevant to the agreement, and they use that private information to their own advantage at the expense of the uninformed party. In the case of online dating, the private information is the quality of each potential date. They know if they are high quality or not, but no one else knows. That could lead to a pooling equilibrium, where every online dating subscriber assumes that everyone they are matched with is low quality (because they can't tell the high quality and low quality dates apart, and assuming that every match is high quality is a recipe for disaster). High quality dates don't want to be treated as if they are low quality, so they drop out of the online dating market. Eventually, the online dating market only has low quality dates. The 'market' for high quality dates fails.

Of course, the market typically hasn't failed. People find love in online dating, so clearly the online dating apps are doing something to reduce the adverse selection problem. One thing that they do is offer the ability for subscribers to chat with the people they are matched with. Chatting allows the subscriber to work out (albeit imperfectly) who is high quality and who is low quality. When applied by the uninformed party, this is referred to as screening.

It also allows the high-quality dates to signal that they are high quality. To be effective, a signal must meet two criteria. First, the signal must be costly (otherwise everyone, even those who are lower quality dates, would provide the signal). And second, it must be costly in a way that makes it unattractive for the lower quality dates to attempt (such as being more costly for them to engage in). Engaging in witty and flirty conversation (or whatever type of conversation a subscriber is looking for, if not witty and flirty) on a dating app is costly - it takes time and effort. It will also be less costly for high-quality dates, for whom the desired type of conversation may come more naturally. It is much easier to be yourself in conversation than it is to pretend to be someone you are not. So, conversation on dating apps is an effective signal, albeit an imperfect one.

The problem with AI is that it eliminates the signalling value of conversation on the app. An uninformed subscriber should no longer find the conversation on the app to be a credible signal of the quality of the match they are engaging with. They can't use the conversation for screening either. After matching, the best option is likely to be, as I suggested earlier, in-person meet-ups.

The shareholders of online dating services should be freaking out right now. Their business model, which is built on providing high-quality matches, is under threat by AI. Once the novelty of watching their AI wingman chatting up their match's AI wingman wears off, subscribers will realise that paying for the dating app has become a waste of time. In fact, here's the share price performance over the last six months of the three biggest dating app companies, Match Group (MTCH) in blue, Bumble (BMBL) in black, and Grindr (GRND) in green (original data taken from here):

While Match is up 10.4 percent over that time, Grinder is down 19.5 percent, and Bumble is down 25.3 percent. It's not looking good.

Read more:

• Who loses status, and who gains status, with ChatGPT
• AI may be the killer app, as well as the killer of (dating) apps

The impact of large language models on job interviews, from both sides

The Financial Times had an interesting article (paywalled) recently about the impact of artificial intelligence on recruitment:

Students applying for graduate jobs this summer can take advantage of a new personal interview coach. If they send over a specific job description they can receive tailored interview questions and answers — and feedback on their own responses — all for free.

The coach, offered by the job search engine Adzuna, is not human but an artificial intelligence bot known as Prepper. It can generate interview questions for more than 1mn live roles at large companies, in industries ranging from technology and financial services to manufacturing and retail.

For a graduate job in PwC’s actuarial practice, the chatbot spits out questions such as: “What skills do you think an actuarial consultant should have?” and “How would you explain actuarial concepts to a client who is not from a finance background?”. When a user answers a question, Prepper generates a score out of 100, and tells them which parts worked well and what was missing.

Prepper is part of a new wave of chatbots powered by generative AI — from ChatGPT to Bard and Claude.

As I noted in my previous post, employers are trying to overcome an adverse selection problem. Job candidates know whether they are high quality or not, but employers do not. High-quality job candidates use signalling to reveal to employers that they are high quality, as I noted in my previous post. Employers use screening to try to identify the quality of job candidates. One screening tool is the job interview. Candidates' likely quality as employees is revealed through the job interview process. However, if AI is prepping job candidates, then that reduces the efficacy of the screening process. If low-quality job candidates can be effectively prepped to seem like they are high-quality job candidates, the screening process fails to solve the adverse selection problem.

On the other hand:

Grace Lordan, an economist at the London School of Economics and director of The Inclusion Initiative, which studies diversity in corporate settings, says companies, particularly technology groups, are experimenting with generative AI to conduct initial interviews.

“One of the biggest areas of bias is actually the interview,” she says. “This is when people’s affinity bias, or representative bias, which means choosing people who look like others in the organisation, comes in.”

AI-conducted interviews could go some way to removing that bias, she says. “Generative AI is quite convincing as an avatar. Using AI as another serious data point will allow pushback from the machines [against human bias].”

So, perhaps there are offsetting benefits of AI on the job interview process, if the AI is being used by the employer. If AI leads to job interviews that are more effectively able to screen for high-quality job candidates, by reducing bias in the interview process, then that can be a good thing. The AI might also be better able to ask the searching questions that distinguish high-quality and low-quality job candidates.

Of course, that assumes that the AI is interviewing a human job candidate. What happens when a job candidate, being interviewed by an AI job interviewer, is being given real-time prompts on how to answer by an AI chatbot? Or, in a Zoom job interview, the job candidate simply replaces themselves with an avatar or a 'deep fake' video of themselves generated in real time, and using an AI-scripted voiceover. If it hasn't happened already, it is going to be happening soon. Will that be the death of the job interview as a screening tool? Time will tell.

Signalling theory suggests that we may need two categories of university degree

In The Conversation last week, Ananish Chaudhuri (University of Auckland) argued the case for having two categories of university degree:

And while cognitive abilities such as reading, writing and maths matter, so too do social skills such as empathy, resilience and an ability to work in diverse groups and with diverse views...

Universities play a crucial role in developing these skills. But the emerging two groups of students – on campus and off – are not getting the same education. The increasing emphasis on online instruction and exams is devaluing degrees...

This suggests we may need to distinguish between online and on-campus students in each of our courses. The course content will be the same, but the assessment methods will be different.

Online students can take tests, quizzes and exams remotely. Some of this may also be available to on-campus students. But on-campus students will be expected to come to lectures regularly, ask questions, write, speak and engage in interactive tasks, including group work.

Would students sign up for on-campus courses but simply not attend? This could be prevented by making sure each student completes tasks that earn participation marks that count toward on-campus credits. If they fail to do so, they will automatically become online students.

Is this unfair to online students? Not necessarily. Many with jobs may prefer it. In any event, they will have to consider whether the benefits of coming to campus are worth it in terms of job prospects or earning potential.

It is worth reviewing the purposes of education from the perspective of the student. On the one hand, education provides useful cognitive and non-cognitive skills that have value in the workforce. In theory, skills development need not necessarily be different between in-person students and online students. However, some 'transferable skills' like teamwork and interpersonal skills are more difficult to develop in an online environment (as Chaudhuri notes).

On the other hand, education provides a signal to employers about the quality of the job applicant. Signalling is necessary because there is an adverse selection problem in the labour market. Job applicants know whether they are high quality or not, but employers do not know. The 'quality' of a job applicant is private information. High-quality (intelligent, hard-working, etc.) job applicants want to reveal to employers that they are hard-working. To do this, they need a signal - a way of credibly revealing their quality to prospective employers.

In order for a signal to be effective, it must be costly (otherwise everyone, even those who are lower quality job applicants, would provide the signal), and it must be costly in a way that makes it unattractive for the lower quality job applicants to attempt (such as being more costly for them to engage in).

Qualifications (degrees, diplomas, etc.) provide an effective signal (they are costly, and more costly for lower quality applicants who may have to attempt papers multiple times in order to pass, or work much harder in order to pass). So by engaging in university-level study, students are providing a signal of their quality to future employers. The qualification signals to the employer that the student is high quality, since a low-quality applicant wouldn't have put in the hard work required to get the qualification. Qualifications confer what we call a sheepskin effect - they have value to the graduate over and above the explicit learning and the skills that the student has developed during their study.

Now we can see where there is a key difference between in-person and online education. In-person education is more costly than online education, even if the tuition fees are the same, because it requires effort for a student to get themselves onto campus and into class each day. Higher-quality students (who will be higher-quality job applicants) are more likely to be conscientious and make this effort than lower-quality students. So, having an in-person education should provide an additional signal of quality to employers, over and above the signal provided by the degree itself.

However, if there is no way for the in-person students to distinguish themselves from the online students, then the value of the signal provided by in-person education is lost. Chaudhuri argues that the solution is to create a two-tiered qualification system. One (in-person) qualification would convey a signal of high quality to employers, and the other (online) qualification would convey a signal of lower quality to employers.

I'd argue that this already happens to some extent. It's the reason why Massive Open Online Courses (MOOCs) haven't displaced traditional education, despite being free or low cost. Employers can tell the signalling value of a degree when compared with studying online using a MOOC. The problem that Chaudhuri notes is that students' university transcripts probably don't clearly identify students who have done their study in-person from those who have done their study online.

Of course, savvy employers don't only rely on job applicants providing signals. Employers engage in screening, in order to reveal the private information for themselves. They do job interviews and subject job applicants to various pre-employment tests, which helps the employer to tell the high-quality and low-quality job applicants apart. It would not surprise me at all to learn that, as part of the standard job interview script, employers now ask how much of their degree a job applicant completed online. However, job applicants can lie in job interviews, so screening is unlikely to be as effective as signalling in solving the adverse selection problem. Perhaps, two categories of university degree is the best solution after all.

Regardless, the takeaway for students is, as I noted in this 2017 post, that they should be acutely aware of the signals that they are sending to future employers. Education is a signal, but the type of education matters as well.

Read more:

• Why all students need to understand adverse selection and signalling

AI may be the killer app, as well as the killer of (dating) apps

In a post a couple of weeks ago about who gains, and who loses, status as a result of ChatGPT (and other large language models), I wrote:

There is one more context I want to highlight, which is a particular favourite of mine when teaching signalling to first-year students: online dating. It is difficult for a person to signal their quality as a potential date on a dating app. Anyone can write a good profile, and use a stock photo. However, one of the few signals that might be effective is the conversation on the app before a first date. A 'good' date should be able to set themselves apart from a 'not-so-good' date, by the things they say during a conversation. However, with ChatGPT in the picture, the signalling value of what people write in dating app conversations is reduced (in contrast to the assertions in this article in The Conversation). I wonder how long it will be before we end up in a situation where one instance of ChatGPT is talking to another instance of ChatGPT, because both dating app users are using ChatGPT at the same time (it has probably happened already). Anyway, good quality dates will lose status as well.

It turns out that I was almost certainly right that ChatGPTs are talking to each other on dating apps. The Washington Post reported earlier this week (paywalled):

Coyne Lloyd, a 35-year-old tech investor, was visiting his family in Upstate New York recently when he decided to set up some dates in the city. He fired up Hinge, his preferred dating app, and swiped on a few interesting women. After receiving a couple of matches, he turned, out of curiosity, to a new AI dating tool called Rizz to break the ice...

Rizz, which is meant to function as a digital wingman, helps users come up with killer opening lines and responses to potential matches. The company behind it is just one of many start-ups trying to transform romance through artificial intelligence by optimizing and automating online dating, now one of the primary ways by which people find romantic connections...

Using dating apps can be a slog. Some people complain that they have to sift through countless matches as others indiscriminately swipe; it is difficult to start conversations with strangers; and many users end up viewing the apps more as a necessary chore than an exciting opportunity to connect with someone new...

That is what drove Dmitri Mirakyan, 28, a data scientist in New York, to develop YourMove.ai, an AI dating tool that helps users begin and respond to messages. “This past summer, I got really tired of sifting through and trying to come up with responses on dating apps,” he said. “So I tried to see if GPT3 could flirt. It turns out it could. A month later, I built the first version [of the platform] on a Saturday.”

Using dating apps is costly. It takes time and effort. However, it is precisely the time and effort involved that makes messaging in a dating app a signal of a date's quality. The messaging function in a dating app provided an opportunity for high-quality dates to signal that they were high quality. High-quality dates are willing to put in the time and effort to write appealing messages. Low-quality dates are not as willing to put in the time and effort. If you eliminate the time and effort required in messaging through a dating app, you eliminate the value of the signal. You end up back in a pooling equilibrium, where anyone you exchange messages with in the app is just as likely to be low-quality (and using AI) as they are to be high-quality. Or worse. Maybe the availability of AI for dating apps crowds out the high-quality dates, leaving only low-quality dates behind. In that case, every person you exchange messages with in the app is low quality.

It is difficult to see how dating apps can restore the previous separating equilibrium, that in-app messaging allowed for (separating the high-quality dates from the low-quality dates). I suggested in my earlier post that in-person meet-ups could become even more important as a tool for sorting out high-quality dates from low-quality dates. That is costly as well, and in an age where ChatGPT can be projected onto a pair of glasses in real time, even the conversation in a real life meet-up may not provide a good signal.

AI may be the killer app of the moment, but it also appears that it may be the killer of (dating) apps. However, markets do adapt. It will be interesting to see how the dating market adapts.

[HT: Marginal Revolution]

Read more:

• Who loses status, and who gains status, with ChatGPT