As I've noted before (see this post and the links at the end of it), on average online and blended learning don't appear to make students any better off, or any worse off, in terms of learning (however, that conclusion hides important heterogeneity, with more engaged students doing better with online learning, and less engaged students doing worse). So, on average, the human capital or skills gained from online learning appear similar for both online and in-person learning. If employers cared only about skills, they shouldn’t care whether those skills were acquired online or in-person.
However, human capital development is only part of the benefits of higher education. In his book The Case Against Education (which I reviewed here), Bryan Caplan presented an estimate that the education premium is 20% human capital and 80% signalling. And as I have noted several times (most recently in this post), the signal from online education is much weaker than the signal from in-person education. Putting that all together, we should expect employers to be more skeptical of online qualifications, and to be less willing to hire graduates who have online qualifications than those who have studied in-person.
This 2021 article by Conor Lennon (University of Louisville), published in the journal ILR Review (ungated version here), uses a correspondence experiment to demonstrate exactly that. A correspondence experiment involves the researcher making job applications with CVs (and sometimes cover letters) that differ in known characteristics. The researcher then counts how often CVs with different characteristics receive callbacks (a positive phone message or email, or an invitation to an interview). A very simple regression model can then be used to estimate the effect of each characteristic on the probability of receiving a callback. That is what Lennon did in this experiment, with the key characteristic being whether the applicant studied online or in person. As he explains:
...I examine employer responses to 1,891 job applications using 100 unique fictitious applicant profiles. The fictitious profiles are based on real résumés, gathered from a major online jobs website, and represent recent college graduates in four broad areas: business, engineering, nursing, and accounting. For each real résumé, names, dates, contact information, addresses, and previous employer and education details were anonymized. At random, for 50 of these résumés, the researcher added the word ‘‘online’’ in parentheses next to the name of the listed college or university. The researcher then used these résumés to apply for suitable job openings... Because employers typically left voicemail messages without specifically offering an interview time, any positive personalized contact is considered a ‘‘callback.’’
To avoid the employers detecting that they were being subjected to research, each job opening received only one randomised application. However, this 'unmatched' design is still appropriate, because randomisation and a large sample size mean that, on average, the only systematic difference between the two groups of résumés is whether the degree is listed as online or in-person. Lennon finds that:
The effect of having an online degree is large and negative in all specifications. Specifically, the estimates... suggest a 7.3 percentage-point difference in callback rates between traditional and online degree holders, all else being equal... Given that the mean callback rate for online degree holders is 8.3%, a 7.3 percentage-point difference suggests that a résumé reflecting a traditional degree will receive almost twice as many callbacks for interviews as a résumé reporting an online degree, all else being equal.
That is a huge effect and, because 'online' versus 'in-person' was randomly assigned across otherwise similar résumés, we can interpret the 7.3 percentage-point difference in callbacks as a causal effect of completing an online qualification.. Lennon then tests whether the effect is larger depending on the gender or race of the (fictitious) applicant, or by profession. The results show some differences, but I wouldn't read too much into them because they’re driven by a small proportion of the sample. On the other hand, the effect on online education does make a difference to the effect of GPA on the probability of getting a callback. Specifically:
...GPA matters significantly but only for in-person degree holders. Put another way, if you earn an online degree, even a 4.0 GPA will not help all that much. This estimate is a confirmation of the main takeaway of this article: Employers currently do not appear to trust online education.
This is a clear indication of the difference in the signalling value between an online qualification and an in-person qualification (note that the qualifications that Lennon chose were those that could be completed online or in-person, and were otherwise identical). GPA is also a signal of quality. If GPA makes no difference to callback rates for an online qualification, then employers aren't distinguishing between high-GPA and low-GPA graduates of the online qualification. Employers don't seem to value GPA as a signal of applicant quality, if the applicant completed an online qualification. In contrast, GPA makes a large and statistically significant difference for in-person qualifications, showing that GPA remains a strong signal for employers when students complete an in-person qualification. Lennon concludes that:
Because learning outcomes appear to vary little between in-person and online instruction... fewer callbacks for those with online degrees would support the idea that employers view having a traditional degree as a better signal of employability... Alternatively, employers may be inferring some socioeconomic characteristics, or they may believe that human capital formation is diminished in online programs relative to traditional degrees (even if it is not), that the individual will be less socially adept, or that a traditional college education gives students something more than just grades written on a piece of paper.
Lennon rightly notes that his results apply to new graduates, and may not apply to second-chance learners, who often have more real-world experience prior to beginning (or returning to) higher education studies. This study was also conducted in 2015-2017, and some things have definitely changed. Large language models may actually make online qualifications even less of a quality signal than they did when this research was conducted.
The new graduate market is important to universities. We need to understand how employers view our graduates. Based on this study, we should be very cautious about encouraging students into online-only qualifications, lest we hamper their chances of employment when they graduate.
