Emerging technology allows organizations to conduct interviews using artificial intelligence. Spoken responses can then be automatically transcribed with machine learning, known as automated speech recognition (ASR). However, concerns remain as to whether ASR can record accurately and whether it demonstrates the same level of accuracy for all subgroups of people. New research (Hickman et al., 2024) examines these concerns.
AUTOMATED TRANSCRIPTION ACCURACY AND BIAS
The researchers used recorded interviews from over 1,000 participants. Participants answered 3-6 interview questions, providing over 140 hours of video content. Their responses were transcribed using multiple platforms (Amazon Transcribe, IBM, and Open AI’s Whisper) and compared against human-generated transcriptions and ratings.
Overall, OpenAI’s Whisper provided the most accurate transcriptions, aligning closely with human transcriptions, whereas IBM and Amazon had lower accuracy. ASR made the most errors in transcription for English as a Second Language (ESL), non-White, and male interviewees. However, these transcription errors did not systematically result in biased predictions of applicant skills and abilities.
PRACTICAL IMPLICATIONS
Although transcription did show more errors for ESL, Black, and male speakers, this may not necessarily translate to decreased predictive ability when this technology is used in the context of employee selection. However, this doesn’t mean that the bias doesn’t matter at all, as there could be implications in other settings. Practitioners should continue to evaluate ASRs to ensure these tools lead to valid hiring decisions and do not disadvantage certain groups of people.
Hickman, L., Langer, M., Saef, R. M., & Tay, L. (2024). Automated speech recognition bias in personnel selection: The case of automatically scored job interviews. Journal of Applied Psychology. Advance online publication.
Image credit: Unsplash+