Beyond General Purpose Machine Translation: The Need for Context-specific Empirical Research to Design for Appropriate User Trust


Machine Translation (MT) has the potential to help people overcome language barriers and is widely used in high-stakes scenarios, such as in hospitals. However, in order to use MT reliably and safely, users need to understand when to trust MT outputs and how to assess the quality of often imperfect translation results. %More research is needed to study and design for building and calibrating trust in MT systems. In this paper, we discuss research directions to support users to calibrate trust in MT systems. We share findings from an empirical study in which we conducted semi-structured interviews with 20 clinicians to understand how they communicate with patients across language barriers, and if and how they use MT systems. Based on our findings, we advocate for empirical research on how MT systems are used in practice as an important first step to address the challenges in building appropriate trust between users and MT tools.

Workshop on Trust and Reliance in AI-Human Teams at CHI 2022