The largest shifts in NLP over the past five years have been the shift to reliance on large pre-trained models (with the advent of the Transformer), followed by the shift to using generative rather than discriminative language models. These shifts each come with serious challenges for ensuring fairness in an NLP system. For the first, the relationship between fairness during pretraining and downstream applications is tenuous and understudied. This causes challenges for where to apply mitigations, and also causes logistical challenges because a different set of engineers creates the pretrained model and the application. For the second shift, generative systems are notoriously hard to evaluate for anything, with poor correlation between automatic metrics and humans, and low agreement scores even among humans. In this talk, I'll present my own research into both of these areas, discuss an overview of current challenges, and make some suggestions for future promising directions of research.
Bio:
Seraphina Goldfarb-Tarrant is the Head of Safety at Cohere, where she works on both the practice and the theory of evaluating and mitigating harms from LLMs. She did her PhD under Adam Lopez in Fairness in Tranfer Learning for NLP, at the Institute for Language, Cognition, and Computation (ILCC) in the Informatics department at the University of Edinburgh. She did her MSc in NLP, with a focus on Natural Language Generation, at the University of Washington under Fei Xia in collaboration with Nanyun Peng. Her research interests include the intersection of fairness with robustness and generalisation, cross-lingual transfer, and causal analysis. She had an industry career before her PhD, where she worked at Google in Tokyo, NYC, and Shanghai. She also spent two years as a sailor in the North Sea.
