Abstract:
The availability of large, high-quality datasets has been one of the main drivers of recent progress in generation tasks like summarization, QA. Such annotated datasets however are difficult and costly to collect, and rarely exist in languages other than English, rendering the technology inaccessible to underrepresented languages. An alternative to building large monolingual training datasets is to leverage pre-trained language models (PLMs). The talk will first discuss an approach, QAmeleon, that tunes a PLM using parameter-efficient fine-tuning methods (PEFT) to synthesize QA data with only five examples per language. Using this data during training delivers accuracy superior to translation-based baselines, and bridges nearly 60% of the gap between an English-only baseline and a fully supervised upper bound trained on almost 50,000 hand-labeled examples. Next, the talk will discuss the cross-lingual transfer approach for a much stricter zero-shot setting to enable generation in unseen languages. Our method composes language and task PEFT modules via element-wise arithmetic operations to leverage unlabeled data and labeled data in other languages. The talk further studied the consistency for cross-lingual generation tasks i.e. the output is in a language different from the source. Here we propose MuPlan which uses intermediate plans resulting in more faithful generation in both fine-tuning and zero-shot setups.
Bio:
Priyanka Agrawal is a Senior Research Scientist at the Google Deepmind in London, formally part of Google Brain, and is focused on building responsible Generative AI models and scaling them to underrepresented languages. Prior to that she was a Senior Researcher and Lead at http://Booking.com and IBM Research Labs, where she was driving work in cross-domain transfer and representation learning. She is an alumni from the Computer Science Department at the Indian Institute of Science. Her work is published at top-tier ML and NLP conferences like NeurIPS, ACL and she holds 25+ US Patents. Priyanka also serves as Area Chair and PC member at these conferences and has been an invited panelist and speaker at various ML/NLP and diversity forums.
