NLIP Seminar Series

Zipf posited that wordforms are optimized to minimize utterances’ communicative costs. He supported this claim by showing that words’ lengths are inversely correlated with their frequencies. This correlation, however, is only expected if one assumes that a words’ communicative cost is given by its length. We explore this...

Abstract not available

Minimum Bayes risk (MBR) decoding outputs the hypothesis with the highest expected utility over the model distribution for some utility function. It has been shown to improve accuracy over beam search in conditional language generation problems and especially neural machine translation, in both human and automatic...

Across linguistic theories, human language structures are represented by graphs (e.g., Chomsky, 1957, Tesnière, 1959, Chomsky, 1995). Much research has focused on the mapping between such graphs and the actual sequences expressing utterances, but less attention has been paid to the shapes that the graphs themselves take...

Pretraining is monolithic. In this talk, I will discuss a collaborative approach to pertaining, by iterative model merging (originally fusing). We will then discuss making evaluation reliable and efficient, to allow anyone to evaluate. We might mention the BabyLM challenge, of pretraining models with human feasible amount...

The largest shifts in NLP over the past five years have been the shift to reliance on large pre-trained models (with the advent of the Transformer), followed by the shift to using generative rather than discriminative language models. These shifts each come with serious challenges for ensuring fairness in an NLP system...

An informal talk around LLM at scale, efficiency and open ML problems we are currently working on at Cohere For AI. I will present some background, some grumpy thoughts about how we can get away from the painfully inefficient formula of just scaling capacity. I'll plan on leaving plenty of time for discussion. Sara Hooker...

Numerical reasoning is a fundamental skill for language models to understand textual input in tasks such as text generation, question answering, and fact checking. The predominant approach for enhancing numerical reasoning has been scaling—larger models trained on more data tend to perform better on relevant benchmarks...

In this talk, we will cover the domain of text-to-speech synthesis with a focus on natural language processing. We will explore the challenges of NLP for TTS and look at some of the methods for overcoming these challenges. This talk is not heavily technical and does not require a background in computational linguistics...

Abstract not available