NLIP Seminar Series

Abstract: State-of-the-art foundation models (FMs) process information as a sequence of internal representations; however, the length of this sequence is fixed and entirely determined by tokenization. This essentially decouples representation granularity from information content, which exacerbates the deployment costs of...

The statistical properties of language and how they may be used in language processing and language acquisition have been studied for many decades. Recently, large language models have demonstrated striking language-learning capabilities, providing evidence for the “richness” of the linguistic stimulus, but are often...

Truth-conditional semantics has been successful in explaining how the meaning of a sentence can be decomposed into the meanings of its parts, and how this allows people to understand new sentences. In this talk, I will show how a truth-conditional model can be learnt in practice on large-scale datasets of various kinds (...

For the first seminar of the year, we welcome new members of the Natural Language & Information Processing group. *Speaker Biographies* *Suchir Salhan* is a first-year PhD student supervised by Professor Paula Buttery, researching efficient and scalable machine learning techniques for small language models. *Paul...

Abstract not available

Recent work has proposed alternative language model architectures (e.g. RWKV, Mamba, Hyena) that are dramatically faster than Attention (e.g. 25x higher throughput). However, it’s unclear how switching to these new architectures might affect the behavior of language models when scaled up. In this talk, we’ll discuss our...

In this talk, we investigate the potential of Large Language Models to serve as model systems for language. Model systems for language should first and foremost perform the relevant function, i.e., use language in the right way. In the first part of the talk we investigate this claim in two ways. First, we critically look...

I will talk about our research towards developing increasingly capable and general AI. Central to this research direction is Open-Endedness: the attempt to create an AI that can endlessly improve and expand its capabilities. In particular, this talk will focus on three areas: training autonomous and robust agents that can...

A survey of methods of interpretability of neural networks: from gender bias mitigation to interpreting BERT embeddings in a psycholinguistic manner. Bio: Giuseppe Attanasio is a postdoctoral researcher affiliated with the Milan Natural Language Processing (MilaNLP) Lab at Bocconi University. His research primarily focuses...

Abstract not available