How much does an LM depend on information provided in-context vs its prior knowledge?

Check out how Vésteinn Snæbjarnarson, Niklas Stoehr, Jennifer White, Aaron Schein, Ryan David Cotterell + I answer this by measuring a *context's persuasiveness* and an *entity's susceptibility*🧵

The newest version is going to be pushed soon!

Monograph on 'Formal Aspects of Language Modeling' from Ryan David Cotterell et al.

arxiv.org/abs/2311.04329

It would be so nice if everyone read this and we had shared foundations. Particularly for interpretability.

Honored to see my name among these amazing colleagues, Association for Psychological Science Rising Stars! psychologicalscience.org/members/awards…

New paper alert! We dive into the world of LLMs and cognitive biases, focusing on how models tackle arithmetic word problems—do they show the same biases as humans? Here’s a summary 🤖📚 #LLMs #MachineLearning #AI
arxiv.org/abs/2401.18070

Zay 💕

Slowly realizing that two days ago I successfully defended my PhD! 🤯

I’m extremely grateful to my supervisors, Isabelle Augenstein and Ryan David Cotterell, my PhD committee, Serge Belongie, Pascale Fung, Ivan Vulić, and all of my colleagues and collaborators!

Massive congrats to Karolina Stanczak for passing her PhD defence with flying colours! 🎊🥂🥳
Very proud of you 🤗🥹
Thanks to Serge Belongie Pascale Fung Ivan Vulić for serving on the committee.

Karolina’s thesis on multilingual gender bias probing:
di.ku.dk/english/resear…
#NLProc

This Friday (05/01), 14:00-15:00 CET, the Pioneer Centre for AI hosts a guest talk by Josef Valvoda titled “When Neural Networks Meet the Law” 🧑‍⚖️(aicentre.dk/events/talk-wh…). The talk will take place in the Seminar Room at P1 (Øster Voldgade 3, 1350 København K).

LLMs are now trained >1000x as much language data as a child, so what happens when you train a 'BabyLM' on just 100M words?

The proceedings of the BabyLM Challenge are now out along with our summary of key findings from 31 submissions: aclanthology.org/volumes/2023.c…

Some highlights 🧵

We derive a concept erasure method that is even more surgical than LEACE, when you have access to ground-truth concept labels at inference time.

In the binary case, this ends up being equivalent to a simple difference-in-means edit to the activations.
blog.eleuther.ai/oracle-leace/

The IBM Research Zürich lab is at #NeurIPS2023 in NOLA! Come chat with us about all things research and life in Zürich, if you haven’t already 😉 IBM Research

If you are interested in knowing how you can do energy-based sampling from language models, make sure to check our #NeurIPS23 paper titled “Structured Voronoi Sampling”...🧵
arxiv.org/pdf/2306.03061…

🤖 Ever wondered what RNN-based language models are truly capable of? Check out our #EMNLP2023 paper which places formal bounds on their capabilities! With Anej Svete, Leo Du, and Ryan (1/7)
arxiv.org/abs/2310.12942

Tianyu and Afra's paper won an Outstanding Paper award at EMNLP 2023 and achieved perfect reviews: 5's for soundness and excitement and 5 from the AC.

It's worth a read!

openreview.net/forum?id=vtqfP…)

Excited to receive an Outstanding Paper award for this work at EMNLP 2024! Thanks to my co-authors George Foster and Markus Freitag! Updated version available here: aclanthology.org/2023.emnlp-mai…

This paper just won an outstanding paper award at #EMNLP2023 and I'm super proud of it! :)

Make sure to chat to Ethan Gotlieb Wilcox about it, if you are in Singapore! Also, feel free to chat with me if you are in New Orleans for #NeurIPS2023

Thank you to #EMNLP2023 chairs for the 😱 two 😱 outstanding paper awards! I am so grateful to have worked on these projects with wonderful colleagues — Tiago Pimentel (who is the first author on one of the papers!), Clara Isabel Meister, Kyle Mahowald and Ryan David Cotterell

I think 2.2 is an example from my LLM class (rycolab.io/classes/llm-s2…). Ruida was a top student.

New preprint! Dana Angluin, I, and Andy Yang Andy J Yang show that masked hard-attention transformers are exactly equivalent to the star-free regular languages. arxiv.org/abs/2310.13897