Controlled zero-shot evals have revealed holes in LMs’ ability to robustly extract and use meaning.

But what happens when you add experimental context (ICL/instructions)? With Allyson Ettinger & Kyle Mahowald, I explore this in the context of semantic property inheritance:

1/13

Coming up: another chance to chat with us about Limits of LLMs as Analyzers of Abstract Meaning Representation, in #EMNLP2023 virtual poster session PS6 happening shortly! Paper here: arxiv.org/pdf/2310.17793…

Joint with Jena Hwang Valentina Pyatkin Chandra Sekhar Yejin Choi

I will virtually give an oral for my work at #EMNLP2023 'Session 4: Interpretability, Interactivity, and Analysis of Models for NLP 1' tomorrow morning. Come to chat with me!

Conference Time: Dec 08, 16:00-17:30 (+08) (Asia/Singapore)
US Time: 2:00AM-3:30AM CST
Place: East

Emergent risk in ChatGPT's situational understanding: tune in to Chenghao's talk coming up soon at #EMNLP2023 or check out our paper at arxiv.org/abs/2310.16135

At Blackbox NLP I’ll present our paper on LLMs and Abstract Meaning Representation, lead by Allyson Ettinger and Jena Hwang : arxiv.org/pdf/2310.17793…

Huge congratulations to Kanishka for his successful (and excellent) defense! So proud.

🚀📢 GPT models have blown our minds with their astonishing capabilities. But, do they truly acquire the ability to perform reasoning tasks that humans find easy to execute? NO⛔️

We investigate the limits of Transformers *empirically* and *theoretically* on compositional tasks🔥

Excited to finally share COMPS, a collection of english minimal pair stimuli to analyze conceptual knowledge in language models!

Work with Julia Rayz and Allyson Ettinger, to be presented at EACL (main)!

Paper: arxiv.org/abs/2210.01963
Code: github.com/kanishkamisra/…
Thread: ⬇️1/

Now in Cognition! Jiaxuan's model simulates an impressive range of N400/P600 results, building on existing psycholinguistic theory while clarifying how difficult-to-explain patterns can be accounted for by interaction of statistical properties of stimuli with posited mechanisms.

Do pre-trained LMs show sensitivity to dialogue response dynamics? Lang Yu / 余浪 and Allyson Ettinger and I explore this by using constructed input sequences that involve at-issueness and ellipsis. Check out our new paper: arxiv.org/abs/2210.02526 We'll be presenting at #COLING2022 !

Check out Kanishka's CogSci paper, to be presented this Saturday!

New list of (so far) 25 North American institutions where faculty are doing #NLProc (-adjacent) computational linguistics research *in a linguistics department*!

Hoping this will, for instance, be useful to prospective #CompLing students.

nschneid.github.io/nlp-in-ling/

Program now live!
compositionalintelligence.github.io

June 29 – Why Compositionality Matters for AI
w/ Allyson Ettinger, Paul Smolensky, Gary Marcus & myself

June 30 – Can Language Models Handle Compositionality?
w/ Dieuwke Hupkes, Tal Linzen, @elliepavlick, Stephanie Chan & Brenden Lake

For this week's Stanford NLP Seminar, we're excited to hear from Allyson Ettinger on controlled examinations of meaning sensitivity in pre-trained models. Join us over zoom on Thursday at 11 am PT. Registration: forms.gle/saouoJ5yBX7cpF…, Abstract: nlp.stanford.edu/seminar/detail…

Celebrating a successful #SCiL2022 - cheers everyone, especially Allyson Ettinger Brandon Prickett and Tim Hunter!

Tomorrow, the last day, starts with Katrin Erik’s plenary at 10am EST.

Reminder that SCiL 2022 (Society for Computation in Linguistics) is this week, Feb 7-9! Come and join us for an awesome lineup of virtual talks and posters. Link for program and registration is below. See you there!

How well are pragmatics captured? well and not well...

The long answer below

Lalchand Pandia cong yan Allyson Ettinger from @ChicagoNLP
arxiv.org/pdf/2109.12951…
#EMNLP2021livetweet #CoNLL2021

Lalchand will be presenting this at the current CoNLL talk session, ten minutes from now!

Happening now! Come and chat with us at poster #2936 in poster session III, Interpretability (east/slightly north section).