When your (model) representations did not turn out quite right. #model_understanding #XAI

So, let me get this straight. I won not one, but two American Psychological Association awards. But, in order to accept these awards at the conference, I need to pay nearly $600 in registration fees.

No one day pass. No fee wavier. No way to attend a 50 min ceremony.

Conferences are a scam. I’m not going.

A great piece in VentureBeat by Sharon Goldman about what we hope to achieve with Cohere For AI.

venturebeat.com/2022/06/14/goo…

Good practical insights on transformer arches.

🗣️I'm looking forward to being on the panel for this Legal Tech Research Talk series . - 'When Research Goes Wrong: Deepfakes'! Join us this Thursday in person (cozy fireplace room) or online via Zoom!

Robert Peharz Yes, much of what we claim to be 'mysterious properties of deep learning' can easily be replicated with the most basic models. We showed double descent (and how closely it's tracked by effective dimension) with a simple linear model as part of arxiv.org/abs/2003.02139.

35. Antiroutine:
To create original output, consume unusual input. Avoid trending videos, NYT bestsellers, widely cited papers. Instead. read ignored texts, plumb the past for forgotten ideas. Step outside the zeitgeist so you can see it with fresh eyes.

h/t: Michael

📣 Looking for a PhD position? 10 PhD fellowships are available via the Danish Data Science Academy ddsa.dk/open-calls/ope…
Caveat: applicants need to have graduated from a 2-year MSc programme by the time of application
Deadline: 20 March 2022
Get in touch if you're interested!

How gorgeous are these astrocytes?!

They're from an iDISCO+ cleared mouse brain hemisphere. The colors represent depth - white is closest, blue farthest.

Interested in how they're connected, and what they're saying? Stay tuned...
#WomeninScience #Science #NeuroTwitter #STEM

More fun publisher surveillance:
Elsevier embeds a hash in the PDF metadata that is *unique for each time a PDF is downloaded*, this is a diff between metadata from two of the same paper. Combined with access timestamps, they can uniquely identify the source of any shared PDFs.

The CFP for the third insights-workshop.github.io is out! Short papers & abstracts can be submitted directly to the workshop by Feb 28, or through ARR by March 21. Any NLP topic is welcome.
/1

This remains one of my favorite papers.

Offers surprising insights into how sparsity impact on generalization is nuanced. While in-distribution, sparsity harms the long-tail and rare features, it can benefit OOD generalization by preventing memorization of irrelevant data.

Wondering how to connect Causality & NLP? Check out Bernhard Schölkopf's keynote at #EMNLP2021 EMNLP 2024 Causal Inference & NLP Workshop: youtu.be/Zwt1jJxVSvg

Some LM pretraining tricks for the knowledge tool box.

Primer-EZ:
Use a better activation function (RelU**2) + a CNN layer in MHSA.

Makes large language model pretraining more efficient and thus a bit more accessible. This is great news.

Primer: Searching for Efficient Transformers for Language Modeling

arxiv.org/abs/2109.08668

We use evolution to design a new Transformer variant, called Primer. Primer has a better scaling law, and is 3X to 4X faster for training than Transformer for language modeling.

Does anybody know of research that shows that large language models can actually do 'out of pretraining domain' zero-shot prediction?

If we pretrain on diverse texts, would we not expect to always be 'partially in domain'. So does 'out of domain ZS prediction' exist?

𝐀𝐬 𝐚 𝐤𝐢𝐝:
My parents to me: 'Don't believe everything you read on the internet.'

𝐀𝐬 𝐚𝐧 𝐚𝐝𝐮𝐥𝐭:
Me to my parents: 'Don't believe everything you read on the internet.'

-- meme found on the internet so it must be true

BSc: This is the answer to this problem.

MSc: How do you answer this slightly different problem?

PhD: I don't know man, people disagree about a lot of things. I've been wrong more than I've been right, man. I just wanna knit or something

Mark Riedl that's why very large pre-trained models should be called 'chonk'

I am looking for a student research assistant for 'Continual Learning ♾ and resource-efficient 🐁learning in NLP 📰 (or vision)'.

20h/week. Remote: applications from anywhere in Germany. Local: Berlin or Potsdam.

For details follow the link.
bit.ly/3j3b3Zf