I had to listen to this three times to try and figure out what Ms. Maher is saying. A few things:

1. Afeni Shakur, member of the Black Panther Party, mother of Tupac Shakur, and a legend in her own right, said that she worshipped the written word. In a way all rappers do. To

It is an incredibly safe assumption that models get much cheaper and smarter from here, yet very few founders are acting like it

We now have an open-source model that is beating Claude 3 Opus...

being served at nearly **300 tokens per second** on Groq Inc.

The applications built off of this tech will be nothing short of revolutionary.

Massimo The original Brahmi numeral system, evolved into the Hindu numerals, & that was later copied to form Arabic numeral system, which got carried over to Europe.

Hindu numerals are a set of 10 symbols 1, 2, 3, 4, 5, 6, 7, 8, 9, 0 — that represent numbers in decimal number system.

This fall, I got an e-mail from a guy named JM Simpson who'd read things I'd written about homelessness. He mentioned he was a photojournalist in Olympia and that he'd been photographing the unhoused population there. He wanted to share some work with me . . . (1)

Simpson told me he talks with, gets to know, and asks permission from any person before he takes their photo. This is good ethical practice. But it also leads to photography that feels intimate and humane rather than removed and voyeuristic (3)

Which is to say, his photography avoids reducing the subjects to mere objects of pity, but also avoids romanticizing their lives (6)

I don’t know if AI “understands” human values but it can certainly compute over them in a sensible way. And—Chinese Room philosophical disputes aside—maybe that’s good enough?

See for instance this cool work: meaningalignment.substack.com/p/the-first-mo…

Think of three people you know personally that you look up to/admire

What are the top 2-3 traits you appreciate most about them?

I agree with Andrej Karpathy that smaller models would be welcome as well… What is the best way to build a quality 1B-3B model that could be used in an MoE configuration?
Is it best to quantize and distill or extract from a big existing model, or is it better to train up a small model

Congrats to AI at Meta on Llama 3 release!! 🎉
ai.meta.com/blog/meta-llam…
Notes:

Releasing 8B and 70B (both base and finetuned) models, strong-performing in their model class (but we'll see when the rankings come in @ lmsys.org :))
400B is still training, but already encroaching

How Faithful are RAG Models?

This new paper aims to quantify the tug-of-war between RAG and LLMs' internal prior.

It focuses on GPT-4 and other LLMs on question answering for the analysis.

It finds that providing correct retrieved information fixes most of the model

my big bets for the future:

- the boundaries between mediums will mostly disappear.
- computation will be embedded in other mediums rather than as a stand alone thing.
- the ability to sketch out rough ideas and progressively enhance them will be incredibly important.
- AI will

Excited to teach the first ever University of Oxford seminar to combine top AI builders + philosophers w/ Prof Philipp Koralus

Our visitors have helped build Midjourney, Imbue, Stripe, Google DeepMind, Anthropic

AI 🤝 philosophy

More on topics & open-sourced syllabus 🧵
👇

Reducing Hallucination in Structured Outputs via RAG

Nice paper by researchers at ServiceNow where they discuss how to deploy an efficient RAG system for structured output tasks.

The RAG system combines a small language model with a very small retriever. It shows that RAG can

😂

'We tend to think that gifted children cruise through school destined for university and successful careers, but this is merely what psychologists call survivorship bias — a tendency to focus on successful gifted people while overlooking the ones who didn’t make it or perhaps did

It’s one of my favorites and an early brick in the foundation of my worldview. I was lucky to encounter it at age ~15.
The sequel Lila is also great.
Highly recommended.

Can you find the rule?

Slack (in the systems sense) seems to both lead to amazing things and flies in the face of efficiency, justification, and rationality.

Over time, we've killed slack in more and more domains in the name of efficiency.

twitter.com/Ben_Reinhardt/…