Easily Fine-tune AI at Meta Llama 3 70B! 🦙 I am excited to share a new guide on how to fine-tune Llama 3 70B with PyTorch FSDP, Q-Lora, and Flash Attention 2 (SDPA) using Hugging Face build for consumer-size GPUs (4x 24GB). 🚀

Blog: philschmid.de/fsdp-qlora-lla…

The blog covers:
👨‍💻

More open computing platform from Meta.

Today at Answer.AI we've got something new for you: FSDP/QDoRA. We've tested it with AI at Meta Llama3 and the results blow away anything we've seen before.

I believe that this combination is likely to create better task-specific models than anything else at any cost. 🧵

Interview express sur Brut.ia
instagram.com/reel/C6BktXNNS…

Llama-3 is really good at English conversation.

Nice

Delighted to share our new preprint! with Alec Marantz, @DavidPoeppel and Jean-Rémi King:
biorxiv.org/content/10.110…

'Hierarchical dynamic coding coordinates speech comprehension in the brain'
Summary below 👇

1/8

Is it possible to produce an AI system that is not biased?

As we saw with Google Gemini, it's clear that bias in our AI models will always be a huge problem. So what's the solution?

Here Yann LeCun nails it - It's the same solution we found with the press: freedom and diversity.

New Hampshire: 'Make Polio Great Again. Live free *and* die.'

Mr. P.C.
youtu.be/Jv5j_Lx2R4g?si…

It's a drone-eat-drone world.

Ouch!

Yann LeCun Yann LeCun delivered a lecture on Objective-Driven AI.

He began with a reality check: 'Machine Learning falls short compared to humans and animals!'

Here's his insight on constructing AI systems that learn, reason, plan, and prioritize safety:

1/5

At this point I feel like we understand pretty well what's going on with LLMs:

- Outputs are roughly equivalent to kernel smoothing over positional embeddings (arxiv.org/pdf/1908.11775…)
- The learned computation model is *probably* bounded by RASP-L (arxiv.org/pdf/2310.16028…)
-

Corvids are known to be smart problem solvers.

This one, for example, plays tic-tac-toe.

[📹voron_gosha_tv]

Agency (and planning) can't be a wart on top of Auto-Regressive LLMs.
It must be an intrinsic property of the architecture.

We started releasing the first chapters of our Geometric Deep Learning book and the accompanying slides from the corresponding Oxford and Cambridge courses.

Planning with a good model of the world and of otter members of the group.

True Story!

One of the many reasons I love open source is it doesn't give a damn about the org chart or 'managing up.' If people outside of FB/Meta didn't use or like our OSS then something was wrong with it.

PyTorch succeeded because of the hyper focus on developer

First throughput tokens/s benchmarks of Llama 3 🐎
Groq Inc is coming fast out of the gate with 284 tokens/s on Llama 3 70B and 876 tokens/s on Llama 3 8B.
Once-again showing the step-change speed benefits of their custom LPU chips.

Congratulations to other providers too,