Today I learned: Amazon recently bought a $650 million nuclear-powered data centre for AWS (!)

“Open source AI has risks, but it’s potentially a much bigger risk that one institution controls the most powerful AI” 🎯🎯🎯🎯🎯🎯🎯🎯🎯🎯🎯🎯🎯

I have become an instant 'Claude 3 Opus' convert.

If finetuning Llama 3 w/ Hugging Face, use Transformers 4.37 or 4.40.

Llama & Gemma in 4.38 & 4.39 don't use PyTorch's Flash Attention 2 kernel, leading to high memory usage.

4.40 uses FA2 in eager mode, but not with torch.compile. I'm working with HF to fully fix this.

> 'I'm not based on LLaMA 3'

I'm surprised that most modern LLMs still aren't being fine tuned to correctly answer basic questions about themselves.

Intuitively, users expect that they can ask an LLM about itself, and they generally trust the answers provided.

These models are incredible, and a massive step forward for OSS AI. Amazing work from Meta team!

On Together AI now at 350 t/s for full precision on 8B and 150 t/s on 70B.

api.together.xyz/playground/cha…

Looks like the answer is yes :D
twitter.com/charliebholtz/…

Claude has a nice trick where you prefill the start of the assistant response, and it continues from there. Anyone know if Llama 3 can do the same thing?

We've just uploaded a GGUF of the 8b llama-3 instruct model on Nous Research's huggingface org:
huggingface.co/NousResearch/M…

Mixtral-8x22B-Instruct-v0.1, going wild on TOOLS & FUNCTION CALLING:

'<unk>'
'<s>',
'</s>',
'[INST]',
'[/INST]',
'[TOOL_CALLS]',
'[AVAILABLE_TOOLS]',
'[/AVAILABLE_TOOLS]',
'[TOOL_RESULT]',
'[/TOOL_RESULTS]',

As predicted in the llama-author-scaling-law, 328 people on the most recent llama paper 🦙

Diff of llama 3 license to llama 2: Mostly around sharing, built with llama 3 branding, agree to meta brand guidelines for distributing trademark
Some minor other differences

Teknium (e/λ) Am I comparing correctly to github.com/openai/simple-… ?

#LLaMA3 is out! It's the same architecture as Llama-2, except for some differences:
1. 128K Tiktoken vocab vs 32K vocab of Llama-2
2. 15 Trillion tokens instead of 2T
3. 8 billion model uses GQA (unlike Llama 7b)
4. 8K Context Length
5. Chinchilla scaling laws - log linear gains!…

Mikel Artetxe I'd just like to interject for a moment. What you're referring to as <model>,
is in fact, Llama 3/<model>, or as I've recently taken to calling it, Llama 3 plus <model>.