I love these tips from Jonathan Whitaker, just posted to Answer.AI, on attacking 'high-surface-area problems'; i.e problems where 'when something doesn’t work it can be hard to find out where the issues may be, let alone what we need to do to fix them.'
answer.ai/posts/2024-04-…

Vik is an extraordinary person. He won the challenging Kaggle essay scoring competition, founded the super successful company Dataquest, created top tier NLP tools like Surya & Marker, & more.

Now he's joining Answer.AI.

Read his journey to deep learning mastery here:

New MoE alert! 🔔 Qwen1.5-MoE-A2.7B just got released. Qwen-MoE is a 14B parameter Mixture of Experts Models matching the performance of 7B models, but with the speed of 2B models. 🏎️⚡️

TL;DR
🧮 14.3B parameters with 2.7B activated during generation
👨‍🏫  60 experts with…

Thanks to the amazing work of Jeremy Howard, the Answer.AI team (special thanks to Jonathan Whitaker & Benjamin Warner for walking through the changes with me) for getting FSDP + QLoRA working.

We've managed to integrate their findings into Axolotl and now have additional…

Together with Hugging Face +
Answer.AI +, in true #FOSS spirit, many amazing volunteers,

we just released bitsandbytes==0.43.0, with these cool new additions:

- 🛫FSDP+QLoRA alpha release
- ⊞🌈 Windows support via pip install
- 📄 huge docs update

Find out more about FSDP:

The 2024 Brain Prize goes to pioneers of computational and theoretical neuroscience: Larry Abbott, Haim Sompolinsky, and Terry Sejnowksi.

It's fabulous to see the field being recognized in a big way, and I can't think of a more deserving group of laureates for it.…

🇯🇵JaColBERTv2 is out 🎉🎉

It's an extension of JaColBERTv1 using distillation - with Omar Khattab's MSMarco scores applied to MMARCO-🇯🇵

JaColBERTv2 tops セコン's JQaRa, doing 5.6% better than the best single-vector model & outperforming models w/ 5x the param count!