Okay I did a first quick pass of naive CUDA kernels for the forward pass of GPT-2 and pushed everything to one file in llm.c, Still only ~1000 lines of code:
github.com/karpathy/llm.c…

Current per iteration timings on my Lambda box <3 A100 40GB PCIe, B=4, T=1024:
- llm.c: 111ms
-

Join Scale in San Francisco on April 20 for the highly anticipated all-women Generative AI Hackathon with Title Developer sponsor GitHub! 🌉👩‍💻

👉 events.scale.com/womxn-gen-ai-h…

Do LLMs hold knowledge that might be dangerous in the hands of a malicious user? Can hazardous knowledge be unlearned?

Introducing WMDP: an open-source eval benchmark of 4,157 multiple-choice questions that serve as a proxy measurement of LLM’s risky knowledge in biosecurity,

Can hazardous knowledge be unlearned from LLMs w/o harming other capabilities?

Scale AI and CAIS are releasing Weapons of Mass Destruction Proxy (WMDP), an eval for catastrophic AI risk & a way to unlearn this knowledge.

📝arxiv.org/abs/2403.03218
🔗wmdp.ai

Scale GenAI Platform is now available on the Microsoft Azure Marketplace!

Azure customers can purchase Scale GP through the Azure Marketplace, leveraging their existing Microsoft Azure Consumption Commitment (MACC).

👉 scl.ai/sgp-on-azure
👉 scale.com/genai-platform

Pinch myself, regularly.

goodareas.co/what-makes-jas…

We are excited that Scale AI has now added Cohere’s Command model and Rerank technology to their GenAI platform to customize business applications with fine-tuning and use RAG for accurate responses. This move will help unlock the powerful potential of AI for enterprise

We’re launching a big update to Scale GenAI Platform, the AI platform for enterprises.

It enables you to use any model for any use case using YOUR data. It comes with:
🤖 model fine-tuning + customization
⚖️ test & eval
📑 RAG
🔌 data connectors
& more!

📣Today, we're launching a major update to the Scale GenAI Platform, the full-stack platform to transform your data into customized enterprise-ready Generative AI applications. Any use case, any model, anywhere 🧵

scale.com/blog/genai-pla…

Scale GP lets technical enterprise leaders focus on scaling up more use cases, not building infrastructure by providing:

👉 Custom Model Builder for LLM fine-tuning
👉 Test and Evaluation
👉 Advanced Retrieval Augmented Generation Tools
👉 Enterprise-grade security & performance

🚀Scale is pleased to collaborate with National Institute of Standards and Technology & U.S. Commerce Dept. on the AI Safety Institute Consortium to establish a new measurement science to enable the identification of proven, scalable, and interoperable measurements and methodologies to promote development of trustworthy,

How do you measure the quality of AI data? How do you reconcile individual bias from multiple raters? Enter: Inter-Rater Reliability (IRR) 🧵

scl.ai/irr-blog

The world we live in: LTE w/ data roaming is significantly faster *and* more reliable than WiFi while traveling… when did this flip?

Scale’s paper “Detecting and Preventing Hallucinations in Large Vision Language Models” has been accepted for publication in AAAI 2024! 🧵

arxiv.org/abs/2308.06394

The Scale AI team put a spotlight⚡️on our work 'Pushing Mixture of Experts to the Limit' arxiv.org/abs/2309.05444 and showcase the impact on LLaMA-2.

Really nice blog post + working implementation. 🔥

scale.com/blog/fine-tuni…

'See this is exactly why we need to regulate it, I can just go online and download these 87GB weights'