Alex Atallah
@xanderatallah
Experiments: @openrouterai + @windowai_io. Previously: co-founder & CTO @OpenSea.
ID:54317637
https://alexatallah.com 06-07-2009 20:12:05
2,9K Tweets
48,5K Followers
2,7K Following
Follow People
Launched lots of perf improvements on OpenRouter, and improved uptime across all models and providers:
- 75% faster routing (by caching our tokenizers more effectively)
- 20% faster auth
- 2% higher uptime across all models and providers (now 99.97%). Reminder: when one…
Launching one of the first mixture-of-expert finetunes: Dolphin 2.5 Mixtral 🐬, by Eric Hartford!
In just a few hours, it's already processed 3.3M tokens. Play with it here: openrouter.ai/models/ehartfo…
Last month’s OpenAI debacle, entertaining as it was, emphasized the need for developers to plan for a multi-model LLM strategy.
I’ve been keeping tabs on OpenRouter, a service that is best facilitating this movement.
LlamaIndex + OpenRouter
OpenRouter is a unified API to access any LLM. It finds the lowest price for any model, and also offers fallbacks in case the primary host is down.
For instance, take mixtral-8x7b: you can see the cost per token, latency, and throughput for each…
Launching some unusual & experimental models today:
1/ 👬 Mistral: Mixtral 8x7B Chat
Eight 7b models connected together into a mixture of experts, for 56b parameters in total, by Mistral AI. Launching a chat version by Fireworks, 100% discounted: openrouter.ai/models/firewor…
This is a great strategy from Mistral AI: trying to own the open-source mixture-of-expert space.
MoE's are much easier to update with new abilities than having to finetune large models. Also faster and more composable.