Alex Atallah(@xanderatallah) 's Twitter Profileg
Alex Atallah

@xanderatallah

Experiments: @openrouterai + @windowai_io. Previously: co-founder & CTO @OpenSea.

ID:54317637

linkhttps://alexatallah.com calendar_today06-07-2009 20:12:05

2,9K Tweets

48,5K Followers

2,7K Following

Follow People
Alex Atallah(@xanderatallah) 's Twitter Profile Photo

Is there any research on ad-supported LLMs? Something where:

1. Completions include links, with some configurable % being ads (which are labeled that way)
2. Something like RLHF is used to maximize link clicks
3. Content publishers like NYT are satisfied

account_circle
Alex Atallah(@xanderatallah) 's Twitter Profile Photo

Launched lots of perf improvements on OpenRouter, and improved uptime across all models and providers:

- 75% faster routing (by caching our tokenizers more effectively)
- 20% faster auth
- 2% higher uptime across all models and providers (now 99.97%). Reminder: when one…

Launched lots of perf improvements on @OpenRouterAI, and improved uptime across all models and providers: - 75% faster routing (by caching our tokenizers more effectively) - 20% faster auth - 2% higher uptime across all models and providers (now 99.97%). Reminder: when one…
account_circle
OpenRouter(@OpenRouterAI) 's Twitter Profile Photo

Launching one of the first mixture-of-expert finetunes: Dolphin 2.5 Mixtral 🐬, by Eric Hartford!

In just a few hours, it's already processed 3.3M tokens. Play with it here: openrouter.ai/models/ehartfo…

Launching one of the first mixture-of-expert finetunes: Dolphin 2.5 Mixtral 🐬, by @erhartford! In just a few hours, it's already processed 3.3M tokens. Play with it here: openrouter.ai/models/ehartfo…
account_circle
Shlok Khemani(@shloked_) 's Twitter Profile Photo

Last month’s OpenAI debacle, entertaining as it was, emphasized the need for developers to plan for a multi-model LLM strategy.

I’ve been keeping tabs on OpenRouter, a service that is best facilitating this movement.

account_circle
LlamaIndex 🦙(@llama_index) 's Twitter Profile Photo

LlamaIndex + OpenRouter

OpenRouter is a unified API to access any LLM. It finds the lowest price for any model, and also offers fallbacks in case the primary host is down.

For instance, take mixtral-8x7b: you can see the cost per token, latency, and throughput for each…

LlamaIndex + @OpenRouterAI OpenRouter is a unified API to access any LLM. It finds the lowest price for any model, and also offers fallbacks in case the primary host is down. For instance, take mixtral-8x7b: you can see the cost per token, latency, and throughput for each…
account_circle
Alex Atallah(@xanderatallah) 's Twitter Profile Photo

Pretty interesting that Mistral, a French company with a predominantly American userbase, is pricing their API in Euros and replying to support emails in French

Pretty interesting that Mistral, a French company with a predominantly American userbase, is pricing their API in Euros and replying to support emails in French
account_circle
Alex Atallah(@xanderatallah) 's Twitter Profile Photo

One of the overlooked details of the Mixtral launch is their PR to vLLM.

Shows the growing community endorsement of vLLM as the source of truth for open-source inference.

One of the overlooked details of the Mixtral launch is their PR to vLLM. Shows the growing community endorsement of vLLM as the source of truth for open-source inference.
account_circle
OpenRouter(@OpenRouterAI) 's Twitter Profile Photo

Launching some unusual & experimental models today:

1/ 👬 Mistral: Mixtral 8x7B Chat
Eight 7b models connected together into a mixture of experts, for 56b parameters in total, by Mistral AI. Launching a chat version by Fireworks, 100% discounted: openrouter.ai/models/firewor…

account_circle
Alex Atallah(@xanderatallah) 's Twitter Profile Photo

This is a great strategy from Mistral AI: trying to own the open-source mixture-of-expert space.

MoE's are much easier to update with new abilities than having to finetune large models. Also faster and more composable.

account_circle
Alex Atallah(@xanderatallah) 's Twitter Profile Photo

Mistral decided to scale horizontally 👀

Seems to be eight 7b-parameter models. The first major open source mixture of experts (similar to GPT-4)?

account_circle
Alex Atallah(@xanderatallah) 's Twitter Profile Photo

LLMs are going to make this kind of scam so easy to scale.

But they'll also help defend against it. Imagine being able to screenshot and submit a sketchy chat.

Who's building that?

account_circle