It's been a wild ride. Just 20 of us, burning through thousands of H100s over the past months, we're glad to finally share this with the world! 💪

One of the goals we’ve had when starting Reka was to build cool innovative models at the frontier. Reaching GPT-4/Opus level was a…

i was suddenly reminded of this article from 2018.

'.. statistical physicists were primed to see power laws everywhere .. there’s a “power law religion.”'

'“We would even squint at the computer screen from an angle to get a better idea if a curve was straight or not,” recalled…

oh i think i figured out:

1. create a new repository on github
2. clone the repository in Lightning AI ⚡️ studio
3. press 'Open' to work directly on the cloned repo

so ... what's the right way to change the folder/repository name from 'this_studio' to something more informative in Lightning AI ⚡️ studio?

washingtonpost.com/world/2024/04/…

Quantized Gemma 2B running at 157 toks/sec in MLX on my M1 Max laptop

Bunch of apps open (slack, chrome, etc) + attached display + screen recording!

Not sped up:

Kyunghyun Cho PyTorch this is true, i can confirm. It wont stay this way, but I can attest that MLX and llama.cpp are moving much faster right now in developing for MacBook GPUs than PyTorch is.

is this true, and will it stay this way, PyTorch ?

... anyhow ... Lightning AI ⚡️ studio saved the day, and all i wanted to create this simple starter script for my friend ...

github.com/kyunghyuncho/q…

'Give a man a fish, and you feed him for a day. Teach a man to fish, and you feed him for a lifetime.'

- Prescient LM (2024)

of course, SFT on the positive responses from a binary preference data cannot work. if y_+, y_- \sim p_{\theta}(y|x), it's probably that y_+ and y_- are deemed similar to each other by the model(\theta), and thus p(y_+|x)↑ iff p(y_-|x)↑.

i heard it's called generalization

'이제 한 사람의 시민의 자리로 돌아갈 것'
imnews.imbc.com/news/2024/poli…

oh happening!

it's taking surprisingly long for an LLM bro to announce the discovery of task oriented dialogue systems