Congrats to AI at Meta on Llama 3 release!! 🎉
ai.meta.com/blog/meta-llam…
Notes:

Releasing 8B and 70B (both base and finetuned) models, strong-performing in their model class (but we'll see when the rankings come in @ lmsys.org :))
400B is still training, but already encroaching

🥁 Llama3 is out 🥁
8B and 70B models available today.
8k context length.
Trained with 15 trillion tokens on a custom-built 24k GPU cluster.
Great performance on various benchmarks, with Llam3-8B doing better than Llama2-70B in some cases.
More versions are coming over the next

We often get questions around why Voltron Data supports the Ibis project -- we've answered them here!

TL;DR: open standards are critical for the composable data ecosystem and tightly coupling Python dataframes to execution engines is bad for everyone

ibis-project.org/posts/why-voda…

The rumors are true! I started a(nother) blog. sympathetic.ink

The first post is an adaption of my talk, recalling the pas 10+ years of building open source standards and the lessons learned along the way. sympathetic.ink/2024/01/24/Ten…

Kaggle's (@Kaggle) latest competition's top 11 highest scoring notebooks all use 🚀@AutoGluon AutoML🚀 to achieve their strong performance!

When I said that AutoGluon 1.0 was the largest jump in the state-of-the-art in 4 years, I meant it.

Competition: kaggle.com/competitions/p…

I am going to speak about ApacheArrow , Apache Parquet and Apache Arrow DataFusion at the Data Council this March. Should be a good conference datacouncil.ai/talks24/buildi…

From the rise of vector databases to SQL:2023 to MariaDB troubles and the FAA outage, 2023 was an exciting year in database history. OtterTune CEO Andy Pavlo (@[email protected]) covers all that, plus database VC funding. ottertune.com/blog/databases…

Thank you so much for the fun keynote, NeurIPS Conference

As in every year, our lab had a blast! We've enjoyed connecting with so many smart, enthusiastic people--and learning about your work. What an exciting time in AI!

Some asked for slides: cs.stanford.edu/~chrismre/pape… and video

What if you could train an MLP with milliseconds instead of hours and still obtain state-of-the-art accuracy?

We introduce HyperFast: a hypernetwork for instant classification of tabular data that matches the accuracy of XGBoost while being much faster!

openreview.net/pdf?id=VRBhaU8…

I'm excited to share our results on MotherNet, a new hyper-network architecture based on TabPFN that can learn an MLP in-context using a single forward pass. This substantially improves prediction times over predicting with TabPFN directly: arxiv.org/abs/2312.08598

Really amazing and inspiring talk by Chris Re! Hope the recording will be available soon, I'll have to re-watch it a couple of times, I think. Also you should follow Dillon Niederhut PhD for amazing NeurIPS coverage.

In other news, I think I'll mostly be leaving this platform for LinkedIn, which seems to have higher quality engagement these days. Though this is obviously the premier platform for trolling and subtweeting (sub-X-ing?)

Christopher Re at his #NeurIPS2023 keynote 'I think Ilya deserves the Touring award. Maybe not Employee of the month, but definitely the touring award'. [Most of the keynote was deeply technical and quite inspiring.]

Since everyone on my Twitter timeline is attending #NeurIPS2023 . I thought it would be helpful to share this Inigo Montoya conference networking tip for initiating conversations, which works great for me.

I'm worried that we're heading into an LLM replicability crisis. How many of the results that we've seen are due to very careful prompts? How many would be robust to a small change?

- Meredith Ringel Morris at #NeurIPS2023

🚨Deadline for ML/AI postdocs at Microsoft Research NYC is extended to December 15. If you are at #NeurIPS2023 and want to chat about these positions, please DM me.

Amazing tutorial by Nihar Shah on the many issues in peer review, definitely check out slides and material! cs.cmu.edu/~nihars/tutori… my take away is that there's really no hope for the current model of peer review. Which was kind of my intuition before... #NeurIPS2023

I'm at #NeurIPS2023 with my amazing team, excited to be presenting 6 papers at the main track and 2 at workshops, as well as a keynote in the table representation learning workshop. Here's all the info in one tweet, ordered by day. Please come by and chat with us 🙂

Tuesday

Just made it to NOLA, can't wait to catch up with everyone at #NeurIPS2023 !

You know how image generation went from blurry 32x32 texture patches to high-resolution images that are difficult to distinguish from real in roughly a snap of a finger? The same is now happening along the time axis (extending to video) and the repercussions boggle the mind just