One week away from ICLR 2024 in Vienna 🤩 I will be presenting two spotlights: why big foundation models generalize so well under the self-supervised setting, and how to leverage massive unlabeled data using a base kernel that encodes inter-sample similarity. Details 👇 (1/3)

Dataset choice is crucial in today's ML training pipeline. We (Mengzhou Xia and I) introduce desiderata for 'good' data and explain how our recent algorithm, LESS, fits into the picture. Huge review of data selection algs for pre-training and fine-tuning!
cs.princeton.edu/~smalladi/blog…

Extremely honored and excited to be starting as a research fellow at Kempner Institute at Harvard University! Looking forward to becoming part of the community; please do reach out if you are around! 😊

Blog post about how to scale training runs to highly distributed settings (i.e., large batch sizes)! Empirical insights from my long-ago work on stochastic differential equations (SDEs). Written to be accessible - give it a shot!
cs.princeton.edu/~smalladi/blog…

Hope everyone had a great time at M3L today! Many thanks to the speakers, authors, reviewers, participants and volunteers for all your contributions that made this workshop fun and successful, we hope to see you again next year! 😃✨

Come join us at M3L tomorrow!! The topics are really fun~! ✨

Presenting this at the poster session for the Generalization in Planning ( #GenPlan ) workshop tomorrow Dec 16 at #NeurIPS2023 ! Come chat about 🧜‍♀️MERMAIDE 🧜‍♀️ where we learn incentive policies with RL for principal-agent problems.

aair-lab.github.io/genplan23/inde…

Please come to our poster at the MATH-AI workshop! 😊
(Room 217-219, 3-4pm)

Will deep learning improve with more data, a larger model, or training for longer?
'Any balanced combination of them' <– in our #NeurIPS2023 spotlight, we reveal this through the lens of gradient-based feature learning in the presence of computational-statistical gaps. 1/5

I’m at NeurIPS Conference!
Giving a talk on this work tomorrow at 4:15pm in Ballroom A-C level 2.

If you’re interested in RLHF or learning from human feedback more generally (in Minecraft ⛏️), stop by. Also come to our poster at 5pm (#1405, Hall B1+B2)!