Gemini 1.5 Pro - A highly capable multimodal model with a 10M token context length

Today we are releasing the first demonstrations of the capabilities of the Gemini 1.5 series, with the Gemini 1.5 Pro model. One of the key differentiators of this model is its incredibly long…

most exciting paper *ever* from our Google AI lab at Princeton University: naman agarwal danielsuo Xinyi Chen
arxiv.org/abs/2312.06837
*** Convolutional filters predetermined by the theory, no learning needed! ***

soon is happening! go use Gemini Ultra gemini.google.com/advanced?hl=en

[LG] Grandmaster-Level Chess Without Search
A Ruoss, G Delétang, S Medapati, J Grau-Moya, L K Wenliang, E Catt, J Reid, T Genewein [Google DeepMind] (2024)
arxiv.org/abs/2402.04494

- The authors train transformer models of different sizes (9M, 136M, 270M parameters) on a dataset…

Google Deepmind presents Grandmaster-Level Chess Without Search

paper page: huggingface.co/papers/2402.04…

largest model reaches a Lichess blitz Elo of 2895 against humans, and successfully solves a series of challenging chess puzzles, without any domain-specific tweaks or explicit…

Gary Marcus TPUs go brrrr

arxiv.org/abs/2402.04494

for anyone looking to hire strong engineers!! bilal did great work with us at Google DeepMind last summer 👌

I’m giving a talk at 2:40pm at the Heavy Tails workshop on Friday. The talk is about DL optimization dynamics in general, and adaptive gradient methods in particular. I’m also around NeurIPS starting Wednesday, and would love to meet people with common interests - send me a DM!

It’s been a privilege to be part of the Gemini pretraining team and overall program, I’m so excited that the world can finally see what we’ve been up to for most of the past year:

tl;dr we’re so back storage.googleapis.com/deepmind-media…

I’m very excited to share our work on Gemini today! Gemini is a family of multimodal models that demonstrate really strong capabilities across the image, audio, video, and text domains. Our most-capable model, Gemini Ultra, advances the state of the art in 30 of 32 benchmarks,…

After 3 years of hard work, our unprecedented neural network training algorithm competition is finally open! The exciting part starts now, seeing what the community can create. 🏆Submit, become the next Adam, and bag $50,000 in prizes!
mlcommons.org/2023/11/mlc-al…

The #ML community is currently unable to reliably identify training algorithm improvements, or even determine the state-of-the-art training algorithm. This has to change if we want to make progress speeding up neural network training mlcommons.org/2023/11/mlc-al…

tl;dr submit a training algorithm* that is faster** than Adam*** and win $10,000 💸🚀

*a set of hparams, self-tuning algorithm, and/or update rule
**see rules for how we measure speed
***beat all submissions, currently the best is NAdamW in wallclock and DistShampoo in steps

To highlight the importance of #ML training & algorithmic efficiency, we’re excited to provide compute resources to help evaluate the best submissions to the MLCommons AlgoPerf training algorithms competition, w/ a chance to win a prize from MLCommons! goo.gle/3N3sHdD

I'm very excited that this paper is out, it has been over 2 years in the making! I started at Google Research speeding up neural net training, but was often frustrated when we didn't know how to declare a win over Adam 🚀

Benchmarking Neural Network Training Algorithms

paper page: huggingface.co/papers/2306.07…

Training algorithms, broadly construed, are an essential part of every deep learning pipeline. Training algorithm improvements that speed up training across a wide variety of workloads (e.g.,…

Is Adam the best optimizer to train neural networks?🤔
We don't know. And we won't know until we test training algorithms properly.

🚀That's why we spent ~2.5 years building AlgoPerf, a competitive, time-to-result training algorithms benchmark using realistic workloads!

#AI173 with 216 passengers stuck in Magadan, Russia since 2 pm IST Tue 6/6 after technical issues. Passengers staying at makeshift accommodation in a nearby school. Conditions have become miserable, no updates from Air India, bad food. Crew not helping. PMO India Dr. S. Jaishankar (Modi Ka Parivar)

Bard is now available in the US and UK, w/more countries to come. It’s great to see early Google AI work reflected in it—advances in sequence learning, large neural nets, Transformers, responsible AI techniques, dialog systems & more.

You can try it at bard.google.com

MusicLM: Generating Music From Text

abs: arxiv.org/abs/2301.11325
project page: google-research.github.io/seanet/musiclm…