Aside from achieving very good results, we also extensively studied the components of our approach to support our claims.

The classifier head builds up the prediction in a cascaded way, by scaling past samples using multiple gating values. It simplifies the regularisation of the model and can be used with any approach, improving the performance, especially when the external memory size is contained

The regularisation schema tries to constrain past probabilities to be lower than the ground truth one.

Combining it with a simple knowledge distillation approach removes the necessity of training on rehearsal samples, improving the plasticity and preserving the stability!

Back in the Continual Learning game with Alessio Devoto and Simone Scardapane!

In this paper, we propose a regularization schema and a novel classifier head. Combining these two will improve the plasticity of your model while actively fighting catastrophic forgetting.

*Conditional computation in NNs: principles and research trends*
by Alessio Devoto Valerio Marsocci Jary Pomponi Pasquale Minervini 🚀 looking for postdocs!

Our latest tutorial on increasing modularity in NNs with conditional computation, covering MoEs, token selection, & early exits.

arxiv.org/abs/2403.07965

Synecdoche, Log Cabin.

Charlie Kaufman, 2008.

My first paper of 2024 is out!

With Matteo Gambella, Manuel Roveri, and Simone Scardapane, we developed NACHOS, the first Neural Architecture Search framework for automatically designing optimal Early Exits NNs.

You can find the paper here: arxiv.org/abs/2401.13330

Herman na prezydenta ♉️🧚‍♀️ GayShit69 Othman ガンコヤドン 🎀☆𝚊𝚔𝚊𝚓𝚒𝚒𝚒☆🎀 Jack Coderre Jacob Ghaz Outro ⪛⦿⫺ Alfonso Gd MrDowgi crara Dissociato Lucky Raja Cyclonic Breezy Barbara Hoffman Voltaire 🌹 YOUSSEF_LFC Jary Pomponi Tilicia alexandra

You can find the paper here: proceedings.mlr.press/v158/qendro21a…

Which comes with a very easy-to-use code: github.com/ajrcampbell/ea…

The paper is currently under review, but we just released it on arXiv: arxiv.org/abs/2402.01262

Also, the source code, based on Avalanche, is available: github.com/jaryP/Cascaded…

Ferenc Huszár ICML Conference I created a repo containing only the modified paper and then anonymized it.

I will share the latter link, and I hope it will work or the reviewer...

Martin Mundt Kristian Kersting Wolfgang Stammer Very interesting, thanks for posting it. I will try to use it in my next Continual Learning research!

Hope to get interesting results