📢We introduce “RefFusion”, a novel inpainting method for scenes reconstructed using 3D Gaussian Splatting.

🔗reffusion.github.io

TLDR: we personalize an image diffusion model to a given reference image and distill its knowledge to 3D through score distillation sampling.

Heading to Chicago for the Midwest Robotics Workshop at TTIC. Drop me a line if you are around and want to chat about AI, CV, and robots. 🦾💬

Thank you BoschGlobal and FZI for hosting me in Renningen and Karlsruhe this week. I had a great time talking about our work and reconnecting with many friends.

So cool to have Fabio Ramos from NVIDIA Robotics & University of Sydney visit University of Toronto Robotics Institute today! He’s giving an exciting talk on “Probabilistic Robotics 2.0”. Thanks for coming!

Such a wonderful collaboration with Guanyu Song under Boris Ivanovic mentorship. 🎉🎉🎊🎊 Our paper has been selected for an Oral presentation at #CVPR2024 !!!! Extremely proud of our team and looking forward to seeing you all in Seattle!!

Had a great time catching up with Luma AI last night. Great to see the people behind the amazing work. Also, this was probably the most remote meeting I had with one of my grad students. 🛫

Excited to meet our incoming faculty Shuang Li on my trip to the Bay Area! We can’t wait to have you join University of Toronto! 🎉

Excited to share our CVPR 2024 work: Producing and Leveraging Online Map Uncertainty in Trajectory Prediction 🚗🚗🚗

Online map estimation makes HD maps more affordable for self-driving. But is it really reliable? We introduce a simple yet general uncertainty formulation to

What if we could show a robot how to do a task?

We present Vid2Robot,  which is a robot policy trained to decode human intent from visual cues and translate it into actions in its environment. 🤖

Website: vid2robot.github.io
Arxiv: arxiv.org/abs/2403.12943

🧵(1/n)

🚨New pre-print alert🎉

Dissecting Deep RL with High Update Ratios: Combatting Value Overestimation and Divergence

We dissect failure modes of the primacy bias in deep RL and show that a major concern is value divergence. Normalization to the rescue!

arxiv.org/abs/2403.05996

The termination of Apple's self-driving car project is sad news. 😢 I would have loved to see it happen. A silver lining for researchers in embodied AI: this type of problems is hard and largely unsolved. There is room for lots of fruitful research. techcrunch.com/2024/02/27/app…

Huge congrats to my students Yash Kant, Ziyi Wu, and Xunjiang Gu (+many collaborators) for getting their papers accepted to #CVPR2024 ! 🎊🍾 Details will follow soon. Also, Xunjiang Gu is applying to graduate school. Check out his application!

Fascinatingly, advances in generative modeling evolve in the opposite order of evolution. It feels like we are reverse engineering nature. First language, now video. And physics is next. It’s the best time to work on AI for Robotics!

Great work led by Yash Kant! Thanks for featuring it AK! 🎉

Wouldn't papers become much more interesting if we ask reviewers to not count anything as a weakness that has already been mentioned as a limitation by the authors?

Thanks for featuring our work! 🎉

Together with David Lindell, we are excited to host NVIDIA AI‘s Zan Gojcic for a talk at the Toronto Vision Seminar!

Huge congrats to my friends and former Autonomous Systems Lab, ETH Zürich colleagues Gregory Hitz, Marcin Dymczyk, Renau Dubé and many more at Sevensense Robotics for being acquired by ABB Robotics! 🎉 ABB is lucky to have you guys!