COLLOQUIUM - 24/04/2025
WHO: Dr. Assaf Shocher, NVIDIA
WHEN: Thursday, April 24th 2025 at 12:00
WHERE: BUILDING 503 (Computer Science) AUDITORIUM
Title:
Projection Based Learning for Adaptive Computer Vision
Abstract:
Every day, somewhere, a researcher mutters, “If only neural networks were linear, this problem would be solved.” Linear operations offer powerful tools: projections onto subspaces, eigen decomposition, and more. Can we adapt these elegant tools to the non-linear world of neural networks, enabling adaptive learning at zero-shot? In this talk, we'll explore generalized projections using idempotent operators—functions satisfying f(f(x)) = f(x)—to bridge linear intuitions and neural network complexity. I'll introduce Deep Internal Learning, where networks are trained entirely at test-time using projection onto patch distributions, solving tasks such as super-resolution, dehazing, and segmentation without external training data. Next, I'll discuss Idempotent Generative Network (IGN), trained to project data onto itself (f(x)=x) and enforce idempotency (f(f(z))=f(z)). IGN generates data directly and projects corrupted inputs onto the target distribution. Projection further motivates Idempotent Test-Time Training (IT³), which adapts models at test-time using only current out-of-distribution input. By enforcing idempotency via a simple consistency objective, IT³ improves robustness across diverse architectures (MLPs, CNNs, GNNs) and tasks (corrupted images, tabular data, facial age prediction). Finally, I'll ask: "Who says neural networks are non-linear?" They're only non-linear relative to standard vector spaces! Ongoing work constructs tailored vector spaces where neural networks become genuinely linear, opening doors to spectral decomposition, zero-shot inverse solutions, and architecture-enforced idempotence.
Bio:
I am a postdoctoral researcher at NVIDIA. Prior to that I was a postdoctoral fellow at UC Berkeley, working with Alyosha Efros, and a visiting researcher at Google. I received my PhD from the Weizmann Institute of Science, where I was advised by Michal Irani. I have bachelor's degrees in Physics and EE from Ben-Gurion University. My prizes and honors include the Rothschild postdoctoral fellowship, the Fulbright postdoctoral fellowship, John F. Kennedy award for outstanding Ph.D. at the Weizmann Institute, and the Blavatnik award for CS Ph.D. graduates.
