 |
Center for Nonlinear Studies |
Deep Equilibrium Models (DEQs) are a class of machine learning models that computes the fixed point of a single nonlinear operator in lieu of a traditional multi-layer network. The resulting models are conceptually simpler than traditional models and benefit from smaller memory requirements due to implicit differentiation, yet often outperform similarly sized feedforward networks. This talk will provide a broad overview of our work in DEQ models, with a focus on some of our recent advances on the topic. I will cover the basic foundations of the methods, approaches to multi-scale modeling. I will then discuss several recent advances in training methods or solvers, plus applications of the methods to domains such as input-optimization in deep networks, implicit layers within neural fields, and methods for improved optical flow.
Host: Wenting Li & Arvind Mohan