Upcoming Events

Wed Feb 8
12 noon ET

Pruning Deep Neural Networks for Lottery Tickets 

Deep learning continues to impress us with breakthroughs across disciplines but comes at severe computational and memory costs that limit the global participation in the development of related technologies. Can we address some of these challenges by finding and training smaller models? The lottery ticket hypothesis has given us hope that this question might be answered by pruning randomly initialized neural networks. In this talk, we will prove a strong version of this hypothesis in realistic settings. Inspired by our theory, we will create a test environment that allows us to highlight current limitations of state-of-the-art algorithms in finding extremely sparse lottery tickets and highlight some opportunities for future progress. 

Wed Feb 22
12 noon ET

TBA

TBA

Wed Mar 1
12 noon ET

Using Algebraic Factorizations for Interpretable Learning

Non-negative Matrix Factorization (NMF) is a fundamental tool for dictionary learning problems, giving an approximate representation of complex data sets in terms of a reduced number of extracted features. In this talk, we will introduce the main concept of NMF, its implementation, and its online and streaming variations. We will showcase how mathematical tools like this can be used for interpretable learning tasks. These applications range from imaging and medicine to forecasting and collaborative filtering. Discussion and questions are welcome.

Wed Mar 15
12 noon ET

TBA

TBA

TBA

Wed Mar 22
12 noon ET

TBA

TBA

TBA

Wed Apr 5
12 noon ET

Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints

Penalty-based regularization is extremely popular in ML. However, this powerful technique can require an expensive trial-and-error process for tuning the penalty coefficient. In this paper, we take sparse training of deep neural networks as a case study to illustrate the advantages of a constrained optimization approach: improved tunability, and a more interpretable hyperparameter. Our proposed technique (i) has a negligible computational overhead, (ii) reliably achieves arbitrary sparsity targets “in one shot” while retaining high accuracy, and (iii) scales successfully to large residual models and datasets. 


In this talk, I will also give a brief introduction to Cooper (https://github.com/cooper-org/cooper/), a general-purpose, deep learning-first library for constrained optimization in Pytorch. Cooper was developed as part of the research direction above, and was born out of the need to handle constrained optimization problems for which the loss or constraints may not be "nicely behaved" or "theoretically tractable", as is often the case in DL. 

Wed Apr 12
12 noon ET

TBA

TBA

TBA

Wed Apr 19
12 noon ET

TBA

TBA

TBA

Wed Apr 26
12 noon ET

TBA

TBA

TBA

Wed May 10
12 noon ET

TBA

TBA

TBA

Wed May 17
12 noon ET

TBA

TBA

TBA

Wed May 24
12 noon ET

TBA

TBA

TBA

Wed May 31
12 noon ET

TBA

TBA

TBA

Wed June 7
12 noon ET

TBA

TBA

TBA

Wed June 14
12 noon ET

TBA

TBA

TBA

Wed June 21
12 noon ET

TBA

TBA

TBA

Wed June 28
12 noon ET

TBA

TBA

TBA