Upcoming Events


Wed May 18
12 noon ET


No Seminar

Wed May 25
12 noon ET

Beyond Regression: Operators and Extrapolation in Machine Learning

In this talk we first suggest a unification of regression-based machine learning methods, including kernel regression and various types of neural networks. We then consider the limitations of such methods, especially the curse-of-dimensionality, and the various potential solutions that have been proposed including: (1) Barron's existence result, (2) leveraging regularity, and (3) assuming special structure in the data such as independence or redundancy. Finally, we consider operator-learning and extrapolation as emerging directions for machine learning. Operator-learning is the more developed of the two, and we show how learning operators allows intrinsic regularization, uncertainty quantification, and can represent many-to-one and one-to-many mappings. However, extrapolation remains the final frontier in machine learning, and we discuss an emerging approach and the mathematics that may underly it.

Wed June 1
12 noon ET


No seminar

Wed June 15
12 noon ET


No seminar

Wed June 22
12 noon ET

SGD Through the Lens of Kolmogorov Complexity

In this talk, we present global convergence guarantees for stochastic gradient descent (SGD) via an entropy compression argument. We do so under two main assumptions: (1. Local progress) The model accuracy improves on average over batches. (2. Models compute simple functions) The function computed by the model has low Kolmogorov complexity. It is sufficient that these assumptions hold only for a tiny fraction of the epochs. Intuitively, our results imply that intermittent local progress of SGD implies global progress. Assumption 2 trivially holds for underparameterized models, hence, our work gives the first convergence guarantee for general, underparameterized models. Furthermore, this is the first result that is completely model agnostic - we don't require the model to have any specific architecture or activation function, it may not even be a neural network.

Wed June 29
12 noon ET

No seminar

Summer break