讲座标题（中文）：理解深度学习中的良性过拟合现象：从隐式偏差与训练动力学视角
讲座标题（英文）: Understanding Benign Overfitting in Modern Deep Learning: Implicit Bias and Training Dynamics
讲座摘要：The phenomenon of benign overfitting, where a trained neural network perfectly fits noisy training data but still achieves near-optimal test performance, has been extensively studied in recent years for linear models and fully-connected/convolutional networks. In this work, we study benign overfitting in a single-head softmax attention model, which is the fundamental building block of Transformers. We prove that under appropriate conditions, the model exhibits benign overfitting in a classification setting already after two steps of gradient descent. Moreover, we show conditions where a minimum-norm/maximum-margin interpolator exhibits benign overfitting. We study how the overfitting behavior depends on the signal-to-noise ratio (SNR) of the data distribution, namely, the ratio between norms of signal and noise tokens, and prove that a sufficiently large SNR is both necessary and sufficient for benign overfitting.
讲者信息：
尚书宁，本科毕业于浙江大学竺可桢学院混合班，目前是普林斯顿大学计算机系的一年级博士生，导师为Sanjeev Arora教授，主要研究方向为机器学习理论与深度学习的科学理解。个人主页：https://nooraovo.github.io/
