A Universal Law in Deep Learning: from MLP to Transformer
报告人:苏炜杰(宾夕法尼亚大学沃顿商学院)
报告时间:2024年7月9日16:00-17:00
报告地点:海纳苑2幢106
摘要: In this talk, we introduce a universal phenomenon that governs the inner workings of a wide range of neural network architectures, including multilayer perceptrons, convolutional neural networks, transformers, and Mamba. Through extensive computational experiments, we demonstrate that deep neural networks tend to process data in a uniform improvement manner across layers. We conclude this talk by discussing how this universal law provides useful insights into practice.
联系人:赖俊(laijun6@zju.edu.cn)