“Deepnet Spectra and the Two Cultures of Data Science”
by Professor David L. Donoho, Department of Statistics, Stanford University
Tuesday, November 12
12:00 p.m. to 1:00 p.m.
Building 9, lecture hall 2, room 2325
Abstract
Machine learning became a remarkable media story of the 2010s largely owing to its ability to focus researcher energy on attacking prediction challenges like ImageNet. Media extrapolations forecast the complete transformation of human existence. Unfortunately, machine learning has a troubled relationship with understanding the foundation of its achievements well enough to face demanding real-world requirements outside the prediction challenge setting. For example, its literature is admittedly corrupted by anti-intellectual and anti-scholarly tendencies. It is beyond irresponsible to build a revolutionary transformation on such a shaky pseudo-foundation. In contrast, more traditional subdisciplines of data science like numerical linear algebra, applied probability and theoretical statistics provide time-tested tools for designing reliable processes with understandable performance. Moreover, positive improvements in human well-being have repeatedly been constructed using these foundations. To illustrate these points, we will review a recent boomlet in the ML literature in the study of eigenvalues of Deepnet Hessians. A variety of intriguing patterns in eigenvalues were observed and speculated about in ML conference papers. We describe the work of Vardan Papyan, which shows that the traditional subdisciplines, properly deployed, can offer insights about these objects that ML researchers had.
About the speaker:
David Donoho is Anne T. and Robert M. Bass professor of humanities and sciences and professor of statistics at Stanford University. He previously was a professor at UC Berkeley and held visiting positions at Universite de Paris (VII and XI), the University of Leiden and the National University of Singapore.
He has also worked in the industry in oil exploration (Western Geophysical), information technology (co-founding BigFix, Inc., which was acquired by IBM) and quantitative finance (Renaissance Technologies).
He received the AB in statistics from Princeton (summa cum laude), where his undergraduate thesis advisor was John Tukey. He earned a Ph.D. in Statistics from Harvard University, where his thesis advisor was Peter Huber. Donoho was awarded honorary degrees from the University of Chicago, Ecole Fédérale Polytechnique de Lausanne, the University of Waterloo and the Technion.
Donoho is a member of the American Academy of Arts and Sciences and the U.S. National Academy of Sciences and is a foreign associate of the French Académie des Sciences. He has received a MacArthur Fellowship and the Committee of Presidents of Statistical Societies Presidents’ award. He was awarded the Norbert Wiener Prize of the American Mathematical Society and Society of Industrial and Applied Mathematics and the Gauss Prize of the International Mathematical Congress and German Mathematical Society. He received the 2013 Shaw Prize in Mathematical Sciences.
For more information about the lecture, please email Professor Marc G. Genton at Marc.Genton@KAUST.EDU.SA.