Today we have Moritz Hardt who will be giving a guest lecture on his work in robust machine learning.
Most applications of machine learning across science and industry rely on the holdout method for model selection and validation. Unfortunately, the holdout method can fail in the now common situation where the data scientist works interactively with the data, iteratively choosing which methods to use by probing the same holdout data many times.
In this lecture, I will review classical methods for model selection and contrast them with some more recent advances in the field. In particular, I will describe a framework for designing reliable machine learning benchmarks, data science competitions, and hyperparameter tuning.