Two perspectives on high-dimensional estimation problems: posterior contraction and median-of-means
Due to the COVID-19 crisis the PhD defence of Gianluca Finocchio will take place (partly) online.
The PhD defence can be followed by a live stream.
Gianluca Finocchio is a PhD student in the research group Mathematics of Operations Research (MOR). His supervisor is prof.dr. A.J. Schmidt-Hieber from the Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS).
This thesis investigates Bayesian and frequentist procedures for challenging high-dimensional estimation problems.
In a Gaussian sequence model, we study the Bayesian approach to estimate the common variance of the observations. A fraction of the means is known to be zero, whereas the non-zero means are treated as nuisance parameters. This model is non-standard in the sense that it induces inconsistent maximum likelihood. We show a general inconsistency result: the posterior distribution does not contract around the true variance as long as the nuisance parameters are drawn from an i.i.d. proper distribution. We also show that consistency is retained by a hierarchical Gaussian mixture prior. For the latter, we recover the asymptotic shape of the posterior in the Bernstein-von Mises sense and show it is non-Gaussian in the case of small means.
In the nonparametric regression model, we study the Bayesian approach to the estimation of a regression function that is characterized by some underlying composition structure, parametrized by a graph and a smoothness index. This model is inspired by deep learning methods, which work well when complex objects have to be built from simpler features. In previous work, a frequentist estimator based on deep neural networks has been shown to be adaptive with respect to the underlying structure and achieve minimax estimation rates. We characterize the contraction rates of the posterior distribution arising from priors induced by the composition of Gaussian processes. With a suitable model selection prior, we show that the posterior achieves the minimax rates of estimation.
In the nonparametric least-squares regression model, we study a frequentist approach to estimate the regression function and the standard deviation of the residuals. The dataset consists of i.i.d. observations contaminated by a small number of outliers, and heavy-tailed residuals. For the case of known standard deviation, robust median-of-means procedures are available, and we extend them to the case of unknown standard deviation. In the sparse linear regression case, the median-of-means estimator yields a robust version of the Lasso, whereas our method yields a robust version of the square-root Lasso thanks to a scale-invariance argument. We also provide an aggregated estimator achieving minimax convergence rates while being adaptive to the unknown sparsity level.
