Variable selection using P-splines

Abstract

Selecting among a large set of variables those that influence most a response variable is an important problem in statistics.When the assumed regression model involves a nonparametric component, penalized regression techniques, and in particular P-splines, are among the commonly used methods. The aim of this paper is to provide a brief review of variable selection methods using P-splines. Starting frommultiple linear regression models,with least-squares regression, and Ridge regression, we review standard methods that perform variable selection, such as LASSO, nonnegative garrote, the SCAD method, etc. We briefly discuss a general framework of penalization and regularization methods. Going toward more flexible regression models, with some nonparametric component(s), we discuss P-splines estimation. For some examples of flexible regression models, we then review a few variable selection methods using P-splines. A brief discussion on grouped regularization techniques and on a robust variable selection method is given. Furthermore, we mention key ingredients in Bayesian approaches, and end the paper by drawing the attention to several other issues in variable selection with P-splines. Throughout the paper we provide some illustrations.