Relationships among sample size, model selection and likelihood regions, and scientifically important differences

Abstract

In multidimensional models, simultaneous confidence or credibility regions for continuous parameters hold the overall frequentist long run or Bayesian posterior probability constant at some level such as 95%. This means that, as the dimensionality of the problem increases, the precision or information required about each individual parameter also rapidly increases so parameters have progressively less chance of being set to 0 in such a model selection procedure. These methods do not appropriately answer most of the inference questions that are generally encountered in applied statistics. Thus, sample size, model selection criteria and the estimation of parameter precision are intimately related. In contrast with frequentist and Bayesian procedures, direct likelihood inference, calibrating acceptable likelihood regions by criteria derived from model selection, such as the Akaike information criterion, holds the precision requirements per parameter constant as the dimensionality grows, thus allowing the series of inferences to remain compatible; Other model selection criteria, such as the Bayes information criterion, that depend on the sample size, maintain compatibility but decrease the precision per parameter as the sample increases, so, in the limit, the null model tends to be chosen (Lindley's paradox).