PRESS statistic

Leave-one-out cross-validation: Illustration of fitting a model and finding the PRESS statistic for n=8 observations

In statistics, the predicted residual error sum of squares (PRESS) is a form of cross-validation used in regression analysis to provide a summary measure of the fit of a model to a sample of observations that were not themselves used to estimate the model. It is calculated as the sum of squares of the prediction residuals for those observations. Specifically, the PRESS statistic is an exhaustive form of cross-validation, as it tests all the possible ways that the original data can be divided into a training and a validation set.

Procedure

Instead of fitting only one model on all data, leave-one-out cross-validation is used to fit N models (on N observations) where for each model one data point is left out from the training set. The out-of-sample predicted value is calculated for the omitted observation in each case, and the PRESS statistic is calculated as the sum of the squares of all the resulting prediction errors:

$\mathrm{PRESS}=\sum _{i=1}^n(y_i-{\hat{y}}_{i,-i}{)}^2$

Usage

Given this procedure, the PRESS statistic can be calculated for a number of candidate model structures for the same dataset, with the lowest values of PRESS indicating the best structures. Models that are over-parameterised (over-fitted) would tend to give small residuals for observations included in the model-fitting but large residuals for observations that are excluded. The PRESS statistic has been extensively used in lazy learning and locally linear learning to speed-up the assessment and the selection of the neighbourhood size.

See also

• Model selection
• Stepwise regression
• Cross-validation (statistics)