Mark Lukas

Generalized cross-validation (GCV) is a popular parameter selection criterion for spline smoothing of noisy data, but it sometimes yields a severely undersmoothed estimate, especially if the sample size is small. Robust GCV (RGCV) and modified GCV are stable extensions of GCV, with the degree of stabilization depending on a parameter (Formula presented.) for RGCV and on a parameter (Formula presented.) for modified GCV. While there are favorable asymptotic results about the performance of RGCV and modified GCV, little is known for finite samples. In a large simulation study with cubic splines, we investigate the behavior of the optimal values of (Formula presented.) and (Formula presented.), and identify simple practical rules to choose them that are close to optimal. With these rules, both RGCV and modified GCV perform significantly better than GCV. The performance is defined in terms of the Sobolev error, which is shown by example to be more consistent with a visual assessment of the fit than the prediction error (average squared error). The results are consistent with known asymptotic results.

The prediction error (average squared error) is the most commonly used performance criterion for the assessment of nonparametric regression estimators. However, there has been little investigation of the properties of the criterion itself. This paper shows that in certain situations the prediction error can be very misleading because it fails to discriminate an extreme undersmoothed estimate from a good estimate. For spline smoothing, we show, using asymptotic analysis and simulations, that there is poor discrimination of extreme undersmoothing in the following situations: small sample size or small error variance or a function with high curvature. To overcome this problem, we propose using the Sobolev error criterion. For spline smoothing, it is shown asymptotically and by simulations that the Sobolev error is significantly better than the prediction error in discriminating extreme undersmoothing. Similar results hold for other nonparametric regression estimators and for multivariate smoothing. For thin-plate smoothing splines, the prediction error's poor discrimination of extreme undersmoothing becomes significantly worse with increasing dimension.

While it is a popular selection criterion for spline smoothing, generalized cross-validation (GCV) occasionally yields severely undersmoothed estimates. Two extensions of GCV called robust GCV (RGCV) and modified GCV have been proposed as more stable criteria. Each involves a parameter that must be chosen, but the only guidance has come from simulation results. We investigate the performance of the criteria analytically. In most studies, the mean square prediction error is the only loss function considered. Here, we use both the prediction error and a stronger Sobolev norm error, which provides a better measure of the quality of the estimate. A geometric approach is used to analyse the superior small-sample stability of RGCV compared to GCV. In addition, by deriving the asymptotic inefficiency for both the prediction error and the Sobolev error, we find intervals for the parameters of RGCV and modified GCV for which the criteria have optimal performance.

In various applications, it is necessary to differentiate a matrix functional w(A(x)), where A(x) is a matrix depending on a parameter vector x. Usually, the functional itself can be readily computed from a triangular factorization of A(x). This paper develops several methods that also use the triangular factorization to efficiently evaluate the first and second derivatives of the functional. Both the full and sparse matrix situations are considered. There are similarities between these methods and algorithmic differentiation. However, the methodology developed here is explicit, leading to new algorithms. It is shown how the methods apply to several applications where the functional is a log determinant, including spline smoothing, covariance selection and restricted maximum likelihood.

In the literature on regularization, many different parameter choice methods have been proposed in both deterministic and stochastic settings. However, based on the available information, it is not always easy to know how well a particular method will perform in a given situation and how it compares to other methods. This paper reviews most of the existing parameter choice methods, and evaluates and compares them in a large simulation study for spectral cut-off and Tikhonov regularization. The test cases cover a wide range of linear inverse problems with both white and colored stochastic noise. The results show some marked differences between the methods, in particular, in their stability with respect to the noise and its type. We conclude with a table of properties of the methods and a summary of the simulation results, from which we identify the best methods.

Generalized cross-validation (GCV) is a widely used parameter selection criterion for spline smoothing, but it can give poor results if the sample size n is not sufficiently large. An effective way to overcome this is to use the more stable criterion called robust GCV (RGCV). The main computational effort for the evaluation of the GCV score is the trace of the smoothing matrix, tr A, while the RGCV score requires both tr A and tr A(2). Since 1985, there has been an efficient O(n) algorithm to compute tr A. This paper develops two pairs of new O(n) algorithms to compute tr A and tr A(2), which allow the RGCV score to be calculated efficiently. The algorithms involve the differentiation of certain matrix functionals using banded Cholesky decomposition.

We consider Tikhonov regularization of linear inverse problems with discrete noisy data containing correlated errors. Generalized cross-validation (GCV) is a prominent parameter choice method, but it is known to perform poorly if the sample size n is small or if the errors are correlated, sometimes giving the extreme value 0. We explain why this can occur and show that the robust GCV methods perform better. In particular, it is shown that, for any data set, there is a value of the robustness parameter below which the strong robust GCV method (R(1)GCV) will not choose the value 0. We also show that, if the errors are correlated with a certain covariance model, then, for a range of values of the unknown correlation parameter, the "expected" R(1)GCV estimate has a near optimal rate as a n -> infinity. Numerical results for the problem of second derivative estimation are consistent with the theoretical results and show that R(1)GCV gives reliable and accurate estimates.

Let fλ be the Tikhonov regularized solution of a linear inverse or smoothing problem with discrete noisy data yi, i = 1, ..., n. To choose λ we propose a new strong robust GCV method denoted by R1GCV that is part of a family of such methods, including the basic RGCV method. R1GCV chooses λ to be the minimizer of γV(λ) + (1 − γ)F1(λ), where V(λ) is the GCV function, F1(λ) is a certain approximate total measure of the influence of each data point on fλ with respect to the regularizing norm or seminorm, and γ in (0, 1) is a robustness parameter. We show that R1GCV is less likely to choose a very small value of λ than both GCV and RGCV. RGCV and R1GCV also have good asymptotic properties for general problems with independent errors. Strengthening previous results for RGCV, it is shown that the (shifted) RGCV and R1GCV functions are consistent estimates of 'robust risk' functions, which place extra weight on the variance of fλ. In addition RGCV is asymptotically equivalent to the modified GCV method. The results of numerical simulations for R1GCV are consistent with the asymptotic results, and, for suitable values of γ, R1GCV is more reliable and accurate than GCV.

This paper extends the direct sensitivity analysis of Shi and Lukas [2005, Sensitivity analysis of constrained linear L 1 regression: perturbations to response and predictor variables. Comput. Statist. Data Anal. 48, 779-802] of linear L 1 (least absolute deviations) regression with linear equality and inequality constraints on the parameters. Using the same active set framework of the reduced gradient algorithm (RGA), we investigate the effect on the L 1 regression estimate of small perturbations to the constraints (constants and coefficients). It is shown that the constrained estimate is stable, but not uniformly stable, and in certain cases it is unchanged. We also consider the effect of addition and deletion of observations and determine conditions under which the estimate is unchanged. The results demonstrate the robustness of L 1 regression and provide useful diagnostic information about the influence of observations. Results characterizing the (possibly non-unique) solution set are also given. The sensitivity results are illustrated with numerical simulations on the problem of derivative estimation under a concavity constraint.