A Note on Test for Model Adequacy in Nonlinear Regression

2004, Vol. 15, No. 3, pp. 689∼694

A Note on Test for Model Adequacy in Nonlinear Regression ¹⁾

Myung-Wook Kahng ²⁾

Abstract

We investigate the test for model adequacy in nonlinear regression. We can expect the usual likelihood ratio statistic to be unaffected by any parametric- effect curvature; only the effect of intrinsic curvature needs to be considered. Multiplicative correction factor is derived for the limiting distribution of test statistic, which is a function of the intrinsic curvature arrays.

Keywords : Correction factor, Intrinsic curvature, Intrinsic curvature array, Model adequacy, Parametric-effect curvature

1. Introduction

Once a nonlinear regression model has been chosen and the unknown parameters estimated, the question of the adequacy of the model to the data arises. Usually one would look at the residuals and carry out various plots. If the intrinsic curvature is negligible, as is often the case, then the usual residual plots and likelihood ratio test carried out for linear models are also appropriate in nonlinear regression analysis. However, if there is substantial intrinsic curvature, then the projected residuals suggested by Cook and Tsai(1985) can be used and its F-approximation can be seriously affected by severe curvature (Bates and Watts 1980, 1981; Hamilton, 1986).

When the model is replicated, the adequacy of the model can be tested directly.

We consider the likelihood ratio statistic for the test of lack of fit. This test is unaffect by any parametric-effect curvature, but the effect of intrinsic curvature 1) This research was supported by the Sookmyung Women's University Research Grants 2003.

2) Professor, Department of Statistics, Sookmyung Women's University, Seoul, 140-742, Korea

E-mail : [email protected]

distribution of test statistic, which is a function of the intrinsic curvature arrays.

If here are no replicates, then there are a number of test procedures available, generally based on the idea of near-replicates. A number of these, which are also suitable for nonlinear models, are given by Miller, Neill and Sherfey(1999).

2. Nonlinear Regression Model

Consider the standard nonlinear regression model

y

= f ( x

: θ ) + ε

, i = 1, 2, …, n , (2.1)

in which the _i -th response y

is related to the q -dimensional vector of known explanatory variable x

through the known nonlinear model function f , which depends on p -dimensional unknown parameter vector θ , and ε

is error. We assume that f is twice continuously differentiable in θ , and errors ε

are independent identically distributed normal random variables with mean 0 and variance σ

. The least squares estimate of θ , denoted by ˆ , minimized the θ error sum of squares

S ( θ ) = ∑

n

i = 1

[ ^y

^{- f ( x}

^{: θ )} ]

^.

Let the p -dimensional parameter vector θ be partitioned in the form θ = ( θ

, θ

)

where θ

is a p

-dimensional nuisance vector and θ

is a p

-dimensional vector of primary interest with p

+ p

= p . Now we consider the testing H

: θ

= θ

. To apply the likelihood ratio procedure we need to minimize S ( θ ), subject to θ

= θ

. Let S ( θ ˜ ) = S ( θ ˜

1

( θ

) , θ

) be the minimum value of S ( θ ) under H

. Then the likelihood ratio statistics is

F

= S( θ ˜ ) - S( θ ˆ )

S( θ ˆ ) ⋅ n - p

p

(2.2)

which is approximately distributed as F

when H

is true.

3. Lack of Fit Test

Suppose a design is replicated, say n

times for point x

, so that our nonlinear model (2.1) now becomes

y

= η

+ e

= f ( x

: θ ) + ε

, i = 1, 2, … , k, j = 1, 2, …, n

(3.1)

where the errors ε

are assumed to be i.i.d. N ( 0, σ

) and k is the number of the distinct setting of the explanatory variables. This model includes the genuine variation between different experiments with the same x observation as well as error of measurements.

Ignoring the structure on η

, we have the usual decomposition

∑

∑

j

( y

- η

)

= ∑

i

∑

j

( y

- y

+ y

- η

)

= ∑

i

∑

j

( y

- y

)

+ ∑

i

n

( y

- η

)

,

(3.2)

where ∑

i

∑

j

( y

- y

)

is usually referred to as the pure error sum of squares.

Defining n = ∑ n

, an unbiased estimate of σ

is

s

= 1 n - k ∑

i

∑

j

( y

- y

)

, and under the normality assumptions of ε

,

(n- k) s

σ

∼ χ

.

To find the least squares estimate ˆ of θ we minimize ∑∑ (y θ

- η

)

with

respect to θ , which is equivalent to minimizing ∑ n

( y

- η

)

, i.e. a

weighted least squares analysis with weight n

. The normal equations for θ ˆ are

therefore

- 2 ∑

i

n

( y

) - f ( x

: θ ) ∂ f ( x

: θ )

∂ θ |

= 0

If η

is replaced by η ˆ

i

= f ( x

: θ ˆ ) , the identity (3.2) still applies, so that

∑

∑

j

( y

- η ˆ

i

)

= ∑

i

∑

j

( y

- y

)

+ ∑

i

n

( y

- η ˆ

i

)

SSE = SS

+ ( SSE - SS

)

= SS

+ SS

The left-hand side, called the residual sum of squares, is therefore split into a pure error sum of squares SS

and a lack of fit sum of squares SS

. For the case when η

is a linear function of θ , a test for the validity of this model is given by Seber(1977),

F = SS

/ ( k - p) SS

/ ( n - k)

= ∑

i

n

( y

- η ˆ

i

)

∑

∑

j

( y

- y

)

⋅ n - k k - p

(3.3)

where F ∼ F

when the model is valid. Because of asymptotic linearity, we find the above F-test is approximately valid for large n when the model is nonlinear.

4. Correction Factors

Since f ( x : θ ) and S ( θ ) are invariant under one-to-one transformations of θ , the likelihood ratio statistic, F

is unaffected by the parameter-effect curvature, however, the effect of intrinsic curvature need to be considered.

Hamilton and Wien(1987) studied second-order approximations for the distribution

of F

and developed correction terms which are functions of the intrinsic

curvature arrays. Assuming normality of the ε

, they showed as σ

→ 0 (rather

than n → ∞ ), the limiting null distribution of F

is, to order σ

,

( 1 - γ σ

) F

(4.1)

where γ = α

/ ( n - p) - α

/ p

and

α

= α

- α

α

= - 1

2 ∑

n - p

i = 1

tr { ^{( A}

⁾

} ^{+ 1} ₄ ∑

n - p

i = 1

{ ^{tr ( A}

⁾ }

α

= - 1

2 ∑

n - p1

i = 1

tr { ^{( A}

⁾

} ^{+ 1} ₄ ∑

n - p1

i = 1

{ ^{tr ( A}

⁾ }

^.

Here the p×p matrix A

, i = 1,2,…, n - p is the i -th face of the intrinsic curvature array A

for the full model (2.1) (Seber and Wild, 1989, p. 142), and the p

×p

matrix A

, i = 1, 2, …, n - p

is the i -th face of the intrinsic curvature array for the restricted model obtained by fixing θ

= θ

and working with θ

. To evaluate γ we use θ

= θ

and θ

= θ ˜

1

( θ

) , the restricted least squares estimator of θ . The variance σ

is estimated by s

= S( θ ˜ )/(n- p) . The computations can be simplified by an appropriate choice of basis for the sample space.

The usual test statistic for evaluating lack of fit in regression is the likelihood ratio statistic for the test

H

: y

= f ( x

: θ ) + ε

H

: y

= g

( x

: θ ) + ε

Let δ

= E ( y | x

) = g

( x

: θ ) , for i = 1, 2, …, k , then the model under H

is a restriction of that under H

, and reparametrization φ = φ ( θ ) can be found

for which the null hypothesis is H

: φ

= 0 , where φ = ( φ

, φ

)

and φ

is p×1 , and φ

is ( k - p) ×1 . Thus the result in (4.1) can be applied, with the

number of the parameters k , and the number of nuisance parameters p .

Fortunately it is not necessary to find φ or the model derivatives with respect to

φ because of the invariance of the intrinsic arrays and α

and α

under

approximately distributed as

[ ^{1 - σ}

( ^{n - k α}

^{- k - p α}

)] ^{⋅ F}

, where α

and α

are calculated for the restricted nonlinear model.

References

1. Bates, D. M. and Watts, D. G. (1980). Relative curvature measures of nonlinearity (with discussion), Journal of the Royal Statistical Society Series, B, Vol. 42, 1-25.

2. Bates, D. M. and Watts, D. G. (1981). Parameter transformations for improved approximate confidence regions in nonlinear least squares, The Annals of Statistics, Vol. 9, 1152-1167.1. Cook, R. D. and Tsai, C.-L. (1985). Residuals in nonlinear regression, Biometrika, Vol. 72, 23-29.

3. Cook, R. D. and Tsai, C.-L. (1985). Residuals in nonlinear regression, Biometrika, Vol. 72, 23-29.

4. Hamilton, D. C. (1986). Confidence regions for parameter subsets in nonlinear regression, Biometrika, Vol. 73, 57-64.

5. Hamilton, D. C. and Wiens, D. (1987). Correction factors for F ratios in nonlinear regression, Biometrika, Vol. 74, 423-425.

6. Miller, F. R., Neill, J. W. and Sherfey, B. W. (1999), Implementation of a Maximin Power Clustering Criterion to Select Near Replicates for

Regression Lack of Fit Tests, Journal of the American Statistical Association, Vol. 94, 610-620.

7. Seber, G. A. F. (1977). Linear Regression Analysis, John Wiley & Sons:

New York.

8. Seber, G. A. F. and Wild, C. J. (1989). Nonlinear Regression, John Wiley & Sons: New York.

[ received date : Jun. 2004, accepted date : Aug. 2004 ]