Estimation of the Number of Change-Points with Local Linear Fit

2 002 , V ol. 13, N o.2 p p . 251～2 60

E s t im at ion o f t h e N um b e r o f Ch an g e - P oin t s w ith Lo c al Lin e ar F it^{1 )}

Jon g T ae K im^{2 )}・H ey m i Ch oi^{3 )}

A b s tra c t

T h e aim of t his pap er is t o con sider of det ect in g t h e location , th e jum p size an d th e n um b er of ch an g e - poin t s in r eg r es sion fun ct ion s by u sin g t h e local lin ear fit w h ich is on e of n onparam etric r egr es sion t echn iqu e s . It is obt ain ed t h e a sy m pt ot ic propert ie s of t h e ch an g e poin t s an d th e ju m p sizes . an d t h e corr espon din g r at e s of conv erg en ce for ch an g e - p oint est im at or s .

K e y w o rd s : Local lin ear r eg r es sion fit ; Ch an g e - poin t ; K ern el sm ooth er .

1 . 서론

N onp ar am et ric r eg r es sion t ech niqu es ar e g en er ally u sed in or der t o obt ain a sm ooth fit of r eg re s sion fun ction w h en ev er th er e is n o su it able p ar am etric m odel av ailable. S om etim es a g en er ally sm oot h fu n ct ion m ig ht con t ain som e is olat ed discont in uity or m ult iple ch an g e p oint s in t h e fun ct ion or in a deriv at iv e, an d in m an y ca ses int er est focu s es on t h e occurr en ce of su ch ch an g e poin t s . T h e an aly sis of ch an g e poin t s u su ally occu rrin g in econ om ics , en gin eerin g m edicin e an d b iological s cien ces h a s r ecen t ly fou n d in cr ea sin g in t er est .

T h e purp ose of t his paper is t o obt ain a sy m pt ot ic dist ribu tion s an d corr esp on din g r at es of conv erg en ce for ch an g e - poin t estim at or s . T h is r esu lt s is v ery im port an t t o su g g est t h e m et h ods for t est in g an d est im at ion t o det ect th e location an d size of ch an g e - point s in r egr es sion fu n ct ion by u sin g th e local lin ear fit

1. T his paper w as support ed by r es earch fund, T aegu Univer sity , 2002.

2. A s s ociat e Pr ofes s or , Dept of St atistics , T aegu Univ er sit y , 712- 714, Kor ea.

E - m ail: jtkim @daegu .ac.kr

3. St atist ics Depart m ent Cas e W es t ern Res erv e univ er s ity 10900 Euclid Av enue, Clev eland, ohio 44106- 7054, USA ,

E - m ail : heym ichoi @y ahoo.com

In t h e lit er at ur e, t estin g an d est im ation ab out ch an g es in t h e n onp ar am etric r eg r es sion fun ction h a s b een stu died by m any au th or s . M u ller (1992) g av e e st im at or s for locat ion an d size of ch an g e - poin t s in n on par am et ric r egr es sion b a sed on a com parison of a left an d righ t on e - sided k ern el sm ooth er s . Loar d (1996) pr oposed an estim at e of t h e locat ion of discon tinu it y b a sed on on e - side n on par am et ric r eg r es sion est im at es of th e m ean fun ct ion . Ch en an d Gupt a (1997) stu died t est in g an d locat in g v arian ce ch an g e point s w it h applicat ion t o st ock price u sin g th e S ch w ar z in form ation crit erion .

2 . E s tim at e s o f lo c ati on an d ju m p s iz e o f c h an g e po in t s

T h e n onp ar am etric r egr es sion m odel con sidered in t his paper is giv en by

Yi= m ( xi ) + i, xi [ 0 , 1] , i = 1, 2 , . . . , n , (2.1) w h er e x_i are fix ed design p oint s an d _i ar e iid error s w it h m ean 0 an d v arian ce

2 < . T h e design p oint s xi ar e a s sum ed t o b e equidist an t , i. e. xi= i / n . m is t h e u nkn ow n regr es sion fun ct ion defin ed on t h e in t erv al [ 0 , 1] .

T h e r eg re s sion est im at or s w e con sider ar e b a sed on t h e local lea st squ are s fitt in g of k ern el w eight ed p oly n om ial r egr es sion fun ction . T h e locally w eigh t ed p oly n om ial r egr es sion est im at or of m is ₀ , t h e solut ion for ₀ t o m in im ize t h e k ern el w eig ht ed local- lik elih ood fun ct ion

n

i = 1{^{Yi -} l = 0^p l( xi - x ) }^{2 K} (^xi h^{- x} )^{. (2.2)}

A s su m e th at s om e ch an g e - point s ex ist for m in th e follow in g sen se. T h er e ex ist s g C²( [ 0 , 1]) su ch t h at

m ( ) = g ( ) +

n c

i = 1 i1 _{( ci)}, (2.3)

w h er e ci, n c ar e un kn ow n an d g ( ) h a s a b oun ded secon d deriv ativ e.

Let K ₊ an d K _- b e on e - sided k ern el fu n ction s w it h su pport ( K₊) [ - 1 , 0 ] an d support ( K _- ) [ 0 , 1] . F or sim plicit y , let

l( F ) =

- y^lF ( y ) dy . T h e k ern el K - ( ) is ( - 1) tim es different iab le on R an d K -^{( - 1)} is ab solu t ely cont inu ou s den sit y fu n ction sat isfy in g

1( K ) = ₃ ( K ) = 0 , ₂ ( K ) > 0 , K _-^{( j )}( 0) = K _-^{( j )}( 1) = 0 , 0 j < , an d K ₊( x ) = K_- ( - x ) . Defin e m₊^{( j )}( x₀) = lim

x x₀m ^{( j )}( x ) , m_-^{( j )}( x₀) = lim

x x₀m ^{( j )}( x ) , j = 0 , 1 , 2 .

( x₀) = m₊( x₀ ) - m_- ( x₀ ) . (2.4)

T h en t h e follow in g h old s , for any x0 [ 0 , 1] ,

m₊( x₀) = m ( x₀) , m ^{( j )}= g ^{( j )}( x₀) , j = 1 , 2 .

( x0) = {^{0i if x}ot h erw is .^{0= ci,} D efin e on e - sided regr es sion estim at e s of m ( x ) ,

m ( x ) = Wj Yj

W_j , (2.5)

w h er e

W_j = (^{x - x}h ^j )^{( S n , 2- ( x - xj ) S n , 1) ,}

S _{n l}= K (^{x - x}h ^j )^{( x - xj )l}, l = 0 , 1 , 2 . In fer en ce for ch an g e poin t s w ill b e b a s ed on t h e follow in g estim at es

( x ) = m₊ ( x ) - m_- ( x ) . (2.6)

w h er e xj is called t o b e a ch an g e poin t if | ( xj) > C , j = 1 , 2 , , n , for s om e con st an t C at a g iv en sig nifican t lev el 1 - w h er e C w ill b e g iv en b elow .

Ob s erv e t h e follow in g b efor e inv e st ig atin g bia s an d v arian ce of ( x ) . P rop o s it i on 2 .1 Defin e S _{n l}= K (^{x - x}h ^j )^{( x - xj )l}, l = 0 , 1 , 2 . .

( 1) S n l = n h^{l + 1}( K ( u ) u^ldy + O ( 1/ n ) ) = n h ^{l + 1}[ l ( K - ) + O ( 1/ n ) ] .

( 2) Wj = S n , 0S n , 2 - ( S n , 1)²= n²h⁴ [ 2( K- ) + O ( 1/ n ) ] .

( 3) ( Wj )²= ( S n , 2)² K² (^{x - x}h ^j )^{+ ( Sn , 1)2 K2} (^{x - x}h ^j )^{( x - xj)2}

- 2 S _{n , 1}S _{n , 2} K² (^{x - x}h ^j )^{( x - xj)}

= n³h⁷ [ ²₀( K ² ) + O ( 1/ n ) ] .

N ow , w e con sider t h e bia s an d v arian ce of ( x₀) , x₀ [ h , 1 - h ] in t h e follow in g .

P rop o s it i on 2 .2 A s sum e t h at th e jum p size is ( x₀) at poin t x₀.

(1) Bia s : E ( ^{( x}0) - ( x₀) ) ^{= o ( h2) .}

(2) V arian ce : V ar ( ( x0 ) ) = 2 ²

n h ( 0( K-²) + o( 1) ) .

P r o of (1 ).

E [ ^{( x}0 ) - ( x0) ]

= E [ ^m+ ( x0) - m - ( x0) ]^{- E [ m}+( x0) - m-( x0) ]

= E [ ^m+ ( x0) - m +( x0) ]^{- E} [ ^m- ( x0) - m- ( x0) ]

= E W_j⁺( Y_j- m₊( x₀ ) )

W_j⁺ - E W_j^- ( Y_j- m_- ( x₀) ) W_j^-

= 1

n²h⁴ [ ₂( K _- ) + O( 1/ n ) ] [ ^{{m ( x}j ) - m₊( x₀) }W_j⁺ - {m ( x_j ) - m_- ( x₀) }W_j^- ]

b y P r op osit ion 2.1- (2). Let n ow b e

R ( x_j) = m ( x_j ) - m ( x₀ ) - ( x_j- x₀ ) m ^' ( x₀ ) . S in ce

( x_j- x₀) W_j = ( x_j - x₀) K (^{x - x}h ^j )^{( S n , 2 - ( x - xj) S n , 1)}

= - S _{n , 2}S _{n , 1} + S _{n , 2}S _{n , 1}= 0 ,

{m ( x_j) - m ( x₀) } w_j = {^{m ( x}j) - m ( x₀) - ( x_j- x₀) m ^' ( x₀) }^wj

an d h en ce w e h av e 1

n R₊( x_j ) K ₊(^{x0- x}h ^j )

= {m ( x ) - m₊( x₀) - ( x - x₀) m₊^'( x )}^K +(^x0h^{- x} )dx + O( 1/ n )

= h

0

- 1{^{m ( x}0- h u ) - m₊( x₀) + h u m₊^' ( x₀)}^K +( u ) d u + O( 1/ n )

= h

0

- 1 ( m₊( x₀) - h u m₊^' ( x₀) + h²u²

2 m₊^{' '}( x₀) - m₊( x₀) + h u m₊^' ( x₀) + o( h²u²) ) K ₊( u ) d u + O( 1/ n )

= h³

2 m₊^{' '}( x₀) ₂( K _- ) + o( h³) + O( 1/ n ) .

1

n R₊( x_j ) ( x₀- x_j ) K ₊(^{x0- x}h ^j )

= {m ( x ) - m+( x0) - ( x - x0 ) m+^'( x ) }^{( x}0- xj ) K +(^x0h^{- x} )dx + O( 1/ n )

= h

0

- 1 {^{m ( x}0- h u ) - m+( x0) + h u m+^'( x0)}^{h u K} +( u ) d u + O( 1/ n )

= h²

0

- 1( m+( x0) - h u m+^'( x0) + h²u²

2 m+^{' '}( x0) - m+ ( x0) + h u m+^'( x0) + o( h²u²) ) u K +( u ) d u + O( 1/ n )

= h⁴

2 m₊^{' '}( x₀) ₃( K ₊) + o( h⁴) + O( 1/ n ) .

T hu s it follow s fr om P r oposit ion 2.1- (1), (2) an d (3 ) t h at ( m ( x_j ) - m₊( x₀) ) W_j⁺ = R₊( x_j) W_j⁺ = n²h⁶

2 [^m +^{' '}( x₀) ²₂( K _- ) + o( 1) ]^.

S im ilarly , ( m ( xj) - m- ( x0) ) Wj^- = n²h⁶

2 [ m+^{' '}( x0) ²2( K- ) + o( 1) ] . T h er efor e

E ( ^{( x}0 ) - ( x0 ) ) ^{= 1}

n²h⁴ { ₂( K _- ) + O ( 1/ n ) }( R+( xj ) Wj⁺ - R- ( xj) Wj^- )

= o( h²) .

P r o of (2 ). F ir st , n ot e t h at C ov ( m+ ( x0) , m- ( x0) ) = 0 sin ce j' s ar e in dep en den t an d ( j : - 1 ( x₀- x_j ) / h < 0 ) an d ( j : 0 ( x₀- x_j ) / h < 1 ) are disj oin t . T hu s , V ar ( ( x0) ) = V ar ( m+ ( x0) ) + V ar ( m- ( x0) ) . S in ce it n ow follow s fr om P r op osit ion 2.1- (2) an d (3 ) th at

V ar ( m ( x₀) ) =

2 ( W_j )²

( Wj )² =

2

n h [ ₀( K ² ) + o( 1) ] , V ar ( ( x₀) ) = 2 ²

n h ( ₀( K _-²) + o( 1) ) .

3 . M ain Re s ult s

In t his s ect ion , w e con sider t h e a sy m pt ot ic dist ribu t ion s an d corr espon din g rat es of con v er g en ce for ch an g e - point est im at or s .

L e m m a 3 .1 . F or fix ed x0 ( 0 , 1) a s n , h 0 in su ch a w ay th at n h

C K _n ( x₀ ) ( x₀ ) - ( x₀ )

n V_n N ( 0 , 1) , (3.1)

w h er e _n is a con sist en t e st im at e of an d

Vn = ( Wj^- )²/ ( Wj^- )² + ( Wj^- )² / ( Wj^- )². In addition ,

n h ( ^{( x}0 ) - ( x₀ ) )

n 2 ₀( K _-²)

d N ( 0 , 1) . (3.2)

P r o of . F or sim plicit y , let a_j den ot e a_j = W_j⁺/ W_j⁺ - W_j^- / W_j^- . T h en

j aj²= Vn an d h en ce C Kn can b e r ew rit t en a s

j aj j / n

2

j aj². S in ce it is sh ow n th at _n^{2 2}, it is su fficien t t o pr ov e (3.1) w ith r espect t o CK_n w ith in place of _n . It follow s , in P r oposition 2.1- (2) an d (3 ), t h at m ax 1 j n

a_j² aj²

= O( 1

n h ) an d h en ce Lin db er g ' s con dit ion is s at isfied. T h erefor e

a sy m pt ot ic n orm alit y of CK _n follow s . S in ce

aj² = 2 0( K-²) / ( n h) + o( 1/ ( n h) ) , th e equ at ion (3.2) follow s .

Let den ot e a locat ion of a ch an g e poin t of m ( x ) . A s sum e t h at w e kn ow A is a clos ed n eig hb orh ood of in w hich is on ly on e ch an g e point of m ( x ) . D efin e t h e e st im at or

= in f{ ^{A : ( ) = su p} x A ( x ) }

for t h e locat ion of a ch an g e p oint . N ow , w e in v est ig at e lim itin g pr opert ies of . In or der t o do th at defin e

( y ) = ( + y h ) = m₊ ( + y h ) - m_- ( + y h )

an d defin e for som e 0 < M < , - M z M , th e sequ en ce of st och a stic pr oces ses

n( z ) = ( n h ) ⁽ + 1) / ( 2 )

( (( n h )^{z1/ ( 2 )} )^{- ( 0 )})^.

N ot e t h at

K (^{x - x}h ^j )^{m ( xj ) Sn , 2 = n2h4 2( K - ) K} ( u ) m ( x - h u ) d u + O( n h⁴) , K (^{x - x}h ^j )^{( x - xj ) m ( xj ) S n , 1 = O ( n h4 ) .}

T hu s w e h av e E { m ( + y h) } = K ( u ) m ( + y h - u h) d u + O ( 1/ n ) an d h en ce

E { ( y ) } =

1

- 1( K ₊( u ) - K _- ( u ) ) g ( + y h - u h) d u + O( 1/ n ) ,

w h ich is t h e sam e a s t h e equ ation (6.2) of M uller (1992). T h er efor e follow in g t h e s am e t echn iqu e s u s ed b elow (6.2) of Lem m a 6.1 in M u ller giv es

E _n( z ) = - z ^{+ 1}K _- ( 0) / ( + 1) ! + o( 1) .

N ow , w e con sider cov arian ce of n( z ) . F ir st , let = ( + 1) / ( 2 ) , = 1/ ( 2 ) ,

S n , l( z ) = K ( + ( z h) / ( n h) - xj

h )⁽ + ( z h) / ( n h) - xj )^l, l = 0 , 1 , 2 . an d

W_{j , z} = K ( + ( z h) / ( n h) - xj

h )^{{S n , 2( z ) - (} + ( z h) / ( n h) - x_j ) ^S n , 1( z )}^.

T h en

n( z ) - E { _n( z ) } = ( n h)

j

W⁺_{j , z}

j W⁺_{j , z} - W⁺_{j , 0}

j W⁺_{j , 0} ^j-

j

W^-_{j , z}

j W^-_{j , z} - W^-_{j , 0}

j W^-_{j , 0} ^j .

Ob serv e n ow for an y z [ - M , M ]

S n , l( z ) = n h^{l + 1}[ l( K - ) + O ( 1/ n ) ] , wj , z = n²h⁴[ 2( K- ) + O( 1/ n ) ] .

T hu s w e h av e

C ov ( n ( z1) , n( z2) )

= ( n h)^{2 2}

n⁴h⁸ 2( K - )^{2 j} [^{( W+j , z}1- W⁺j , 0) ( W⁺j , z₂- W⁺j , 0) - ( W⁺j , z₁- W⁺j , 0)( W^-j , z₂- W^-j , 0) - ( W^-j , z₁- W^-j , 0) ( W⁺j , z₂- W⁺j , 0) + ( W^-j , z₁- W^-j , 0) ( W^-j , z₂- W^-j , 0)]

= ( n h)² n h²

2{ {[^{K +}( ^{+ ( z1h) / ( n h)h - x} )^{- K +}( ^{- x}h )] [^{K +}( ^{+ ( z2h) / ( n h)h - x} )^{- K +}( ^{- x}h )]

- [^{K +}( ^{+ ( z1}h) / ( n h) - x

h )^{- K+}( ^{- x}h )]

[^{K -} ( ^{+ ( z2}h) / ( n h) - x

h )^{- K-} ( ^{- x}h )]

+ [^{K -} ( ^{+ ( z1}h) / ( n h) - x

h )^{- K +}( ^{- x}h )]

[^{K -} ( ^{+ ( z2}h) / ( n h) - x

h )^{- K -} ( ^{- x}h )]}dx + O( 1/ n )}^.

By th e a s su m pt ion s for K , th en K ( + ( z h) / ( n h) - x

h )^{- K} ( ^{- x}h )

= K ^' ( ^{- x}h ) ( n h)^z + O(( n h)^{1 2} ) ¹ _{^{K '}₍ ^{- x}_{h )}⁰_}{^{K '}( + ( z h ) / ( n h ) - x

h )⁰}

S i n c e 1

{^K' (^{- xh} )⁰}^{K '}(+ ( z h ) / ( n h ) - x

h )⁰}dx = O ( h) . K ^'2( ^{- x}h )^{dx = h K -'2}( u ) d u , an d K ₊^' ( ^{- x}h )^{K -'}( ^{- x}h )dx = 0 , t h en

C ov ( n( z1) , n( z2) ) = 2z1z2

2 K -^'2( u ) d u + O ( 1/ ( n h) ) . (3.3) Let

a_j = ( n h) W⁺j , z

j W⁺_{j , z} - W⁺j , 0

j W⁺_{j , 0} - W^-j , z

j W^-_{j , z} - W^-j , 0

j W^-_{j , 0} .

T h en _n ( z ) - E _n( z ) = a_{j j}. S in ce m ax _{1 j n} a_j²/ a_j² 0 ,

n( z ) - E _n( z ) sat isfies Lin db erg ' s con dit ion . T hu s w e h av e a sy m pt ot ic n orm alit y of _n( z ) a s follow s .

L e m m a 3 .2 . F or fix ed z [ - M , M ] ,

n( z ) - E _n ( z ) d

N (^{0 , 2 z2 2 K -'2( u ) d u})^.

F or fix ed z1 ,z2 , , zl [ - M , M ] ,

( _n( z₁) - E _n( z₁) , , _n( z_l) - E _n( z_l) ) d

N ( 0 , ) , w h er e = ( i , j ) an d i , j = 2 z1z2

2 K -^'2( u ) d u .

By follow in g t h e sim ilar lin es t o t h e pr oof of Lem m a 6.5 in M u ller (1992), it can b e sh ow n t h at th e sequ en ce _n = _n( z ) - E _n( z ) is t ig ht . T h is an d Lem m a 3.2 t og et h er im ply t h e follow in g .

T h e o re m 3 .1 F or fix ed z [ - M , M ] ,

n

w on C ( [ - M , M ]) , (3.4)

w h er e is a cont in u ou s Gau s sian pr oces s w it h

E ( z ) = - z ^{+ 1}K _- ( 0) / ( + 1) .

C ov ( ( z₁) , ( z₂) ) = 2 z₁z₂ ² K _-^'2( u ) d u . N ot e t h at ( z ) in ab ov e th eor em can b e w rit t en equ iv alen tly a s

( z ) = - z ^{+ 1}K _- ( 0) / ( + 1) ! + X z , w h er e X N ( 0 , ² K _-^'2( u ) d u ) an d h a s a u niqu e m ax im u m at

Z^*= [ x ! / ( ( ) K _- ( 0) ) ]^1/ . Let Z_n b e t h e location of th e m ax im um of _n. T h en

= + Zn h/ ( n h) ^{1/ ( 2 )}. S in ce Zn

d Z ^* by T h eor em 3.1 th e a sy m pt otic n orm ality of follow s . C o ro ll ary 3 .1

n h ( h^- ) ^{d N} (^{0 , 2 2} ( ^{( ) K!}-^{( )}( 0) )^{2 K -'2( u ) d u})^{. (3.6)}

A pply in g t h e fun ct ion al m appin g th eor em giv es t h at _n ( Z_n) d

( Z ^*) an d h en ce ( n h) ^{1/ 2} _n( Z_n) / ( n h) ⁽ + 1) / ( 2 )

0 . T his im plies

( n h) ^{1/ 2} { ( ) - ( ) } ^p 0 by t h e defin it ion s of n( ) , Zn, ( ) an d . M or eov er sin ce n h n Vn

p K , for s om e finit e con st ant K ,

{ ( ) - ( ) }/ _n V_n p

0 . T h erefor e com binin g t his w it h Lem m a 3.1, w e h av e t h e a sy m pt ot ic n orm alit y of ( ) .

C o ro ll ary 3 .2

C K ^*_n ( ) = ( ) - ( )

n V_n

d N ( 0 , 1) . (3.7)

A sy m pt ot ic 100 ( 1 - ) % confiden ce in t erv als of an d ( ) are

h [ ^{- 1}( 1 - / 2) ! _n / ( ( ) K_- ( 0) ) ]^1/ [ 2 K -^'2( u ) d u / ( n h) ]^{1/ ( 2 )}, (3.8 )

( ) ^{- 1}( 1 - / 2) _n V_n. . (3.9)

By Cor ollary 3.2. th e nu m b er of ch an g e p oint s is equ iv alen t t o t h e n um b er of t h e clu st er s of th e closed n eig hb orh ood s atisfied a s

C K ^*_n ( ) > ^{- 1}( 1 - / 2) .

Re f e ren c e

1. Ch en , J . an d Gupt a , A . K . (1997 ). T est in g an d Locatin g V arian ce Ch an g epoin t s W ith A pplicat ion t o S t ock P rice, J ournal of A m er ican S ta tis tical A s s ocia tion , 92, 739 - 747.

2. J ose, C. T . an d Ism ail, B . (1999 ). Ch an g e p oint s in n onp ar am etric r eg r es sion fu n ct ion s , Com m un ica t ion in S ta tis tics - T he ory an d M e th od , 28, 1883 - 1902.

3. Loader , C. (1996 ). Ch an g e p oint E st im at ion u sin g n on par am et ric r egr es sion , A nnals of S ta tis tics , 24, 1667 - 1678.

4. M u ller , H . G. (1992). Ch an g e p oint s in n onparam etric r egr es sion an aly sis , A nnals s ta t is tics , 20, 737 - 761.

[ 2002년 9월 접수, 2002년 10월 채택 ]