Advanced Econometrics - Part I - Chapter 3: Stochastic Regression Model

Advanced Econometrics Chapter 3: Stochastic Regression Model Nam T. Hoang University of New England - Australia 1 University of Economics - HCMC - Vietnam Chapter 3 STOCHASTIC REGRESSION MODEL I. CONSISTENCY: 1. Definition: • Let nθˆ be a random variable. If for any 0>∀ε we have 0}0ˆlim{ =>−θθn then θ is probability limit of nθˆ . • If nθˆ is an estimator for θ , then nθˆ is said a consistent estimator of θ . notation: θθ = ∞→ nn p ˆlim Note: A sufficient condition for this to hold is if Bias ( θθ →)nˆ and Var( θθ →)nˆ when ∞→n 2. Cramer Theorem: If:     = = ∞→ ∞→ 0)ˆ(lim)( )ˆ(lim)( nn nn Varii Ei θ θθ then θθ = ∞→ nn p ˆlim θ )ˆ( nf θ )ˆ( 100θf )ˆ( 50θf )ˆ( 10θf Advanced Econometrics Chapter 3: Stochastic Regression Model Nam T. Hoang University of New England - Australia 2 University of Economics - HCMC - Vietnam Example: niNxi ,3,2,1),(~ 2 =σµ sample size. Get ∑ = = n i ixn X 1 1 →     = = µ σ )( )( 2 XE n XVar So     = = ∞→ ∞→ µ)(lim 0)(lim XE XVar n n then X is a consistent estimator of µ: µ= ∞→ )(lim Xp n Note: • If an estimator is "inconsistent", then it is a useless estimator (unreliable). • There are many situations where OLS estimator is inconsistent. Need to be clear with this. 3. Slutsky Theorem: Let F() be a continuous function, then: )]ˆ(lim),...,ˆ(lim),ˆ(lim[)ˆ,...,ˆ,ˆ(lim ,,2,1,,2,1 nknnnnnnknnn pppFFp θθθθθθ ∞→∞→∞→∞→ = EX: if Cp n =)ˆlim(θ → )()]ˆ(lim[ CFFp n =θ Cp n /1]ˆ/1lim[ =θ 33 ]ˆlim[ Cp n =θ Cn ep =)ˆlim[exp(θ ... )ˆlim().ˆlim()ˆ.ˆlim( ,2,1,1,1 nnnn ppp θθθθ = A and B are stochastic matrices: )lim().lim()lim( BpApABp = also 11 )lim()lim( −− = ApAp if A is non-singular. Advanced Econometrics Chapter 3: Stochastic Regression Model Nam T. Hoang University of New England - Australia 3 University of Economics - HCMC - Vietnam II. CLASSICAL STOCHASTIC REGRESSION MODEL: Now, consider the LS model, first under our standard assumption. However, we will relax some of them. • Don't need normality. • X can be random, just assume that {xi, εi} is a random & independent sequence. Model: (1) εβ += ×kn XY (2) X and ε are generated independently of each other and kXRank =)( . (3) E(ε|X) = 0 (4) E(εε'|X) = σ2I (5) X consists of stationary random variables with: XX k i kn i XXE Σ=      ′ ×× 1 and ∑ = ′=′ n i ii XXn pXX n p 1 1lim)1lim( = XXii XXE Σ=′)( (Because X now is random). Stationary random variable: Xi =                 ik i i x x x  3 2 1 First and second moments are constants: 11 ×× =      k x k iXE µ 1×k iX : the ith row of 1×k X = ith observation on all k variables Advanced Econometrics Chapter 3: Stochastic Regression Model Nam T. Hoang University of New England - Australia 4 University of Economics - HCMC - Vietnam ])'()([ 1 )( 11 3 2   k Xi k Xi ik i i i XXE x x x VarCovXVarCov ×× −−=                 = µµ = matrix of constants. XXΣ = population 2 nd moment matrix. XX n '1 = sample 2nd moment matrix. Recall:               = ∑∑∑∑ ∑∑ ∑∑∑∑∑∑ ∑∑∑ 2 32 232 2 22 32 ... ... ... ' ikiikiikik ikiiiii ikii XXXXXX XXXXXX XXXn XX  → ∑ = ′= n i ii XXXX 1 ' 1. Unbiasedness of OLSβˆ : εββ  random XXX ′′+= −1)(ˆ ( YXXX ′′= −1)(βˆ ) → ])/ ˆ([)ˆ(  XEEE X ββ ↓ = (law of iterated expectation). expectation of βˆ conditional on X. XE : Expectation over value of X. Advanced Econometrics Chapter 3: Stochastic Regression Model Nam T. Hoang University of New England - Australia 5 University of Economics - HCMC - Vietnam → }])({[) ˆ( 1 XXXXEEE X εββ ′′+= − }]){([ 1 XXXXEEX εβ ′′+= − ])(}.){([ 0 1  XEXXXXEEX εβ ′′+= − (by assumption 2: X & ε are independent). ]0.)([ 1 XXXEX ′′+= −β ββ =+= )]0(XE 2. VarCov of OLSβˆ : ]))ˆ(ˆ())ˆ(ˆ([)ˆ( ′ −−=  ββ βββββ EEEVarCov ])()[( 11 −− ′′′′= XXXXXXE εε }])(){([ 11 XXXXXXXEEX −− ′′′′= εε }])({}(}){([ 11 XXXXEXEXXXXEEX −− ′′′′= εε ])('.)[( 121 −− ′′′= XXXIXXXEX εσ IXXEX 21)( εσ −′= )ˆ(βE) ˆ( 1XXE =β )ˆ( 2XXE =β )ˆ( 3XXE =β Advanced Econometrics Chapter 3: Stochastic Regression Model Nam T. Hoang University of New England - Australia 6 University of Economics - HCMC - Vietnam 3. Consistency of OLSβˆ :              += − n X n XXpp εββ ''limˆlim 1 Note: εββ XXX ′′+= −1)(ˆ = n X n XX ε β '' 1−      +            += − n XpXX n pp εββ 'lim'1limˆlim 1 (by Slutsky theorem: plimf(x) = f(plimx))      Σ+= − n Xpp XX ε ββ 'limˆlim 1 Note:     =      ′=Σ=′ n XXpXX n pXXE iiXXii 'lim1lim)( Apply the Cramer theorem to ε'1 X n (i) 0'1lim =      ∞→ εX n E n Because:            =      XX n EEX n E X εε ' 1'1 ( )              =  0 '1 XEXX n EEX ε 1 00'.1 × =     = kX X n E (ii)       ∞→ ε'1lim X n CovVar n      = ∞→ n XX n E n 1''1lim εε ( )    = ∞→ 2 1''lim n XXXEEXn εε ( )    = ∞→ 2 2 1'lim n IXXEXn εσ Advanced Econometrics Chapter 3: Stochastic Regression Model Nam T. Hoang University of New England - Australia 7 University of Economics - HCMC - Vietnam ( )       = ∞→ 2 2 'lim n XXEXn εσ       ×= ∞→ nn XXEXn 2'lim εσ Note: XXXX n i iiX n i iiX nn XXE n XX n EXX n E XX Σ=Σ=        ′=      ′=      ∑∑ = Σ= ..1)(11'1 11  Then: =      × ∞→ nn XXEXn 2'lim εσ 01lim 2 =Σ ∞→ XXn n σ We have:       =      =      ∞→ ∞→ 0'1lim 0'1lim ε ε X n VarCov X n E n n → 0'1lim =      εX n p (Cramer's Theorem). →      Σ+= − n Xpp XX ε ββ 'limˆlim 1 ββ =Σ+= − 0.1XX → βˆ is a consistent estimator of β III. LIMITING DISTRIBUTIONS AND ASYMPTOTIC DISTRIBUTIONS: 1. Definition: Let zn be a random variable with probability distribution F(zn) and let z be another random variable with probability distribution F(z). If Fn(zn) converges to F(z) then F(z) is the limiting distribution of Fn(zn). Converges means: 0)()(lim =− ∞→ zFzF nnn Advanced Econometrics Chapter 3: Stochastic Regression Model Nam T. Hoang University of New England - Australia 8 University of Economics - HCMC - Vietnam Notation )()( zFzF dn → zz dn → or equivalently write: )(zFz d n → Example: )1,0(][ Nt d r → 2. Central limit theorem: If X1, X2, ... Xn is a random sample from some distribution with mean µ, variance σ2. Then: ),0()( 2σµ NXn d→− . 3. Proposition: Let wn be a random variable with plimwn = w and zn has limiting distribution of F(z). Then the limiting distribution of wnzn is equal to w.F(z) = plimwn.F(z). The asymptotic distribution of X is defined in terms of the limiting distribution of a related random variable )( µ−Xn , which has a non-degenerate limiting distribution ),0()( 2σµ NXn d→−       →− n NX d 2 ,0)( σµ is the asymptotic distribution of )( µ−X →       n NX a 2 ,~ σµ IV. ASYMPTOTIC DISTRIBUTION OF βˆ Recall: εββ XXX ′′+= −1)(ˆ ββ =)ˆ(E ββ =)ˆlim(p → consistency. 12 )'()ˆ( −= XXEVarCov Xεσβ 0)'( =εXE Advanced Econometrics Chapter 3: Stochastic Regression Model Nam T. Hoang University of New England - Australia 9 University of Economics - HCMC - Vietnam 12 )'()ˆ( −= XXEVarCov Xεσβ =      XX n E ''1 εε XXΣ 2 εσ Recall: ),0()( 2σµ NXn d→− →       n NX a 2 ,~ σµ ),0()ˆ( 1 kkk d n QNn ×× →−θθ →       Q n N a n 1,~ˆ θθ where Q n 1 : ( )nasyVarCov θˆ For βˆ : =           =− −Σ − nX n XX n n XXp εββ '1'1)ˆ( 1lim 1  Because: 0'1 =      εX n E (E(X'ε) = 0 =      n XX n E 1''1 εε XXXXn IE Σ=      22 '1 εε σσ Then by the central limit theorem:       ε'1 X n n ~ ),0( 2 1 kk XXk N ×× Σεσ → n i X n 1 '1 =       ε is a random sample from some distribution with mean 0 & variance kk XX × Σ2εσ Consider: ∑ = = n i iwX 1 'ε → wi = Xiεi = i ki i i X X X ε                  3 2 1 → 0)()()()( 0 ===  iiiXiii XEXEXEwE X εε µ Advanced Econometrics Chapter 3: Stochastic Regression Model Nam T. Hoang University of New England - Australia 10 University of Economics - HCMC - Vietnam =′= )()( iiiii XXEwVarCov εε ( ) XXXXE Σ= 22 ' εε σσ Then by the central limit theorem:       ε'1 X n n = ~01 11       − × = ∑ k n i iwn n ),0( 2 1 kk XXk N ×× Σεσ Then: nX n XX n n            =− − εββ '1'1)ˆ( 1 1−Σ→ XX d ),0( 2 1 kk XXk N ×× Σεσ →d )',0( 211 1 ε σ−− × ΣΣΣ XX I XXXXk N  ( XXΣ symmetric). Note: Z ~ ),0( XXN Σ W = cZ → w ~ )',0( ccN XXΣ XXΣ symmetric )( γµβµγ X+→ So: →− dn )ˆ( ββ ),0( 21 εσ −Σ XXN a ~βˆ ),( )ˆ( 2 1  β εσβ asyVarCov XX n N −Σ nXX 2 1 εσ−Σ = 1 '1lim −       XX n p n 2 εσ = ( ) 1 'lim −     XXpn n 2 εσ = )'( XXE 2εσ Note: XXΣ = )( iiX XXE ′ ( ) kk XX n i XX n i ii n i ii nXXEXXEXXE ×=== Σ=Σ=      ′=      ′= ∑∑∑ 111 )(' == − 21)'()( εσXXEwVarCov i 12 ][ −Σ XXnεσ = nXX 2 1 εσ−Σ Advanced Econometrics Chapter 3: Stochastic Regression Model Nam T. Hoang University of New England - Australia 11 University of Economics - HCMC - Vietnam Remember: n XXEXXE iiXX 1)'()( =′=Σ → XXnXXE Σ=)'( → The more observations we have, the smaller variance of βˆ are.