Advanced Econometrics - Chapter 7: Generalized Linear Regression Model

Advanced Econometrics Chapter 7: Generalized Linear Regression Model Nam T. Hoang University of New England - Australia 1 University of Economics - HCMC - Vietnam Chapter 7 GENERALIZED LINEAR REGRESSION MODEL I. MODEL: Our basic model: Y = X.β + ε with ],0[~ 2 IN σε We will now generalize the specification of the error term. E(ε) = 0, E(εε') = Σ=Ω×nn 2σ . This allows for one or both of: 1. Heteroskedasticity. 2. Autocorrelation. The model now is: (1) εβ += ×knXY (2) X is non-stochastic and kXRank =)( . (3) E(ε) = 1 0 ×n (4) E(εε') = nn× Σ nn× Ω= 2εσ Heteroskedasticity case:             =Σ 2 2 2 2 1 00 00 00 nσ σ σ     Advanced Econometrics Chapter 7: Generalized Linear Regression Model Nam T. Hoang University of New England - Australia 2 University of Economics - HCMC - Vietnam Autocorrelation case:             =Σ −− − − 1 1 1 21 21 11 2     nn n n ρρ ρρ ρρ σε ),( itti Corr −= εερ = correlation between errors that are i periods apart. II. PROPERTIES OF OLS ESTIMATORS: 1. YXXX ′′= −1)(βˆ = )()( 1 εβ +′′ − XXXX εββ XXX ′′+= −1)(ˆ βεββ =′′+= − )()()ˆ( 1 EXXXE βˆ is still an unbiased estimator 2. )ˆ(βVarCov = ])'ˆ)(ˆ[( ββββ −−E = ])'))(()[( 11 εε XXXXXXE ′′′′ −− = ])(')[( 11 −− ′′′ XXXXXXE εε = 11 )()'()( −− ′′′ XXXEXXX εε = 121 )()()( −− ′Ω′′ XXXXXX σ 12 )( −′≠ XXσ so standard formula for βσ ˆˆ no longer holds and βσ ˆˆ is a biased estimator of true βσ ˆˆ . → βˆ ~ N[β, 112 )())( −− ′Ω′′ XXXXXXσ ] so the usual OLS output will be misleading, the std error, t-statistics, etc will be based on 12 )'(ˆ −XXεσ not on the correct formula. 3. OLS estimators are no longer best (inefficient). Advanced Econometrics Chapter 7: Generalized Linear Regression Model Nam T. Hoang University of New England - Australia 3 University of Economics - HCMC - Vietnam Note: for non-stochastic X, we care about the efficient of βˆ . Because we know if n↑ → Var( jβˆ ) ↓ → plim βˆ = β, βˆ is consistent. 4. If X is stochastic: - OLS estimators are still consistent (when E(ε|X) = 0. - IV estimators are still consistent (when E(ε|X) ≠ 0). - The usual covariance matrix estimator of VarCov( βˆ ) which is 12 )'(ˆ −XXεσ will be inconsistent (n →∞) for the true VarCov( βˆ ). We need to know how to deal with these issues. This will lead us to some generalized estimator. III. WHITE'S HETEROSCEDASCITY CONSISTENT ESTIMATOR OF VarCov( βˆ ). (Or Robust estimator of VarCov( βˆ ) If we knew σ2Ω then the estimator of the VarCov( βˆ ) would be: V = 121 )()()( −− ′Ω′′ XXXXXX σ = 1 2 1 1)(111 −−       ′    Ω′      ′ XX n XX n XX nn σ = 11 1)111 −−       ′    Σ′      ′ XX n XX n XX nn If Σ is unknown, we need a consistent estimator of     Σ′ XX n )1 (Note that the number of unknowns is Σ grows one-for-one with n, but [ ]XX )Σ′ is k×k matrix it does not grow with n). Let: XX n Σ′=Σ 1* ∑∑ = = ×× ′=Σ n i n j k j k iij XXn 1 1 11 * 1 σ Advanced Econometrics Chapter 7: Generalized Linear Regression Model Nam T. Hoang University of New England - Australia 4 University of Economics - HCMC - Vietnam In the case of heteroskedasticity             =Σ 2 2 2 2 1 00 00 00 nσ σ σ     ∑ = ′=Σ n i iii XXn 1 2* 1 σ White (1980) showed that if: ∑ = ′=Σ n i iii XXen 1 2 0 1 then plim(Σ0) = plim(Σ*) so we can estimate by OLS and then a consistent estimator of V will be: 1 1 2 1 1111ˆ − = −       ′      ′      ′= ∑ XXnXXenXXnnV n i iii ( ) ( ) 101ˆ −− ′Σ′= XXXXnV Vˆ is consistent estimator for V, so White's estimator for VarCov( βˆ ) is: ( ) ( ) VXXXXXXVarCov ˆˆ')ˆ( 11 =′Σ′= −−β where:             =Σ 2 2 2 2 1 00 00 00 ˆ ne e e     (Note )ˆ10 Σ=Σ n Vˆ is consistent for ( ) ( ) 121 −− ′Ω′= XXXXnV σ regardless of the (unknown) form of the heteroskedasticity (only for heteroskedasticity). Newey - West produced a corresponding consistent estimator of V when there is autocorrelation and/or heteroskedasticity. Note that White's estimator is only for the case of heteroskedasticity and autocorrelation. White's estimator just modifies the covariance matrix estimator, not βˆ . The t-statistics, F-statistics, etc will be modified, but only in a manner that is appropriate asymptotically. So if we have heteroskedasticity or autocorrelation, whether we modify the covariance matrix estimator or not, the usual t-statistics will be unreliable in finite samples (the Advanced Econometrics Chapter 7: Generalized Linear Regression Model Nam T. Hoang University of New England - Australia 5 University of Economics - HCMC - Vietnam white's estimator of VarCov( βˆ ) only useful when n is very large, n → ∞ the →Vˆ VarCov( βˆ ). → βˆ is still inefficient. → To obtain efficient estimators, use generalized lest squares - GLS A good practical solution is to use White's adjustment, then use Wald test, rather than the F-test for exact linear restrictions. Now let's turn to the estimation of β, taking account of the full process for the error term. IV. GENERALIZED LEAST SQUARES ESTIMATION (GLS): OLS estimator will be inefficient in finite samples. 1. Assume E(εε') = nn×Σ is known, positive definite. → there exists 1×n jC and 1×n jλ j = 1,2, ... ,n such that nn× Σ 1×n jC = 1×n jC 1×n jλ (characteristic vector C, Eigen-value λ). → before C'ΣC = Λ where [ ]nn CCCC 211 =×             =Λ nλ λ λ     00 00 00 2 1               =Λ nλ λ λ     00 00 00 2 1 2/1 )()'(' 2/12/1 ΛΛ=Λ=ΣCC → =ΛΣΛ −−  ' 2/1 ' 2/1 )(')( HH CC I=ΛΛΛΛ −− ))()()(( 2/12/12/12/1 → IHH =Σ ' → 1111 '' −−−− ==Σ HHIHH → HH '=Σ '2/1 CH −Λ= Advanced Econometrics Chapter 7: Generalized Linear Regression Model Nam T. Hoang University of New England - Australia 6 University of Economics - HCMC - Vietnam Our model: εβ += XY Pre-multiply by H:    *** ε εβ HHXHY XY += → *** εβ += XY ε* will satisfy all classical assumption because: E(ε*ε*') = E[H(εε')H'] = HΣH' = I. Since transformed model meets classical assumptions, application of OLS to (Y*, X*) data yields BLUE. → **1** ')'(ˆ YXXXGLS −=β →  YHHXXHHX 11 '')''( 1 −− Σ − Σ = → YXXXGLS 111 ')'(ˆ −−− ΣΣ=β Moreover: [ ] [ ]1******1** )'()'(')'()ˆ( −−= XXXEXXXVarCov GLS εεβ 1** )'( −= XX 11 )'( −−Σ= XX =) ˆ( GLSVarCov β 11 )'( −−Σ XX → [ ])'(,~ˆ 1 XXNGLS −Σββ Note that: GLSβˆ is BLUE of βˆ → ββ =) ˆ( GLSE GLS estimator is just OLS, applied to the transformed model → satisfy all assumptions. Gauss - Markov theorem can be applied → GLSβˆ is BLUE of βˆ . → OLSβˆ must be inefficient in this case. → ) ˆ()ˆ( OLSjGLSj VarVar ββ ≤ . Advanced Econometrics Chapter 7: Generalized Linear Regression Model Nam T. Hoang University of New England - Australia 7 University of Economics - HCMC - Vietnam Example:             =Σ 2 2 2 2 1 00 00 00 n known σ σ σ                 =Σ→ − 2 2 2 2 1 1 /100 0/10 00/1 nσ σ σ     H'H = Σ-1             =→ 2 2 2 2 1 /100 0/10 00/1 n H σ σ σ     * 2 2 22 2 11 2 1 2 2 2 2 1 / / / /100 0/10 00/1 Y Y Y Y Y Y Y HY nnnn =             =                         = σ σ σ σ σ σ                  == nnknnn k k XX XX XX HXX σσσ σσσ σσσ ///1 ///1 ///1 2 222222 111121 *     Transformed model has each observations divided by σi:       +      ++      +      =      i i i ik k i i ii i XXY σ ε σ β σ β σ β σ 221 1 Apply OLS to this transformed equation → "Weighted Least Squares": Let: βˆ = GSL estimator. GLSXY βε ˆˆ ** −= kn − = εεσ ˆ'ˆˆ Then to test: H0: Rβ = q (F Wald test). ),(2 11** ~ ˆ ]ˆ[]')'([]'ˆ[ knr r kn Fr qRRXXRqR F − −− − −− = σ ββ if H0 is true. Advanced Econometrics Chapter 7: Generalized Linear Regression Model Nam T. Hoang University of New England - Australia 8 University of Economics - HCMC - Vietnam and )( ˆ'ˆ ]ˆ'ˆˆˆ[ kn rF cc r kn − −′ =− εε εεεε where: GLScc XY βε ˆˆ ** −= ) ˆ(]')'([')'(ˆˆ 11111 qRRXXRRXX GLSGLSGLSc −ΣΣ−= −−−−− βββ is the "constrained" GLS estimator of β. 2. Feasible GLS estimation: In practice, of course, Σ is usually unknown, and so βˆ cannot be constructed, it is not feasible. The obvious solution is to estimate Σ, using some Σˆ then construct: YXXXGLS 111 ˆ')ˆ'(ˆ −−− ΣΣ=β A practical issue: Σ is an (n×n), it has n(n+1)/2 distinct parameters, allowing for symmetry. But we only have "n" observations → need to constraint Σ. Typically Σ = Σ(θ) where θ contain a small number of parameters. Ex: Heteroskedasticity var(εi) = σ2(θ1+θ2Zi).             + + + =Σ n n z z z 21 21 121 00 00 00 θθ θθ θθ     just 2 parameters to be estimated to form Σˆ . Serial correlation: )( 1 1 1 2 2 1 1 1 ρ ρρ ρρ ρρ Σ=             =Σ − − − −     n n n n only one parameter to be estimated. • If Σˆ is consistent for Σ then β will be asymptotically efficient for β. • Of course to apply β we want to know the form of Σ → construct tests.