Advanced Econometrics - Chapter 13: Generalized Method of Moments

Advanced Econometrics Chapter 13: Generalized Method of Moments Nam T. Hoang UNE Business School 1 University of New England Chapter 13 GENERALIZED METHOD OF MOMENTS (GMM) I. ORTHOGONALITY CONDITION: The classical model: ( 1) ( ) ( 1)( 1)n n k nk Y X β ε × × ×× = + (1) ( ) 0E Xε = (2) / 2( )E X Iεε σ= (3) X and ε are generated independently. If ( ) 0i iE Xε = then (for equation i: (1 ) ( 1) i ik k Y X β ε × × = + ) ( )i iE X ε ( ) iX i i i E E X Xε =   0 ( ) iX i i i E E X Xε    =     [ ] ( 1) 0. 0 iX i k E X × = = → Orthogonality condition. Note: ( )( )/ / /( , ) ( ) ( )i i i i i iCov X E X E X Eε ε ε = − −  ( )/ /( )i i iE X E X ε = −  ( )  / / 0 ( ) ( )i i i iE X E X Eε ε= − / ( 1)( 1) (1 1) 0i i kk E X ε ×× ×   = =    if ( ) 0E Xε = So for the classical model: / ( 1)( 1) (1 1) 0i i kk E X ε ×× ×   =    Advanced Econometrics Chapter 13: Generalized Method of Moments Nam T. Hoang UNE Business School 2 University of New England II. METHOD OF MOMENTS Method of moments involves replacing the population moments by the sample moment. Example 1: For the classical model: ( ) ( ) / / ( 1) : 0 0 i i i i i k population moment Population E X E X Y X ε β × =  → − =   Sample moment of this: / / ( ) 1( 1) 1 11 ( 1) 1 1ˆ ˆ( ) ( ) n i i i k n nki k X Y X X Y X n n β β × ×× ×= ×   − = −    ∑  Moment function: A function that depends on observable random variables and unknown parameters and that has zero expectation in the population when evaluated at the true parameter. ( )m β - moment function – can be linear or non-linear. ( 1)k β × is a vector of unknown parameters. [ ( )] 0E m β = population moment. Method of moments involves replacing the population moments by the sample moment. Example 1: For the classical linear regression model: The moment function: / ( )i iX mε β= The population function: ( )/ 0i iE X ε = [ ] ( )/ ( 1) ( ) 0i i i kE m E X Y Xβ β × = − =  The sample moment of ( )i iE X ε is / / ( ) 1( 1) 1 11 ( 1) 1 1ˆ ˆ( ) ( ) n i i i k n nki k X Y X X Y X n n β β × ×× ×= ×   − = −    ∑  Replacing sample moments for population moments: Advanced Econometrics Chapter 13: Generalized Method of Moments Nam T. Hoang UNE Business School 3 University of New England ( )/1 ˆ 0X Y Xn β − =  / / ˆ 0X Y X X β− = / /ˆX X X Yβ = / 1 /ˆ ˆ( )MOM OLSX X X Yβ β −= = Example 2: If (1 ) i k X × are endogenous → /( , ) 0iCov X ε ≠ Suppose ( )1 2 (1 ) i i i Li L Z Z Z Z × =  is a vector of instrumental variables for (1 ) i k X × Zi satisfies: (1 ) ( ) 0i i L E Zε × = → / ( 1) ( ) 0i i LE Z ε ×= and / ( 1) ( ) 0i i LCov Z ε ×= We have: / ( 1)( 1) (1 1) ( ) 0i i LL E Z ε ×× × = ( )/ ( 1) 0i i i L population moment E Z Y X β ×   − =     The sample moment for ( )/i i iE Z Y X β −  is ( )/ 1 1 ˆ n i i i i Z Y X n β =  − ∑ = / ( 1)( ) ( 1) ( 1) 1 ˆ kL n n L Z Y X n β ×× × ×     −           Replacing sample moments for population moments:  / ( 1)( ) ( 1) ( 1) 1 ˆ 0 kL n L n Z Y X n β ×× × ×     − =          (*) a) If L < k (*) has no solution b) If L = k exact identified. Advanced Econometrics Chapter 13: Generalized Method of Moments Nam T. Hoang UNE Business School 4 University of New England  / ( ) ( 1)( ) ( 1) ˆ 0 n k kL n k Z Y X β × ×× ×  − =    / /ˆZ X Z Yβ = / 1 /ˆ ˆ( )MOM IVZ X Z Yβ β −= = c) If L > k → k parameters, L equations → There is NO unique solution because of "too many" equations → GMM. III. GENERALIZED METHOD OF MOMENTS: 1. The general case: Denote: ˆ( )m β/ is the sample moment of the population moment [ ( )] 0E m β = Method of moments: ( 1) ( 1) ˆ( ) k L m β × × /  = ( 1) 0 L× 1 2 ˆ ˆˆ ˆ k β β β β      =         a) If L < k: no solution for βˆ b) If L = k: unique solution for βˆ as ( 1) ( 1) ˆ( ) k L m β × × /  = ( 1) 0 L× c) If L > k, how do we estimate β ? Hausen (1982) suggested that instead of solving the equations for βˆ ( 1) ˆ( ) k m β × / = ( 1) 0 L× We solve the minimum problem: / ˆ ( ) (1 ) ( 1) ˆ ˆmin ( ) ( ) L L L L m W m β β β × × ×    / /    (**) Where W is any positive definite matrix that may depend on the data. Note: If ( )n n X × is a positive definite matrix then for any vector 1 2( )na a a a=  / ( )(1 ) (1 ) 0 n nn n a X a ×× × > Advanced Econometrics Chapter 13: Generalized Method of Moments Nam T. Hoang UNE Business School 5 University of New England βˆ that minimize (**) is called generalized method of moments (GMM) estimator of βˆ , denote ˆGMMβ . Hausen (1982) showed that ˆGMMβ from (**) is a consistent estimator of β. ( 1) ˆlim GMM n k p β β →∞ × = Problem here: What is the best W to use ? - Hansen (1982) indicated: 1 ( ) ˆ( ( )) L L W VarCov n m β − ×  =   - With this W, ˆGMMβ is efficient → has the smallest variance: 1/1ˆ( )GMMVarCov G WGn β −  =   where: /( ) ˆ( )lim ˆL k mG p β β× ∂ = ∂ ( ) 1 1/ / / /1ˆ ˆ( )GMMVarCov G WG G W VarCov n m WG G WGnβ β − −    =      2. The linear model: ( )/ 1 1 n i i i i Z Y X n β =  − ∑ = ( )m β/ The sample moment becomes: ( ) ( ) / / / ˆ ( )1 1 (1 ) ( 1) min n n i i i i i iL Li i L L Z Y X W Z Y X β β β × = = × ×       − −          ∑ ∑ First- order condition: ( ) / / / ( ) ( 1)( 1) (1 )1 1 (1 ) ( 1) 0 n n i i i i iL L kL ki i L L Z X W Z Y X β × ×× ×= = × ×     − =        ∑ ∑ k equations: ( ) ( )// / / ˆ 0Z X W Z Y Z Xβ→ − = ( ) ( )/ // / / / ˆ. .Z X W Z Y Z X W Z X β→ = Advanced Econometrics Chapter 13: Generalized Method of Moments Nam T. Hoang UNE Business School 6 University of New England ( ) ( ) 1/ // / / /ˆ . .GMM Z X W Z X Z X W Z Yβ −    → =        For the linear regression model: ( ) ( ) ( ) ( ) 1 / / / / ( ) ( )( 1) ( ) ( 1) ( ) ( 1) ˆ GMM L L L Lk k L L k L L X Z W Z X X Z W Z Yβ − × ×× × × × ×     =            IV. GMM AND OTHER ESTIMATORS IN THE LINEAR MODELS: 1. Notice that if L = k (exact identified) then /X Z is a square matrix ( )k k× so that: ( ) ( ) ( ) ( )1 1 1/ / / 1 /. . .X Z W Z X Z X W X Z− − −−   =     and ( ) ( )1/ /ˆGMM Z X Z Yβ − = which is the IV estimator → The IV estimator is a special case of the GMM estimator. 2. If Z = X then ( ) ( )1/ /ˆ ˆGMM OLS X X X Yβ β − = = 3. If L > k over-identification, the choice of matrix W is important. W is called weight matrix. ˆ GMMβ is consistent for any W positive definite. The choice of W will affect the variance of ˆGMMβ → We could choose W such that ˆ( )GMMVar β is the smallest → efficient estimator. 4. If / 1( )W Z Z −= then: ( ) ( ) ( ) ( ) ( ) ( ) 11 1/ / / / / /ˆ GMM X Z Z Z Z X X Z Z Z Z Yβ −− −   =        which is the 2SLS estimator is also a special case of the GMM estimator. 5. From Hausen (1982), the optimal W in the case of linear model is: ( ) 1 1/ / /1 1 i i i iW Z Z E Z Zn n ε ε − − = Σ =    ( ) 1 /1 i iW VarCov Zn ε −  =     Advanced Econometrics Chapter 13: Generalized Method of Moments Nam T. Hoang UNE Business School 7 University of New England 6. The next problem is to estimate W: a) If no heteroskedasticity and no autocorrelation: ( ) 1 2 / 1 1ˆ n i i i W Z Z nε σ − =   =     ∑ = ( ) 1 2 /1ˆ Z Z nε σ −      ( ) ( ) ( ) ( ) ( ) ( ) 11 1/ / / / / /ˆ GMM X Z Z Z Z X X Z Z Z Z Yβ −− −   =        We get the 2SLS estimator → there is no different between 2ˆ SLSβ and ˆGMMβ in the case of no heteroskedasticity and no autocorrelation. b) If there is heteroskedasticity in the error terms (but no autocorrelation) in unknown forms. ( ) 1 2 / 1 1ˆ n i i i i W e Z Z n − =   =     ∑ (White's estimator) → efficient gain over 2ˆ SLSβ . c) If there are both heteroskedasticity and autocorrelation in unknown forms, use: Wˆ = Newey - West estimator) . 1 /1Wˆ Z Z n −  = Σ    ( ) ( ) 1 / 2 / / 1 1 1 1ˆ n L n i i i j i i j i i j i j i i j i j W Z Z e w e e Z Z Z Z n − − − − = = = +    = + +      ∑ ∑∑ 1 1j jw L  = − +  → efficient gain over 2ˆ SLSβ Notes: /( )i iE ε εΣ = if heteroskedasticity and autocorrelation forms are known EX: 2 1 ( )i i t t t f X u σ ε ρε −  =  = + then /( )i iE ε εΣ = can be consistently estimated and we could perform GLS to get the efficient estimators ˆGLSβ (using instrumental variables), GMM is not necessary here. Usually the form of autocorrelation and heteroskedasticity are not known → GMM estimator is an important improvement in these cases. Advanced Econometrics Chapter 13: Generalized Method of Moments Nam T. Hoang UNE Business School 8 University of New England V. GMM ESTIMATION PROCEDURE: Step 1: Use W = I or / 1( )W Z Z −= to obtain a consistent estimator of β. Then estimate Wˆ by White's procedure (heteroskedasticity case) or Newey-West procedure (general case). Step 2: Use the estimated Wˆ to compute the GMM estimator: ( ) ( ) ( ) ( )1/ / / /ˆ ˆ ˆGMM X Z W Z X X Z W Z Yβ −    =     Note: We always need to construct Wˆ in the first step. VI. THE ADVANTAGES OF GMM ESTIMATOR: 1. If we don't know the form/patent of heteroskedasticity or autocorrelation → correct by Robust standard error (White or Newey-West) → stuck with inefficient estimators. 2. 2-SLS estimator is consistent but still inefficient if error terms are correlated/ heteroskedasticity. 3. GMM brings efficient estimator in the case of unknown heteroskedasticity and autocorrelation forms/ correct standard errors also. Potential drawbacks: 1. Definition of the weight matrix W for the first is arbitrary, different choices will lead to different point estimates in the second step. One possible remedy is to not stop after iterations but continue to update the weight matrix W until convergence has been achieved. This estimator can be obtained by using the "cue" (continuously updated estimators) option within ivreg 2 2. Inference problem because the optimal weight matrix is estimated → can lead to sometime downward bias in the estimated standard errors for GMM estimator. Advanced Econometrics Chapter 13: Generalized Method of Moments Nam T. Hoang UNE Business School 9 University of New England VII. VARIANCE OF THE GMM ESTIMATOR FOR LINEAR MODELS: Note: ( ) ( ) ( ) 1/ /1ˆ ˆGMMVarCov X Z W Z Xnβ −  =   / / 11 ˆ .n X Z Z X Q W Q n −→∞  →   0n→∞→ → consistency. ( )ˆ LGMMn β β− → ( )/ / 1ˆ0, X Z Z XN Q WQ −     so that ˆGMMβ is consistent estimator. Estimated: ( ) ( ) ( ) 1/ /ˆ ˆGMMVarCov n X Z W Z Xβ − =   In practice Wˆ is noise, since the residual in the first step are affected by sampling error. The upshot is that step 2 standard errors tend to be too good. Methods now exist that enable you to correct for sampling error in the first step (Windmeijer procedure). VIII. SPECIFICATION TESTS WITH GMM: ˆ R GMMβ : Restricted estimator (under constraints). ˆ GMMβ : Unrestricted estimator (no constraints). / / / / 2 2 2 1 1 1 1 ˆ ˆˆ ˆ ˆ ˆ( ) ( ) / n n n n i i i i i i i i q i i i i J Z R W Z R Z W Z nε ε ε ε χ = = = =   = −    ∑ ∑ ∑ ∑  q: number of restrictions.