Advanced Econometrics - Part II - Chapter 1: Review Of Least Squares & Likelihood Methods

Advanced Econometrics - Part II Chapter 1: Review Of Least Squares & Likelihood Methods Nam T. Hoang UNE Business School 1 University of New England Chapter 1 REVIEW OF LEAST SQUARES & LIKELIHOOD METHODS I. LEAST QUARES METHODS: 1. Model: - We have N observations (individuals, forms, ) drawn randomly from a large population i = 1, 2, , N - On observation i: Yi and K-dimensional column vector of explanatory variables ),...,,( 21 ikiii XXXX = and assume 1ieX = for all i = 1, 2, , N. - We are interested in explaining the distribution of iY in terms of the explanatory variables iX using linear model: )),...,((' 1 kiii XY βββεβ =+= In matrix notation: εβ += XY iikkiii XXXY εββββ +++++= ...33221 Assumption 1: n 1iii }Y,X{ = are independent and identically distributed Assumption 2: iε ׀ iX ~ N ( 0, σ 2 ) Assumption 3: ii X⊥ε (∑ = = n i ijX 1 i 0ε ) Assumption 4: E[ iε ׀ iX ] = 0 Assumption 5: E[ iε * iX ] = 0 Advanced Econometrics - Part II Chapter 1: Review Of Least Squares & Likelihood Methods Nam T. Hoang UNE Business School 2 University of New England The Ordinary Least Squares (OLS) estimator for β solves: 2 1 )'(min∑ = − n i ii XY ββ This leads to: )'()'('ˆ 1 1 1 1 YXXXYXXX n i ii n i ii − = − = =            = ∑∑β The exact distribution of the OLS estimation under the normality assumption is: ])'.(,[~ˆ 12 −XXN σββ • Without the normality of the ε it is difficult to derive the exact distribution of βˆ . However we can establish asymptotic distribution: )]'[.,0()ˆ( 12 −→− XXENN d σββ • We do not know 2σ , we can consistently estimate it as 2 1 2 )'ˆ( 1 1ˆ ∑ = − −− = n i ii XYkn βσ • In practice, whether we have exact normality for the error terms or not, we will use the following distribution for βˆ : ),(ˆ VN ββ ≈ where: 12 ))'(.( −= XXEV σ estimate V by: 1 1 2 )'.(ˆˆ − = ∑= N i ii XXV σ • If we are interested in a specific coefficient: )ˆ,ˆ(ˆ kkkk VN ββ ≈ ijVˆ is the (i,j) element of the matrix Vˆ • Confidence intervals for kβ would be (95%)     +− kkkkkk VV ˆ96.1ˆ;ˆ96.1ˆ ββ • Test a hypothesis whether αβ =k Advanced Econometrics - Part II Chapter 1: Review Of Least Squares & Likelihood Methods Nam T. Hoang UNE Business School 3 University of New England )1,0(~ ˆ N V t kk k αβ −= 2. Robust Variances: If we don’t have the homoscedasticity assumption then: -12-1 ])(E[XX' ])XX'(E[ ])(E[XX',0()ˆ( εββ Nn →− We can estimate the heteroskedasticity – consistent variance as: (White’s estimator) 1 11 2 1 1 '1'ˆ1'1ˆ − == − =                   = ∑∑∑ N i ii N i ii N i ii XXN XX N XX N V ε II. MAXIMUM LIKELIHOOD ESTIMATION: 1. Introduction: • Linear regression model: iii XY εβ += ' with ε׀Xi ~ N ( 0, σ2 ) OLS: 2 1 ˆ arg min ( ' ) n i i i Y X β β β = = −∑             =→ ∑∑ = − = n i ii n i ii YXXX 1 1 1 'βˆ • Maximum likelihood estimator: 2 2 2 , ˆ ˆ( , ) arg max ( , )MLE L β σ β σ β σ= Where: ∑ ∑ = = −−−=       −−−= n i ii n i ii XYn XYL 1 2 2 2 1 2 2 22 )'( 2 1)2ln( 2 )'( 2 1)2ln( 2 1),( β σ πσ β σ πσσβ Note: →),N(~ 2σµX density function of X: 2 2 ( ) 2 2 1( ) 2 X f X e µ σ πσ − − = Advanced Econometrics - Part II Chapter 1: Review Of Least Squares & Likelihood Methods Nam T. Hoang UNE Business School 4 University of New England • This lead to the same estimator for β as in OLS and the MLE approach is a systematic way to deal with complex nonlinear model. ∑ = −= n i iiMLE XYn 1 22 )'(1ˆ βσ 2. Likelihood function: • Suppose we have independent and identically distributed random variables nZZ ,...,1 with common density ),( θiZf . The likelihood function given a sample nZZZ ,...,, 21 is 1 ( ) ( , ) n i i f Zθ θ = =∏ • The log – likelihood function: i 1 ( ) ln ( ) ln ( , ) n iL f Zθ θ θ = = = ∑ • Building a likelihood function is first step to job search theory model • An example of maximum likelihood function:  An unemployed individual is assumed to receive job offers.  Arriving according to rate λ such that the expected number of job offers arriving in a short interval of length dt is dtλ  Each offer consist of some wage rate w, draw independently of previous wages, with continuous distribution function )(wFw  If the offer is better than the reservation wage w , that is with probability )(1 wF− , the offer is accepted.  The reservation wage is set to maximize utility.  Suppose that the arrival rate is constant over time.  Optimal reservation wage is also constant over time.  The probability of receiving an acceptable offer in a short time dt is dt.θ with ))(1.( wF−= λθ Advanced Econometrics - Part II Chapter 1: Review Of Least Squares & Likelihood Methods Nam T. Hoang UNE Business School 5 University of New England  The constant acceptance rate θ implies that the distribution for the unemployment duration is exponential with mean θ1 and density function: )()( θθ yeyf −=→ y: unemployment duration - random variable  Mean & variance 2 1,1 θθ  )()(1)( θyeyFyS −=−= : survivor function  θ= < +<< == → )( )Pr(lim )( )()( 0 YyP dyyYy yS yfyh dy Y : hazard function (The rate at which a job is offered and accepted) Likelihood function: a) If we observe the exact unemployment duration iy n i i 1 1 ( ) f(y , ) ( )S( ) n i i i h y yθ θ θ θ = = → = =∏ ∏ b) We observe a number of people all becoming unemployed at the same point in time, but we only observe whether they exited unemployment before a fixed point in time, say c: n n 1-di 1-di i 1 i 1 ( ) F( ) .(1 ( )) (1-S( )) . ( )di dic F c c S cθ θ θ θ θ = = = − =∏ ∏ 1=di denotes that individual i left unemployment before c and 0=di to denote this individual was still unemployed at time c. c) If we observe the exact exit or failure time if it occurs before c, but only an indicator of exit occurs after c n 1-di di di 1-di i i 1 1 ( ) f(y , ) . ( ) ( ) .S( ) . ( ) n di i i i S c h y y S cθ θ θ θ θ θ = = = =∏ ∏ d) Denote ic is the specific censoring time of individual i. Letting t denote the minimum of the exit time iy and censoring time ic , ),min( iii cyt = n 1-di di 1-di i i 1 1 1 ( ) f(y , ) . ( ) ( ) .S( ) ( ) . ( ) n n di di i i i i i i i S c f t t h t S tθ θ θ θ θ θ θ = = = = = =∏ ∏ ∏ Advanced Econometrics - Part II Chapter 1: Review Of Least Squares & Likelihood Methods Nam T. Hoang UNE Business School 6 University of New England 3. Properties of MLE 1 ˆ arg max ln ( , ) n MLE i i f Z θ θ θ →Θ = = ∑ a. Consistency: For all ε > 0 ˆlim Pr( ) 0MLEn θ θ ε→∞ − > = b. Asymptotic normality: 12 0 0 ˆ( ) 0, ( , ) 'MLE i Ln N E Zθ θ θ θ θ −  ∂ − → −   ∂ ∂   4. Computation of the maximum likelihood estimator: Newton – Raphson method: • Approximate the objective function )()( θθ LQ −= around some starting value 0θ by a quadrate function and find the exact minimum for that quadrate approximation. Call this 1θ • Redo the quadrate