Three Lectures on Stochastic Decision Processes
University of Texas at Austin
March, 2005

John Rust, University of Maryland

Optional (but Recommended) Texts: Dynamic Economics By Jerome Adda and Russell Cooper (2003) MIT Press.

Handbook Chapter: Structural Estimation of Markov Decision Processes by John Rust (1994) in R. Engle and D. McFadden Handbook of Econometrics volume 4, Elsevier, North Holland.

Lecture 1: Discrete Decision Processes

Discrete Decision Processes are Problems where the choice variable is restricted to a finite set of alternatives. I describe parametric econometric methods for inferring the unknown parameters of these processes, particularly the method of maximum likelihood and survey some of the numerous applications of these models in many different parts of economics. I discuss the identification problem and show that these problems are non-parametrically unindentified, and discuss the implications of this result for empirical work in this area. I also discuss recent work on Bayesian estimation and simulation and other ``quasi monte carlo'' methods for solving and estimating these models.

Readings:

Lecture 2: Continuous Decision Processes

Continuous Decision Processes are Problems where the choice variable can take on a continuum of possible values. I describe parametric econometric methods for inferring the unknown parameters of these processes based on the "Euler Equation" and via parametric maximum likelihood and simulated method of moments approaches. I discuss in particular a problem arising in modeling optimal commodity price speculation and the problem of endogenous sampling of prices and the econometric problems this creates when one tries to estimate the model via maximum likelihood methods. However I show that the problem is quite tractable when one adopts a simulated minimum distance esimator. The general lesson is that via simulation methods, a huge range of endogeneity, measurement error, attrition, selectivity bias and other types of econometric problems can be handled in a very natural way, provided one is willing to do some parametric modeling. I discuss the identification problem, which can also be dicey when one combines behavioral modeling assumptions with assumptions about processes leading to endogenous attrition, participation, reporting and so forth. However for certain types of problems, particularly for case of risk neutral profit maximizers, results on non-parametric identification of unknows may be available.

Readings:

Lecture 3: Estimation of Dynamic Games

I extend the single agent decision framework to multi-agent dynamic games. This is much harder and is at the current frontier of research in this area. We will discuss several recent papers that make substantial headway on these topics.

Readings: