This post deals with the Theil mixed estimator which uses the prior information as well as the sample information.

The Cosmic Cliffs shows the star-forming Carina Nebula in exquisite detail, revealing newborn stars and their dynamic influence on their environment. NASA, ESA, CSA, and STScI

Theil mixed estimator

Theil mixed estimator incorpoate stochastic non-sample information into a linear model. This estimator mixes sample and prior information in a generalized least squares sense. Information regarding GLS estimator can be found at the following previous post.

Generalized Least Squares (GLS) estimator

Linear model with sample information

Sample information is represented by the following linear model.
\[\begin{align} y &= X \beta + u\\ \text{E}(u) &= 0, \quad \text{E}(uu^{\prime}) = \sigma^2 I \end{align}\]

Linear model with sample and prior information

Prior information has the following form \[\begin{align} r & = R \beta + \nu \\ \text{E}(\nu) &= 0, \quad \text{E}(\nu {\nu}^{\prime}) = \Psi \end{align}\] where \(R\) is \(J \times K \) and \(r\) is \(J \times 1\). \(\nu\) is a \(J \times 1\) normally distributed random error vector.

Incorporating the prior information into the sample information leads to the following model specification.
\[\begin{align} \begin{bmatrix} y \\ r \end{bmatrix} = \begin{bmatrix} X \\ R \end{bmatrix} \beta + \begin{bmatrix} u \\ \nu \end{bmatrix} \end{align}\] \[\begin{align} \text{E}(\begin{bmatrix} u \\ \nu \end{bmatrix}\begin{bmatrix} u^{\prime} \color{white}{0} \nu^{\prime} \end{bmatrix} ) = \begin{bmatrix} \sigma^2 I & 0 \\ 0 & \Phi \end{bmatrix} \end{align}\]

GLS estimator

The GLS estimator is as follows.
\[\begin{align} &\begin{bmatrix} X^{\prime} \color{white}{0} R^{\prime} \end{bmatrix} \begin{bmatrix} \sigma^2 I & 0 \\ 0 & \Phi \end{bmatrix}^{-1} \begin{bmatrix} y \\ r \end{bmatrix} \\ = &\begin{bmatrix} X^{\prime} \color{white}{0} R^{\prime} \end{bmatrix} \begin{bmatrix} \sigma^2 I & 0 \\ 0 & \Phi \end{bmatrix}^{-1} \begin{bmatrix} X \\ R \end{bmatrix} \hat{\beta}_{GLS} \end{align}\]
By using matrix multiplications, this result can be simplified to
\[\begin{align} \phi X^{\prime} y + R^{\prime} \Phi^{-1} r = (\phi X^{\prime}X + R^{\prime} \Phi^{-1} R)\hat{\beta}_{GLS} \end{align}\] where \(\phi\) stands for the precision of the regression model : \(\phi = 1/\sigma^2\).

By inverting \(\phi X^{\prime}X + R^{\prime} \Phi^{-1} R\), \(\hat{\beta}_{GLS}\) and its variance can be obtained.

\[\begin{align} \hat{\beta}_{GLS} &= (\phi X^{\prime}X + R^{\prime} \Phi^{-1} R)^{-1} (\phi X^{\prime} y + R^{\prime} \Phi^{-1} r) \\ Var(\hat{\beta}_{GLS}) &= (\phi X^{\prime}X + R^{\prime} \Phi^{-1} R)^{-1} \end{align}\]
This is the Theil mixed estimator.