<?xml version="1.0" encoding="UTF-8"?>
<quiz>
  <question type="description">
    <name>
      <text>43.1.3.1 Least squares - modelling</text>
    </name>
    <questiontext format="html">
      <text><![CDATA[<h3>The method of least squares -  a modelling view</h3>

We take the dependent variable \(Y\) to be a random variable whose value, for a fixed value of \(x\) depends on the value of \(x\) and a random error component say \(e\) and we write
\[Y = \alpha + \beta x + e.\]
Adopting the notation of conditional probability, we are looking for the expected value of \(Y\) for a
given value of \(x\). The expected value of \(Y\) for a given value of \(x\) is denoted by
\[ E \left( Y \lvert x \right)\ = E \left( \alpha + \beta x + e \right) =  E \left( \alpha + \beta x \right) + E \left( e \right). \]
The variance of \(Y\) for a given value of \(x\) is given by the relationship
\[ V \left( Y \lvert x \right)\ = V \left( \alpha + \beta x + e \right) =  V \left( \alpha + \beta x \right) + V \left( e \right),\]
assuming independence.&nbsp;<p><br></p><p>If \(\mu_{Y \lvert_{x}} \) represents the true mean value of \(Y\) for a given value of \(x\) then
\[ \mu_{Y \lvert_{x}} = \alpha + \beta x, \]
assuming a linear relationship holds, is a straight line of mean values. If we now assume that the errors \(e\) are distributed with mean \(0\) and variance \(\sigma^2\) we may write
\[ E \left( Y \lvert x \right)\ = E \left( \alpha + \beta x \right) + E \left( e \right) = \alpha + \beta x \]
since \( E(e) = 0 \). 
Also,
\[ V \left( Y \lvert x \right)\ = V \left( \alpha + \beta x \right) + V \left( e \right) =\sigma^2 \]
since \( V(\alpha + \beta x) = 0 \). 

This implies that for each value of \(x\), \(Y\) is distributed with mean \( \alpha + \beta x \) and variance \(\sigma^2\). Hence when the variance is small the observed values of \(Y\) will be close to the regression line and when the variance is large, at least some of the observed values of \(Y\) may not be close to the line. Note that the assumption that the errors \(e\) are distributed with mean \(0\) may be made without loss of generality. If the errors had any other mean, we could subtract it and then add the mean to the value of \(c\). The ideas are illustrated in the diagram below.

<img src="@@PLUGINFILE@@/43-1-fig-5-error.svg" alt="" width="431" height="247" role="presentation" class="img-responsive atto_image_button_text-bottom">&nbsp;&nbsp;</p><p><br></p><p>&nbsp;The regression line is shown passing through the means of the distributions for the individual values of \(x\). The value of \(y\) corresponding to the \(x\)-value \(x_i\) can be represented by the equation
\[ y_i = \alpha + \beta x_i + e_i , \]
where \(e_i\) is the error of the observed value of \(y\), that is the difference from its expected value, namely
\[ E \left( Y \lvert x_i \right) = \mu_{y \lvert _{x_i}} = \alpha + \beta x_i .\]
Now, if we estimate \(\alpha\) and \(\beta\) with \(a\) and \(b\), the <i>residual</i>, or estimated error, becomes
\[ \hat{e}_i = y_i - a - bx_i , \]
so that the sum of the squares of the residuals is given by
\[ S = \sum \hat{e}_i^2 = \sum \left( y_i - a -b x_i \right)^2, \]
and we may minimize the quantity \(S\) by using the method of least squares as before. The mathematical details are omitted as before and the equations obtained for \(b\) and \(a\) are as before, namely
\[ b = \dfrac{ \dfrac{ \sum xy}{n} - \dfrac{ \sum x}{n}\dfrac{ \sum y}{n}  }{ \dfrac{\sum x^2}{n} - \left( \dfrac{\sum x}{n} \right)^2 }, \text{    and    } a = \bar{y} - b\bar{x}. \]

Note that since the error \(e_i\) in the \(i\)th observation essentially describes the error in the fit of the model to the \(i\)th observation, the sum of the squares of the errors \( \sum  e^2_i \) will now be used to allow us to comment on the adequacy of fit of a linear model to a given data set.

</p>]]></text>
<file name="43-1-fig-4-least-squares.svg" path="/" encoding="base64">[encoded data, 15448 characters]</file>
<file name="43-1-fig-5-error.svg" path="/" encoding="base64">[encoded data, 16552 characters]</file>
<file name="43-fig-1-scatter.svg" path="/" encoding="base64">[encoded data, 13776 characters]</file>
<file name="43-fig-5-errors.svg" path="/" encoding="base64">[encoded data, 29408 characters]</file>
    </questiontext>
    <generalfeedback format="html">
      <text/>
    </generalfeedback>
    <defaultgrade>0</defaultgrade>
    <penalty>0</penalty>
    <hidden>0</hidden>
    <idnumber/>
  </question>
</quiz>
