Higher moments and Variance

k-th moment of a R.V. X

• Definition: $E[X^k]$
• This is simply a generalization of the concept of the first moment $E[X]$ (=mean). Strictly, $E[X^k]$ should be called k-th moment about the origin (0).
• More generally k-th moment about the point (c) can be defined as $E[(X-c)^k]$.
• We mostly use the simpler form $E[X^k]$ because if you know the first r moments about the origin (or any point), it's straightforward to obtain $E[X-c]$ to $E[(X-c)^r]$ for any other points c (an example in the section ).
• Higher moments, useful for approximating the distribution of complex random variables.
• Calculation:
  $E[(X-c)^r] = (x_1 - c)^r Prob[X=x_1] + (x_2 - c)^r Prob[X=x_2] + ....$

Variance

• The Variance is a measure of how far typical values of X are from the mean.
• $Var[X] = E[(X-\mu)^2]$
• 2nd moment around the mean (central moment).
• This 2nd central moment can be expressed in terms of moments about the origin.
  $Var[X] = E[X^2] - (E[X])^2$
  Example: For the binomial distribution:
  $$\begin{eqnarray*} Var[Y] & = & E[(Y-\mu)^2]\\ & = & E[Y^2] - \mu^2\\ & = & E... ...2\\ & = & n (n - 1) p^2 + n p - (n p)^2\\ & = & n p (1-p)\\ \end{eqnarray*}$$
  
  
  Hint: $(a + b)^n = \sum_{i=0}^n {n \choose i} a^i b^{n-i}$, sect. for Prob[Y=k].

Covariance

For two R.V.'s $X_1$ and $X_2$, $(X_1 - E[X_1]) (X_2 - E[X_2])$ is another random variable. We call its expectation the covariance between $X_1$ and $X_2$.

$$Cov[X_1, X_2] = E[(X_1 - E[X_1]) (X_2 - E[X_2])]$$

$Cov[X_1, X_2]=0$ when $X_1$ and $X_2$ are independent.

Mean, Variance, Covariance, higher moments are summarization of distributions.