Covariance Definition

Covariance is a measure of association between two random variables. Let X and Y be two random variables. Then the covariance is defined as
\[Cov\left( X,Y \right)=E\left[ \left\{ X-E\left( X \right) \right\}\left\{ Y-E\left( Y \right) \right\} \right]\]
\[\Rightarrow Cov\left( X,Y \right)=E\left[ XY-XE\left( Y \right)-YE\left( X \right)+E\left( X \right)E\left( Y \right) \right]\]
\[\Rightarrow Cov\left( X,Y \right)=E\left( XY \right)-E\left( X \right)E\left( Y \right)-E\left( X \right)E\left( Y \right)+E\left( X \right)E\left( Y \right)\]
\[\therefore Cov\left( X,Y \right)=E\left( XY \right)-E\left( X \right)E\left( Y \right)\]

Note:

If X and Y are independent random variables, then Cov(X, Y) = 0

Properties of Covariance:

\[(i)~Cov\left( aX,bY \right)=abCov\left( X,Y \right)\]
\[(ii)Cov\left( X+a,Y+b \right)=Cov\left( X,Y \right)\]
\[(iii)Cov\left( \frac{X-\overline{X}}{\sigma X},\frac{Y-\overline{Y}}{\sigma Y} \right)=\frac{1}{\sigma X\sigma Y}\]
\[(iv)Cov\left( aX+b,cY+d \right)=acCov\left( X,Y \right)\]
\[(v)Cov\left( X+Y,Z \right)=Cov\left( X,Z \right)+Cov\left( Y,Z \right)\]
\[(vi)Cov\left( aX+bY,cX+dY \right)=ac{{\sigma }_{X}}^{2}+bd{{\sigma }_{Y}}^{2}+\left( ad+bc \right)Cov\left( X,Y \right)\]
\[(vii)\text{If}~~{{X}_{1}},{{X}_{2}},…,{{X}_{n}}~~\text{be n random variables},\text{ then}\]
\[\text{V}\left( \sum\nolimits_{i=1}^{n}{{{a}_{i}}{{X}_{i}}} \right)=\sum\nolimits_{i=1}^{n}{{{a}_{i}}^{2}V\left( {{X}_{i}} \right)}+2\sum\nolimits_{i=1}^{n}{\sum\nolimits_{j=1}^{n}{{{a}_{i}}{{a}_{j}}}}Cov\left( {{X}_{i}},{{X}_{j}} \right)\]
\[(viii)\text{If}~~{{X}_{1}},{{X}_{2}},…,{{X}_{n}}~~\text{be n independent random variables},\text{ then}\]
\[\text{V}\left( \sum\nolimits_{i=1}^{n}{{{a}_{i}}{{X}_{i}}} \right)=\sum\nolimits_{i=1}^{n}{{{a}_{i}}^{2}V\left( {{X}_{i}} \right)}\]

The score of 12 students in mathematics and physics classes are

Find the covariance of the distribution.

Solution:

We know that,
\[Cov\left( X,Y \right)=E\left( XY \right)-E\left( X \right)E\left( Y \right)\]
Now do it yourself. You can take the help of “mathematical expectation” article to solve this sum.

Let X and Y be two random variables such that Var(X) = 4, Cov(X, Y) = 2. Compute the Cov(3X, X+3Y).

Solution:

By the properties of covariance operator we have,
\[Cov\left( 3X,X+3Y \right)=3Cov\left( X,X+3Y \right)\]
\[\Rightarrow Cov\left( 3X,X+3Y \right)=3Cov\left( X,X \right)+9Cov\left( X,Y \right)\]
\[\Rightarrow Cov\left( 3X,X+3Y \right)=3Var\left( X \right)+9Cov\left( X,Y \right)\]
\[\therefore Cov\left( 3X,X+3Y \right)=3\times 4+9\times 2=30\]

Let (X, Y) be an absolutely continuous random variable with support R_XY = [0, ∞) X [1, 4] and its joint probability density function be
\[{{f}_{XY}}=\frac{1}{3}y.\exp \left( -xy \right),if~~x\in [0,\infty )\And y\in [1,4]\]
\[{{f}_{XY}}=0,~~otherwise\]
Compute covariance between X and Y.

Solution:

Note that the support of Y is R_Y = [1, 4]. When y does not belongs to [1, 4], the marginal probability density function of Y is 0, while yЄ[1, 4], the marginal probability density function of Y is
\[{{f}_{Y}}\left( y \right)=\int\limits_{-\infty }^{\infty }{{{f}_{XY}}\left( x,y \right)}dx\]
\[\Rightarrow {{f}_{Y}}\left( y \right)=\int\limits_{0}^{\infty }{\frac{1}{3}y.\exp \left( -xy \right)}dx\]
\[\Rightarrow {{f}_{Y}}\left( y \right)=\frac{1}{3}y.\left[ \frac{\exp \left( -xy \right)}{-y} \right]_{0}^{\infty }\]
\[\therefore {{f}_{Y}}\left( y \right)=\frac{1}{3}\left[ 0-\left( -1 \right) \right]=\frac{1}{3}\]
The expected value of Y is
\[E\left( Y \right)=\int\limits_{-\infty }^{\infty }{y}{{f}_{Y}}\left( y \right)dy=\int\limits_{1}^{4}{y.}\frac{1}{3}dy=\frac{1}{3}\left[ \frac{{{y}^{2}}}{2} \right]_{1}^{4}=\frac{15}{6}\]
Now, the support of X is R_X = [0, ∞). When y does not belongs to [0, ∞), the marginal probability density function of X is 0, while xЄ[0, ∞), the marginal probability density function of X is
\[{{f}_{x}}\left( x \right)=\int\limits_{-\infty }^{\infty }{{{f}_{XY}}\left( x,y \right)dy}\]
\[\Rightarrow {{f}_{x}}\left( x \right)=\int\limits_{1}^{4}{\frac{1}{3}y.\exp \left( -xy \right)dy}\]
The expected value of X is
\[E\left( X \right)=\int\limits_{-\infty }^{\infty }{x{{f}_{x}}\left( x \right)}dx=\int\limits_{0}^{\infty }{x\left\{ \int\limits_{1}^{4}{\frac{1}{3}y.\exp \left( -xy \right)dy} \right\}}dx\]
\[\Rightarrow E\left( X \right)=\frac{1}{3}\int\limits_{1}^{4}{\left\{ \int\limits_{0}^{\infty }{xy.\exp \left( -xy \right)dx} \right\}}dy\]
\[\left( \text{by changing the order of integration} \right)\]
\[\Rightarrow E\left( X \right)=\frac{1}{3}\int\limits_{1}^{4}{\left\{ \frac{1}{y}\int\limits_{0}^{\infty }{t\exp (-t)dt} \right\}}dy\]
\[\left( \text{by changing the variable in the inner integral},\text{ t }=\text{ xy} \right)\]
\[\Rightarrow E\left( X \right)=\frac{1}{3}\int\limits_{1}^{4}{\frac{1}{y}\left\{ \left[ -t\exp (-t) \right]_{0}^{\infty }+\int\limits_{0}^{\infty }{\exp (-t)dt} \right\}}dy\]
\[\Rightarrow E\left( X \right)=\frac{1}{3}\int\limits_{1}^{4}{\frac{1}{y}\left\{ 0+\left[ \exp (-t) \right]_{0}^{\infty } \right\}}dy\]
\[\Rightarrow E\left( X \right)=\frac{1}{3}\int\limits_{1}^{4}{\frac{1}{y}dy=}\frac{1}{3}\left[ \log y \right]_{1}^{4}=\frac{1}{3}\log 4\]
The expected value of the product XY can be computed using the transformation theorem
\[E\left( XY \right)=\int\limits_{-\infty }^{\infty }{\int\limits_{-\infty }^{\infty }{xy~~{{f}_{xy}}\left( xy \right)dydx}}\]
\[\Rightarrow E\left( XY \right)=\int\limits_{0}^{\infty }{\left\{ \int\limits_{1}^{4}{xy.}\frac{1}{3}y.\exp \left( -xy \right)dy \right\}dx}\]
\[\Rightarrow E\left( XY \right)=\frac{1}{3}\int\limits_{1}^{4}{y\left\{ \int\limits_{0}^{\infty }{xy.}\exp \left( -xy \right)dx \right\}}dy\]
\[\left( \text{by changing the order of integration} \right)\]
\[\Rightarrow E\left( XY \right)=\frac{1}{3}\int\limits_{1}^{4}{y\left\{ \frac{1}{y}\int\limits_{0}^{\infty }{t.}\exp \left( -t \right)dt \right\}}dy\]
\[\left( \text{by changing the variable in the inner integral},\text{ t }=\text{ xy} \right)\]
\[\Rightarrow E\left( XY \right)=\frac{1}{3}\int\limits_{1}^{4}{\left\{ \left[ t\exp \left( -t \right) \right]_{0}^{\infty }+\int\limits_{0}^{\infty }{\exp \left( -t \right)d}t \right\}}dy\]
\[\Rightarrow E\left( XY \right)=\frac{1}{3}\int\limits_{1}^{4}{\left\{ 0+\left[ -\exp \left( -t \right) \right]_{0}^{\infty } \right\}}dy\]
\[\therefore E\left( XY \right)=\frac{1}{3}\int\limits_{1}^{4}{dy=}\frac{1}{3}\left[ y \right]_{1}^{4}=1\]
Hence, using the covariance formula, the covariance between X and Y is
\[Cov\left( X,Y \right)=E\left( XY \right)-E\left( X \right)E\left( Y \right)\]
\[\therefore Cov\left( X,Y \right)=1-\frac{1}{3}\log 4.\frac{15}{6}=1-\frac{5}{6}\log 4\]

Let [X, Y] be an absolutely continuous random vector with support R_XY = {(x, y): 0≤x≤y≤2} i.e., R_XY is the set of all couples (x, y) such that 0≤y≤2 & 0≤x≤y. Let the joint probability density function of [X, Y] be
\[{{f}_{XY}}\left( x,y \right)=\frac{3}{8}y,~~if~~(x,y)\in {{R}_{XY}}\]
\[{{f}_{XY}}=0,~~otherwise\]
Compute the covariance between X and Y.

Solution:

Do it yourself, like above example.

<< Previous     Next>>

 ;