Bivariate Distributions

Probability and Statistics #math pns

---

• of The Discrete Type
  • joint pmf
  • marginal pmf
    • independent random variables
  • mathematical expectation
• The Correlation Coefficient
  • Correlation and regression
• Conditional Distributions
• of The Continuous Type
• Bivariate Normal Distribution

---

Discrete Bivariate Distributions

So far our random variables were only taking one input variable, but what if we want to correlate 2 input fields? Or predict a a third quantity based on multiple inputs. e.g. Based on 12th board marks and JEE Rank, can we predict CGPA at the end of first semester?

Joint Probability Mass Function:

Let $X$ and $Y$ be two random variables defined on a discrete space. Let $S$ denote the corresponding two-dimensional space of $X$ and $Y$. The probability that $X=x$ and $Y=y$ is given by the joint pmf $f(x,y)=P(X=x,Y=y)$ where

1. $0\le f(x,y)\le 1$
2. ${\sum \sum }_{(x,y)\in S}f(x,y)=1$
3. $P[(X,Y)\in A]={\sum \sum }_{(x,y)\in A}f(x,y)$

Marginal Probability Mass Function:

The pmf of $X$ alone is called the marginal pmf of $X$ and is given by $f_x(x)={\sum }_{y}f(x,y)=P(X=x)\text{ when }x\in S_x$ , where the summation is taken over all possible $y$ values for every $x\in S_x$

Independence: The random variables $X$ and $Y$ are independent if and only if for $x\in S_x$ and every $y\in S_y$ : $P(X=x,Y=y)=P(X=x)P(Y=y)$ or equivalently $f(x,y)=f_x(x)f_y(y)$ Otherwise $X$ and $Y$ are said to be dependent

Note: Whenever the support of $X$ and $Y$ is not rectangular, they are dependent

Mathematical Expectation:

$E[u(X_1,X_2)]={\sum \sum }_{(x_1,x_2)\in S}u(x_1,x_2)f(x_1,x_2)$ is the mathematical expectation of $u(X_1,X_2)$

1. if $u_i(X_1,X_2)=X_i$ then $E(u_i)={\mu }_i$ is called the mean of $X_i$
2. if $u_i(X_1,X_2)=(X_i-{\mu }_i{)}^2$ then $E(u_i)={\sigma }^2=Var(X_i)$ is the variance of $X_i$

Correlation Coefficient

${\mu }_X=E(X)$ and ${\sigma }_X^2=E[(X-{\mu }_X{)}^2]$ are the mean and variance

We now introduce covariance and the correlation coefficient

1. if $u(X,Y)=(X-{\mu }_X)(Y-{\mu }_Y)$ then $E[(u(X,Y)]={\sigma }_{XY}=Cov(X,Y)$ is the covariance
2. if that standard deviations are positive (not imaginary) then $\rho =\frac{{\sigma }_{XY}}{{\sigma }_X{\sigma }_Y}$ is the correlation coefficient

Correlation and Regression:

good resource: MIT Notes

let $Y$ and $X$ be our two random variables, $\rho$ is a measure of how correlated $Y$ and $X$ are. i.e. When we form our least squares linear regression line for the graph of $Y\text{ vs }X$ then how linear of a relation do we get?

The slope of the line $y=ax+b$ is $a=\frac{\rho {\sigma }_y}{{\sigma }_x}$, the intercept $b={\mu }_y-\frac{\rho {\sigma }_y}{{\sigma }_x}{\mu }_x$

And the minimum value of the square of the error is $E\{[Y-(aX+b){]}^2\}={\sigma }_y^2(1-{\rho }^2)$

This gives us the property $-1\le \rho \le 1$

we can say that $\rho$ measures the amount of linearity of a bivariate probability distribution

Independence implies zero correlation, but zero correlation does not necessarily imply independence.

it is possible that $X$ and $Y$ are not linearly related, but still related nonetheless