Random Variables

CDF:
$Prob(X(w)\le x)=F_X(x)=P(X\le x)$
props

1.

$0\le F_X(x)\le 1$

2.

F_X(x) is non decreasing

3.

$\lim_{x\rightarrow -\infty} F_X(x)=0\quad \lim_{x\rightarrow \infty} F_X(x)=1$

4.

F_X(x) is continuous from right

$$\lim_{\epsilon\rightarrow 0} F_X(x+\epsilon)=F_X(x)$$

5.

$P(a<X\le b)=F_X(b)-F_X(a)$

6.

P(X=a)=F_X(a)-F_X(a^-)

pdf:

7.

$f_X(x)=\frac{d F_X(x)}{dx}$ , if discrete we get impulses props

(a)

$f_X(x)\ge 0$

(b)

$\int_{-\infty}^{\infty} f_X(x)dx=1$

(c)

$\int_{a^+}^{b^+}f_X(x)dx=P(a<X\le b)$

(d)

$P(X\in A)=\int_A f_X(x)dx$

(e)

$F_X(x)=\int_{-\infty}^{x^+} f_X(x)dx$

Useful RV distributions

1.

Bernoulli: biasd coin flip
p - prob of success
1-p - prob of failure

$$p_K(k)=\left\{\begin{array}{ll} \displaystyle {p} & {\qquad k=1} \\ \displaystyle {1-p} & {\qquad k=0} \end{array}\right.$$

2.

Binomial (a sequence of N indept Bernoulli trials)

$$P_K(k)={N \choose k} p^k(1-p)^{N-k}\qquad 0\le k \le N$$

3.

uniform

4.

Gaussian

$$f_X(x)=\frac{1}{\sqrt{2\pi\sigma^2}}e^{-(x-m_X)^2/2\sigma^2}$$

5.

Poisson

$$P_K(k)=\frac{(\lambda T)^k}{k!}e^{-\lambda T}\quad k=0,1,...$$

6.

Exponential

$$f_X(x)=\lambda e^{-\lambda x} \quad x\ge 0$$

Functions of RV
no time to review here
But y=g(x) and f_X(x) known,
first find $F_Y(y)=\int_{g(x)\le y} f_X(x)dx$
then differentiate
THIS ALWAYS WORKS! (book method doesn't always work)

More Defs

$$E(X)=\int_{-\infty}^{\infty} xf_X(x)dx$$

$$E(g(X))=\int_{-\infty}^{\infty}g(x)f_X(x)dx$$

nth moment

$$m^{(n)}_x=\int_{-\infty}^{\infty}x^n f_X(x)dx$$

nth central moment

$$\sigma^{(n)}_x=\int_{-\infty}^{\infty}(x-m_X)^nf_X(x)dx$$

$n=2 \rightarrow$ variance $\sigma^2_x$

1.

E(cX)=cE(X)

2.

E(c)=c

3.

E(X+c)=E(X)+c

4.

$var(cX)=c^2\sigma^2_x$

5.

var(c)=0

6.

$var(X+c)=var(X)=\sigma^2_x$

Characteristic functions

$$\phi_X(s)=\int_{-\infty}^{\infty}f_X(x) e^{j s x}dx\quad \mbox{look like Laplace}$$

props

1.

$$m_X^n=\left.\frac{1}{j^n}\frac{d^n \phi_X(s)}{ds^n}\right\vert _{s=0}$$

2.

$X~ N(m_X, \sigma)$ Gaussian

$$\phi_X(s)=e^{js m_X-\frac{s^2\sigma^2}{2}}$$

More than 1 RV, some basic idea

$$F_{X,Y}(x,y)=P(X\le x, Y\le y)$$

$$f_{X,Y}(x,y)=\frac{\partial}{\partial x \partial y}F_{X,Y}(x,y)$$

Look at props (top p.155)

Covariance

E((x-m_X)(y-m_y))=cov(X,Y)

correlation coefficient

$$\rho=\frac{cov(X,Y)}{\sigma_X \sigma_Y}$$

Turns out that $-1\le\rho\le1$ so its a nice measure of how correlated X+Y are.
no corr.        $\rho=0$ (not indept. necessarily!!!)
same RV         $\rho=1$ (only if Gaussian)
liar         $\rho=-1$

prop

1.

$$E(\sum_ic_iX_i)=\sum_i c_iE(X_i)$$

2.

$$\begin{eqnarray*}var(\sum_i c_i X_i)&=&\sum_{i,j}cov(X_i X_j)c_i c_j\\ &=&\sum_i c_i^2 var(X_i)+\sum_i\sum_{j\ne i} cov(X_iX_j)c_i c_j \end{eqnarray*}$$

Jointly Gaussian
2 vars

$$f_{X,Y}(x,y)=\frac{1}{\sqrt{(2\pi)^2\sigma_X^2\sigma_Y^2(1-\r... ...gma_Y^2}-\frac{2\rho (x-m_X)(y-m_y)}{\sigma_X\sigma_Y}\right)}$$

general

$$f_X(x)=\frac{1}{\sqrt{(2\pi)^2\vert C\vert}} e^{-\frac{1}{2}(... ...ine{x}-\underline{m_X})^TC^{-1}(\underline{x}-\underline{m_X})}$$

where

x=[x₁ x₂ ... x_n]^T

C = cov matrix, |C|=det(C)

Props (Important since will mostly use gaussians)

1.

any subset of jointly gaussian RV's is gaussian as well

2.

completely characterized by their first and second moments

3.

conditioning still makes them jointly gaussian
i.e. f_X1,X2|X3,X4 is jointly gaussian

4.

$\sum a_i X_i$ is gaussian. set of linear combos is gaussian

5.

uncorrelated jointly gaussian RV's are INDEPENDENT

RV sums

1.

Indept. RV's X+Y w/ f_X(x), f_Y(y)

$$Z=X+Y\rightarrow f_Z(z)=f_X(x)\ast f_Y(y)$$

Prove it!

2.

weak law of large numbers, X₁,X₂,... uncorrelated each w/ mean m_X and $\sigma_X^2 < \infty$ then

$$\epsilon>0, \lim_{n\rightarrow\infty} Prob\left[\left\vert\frac{1}{n}\sum_{i=1}^n X_i -m_X \right\vert > \epsilon \right]=0$$

The empirical average converges in probability to the mean

3.

Central limit thm. X₁,...,X_n indept. w/ means ( m₁,...,m_n) and ( $\sigma_1^2,...,\sigma_n^2$)

$$Y=\frac{1}{\sqrt{n}}\sum_{i=1}^n\frac{X_i-m_i}{\sigma_i}\qquad n\rightarrow\infty\Rightarrow f_Y(y)=N(0,1)$$

IID? $\rightarrow Z=\frac{1}{N}\sum_{i=1}^n X_i\Rightarrow f_Z(z)=N(m_X,\sigma_x^2/n)$ when $n\rightarrow\infty$
i.e. Large sums approach a gaussian!
Very useful since physical world is often a superposition of uncorrelated events!