Class Notes, Class 4

Review outline of Chapter 3:

Random variable
Discrete random variable, probability (mass) function
p(x) or f(x)
Continuous random variable, probability density function
f(x)
Joint probability function, joint probability density function
intersection
Marginal probability, marginal probability density
sum or integrate over other variable(s)
Conditional probability, conditional probability density
take a "slice", divide by marginal to "renormalize" (sum to 1)
i.e. divide joint by marginal

Multiplicative Rule - always true
follows from definition of conditional
Bayes' Theorem - use multiplicative rule more than once

Independence - only true sometimes
p(B|A) = p(B) (alt. p(A|B) = p(A))

Multivariate independence

Basic concepts of statistics, based on commonsense ideas, that we will refine: expectation, estimation, decision criterion

The Algebra of Expectation

These are the building blocks that let us calculate our expectations for samples, given the underlying distribution and our sampling scheme.

Multiple notations for the same thing:

Mean of a random variable
analogy with weighted mean for a sample
(analogy with center of gravity)
Alternate name: the Expected Value of the random variable

Expected value of a random variable

Discrete
Continuous

 
New random variable, g(X), that depends on X; i.e. a transform of X
(the values change, but the probabilities or probability densities don't)
New random variable, g(X, Y), that depends on the joint random variable, (X,Y)
(the values change and go from being pairs to being single values, but the probabilities or probability densities don't)

Examples p115 4.4 4.5

Variance and covariance

Variance of random variable X
(think of Mean Squared Error, MSE)
Standard deviation of random variable X
(think of Root Mean Squared Error, RMS error)
Covariance of a joint random variable (X,Y)
The covariance of (X, Y) is a measure of the linear relationship between X and Y.
(Note Variance is just Covariance with itself)
Correlation of a joint random variable (X,Y)
Correlation is normalized covariance

Exercises p127 4.49, 4.43 (see reminder at bottom of page), Example p124 4.13

Expected value for linear combinations of random variables

Means
a, b constants
E(b) = b
E(aX) = aE(X)
E(X + b) = E(X) + b
E(aX + b) = aE(X) + b
X, Y random variables
E(X + Y) = E(X) + E(Y)
E(X - Y) = E(X) - E(Y)
E(aX + bY) = aE(X) + bE(Y)
g(X), h(X) functions of X
E[g(X) + h(X)] = E[g(X))] + E[(h(X)]
E[g(X) - h(X)] = E[g(X))] - E[(h(X)]
X, Y independent random variables
E(XY) = E(X)E(Y)

Examples pp128,129 4.17 4.18 4.19

Variances
a, b constants
V(b) = 0
V(aX) = a2V(X)
V(X + b) = V(X)
V(aX + b) = a2V(X)
X, Y joint independent random variables
V(X + Y) = V(X) + V(Y)
V(X - Y) = V(X) + V(Y)
V(aX + bY) = a2V(X) + b2V(Y)
C(X, Y) = 0
If X,Y independent then Cov = 0
If Cov = 0, then X,Y not necessarily independent
X, Y joint random variables, not necessarily independent
V(X + Y) = V(X) + V(Y) + 2C(X,Y)
V(X - Y) = V(X) + V(Y) - 2C(X,Y)
V(aX + bY) = a2V(X) + b2V(Y) + 2abC(X,Y)

Examples p133 4.22 4.23


Quick reminder:


Class Home