1. Descriptive Statistics
Measures of Central Tendency
-
Mean:
-
Ungrouped: $$\displaystyle \bar{x} = \frac{\sum x_i}{n} $$
-
Grouped: $$\displaystyle \bar{x} = \frac{\sum f_i x_i}{\sum f_i} $$ (where $$\displaystyle x_i $$ = class midpoints)
-
-
Median:
-
Ungrouped: value at position $(n+1)/2$ (sorted data).
-
Grouped: $$\displaystyle L + \left(\frac{n/2 - cf}{f}\right) \times h $$ (L = lower limit of median class, cf = cumulative freq before, f = freq of median class, h = class width).
-
-
Mode:
- Grouped: $$\displaystyle L + \left(\frac{f_1 - f_0}{2f_1 - f_0 - f_2}\right) \times h $$ (f₁ = freq of modal class, f₀, f₂ = adjacent class freqs).
-
Merits of Mean: Uses all data, mathematically tractable, suitable for further analysis.
-
Demerits of Mean: Affected by outliers, not definable for open-ended distributions.
Measures of Dispersion
-
Variance:
-
Population: $$\displaystyle \sigma^2 = \frac{\sum (x_i - \mu)^2}{N} $$
-
Sample: $$\displaystyle s^2 = \frac{\sum (x_i - \bar{x})^2}{n-1} $$
-
-
Standard Deviation: $$\displaystyle \sigma = \sqrt{\sigma^2} $$ (population), $$\displaystyle s = \sqrt{s^2} $$ (sample).
-
Mean Deviation: $$\displaystyle \frac{\sum |x_i - \text{central tendency}|}{n} $$ (usually about median or mean).
-
Calculations: For grouped data, replace $$\displaystyle x_i $$ with midpoints and use frequencies.
Measures of Shape
-
Skewness:
-
Pearson's coefficients:
-
First: $$\displaystyle \frac{\text{Mean} - \text{Mode}}{\text{SD}} $$
-
Second: $$\displaystyle \frac{3(\text{Mean} - \text{Median})}{\text{SD}} $$
-
-
Moment-based: $$\displaystyle \gamma_1 = \frac{\mu_3}{\sigma^3} $$ (skewness). Positive → right-skewed, negative → left-skewed.
-
-
Kurtosis:
-
$$\displaystyle \beta_2 = \frac{\mu_4}{\sigma^4} $$ (kurtosis). For normal distribution, $$\displaystyle \beta_2 = 3 $$.
-
Excess kurtosis = $$\displaystyle \beta_2 - 3 $$.
-
$$\displaystyle \beta_1 = \frac{\mu_3^2}{\mu_4} $$ (also related to skewness measure).
-
-
Tests: Based on sample moments; compare with normal values ($$\displaystyle \gamma_1 \approx 0 $$, $$\displaystyle \beta_2 \approx 3 $$).
Rank Correlation
- Spearman's Rank Correlation Coefficient:
$$ \rho = 1 - \frac{6\sum d_i^2}{n(n^2-1)} $$
where $$\displaystyle d_i $$ = difference between ranks of paired observations.
-
For tied ranks: assign average ranks.
-
Properties: Non-parametric, measures monotonic relationship, robust to outliers.
-
Comparison with Pearson:
| Pearson | Spearman | |---|---| | Measures linear correlation | Measures monotonic correlation | | Uses raw data | Uses ranks | | Sensitive to outliers | Less sensitive | | Parametric | Non-parametric |
2. Probability Fundamentals
-
Events and Sample Spaces: Sample space $S$ = set of all outcomes; events = subsets of $S$.
-
Conditional Probability:
$$ P(A|B) = \frac{P(A \cap B)}{P(B)}, \quad P(B) > 0 $$
-
Independence: Events $A, B$ independent iff $$\displaystyle P(A \cap B) = P(A)P(B) $$.
-
Bayes' Theorem:
$$ P(A_i|B) = \frac{P(B|A_i)P(A_i)}{\sum_j P(B|A_j)P(A_j)} $$
[!TIP] Common in medical diagnosis, defective item problems, and updating probabilities.
Proof: From $$\displaystyle P(A_i \cap B) = P(B|A_i)P(A_i) $$ and $$\displaystyle P(B) = \sum_j P(B|A_j)P(A_j) $$.
3. Random Variables
-
Discrete Random Variables:
-
PMF: $$\displaystyle p(x) = P(X = x) $$, with $p(x) \ge 0$, $$\displaystyle \sum_x p(x) = 1 $$.
-
CDF: $$\displaystyle F(x) = P(X \le x) = \sum_{t \le x} p(t) $$.
-
-
Continuous Random Variables:
-
PDF: $f(x) \ge 0$, $$\displaystyle \int_{-\infty}^\infty f(x)dx = 1 $$.
-
CDF: $$\displaystyle F(x) = \int_{-\infty}^x f(t)dt $$.
-
-
Independence: Joint PMF/PDF factorizes: $$\displaystyle p(x,y) = p_X(x)p_Y(y) $$ or $$\displaystyle f(x,y) = f_X(x)f_Y(y) $$.
-
Expectation:
-
Discrete: $$\displaystyle E(X) = \sum_x x p(x) $$.
-
Continuous: $$\displaystyle E(X) = \int_{-\infty}^\infty x f(x)dx $$.
-
Linearity: $$\displaystyle E(aX + b) = aE(X) + b $$, and $$\displaystyle E(X_1 + \cdots + X_n) = E(X_1) + \cdots + E(X_n) $$ (no independence required).
-
-
Variance:
$$ V(X) = E[(X - \mu)^2] = E(X^2) - [E(X)]^2 $$
-
Properties: $$\displaystyle V(aX + b) = a^2 V(X) $$, $$\displaystyle V\left(\sum_{i=1}^n X_i\right) = \sum_{i=1}^n V(X_i) $$ if $$\displaystyle X_i $$ are independent.
-
Moments:
-
Non-central: $$\displaystyle \mu_r' = E(X^r) $$.
-
Central: $$\displaystyle \mu_r = E[(X - \mu)^r] $$.
-
-
Moment Generating Function (MGF):
$$ M_X(t) = E(e^{tX}) $$
-
Properties: $$\displaystyle M_{aX+b}(t) = e^{bt} M_X(at) $$; $$\displaystyle M_X^{(r)}(0) = \mu_r' $$.
-
Characteristic Function (CF):
$$ \varphi_X(t) = E(e^{itX}) $$
- Always exists; $$\displaystyle \varphi_X(t) = M_X(it) $$ if MGF exists.
4. Common Probability Distributions
Discrete Distributions
-
Binomial $X \sim \text{Bin}(n, p)$:
-
PMF: $$\displaystyle P(X = k) = \binom{n}{k} p^k (1-p)^{n-k} $$, $$\displaystyle k = 0,1,\dots,n $$.
-
Mean: $np$, Variance: $np(1-p)$.
-
MGF: $$\displaystyle M(t) = (1-p + pe^t)^n $$.
-
-
Poisson $X \sim \text{Pois}(\lambda)$:
-
PMF: $$\displaystyle P(X = k) = e^{-\lambda} \frac{\lambda^k}{k!} $$, $$\displaystyle k = 0,1,2,\dots $$
-
Mean: $\lambda$, Variance: $\lambda$.
-
MGF: $$\displaystyle M(t) = e^{\lambda(e^t - 1)} $$.
-
Limiting case of Binomial: As $n \to \infty$, $p \to 0$, $$\displaystyle np = \lambda $$, binomial PMF $\to$ Poisson PMF.
-
-
Hypergeometric $X \sim \text{Hyper}(N, K, n)$:
-
PMF: $$\displaystyle P(X = k) = \frac{\binom{K}{k}\binom{N-K}{n-k}}{\binom{N}{n}} $$.
-
Mean: $$\displaystyle n\frac{K}{N} $$, Variance: $$\displaystyle n\frac{K}{N}\left(1-\frac{K}{N}\right)\frac{N-n}{N-1} $$.
-
Continuous Distributions
-
Exponential $X \sim \text{Exp}(\theta)$ (rate $\theta$):
-
PDF: $$\displaystyle f(x) = \theta e^{-\theta x} $$, $x \ge 0$.
-
Mean: $$\displaystyle \frac{1}{\theta} $$, Variance: $$\displaystyle \frac{1}{\theta^2} $$.
-
MGF: $$\displaystyle M(t) = \frac{\theta}{\theta - t} $$, $$\displaystyle t < \theta $$.
-
Median: $$\displaystyle \frac{\ln 2}{\theta} $$, Mode: $0$.
-
-
Gamma $X \sim \text{Gamma}(\alpha, \beta)$ (shape $\alpha$, rate $\beta$):
-
PDF: $$\displaystyle f(x) = \frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x} $$, $$\displaystyle x > 0 $$.
-
Mean: $$\displaystyle \frac{\alpha}{\beta} $$, Variance: $$\displaystyle \frac{\alpha}{\beta^2} $$.
-
MGF: $$\displaystyle M(t) = \left(\frac{\beta}{\beta - t}\right)^\alpha $$, $$\displaystyle t < \beta $$.
-
CF: $$\displaystyle \varphi(t) = \left(\frac{\beta}{\beta - it}\right)^\alpha $$.
-
Exponential is Gamma with $$\displaystyle \alpha = 1 $$.
-
-
Normal $$\displaystyle X \sim N(\mu, \sigma^2) $$:
-
PDF: $$\displaystyle f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}} $$.
-
Properties: symmetric about $\mu$, mean = $\mu$, variance = $$\displaystyle \sigma^2 $$.
-
MGF: $$\displaystyle M(t) = e^{\mu t + \frac{1}{2}\sigma^2 t^2} $$.
-
CF: $$\displaystyle \varphi(t) = e^{i\mu t - \frac{1}{2}\sigma^2 t^2} $$.
-
Standard normal: $$\displaystyle Z = \frac{X - \mu}{\sigma} \sim N(0,1) $$.
-
Finding parameters from percentiles: use $z$-scores and solve $$\displaystyle \mu + z\sigma = x $$.
-
-
Chi-square $$\displaystyle \chi^2 \sim \chi^2(k) $$:
-
Special case of Gamma: shape $k/2$, rate $1/2$.
-
Mean: $k$, Variance: $2k$.
-
CF: $$\displaystyle \varphi(t) = (1 - 2it)^{-k/2} $$.
-
5. Advanced Distribution Topics
- Poisson as Limiting Case of Binomial:
$$ \lim_{n \to \infty, p \to 0, np = \lambda} \binom{n}{k} p^k (1-p)^{n-k} = e^{-\lambda} \frac{\lambda^k}{k!} $$
[!TIP] Use when $n$ large, $p$ small, and $$\displaystyle \lambda = np $$ moderate.
-
Sum of Independent Normal Random Variables:
If $$\displaystyle X_i \sim N(\mu_i, \sigma_i^2) $$ independent, then $$\displaystyle \sum_{i=1}^n X_i \sim N\left(\sum \mu_i, \sum \sigma_i^2\right) $$.
-
Chebyshev's Inequality:
$$ P(|X - \mu| \ge k\sigma) \le \frac{1}{k^2} \quad \text{for any } k > 0 $$
[!TIP] Holds for any distribution; useful for bounds without distributional assumptions.
Proof: Apply Markov's inequality to $$\displaystyle Y = (X-\mu)^2 $$: $$\displaystyle P(Y \ge k^2\sigma^2) \le \frac{E(Y)}{k^2\sigma^2} = \frac{\sigma^2}{k^2\sigma^2} = \frac{1}{k^2} $$.
6. Bivariate and Multivariate Analysis
-
Bivariate Distributions:
-
Joint PMF/PDF: $f(x,y)$ with $$\displaystyle \sum_{x,y} f(x,y) = 1 $$ (discrete) or $$\displaystyle \int\int f(x,y)dxdy = 1 $$ (continuous).
-
Marginal: $$\displaystyle f_X(x) = \sum_y f(x,y) $$ or $$\displaystyle \int f(x,y)dy $$.
-
Conditional: $$\displaystyle f_{Y|X}(y|x) = \frac{f(x,y)}{f_X(x)} $$.
-
Independence: $$\displaystyle f(x,y) = f_X(x)f_Y(y) $$ for all $x,y$.
-
-
Covariance and Correlation:
-
Covariance: $$\displaystyle \text{Cov}(X,Y) = E[(X-\mu_X)(Y-\mu_Y)] = E(XY) - E(X)E(Y) $$.
-
Pearson's Correlation Coefficient:
-
$$ \rho = \frac{\text{Cov}(X,Y)}{\sigma_X \sigma_Y}, \quad -1 \le \rho \le 1 $$
-
Types: positive ($$\displaystyle \rho > 0 $$), negative ($$\displaystyle \rho < 0 $$), zero ($$\displaystyle \rho = 0 $$).
-
Regression Analysis:
- Regression coefficients:
$$ b_{yx} = \frac{\text{Cov}(X,Y)}{\text{Var}(X)}, \quad b_{xy} = \frac{\text{Cov}(X,Y)}{\text{Var}(Y)} $$
-
Properties: $$\displaystyle b_{yx} \cdot b_{xy} = \rho^2 $$, both have same sign as $\rho$.
-
Regression lines:
-
$y$ on $x$: $$\displaystyle y - \bar{y} = b_{yx}(x - \bar{x}) $$.
-
$x$ on $y$: $$\displaystyle x - \bar{x} = b_{xy}(y - \bar{y}) $$.
-
-
Partial and Multiple Correlation: Measure relationship between one variable and a set of others while controlling for additional variables.
-
Curve Fitting (Least Squares):
- Straight line $$\displaystyle y = a + bx $$:
$$ \begin{cases} \sum y = na + b\sum x \\ \sum xy = a\sum x + b\sum x^2 \end{cases} $$
- Parabola $$\displaystyle y = a + bx + cx^2 $$:
$$ \begin{cases} \sum y = na + b\sum x + c\sum x^2 \\ \sum xy = a\sum x + b\sum x^2 + c\sum x^3 \\ \sum x^2 y = a\sum x^2 + b\sum x^3 + c\sum x^4 \end{cases} $$
-
Bivariate Normal Distribution:
- PDF:
$$ f(x,y) = \frac{1}{2\pi\sigma_X\sigma_Y\sqrt{1-\rho^2}} \exp\left(-\frac{1}{2(1-\rho^2)}\left[\frac{(x-\mu_X)^2}{\sigma_X^2} - 2\rho\frac{(x-\mu_X)(y-\mu_Y)}{\sigma_X\sigma_Y} + \frac{(y-\mu_Y)^2}{\sigma_Y^2}\right]\right) $$
- Properties: marginals are normal, conditionals are normal, $\rho$ is the correlation.
7. Inferential Statistics
Sampling and Estimation
-
Point Estimation: Use sample statistic (e.g., $\bar{x}$ for $\mu$, $$\displaystyle s^2 $$ for $$\displaystyle \sigma^2 $$).
-
Confidence Intervals:
-
Mean (known $\sigma$): $$\displaystyle \bar{x} \pm z_{\alpha/2} \frac{\sigma}{\sqrt{n}} $$.
-
Mean (unknown $\sigma$): $$\displaystyle \bar{x} \pm t_{\alpha/2, n-1} \frac{s}{\sqrt{n}} $$.
-
Proportion: $$\displaystyle \hat{p} \pm z_{\alpha/2} \sqrt{\frac{\hat{p}(1-\hat{p})}{n}} $$.
-
Hypothesis Testing
-
Null and Alternative Hypotheses: $$\displaystyle H_0 $$ (status quo), $$\displaystyle H_1 $$ (claim).
-
Test Statistics:
-
$$\displaystyle z = \frac{\bar{x} - \mu_0}{\sigma/\sqrt{n}} $$ (large $n$ or known $\sigma$).
-
$$\displaystyle t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}} $$ (small $n$, unknown $\sigma$).
-
$$\displaystyle \chi^2 = \sum \frac{(O_i - E_i)^2}{E_i} $$ (goodness of fit/independence).
-
$$\displaystyle F = \frac{s_1^2}{s_2^2} $$ (equality of variances, larger variance in numerator).
-
-
Errors:
-
Type I: Reject $$\displaystyle H_0 $$ when true (probability $\alpha$).
-
Type II: Fail to reject $$\displaystyle H_0 $$ when false (probability $\beta$).
-
-
Common Tests:
-
$z$-test for single mean/proportion.
-
$t$-test for single mean (small samples).
-
Difference of means (large samples):
-
$$ z = \frac{(\bar{x}_1 - \bar{x}_2) - (\mu_1 - \mu_2)}{\sqrt{\sigma_1^2/n_1 + \sigma_2^2/n_2}} $$
- Proportions (pooled):
$$ z = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\hat{p}(1-\hat{p})(1/n_1 + 1/n_2)}}, \quad \hat{p} = \frac{x_1 + x_2}{n_1 + n_2} $$
-
$F$-test: $$\displaystyle F = \frac{\text{larger } s^2}{\text{smaller } s^2} $$, compare with $$\displaystyle F_{\alpha, df_1, df_2} $$.
-
Chi-square Test:
-
Goodness of Fit: $$\displaystyle H_0 $$: data fits specified distribution.
-
Conditions: random sample, categorical data, expected frequencies $$\displaystyle E_i \ge 5 $$ (no more than 20% below 5), independent observations.
-
Procedure: Compute $$\displaystyle \chi^2 = \sum \frac{(O_i - E_i)^2}{E_i} $$, $$\displaystyle df = k - 1 - m $$ (k categories, m estimated parameters).
-
-
Test of Independence: $$\displaystyle H_0 $$: two categorical variables independent.
-
Contingency table: $$\displaystyle df = (r-1)(c-1) $$.
-
Same $$\displaystyle \chi^2 $$ statistic and conditions.
-
-
8. Special Topics and Problem-Solving
-
Finding Parameters from Percentiles (Normal distribution):
Given $$\displaystyle P(X < x_1) = p_1 $$, $$\displaystyle P(X < x_2) = p_2 $$, find $$\displaystyle z_1, z_2 $$ from standard normal table, then solve:
$$ \mu + z_1\sigma = x_1, \quad \mu + z_2\sigma = x_2. $$
-
Applications of Bayes' Theorem:
-
Defective items: $$\displaystyle P(\text{source}|\text{defective}) = \frac{P(\text{defective}|\text{source})P(\text{source})}{\sum P(\text{defective}|\text{source}_i)P(\text{source}_i)} $$.
-
Medical diagnosis: Update probability of disease given test result.
-
-
Using Poisson for Rare Events: When events occur independently at constant average rate $\lambda$, and $n$ large, $p$ small.
-
Comparison of Correlation and Regression:
| Correlation | Regression | |---|---| | Measures strength/direction of linear association | Predicts value of dependent variable | | Unitless ($\rho$) | Coefficients have units (slope) | | Symmetric: $$\displaystyle \rho_{XY} = \rho_{YX} $$ | Not symmetric: regression lines differ | | $\rho$ between -1 and 1 | No such bound on regression coefficients |
-
Methodology for Difference of Means (Large Samples):
-
Use $z$-test assuming $$\displaystyle \sigma_1, \sigma_2 $$ known or large samples (CLT).
-
If $$\displaystyle \sigma_1, \sigma_2 $$ unknown, use sample variances $$\displaystyle s_1^2, s_2^2 $$ as estimates.
-
Test statistic:
-
$$ z = \frac{(\bar{x}_1 - \bar{x}_2) - (\mu_1 - \mu_2)}{\sqrt{s_1^2/n_1 + s_2^2/n_2}} $$
- Confidence interval: $$\displaystyle (\bar{x}_1 - \bar{x}_2) \pm z_{\alpha/2} \sqrt{s_1^2/n_1 + s_2^2/n_2} $$.