Advanced Statistical Analytics (AD-703 (C)) Unit 1 Notes

How unit 1 is examined

This unit covers populations, samples, sampling distributions, the finite population correction, the normal-sampling results and the Central Limit Theorem; no topic was asked in recent papers, so all are short.

Population and Sample

Not asked since 2022

Definition. A population is the complete collection of all units under study; a sample is a part of the population selected to draw conclusions about it. A sample is studied so that inference about the whole population can be made cheaply and quickly.

Key points.

A population may be finite, with $N$ countable units, or infinite.
The number of units in a sample is the sample size $n$.
A good sample is random and representative, so that every unit has a known chance of selection.
Sampling is used when a census is costly, slow or destructive.

Random Sampling from finite population (SRSWR and SRSWOR)

Definition. Simple random sampling gives every unit an equal chance of selection. SRSWR draws with replacement, so there are $N^n$ equally likely samples; SRSWOR draws without replacement, so there are $\binom{N}{n}$ equally likely samples.

Key points.

In SRSWR the same unit may appear more than once and the draws are independent.
In SRSWOR a unit cannot repeat, so the draws are dependent.
In SRSWR each unit has probability $1/N$ at every draw; in SRSWOR each sample has probability $1/\binom{N}{n}$.
Selection is done by lottery or random number tables.

Parameter and Statistic

Definition. A parameter is a numerical characteristic of the population, such as $\mu$ or $\sigma^2$; a statistic is a function of the sample values, such as $\bar{x}$ or $s^2$. A parameter is fixed and usually unknown, whereas a statistic is a random variable that changes from sample to sample.

Key points.

Parameters are written in Greek letters and statistics in Roman letters.
A statistic used to guess a parameter is called an estimator.
Sample mean is $\bar{x}=\frac{1}{n}\sum x_i$ and sample variance is $s^2=\frac{1}{n-1}\sum (x_i-\bar{x})^2$.
Population proportion is $P$ and sample proportion is $p$.

Sampling distribution of a statistic in the context of a finite population

Definition. The sampling distribution of a statistic is the probability distribution of its values over all possible samples of size $n$. The standard deviation of a sampling distribution is called the standard error.

Key points.

List every possible sample, compute the statistic for each, and tabulate the values with their probabilities.
For $N=3$ and $n=2$ there are 9 samples under SRSWR and 3 under SRSWOR.
The sampling distribution shows how much the statistic varies from sample to sample.
Standard error measures that variation; a smaller standard error means a more reliable statistic.

Sampling distribution of sample mean and sample proportion while sampling from a finite population

Definition. For a population with mean $\mu$ and variance $\sigma^2$, the sample mean $\bar{x}$ has mean $\mu$ under both schemes, and the sample proportion $p$ has mean $P$. Under SRSWOR the variance carries the finite population correction $(N-n)/(N-1)$.

Formula. $$\text{SRSWR: } Var(\bar{x})=\frac{\sigma^2}{n}\qquad \text{SRSWOR: } Var(\bar{x})=\frac{\sigma^2}{n}\cdot\frac{N-n}{N-1}$$ $$E(p)=P,\qquad Var(p)=\frac{PQ}{n}\ (\text{SRSWR}),\qquad Var(p)=\frac{PQ}{n}\cdot\frac{N-n}{N-1}\ (\text{SRSWOR})$$

Example. Population $\{2,4,6\}$: $\mu=4$, $\sigma^2=8/3$, $n=2$.

Scheme	Sample means	$E(\bar{x})$	$Var(\bar{x})$
SRSWR (9 samples)	2, 3, 4, 3, 4, 5, 4, 5, 6	4	$\frac{8/3}{2}=\frac{4}{3}$
SRSWOR (3 samples)	3, 4, 5	4	$\frac{4}{3}\cdot\frac{1}{2}=\frac{2}{3}$

Answer. $\bar{x}$ is unbiased under both schemes; $Var(\bar{x})$ is $4/3$ for SRSWR and $2/3$ for SRSWOR.

Random sampling from an infinite population

Definition. A random sample from an infinite population is a set $X_1,\dots,X_n$ of independent and identically distributed (i.i.d.) random variables with the population distribution. Every draw is independent, so no finite correction is needed.

Key points.

Removing units does not change an infinite population, so with and without replacement coincide.
For i.i.d. draws with mean $\mu$ and variance $\sigma^2$, $E(\bar{X})=\mu$ and $Var(\bar{X})=\sigma^2/n$.
The standard error of the mean is $\sigma/\sqrt{n}$.
A finite population that is very large compared with $n$ is treated as infinite.

Sampling Distribution of sample mean and sample variance when the sample is drawn from a Normal distribution

Definition. If $X_1,\dots,X_n$ is a random sample from $N(\mu,\sigma^2)$, then $\bar{X}\sim N(\mu,\sigma^2/n)$ exactly. $\frac{(n-1)s^2}{\sigma^2}$ follows a chi-square distribution with $n-1$ degrees of freedom and is independent of $\bar{X}$.

Key points.

$Z=\frac{\bar{X}-\mu}{\sigma/\sqrt{n}}$ follows the standard normal distribution $N(0,1)$.
$\chi^2=\frac{(n-1)s^2}{\sigma^2}\sim\chi^2_{n-1}$, with $E(s^2)=\sigma^2$ and $Var(s^2)=\frac{2\sigma^4}{n-1}$.
$t=\frac{\bar{X}-\mu}{s/\sqrt{n}}$ follows Student's t with $n-1$ degrees of freedom, because $\bar{X}$ and $s^2$ are independent.
Both results are exact for any $n$, not only large $n$.

Problems on sampling distributions of statistics from finite and infinite populations

Definition. These numericals find the mean, standard error and probability of a sample statistic from the population values. Use $\sigma/\sqrt{n}$ for infinite or with-replacement sampling and multiply by $\sqrt{(N-n)/(N-1)}$ for finite sampling without replacement.

Steps.

Identify $N$, $n$, $\mu$, $\sigma$ and the scheme.
Compute the standard error with or without the correction.
Standardise: $Z=\frac{\bar{x}-\mu}{SE}$.
Read the probability from normal tables.

Example. $\mu=50$, $\sigma=10$, $n=25$, infinite population: $SE=10/5=2$, so $P(\bar{X}>52)=P(Z>1)=0.1587$.

Statement of Lyndeberg-Levy Central Limit Theorem (CLT) and its applications

Definition. If $X_1,\dots,X_n$ are i.i.d. with finite mean $\mu$ and finite variance $\sigma^2$, then $\frac{\bar{X}-\mu}{\sigma/\sqrt{n}}$ tends to $N(0,1)$ as $n\to\infty$. For large $n$, the sample mean is approximately normal whatever the population distribution.

Key points.

The variables must be independent, identically distributed and have finite variance.
A sample size $n\ge 30$ is the usual rule of thumb for a good approximation.
It justifies normal tests and confidence intervals for means with large samples.
It gives the normal approximation to the binomial and Poisson distributions.

Last-minute revision

Parameter is a population value; statistic is a sample value and a random variable.
SRSWR has $N^n$ samples; SRSWOR has $\binom{N}{n}$ samples.
$E(\bar{x})=\mu$ in both schemes.
$Var(\bar{x})=\sigma^2/n$ for SRSWR and infinite populations.
SRSWOR multiplies the variance by $(N-n)/(N-1)$.
Standard error of the mean is $\sigma/\sqrt{n}$.
$Var(p)=PQ/n$.
$(n-1)s^2/\sigma^2\sim\chi^2_{n-1}$; $E(s^2)=\sigma^2$.
$t=(\bar{X}-\mu)/(s/\sqrt{n})$ has $n-1$ degrees of freedom.
CLT: $\bar{X}$ is approximately $N(\mu,\sigma^2/n)$ for large $n$.
For $\{2,4,6\}$, $n=2$: $Var(\bar{x})$ is $4/3$ (SRSWR) and $2/3$ (SRSWOR).

Memory hooks

WR means With Replacement, so more samples ($N^n$) and no correction.
Greek for parameter, Roman for statistic.
"Standard error is the standard deviation of a statistic."
CLT: large $n$ makes the mean normal, whatever the parent.

Coverage checklist

Population and Sample: covered, no past questions.
Random Sampling from finite population (SRSWR and SRSWOR): covered, no past questions.
Parameter and Statistic: covered, no past questions.
Sampling distribution of a statistic in the context of a finite population: covered, no past questions.
Sampling distribution of sample mean and sample proportion while sampling from a finite population: covered, no past questions.
Random sampling from an infinite population: covered, no past questions.
Sampling Distribution of sample mean and sample variance when the sample is drawn from a Normal distribution: covered, no past questions.
Problems on sampling distributions of statistics from finite and infinite populations: covered, no past questions.
Statement of Lyndeberg-Levy Central Limit Theorem (CLT) and its applications: covered, no past questions.

Account

Advanced Statistical Analytics (AD-703 (C)) - Unit 1 Short Notes

How unit 1 is examined

Population and Sample

Random Sampling from finite population (SRSWR and SRSWOR)

Parameter and Statistic

Sampling distribution of a statistic in the context of a finite population

Sampling distribution of sample mean and sample proportion while sampling from a finite population

Random sampling from an infinite population

Sampling Distribution of sample mean and sample variance when the sample is drawn from a Normal distribution

Problems on sampling distributions of statistics from finite and infinite populations

Statement of Lyndeberg-Levy Central Limit Theorem (CLT) and its applications

Last-minute revision

Memory hooks

Coverage checklist