Probability and Statistics for Data Science (AD-302) - Unit 3 Short Notes

How unit 3 is examined

Binomial, Poisson and Normal distributions (with their proofs and Z-table numericals) and least-squares fitting of a line and a parabola carry almost all the marks; every question is 7 marks.

Theoretical Distribution: Discrete Distribution- Binomial Distribution

<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">High weight</span>

Definition. ==A random variable $X$ follows the binomial distribution $B(n,p)$ if it counts the successes in $n$ independent trials, each with the same success probability $p$, and $P(X=r)=\binom{n}{r}p^r q^{n-r}$ for $r=0,1,\dots,n$, where $q=1-p$.==

Key points.

1. Each trial has exactly two outcomes, success with probability $p$ and failure with probability $q=1-p$.
2. The number of trials $n$ is fixed and the trials are independent, so $p$ stays constant.
3. The parameters are $n$ and $p$, and the probabilities sum to $(q+p)^n=1$.
4. Mean $=np$, variance $=npq$, so the mean is always greater than the variance.
5. It is symmetric when $p=q=\tfrac12$, skewed right when $p<\tfrac12$ and skewed left when $p>\tfrac12$.
6. For $N$ repetitions of the experiment, the expected frequency of $r$ successes is $N\cdot P(X=r)$.

Derivation of mean and variance. Use the identities $r\binom{n}{r}=n\binom{n-1}{r-1}$ and $r(r-1)\binom{n}{r}=n(n-1)\binom{n-2}{r-2}$. $$E[X]=\sum_{r=0}^{n} r\binom{n}{r}p^rq^{n-r}=np\sum_{r=1}^{n}\binom{n-1}{r-1}p^{r-1}q^{n-r}=np(q+p)^{n-1}=np$$ $$E[X(X-1)]=\sum r(r-1)\binom{n}{r}p^rq^{n-r}=n(n-1)p^2(q+p)^{n-2}=n(n-1)p^2$$ $$\text{Var}(X)=E[X(X-1)]+E[X]-(E[X])^2=n(n-1)p^2+np-n^2p^2=np-np^2=npq$$

Example. 5 births, $p=q=\tfrac12$: $P(X=r)=\binom5r/32$.

$P(X\ge1)=1-\tfrac1{32}=\tfrac{31}{32}$; $P(X\le3)=1-\tfrac6{32}=\tfrac{26}{32}=\tfrac{13}{16}$. Of 960 families: $960\times\tfrac{31}{32}=930$ and $960\times\tfrac{26}{32}=780$. Ten coins, $n=10$, $p=\tfrac12$: $P(X\ge7)=\dfrac{120+45+10+1}{1024}=\dfrac{176}{1024}=\dfrac{11}{64}$.

Answer frame. Open with the definition and the pmf; for the derivation, write $E[X]$ then $E[X(X-1)]$ then the variance, closing with $np$ and $npq$; for numericals, list $n,p,q$, write the pmf, compute the terms and box the answer.

Pitfall: "At least" means the sum of upper terms; use the complement ($1-P(X=0)$) when it is shorter.

Asked: [7 marks] (Dec 2023, Dec 2025) Ten coins are thrown simultaneously. Find the probability of getting at least seven heads. Asked: [7 marks] (Nov 2022) Assuming the probability of male birth as 1/2, find the probability distribution of the number of boys out of 5 births; probability that a family of 5 children has (I) at least one boy, (II) at most 3 boys; out of 960 families the expected number in each case. Asked: [7 marks] (Dec 2025) Derive mean and variance of the Binomial distribution.

Poisson Distribution

<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">High weight</span>

Definition. ==A discrete random variable $X$ follows the Poisson distribution with parameter $\lambda$ if $P(X=x)=\dfrac{e^{-\lambda}\lambda^x}{x!}$ for $x=0,1,2,\dots$; it is the limit of the binomial when $n\to\infty$, $p\to0$ and $np=\lambda$ stays finite.==

Key points.

1. It counts rare events in a fixed interval of time, length or area, with $\lambda$ as the average number of occurrences.
2. Mean $=$ variance $=\lambda$, which is its identifying property.
3. It has only one parameter $\lambda$, and it is skewed to the right with mode at the integer part of $\lambda$ (two modes, $\lambda-1$ and $\lambda$, when $\lambda$ is an integer).
4. The sum of independent Poisson variables with $\lambda_1,\lambda_2$ is Poisson with $\lambda_1+\lambda_2$ (additive property).
5. Applications: calls arriving at an exchange, radioactive decay, accidents or traffic arrivals, misprints per page, defects in manufactured items.
6. Use it in place of the binomial when $n$ is large and $p$ is small, with $\lambda=np$.

Proof (limit of binomial). Put $p=\lambda/n$: $$P(X=x)=\frac{n(n-1)\cdots(n-x+1)}{x!}\frac{\lambda^x}{n^x}\left(1-\frac\lambda n\right)^{n-x}=\frac{\lambda^x}{x!}\left(1-\frac1n\right)\cdots\left(1-\frac{x-1}{n}\right)\frac{(1-\lambda/n)^n}{(1-\lambda/n)^x}$$ As $n\to\infty$ each factor $(1-\tfrac kn)\to1$, $(1-\tfrac\lambda n)^x\to1$ and $(1-\tfrac\lambda n)^n\to e^{-\lambda}$, so $P(X=x)\to\dfrac{e^{-\lambda}\lambda^x}{x!}$.

Example. Six coins tossed 6400 times: $p=(\tfrac12)^6=\tfrac1{64}$, $\lambda=6400\times\tfrac1{64}=100$, so $P(r\text{ six-head throws})=\dfrac{e^{-100}100^r}{r!}$. 520 pages with 390 errors: $\lambda=390/520=0.75$ per page, so for 5 pages $\lambda=3.75$ and $P(0)=e^{-3.75}=0.0235$. Condensers, 1% defective, box of 100: $\lambda=1$, $P(X\ge4)=1-e^{-1}\left(1+1+\tfrac12+\tfrac16\right)=1-0.9810=0.0190$.

Sketch (bar chart). Draw bars at $x=0,1,2,\dots$ with heights $P(x)$; the tallest bar stands near $x=\lambda$, the bars on the left fall quickly to $x=0$, and the bars on the right shrink slowly into a long right tail, which shows the positive skew (for $\lambda=2$: 0.135, 0.271, 0.271, 0.180, 0.090, 0.036).

Answer frame. For the list-and-properties question, open with the pmf, give the applications, then properties 1-4. For the proof, state the binomial pmf, substitute $p=\lambda/n$, take limits, and close with the Poisson pmf. For numericals, find $\lambda$ first, then substitute.

Asked: [7 marks] (Jun 2023, Dec 2023, Dec 2024) Six coins are tossed 6,400 times; using the Poisson distribution find the probability of getting six heads $r$ times. Also: 390 errors in 520 pages, probability that 5 pages contain no error; 1% defective condensers in boxes of 100, probability of 4 or more faulty. Asked: [7 marks] (Nov 2022) List some important areas where the Poisson distribution is used; state its properties. Asked: [7 marks] (Dec 2024) Prove the Poisson distribution is a limiting case of the binomial distribution.

Continuous Distribution –Rectangular and Normal distribution

<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">High weight</span>

Definition. ==A continuous variable $X$ is normal $N(\mu,\sigma^2)$ if its pdf is $f(x)=\dfrac1{\sigma\sqrt{2\pi}}e^{-(x-\mu)^2/2\sigma^2}$, $-\infty<x<\infty$; the standard variable is $Z=\dfrac{X-\mu}{\sigma}\sim N(0,1)$.== The rectangular (uniform) distribution on $[a,b]$ has $f(x)=\dfrac1{b-a}$, mean $\dfrac{a+b}2$ and variance $\dfrac{(b-a)^2}{12}$.

Key points.

1. The curve is bell-shaped and symmetric about $x=\mu$, so mean = median = mode $=\mu$.
2. The total area under the curve is 1, and it has points of inflection at $\mu\pm\sigma$.
3. Areas: $\mu\pm\sigma$ holds 68.27%, $\mu\pm2\sigma$ holds 95.45% and $\mu\pm3\sigma$ holds 99.73%.
4. Quartile deviation $=0.6745\sigma\approx\tfrac23\sigma$; mean deviation $=\sqrt{2/\pi}\,\sigma=0.7979\sigma\approx\tfrac45\sigma$; SD $=\sigma$.
5. The sum or difference of independent normals is normal: means add or subtract, variances always add.
6. For sample means, $\sigma_{\bar X}=\sigma/\sqrt n$ and $Z=(\bar X-\mu)/\sigma_{\bar X}$.
7. A valid pdf needs $f\ge0$ and $\int f\,dx=1$; the median $m$ satisfies $\int_{-\infty}^m f=\tfrac12$.

Proof of QD : MD : SD = 10 : 12 : 15. The quartiles are $\mu\pm q\sigma$ where $P(0<Z<q)=0.25$; the normal table gives $q=0.6745$, so QD $=\tfrac{Q_3-Q_1}2=0.6745\sigma\approx\tfrac23\sigma$. MD $=\displaystyle\int_{-\infty}^{\infty}|x-\mu|\,\frac1{\sigma\sqrt{2\pi}}e^{-(x-\mu)^2/2\sigma^2}dx$; put $z=\dfrac{x-\mu}{\sigma}$, $dx=\sigma\,dz$, and since $|z|e^{-z^2/2}$ is even the integral is $2\times\int_0^\infty$, so MD $=\dfrac{2\sigma}{\sqrt{2\pi}}\int_0^\infty z\,e^{-z^2/2}dz=\dfrac{2\sigma}{\sqrt{2\pi}}\left[-e^{-z^2/2}\right]_0^\infty=\dfrac{2\sigma}{\sqrt{2\pi}}\times1=\sqrt{2/\pi}\,\sigma\approx\tfrac45\sigma$, and SD $=\sigma$, so the ratio is $\tfrac23:\tfrac45:1$; multiplying by 15 gives $10:12:15$.

Examples.

• Coffee, $\mu=200,\sigma=4$: $P(X\ge200)=0.5$; $Z=1.5$, $P(200\le X\le206)=0.4332$. Resistors ($\mu=100,\sigma=2$): $P(98<X<102)=P(-1<Z<1)=2\times0.3413=0.6826$, i.e. 68.26%.
• Engines, $n=100$: $\sigma_{\bar X}=1.5$, $Z=\dfrac{217-220}{1.5}=-2$, $P=0.5-0.4772=0.0228$.
• $Z=X-Y$: mean $1-2=-1$, variance $9+16=25$, SD 5, median $-1$; pdf $f(z)=\dfrac1{5\sqrt{2\pi}}e^{-(z+1)^2/50}$; $P(Z+1\le0)=P(Z\le-1)=0.5$.
• Income $\mu=750,\sigma=50$: $X=688\Rightarrow Z=-1.24$, $P(X>688)=0.5+0.3925=0.8925$ (89.25%, not 95%), so the paper almost certainly misprints Rs 668: $Z=\tfrac{668-750}{50}=-1.64$, $P(X>668)=0.5+0.4495=0.9495\approx95\%$, matching the 5% tail above 832; show both values and state the misprint; $X=832\Rightarrow Z=1.64$, $P=0.0505\approx5\%$; top $100/10000=1\%$ gives $Z=2.33$, $x=750+2.33\times50=\text{Rs }866.5$.
• Normal table extract (area from 0 to $z$): $1.00\to0.3413$, $1.24\to0.3925$, $1.50\to0.4332$, $1.64\to0.4495$, $2.00\to0.4772$, $2.33\to0.4901\approx0.49$.
• $f=cx^2$: $\int_0^1cx^2dx=\tfrac c3=1\Rightarrow c=3$; $P(\tfrac13<x<\tfrac12)=[x^3]_{1/3}^{1/2}=\tfrac18-\tfrac1{27}=\tfrac{19}{216}$.
• $f=6x(1-x)$: $\int_0^16(x-x^2)dx=1$ and $f\ge0$, so valid; $3b^2-2b^3=\tfrac12\Rightarrow 4b^3-6b^2+1=0$. Quick route: $f$ is symmetric about $x=\tfrac12$, so $b=\tfrac12$. Check: $4b^3-6b^2+1=(2b-1)(2b^2-2b-1)$, and the other roots $b=\tfrac{1\pm\sqrt3}2\approx1.366,-0.366$ lie outside $[0,1]$, so $b=\tfrac12$ is the only valid root.

Answer frame. Open with "Standardise using $Z=(X-\mu)/\sigma$"; draw the bell curve with $\mu$, the shaded area and the Z-limits; read the table value; close with the boxed probability. For the proof, state the three measures in terms of $\sigma$ and form the ratio. For pdf questions, integrate to 1 first.

Pitfall: Variances add for $X-Y$ too; never subtract them.

Asked: [7 marks] (Nov 2022, Dec 2024) Coffee pouches, mean 200 gm, SD 4 gm: probability of at least 200 gm and between 200 and 206 gm; also resistors, percentage between 98 and 102 ohms. Asked: [7 marks] (Nov 2022) Engines, mean 220 hp, SD 15 hp, sample of 100: probability that the sample mean is less than 217 hp. Asked: [7 marks] (Dec 2023) Prove that for the normal distribution QD, MD and SD are approximately 10 : 12 : 15. Asked: [7 marks] (Jun 2023) X and Y independent normal, means 1 and 2, SD 3 and 4; $Z=X-Y$: write the pdf, median, SD, mean; find $P(Z+1\le0)$. Asked: [7 marks] (Dec 2024) Income of 10,000 persons normal, mean 750, SD 50: about 95% exceed 688, 5% exceed 832; lowest income among the richest 100. Asked: [7 marks] (Dec 2025) If $f(x)=cx^2$, $0<x<1$, find $c$ and $P(1/3<x<1/2)$. Asked: [7 marks] (Jun 2023) Cable diameter pdf $f(x)=6x(1-x)$: check it is a pdf; find $b$ with $P(X<b)=P(X>b)$. Asked: [7 marks] (Nov 2022) Differentiate between Binomial, Poisson and Normal distributions. Asked: [7 marks] (Dec 2025) Define Binomial, Poisson and Normal distributions.

Curve fitting and Methods of Least square

<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">Not asked since 2022</span>

Definition. ==Curve fitting finds the equation of a curve $y=f(x)$ that best represents a set of data points, and the method of least squares chooses the constants so that $\sum(y_i-f(x_i))^2$ is minimum.==

Key points.

1. The difference $y_i-f(x_i)$ is the residual, and least squares minimises the sum of the squared residuals.
2. Setting the partial derivatives with respect to each constant to zero gives the normal equations.
3. For a line $y=a+bx$: $\sum y=na+b\sum x$ and $\sum xy=a\sum x+b\sum x^2$.
4. The number of normal equations equals the number of constants.

fitting a Straight line and a Parabola

<span style="display:inline-block;padding:.16em .6em;border:1.5px solid currentColor;border-radius:999px;font-size:.68em;font-weight:700;letter-spacing:.06em;text-transform:uppercase;opacity:.75">High weight</span>

Definition. ==Fitting a straight line $y=a+bx$ or a parabola $y=a+bx+cx^2$ by least squares means solving the normal equations obtained from minimising $\sum e^2$.==

Formulas. $$\text{Line: }\sum y=na+b\sum x,\quad \sum xy=a\sum x+b\sum x^2$$ $$\text{Parabola: }\sum y=na+b\sum x+c\sum x^2,\ \ \sum xy=a\sum x+b\sum x^2+c\sum x^3,\ \ \sum x^2y=a\sum x^2+b\sum x^3+c\sum x^4$$

Key points.

1. Prepare a table of $x,y,x^2,xy$ (and $x^3,x^4,x^2y$ for a parabola) and total each column.
2. For equally spaced data shift the origin to the middle, $u=x-\text{middle}$ (for an even count use $u=2(x-\text{middle})$), so that $\sum u=0$ and $\sum u^3=0$ and the equations separate.
3. With $\sum u=0$ the line gives $a=\bar y$ and $b=\sum uy/\sum u^2$.
4. Transform back by substituting $u$ in terms of $x$ at the end.

Example 1 (line). Years 2012-2017, sales 7, 10, 12, 14, 17, 24. Middle is 2014.5, take $u=2(X-2014.5)=-5,-3,-1,1,3,5$: $\sum u=0$, $\sum u^2=70$, $\sum Y=84$, $\sum uY=108$. $a=84/6=14$, $b=108/70=1.543$, so $Y=14+1.543u$. Rule: when $u=(X-\text{mid})/h$, the slope per year is $b_u/h$; here $h=\tfrac12$, so the $u$-coefficient is doubled: $Y=14+3.086(X-2014.5)$, a rise of 3.086 thousand sets a year. Trend values: $Y(2012)=14-3.086\times2.5=6.29$, then 9.37, 12.46, 15.54, 18.63, 21.71 for 2013-2017.

Example 2 (parabola). $x=1..7$, $y=2.3,5.2,9.7,16.5,29.4,35.5,54.4$. Take $u=x-4$ ($\sum u^3=0$):

$7a+28c=153$; $28b=236.6$; $28a+196c=712.2$. So $b=8.45$, $c=1.1929$, $a=17.0857$. Put $u=x-4$: $a+b(x-4)+c(x-4)^2$ has constant $a-4b+16c=17.086-33.8+19.086=2.371$ and $x$-coefficient $b-8c=8.45-9.543=-1.093$, with $x^2$-coefficient $c=1.193$. $y=17.086+8.45u+1.193u^2$, i.e. $y=2.371-1.093x+1.193x^2$.

Variant 1 (line). $x=1,2,3,4,6,8$, $y=2.4,3,3.6,4,5,6$: $\sum x=24$, $\sum y=24$, $\sum x^2=130$, $\sum xy=113.2$. Normal equations $24=6a+24b$, $113.2=24a+130b$ give $b=0.506$, $a=1.976$, so $y=1.976+0.506x$.

Variant 2 (line). $x=1..6$, $y=5.04,8.12,10.64,13.18,16.2,20.04$: $\sum x=21$, $\sum y=73.22$, $\sum x^2=91$, $\sum xy=307.16$. Normal equations $73.22=6a+21b$, $307.16=21a+91b$ give $b=2.908$, $a=2.025$, so $y=2.025+2.908x$.

Variant 3 (parabola). $x=0..4$, $y=1,1.8,1.3,2.5,6.3$: $\sum x=10$, $\sum x^2=30$, $\sum x^3=100$, $\sum x^4=354$, $\sum y=12.9$, $\sum xy=37.1$, $\sum x^2y=130.3$. $5a+10b+30c=12.9$; $10a+30b+100c=37.1$; $30a+100b+354c=130.3$. Eliminating $a$: $10b+40c=11.3$ and $40b+174c=52.9$, so $14c=7.7$, $c=0.55$, $b=-1.07$, $a=1.42$: $y=1.42-1.07x+0.55x^2$. (If a key prints $\sum xy=39.1$, $\sum x^2y=139.5$, the same steps give $y=1.191-1.213x+0.636x^2$.)

Answer frame. Open with "Let the line be $y=a+bx$; normal equations are ..."; draw the tabulation of sums; substitute the totals into the normal equations; solve for the constants; close with the fitted equation.

Pitfall: Column totals decide everything; recheck $\sum xy$ and $\sum x^2y$ before solving.

Asked: [7 marks] (Nov 2022, Dec 2023, Dec 2024) Fit a straight line by least squares to the sales data (2012-2017); also x = 1,2,3,4,6,8 with y = 2.4,3,3.6,4,5,6; and x = 1..6 with y = 5.04 ... 20.04. Asked: [7 marks] (Jun 2023, Dec 2023, Dec 2025) Fit a parabola $y=a+bx+cx^2$ to x = 1..7, y = 2.3, 5.2, 9.7, 16.5, 29.4, 35.5, 54.4; also a second-degree parabola to x = 0..4, y = 1, 1.8, 1.3, 2.5, 6.3.

Last-minute revision

• Binomial: $P(r)=\binom nr p^rq^{n-r}$, mean $np$, variance $npq$.
• Poisson: $P(x)=e^{-\lambda}\lambda^x/x!$, mean = variance = $\lambda$; use $\lambda=np$.
• Poisson is the limit of the binomial as $n\to\infty$, $p\to0$, $np=\lambda$.
• Ten coins, at least 7 heads: $176/1024=11/64$.
• Six coins in 6400 tosses: $\lambda=100$.
• Normal: $Z=(X-\mu)/\sigma$; 68.27%, 95.45%, 99.73% within 1, 2, 3 SD.
• $\Phi(1.5)-0.5=0.4332$; $P(Z<-2)=0.0228$.
• QD : MD : SD $=10:12:15$.
• Uniform: mean $(a+b)/2$, variance $(b-a)^2/12$.
• Line normal equations: $\sum y=na+b\sum x$, $\sum xy=a\sum x+b\sum x^2$.
• Parabola needs three equations and sums up to $\sum x^4$ and $\sum x^2y$.

Memory hooks

• Binomial = "B for Both": mean > variance ($np>npq$).
• Poisson = "Pair": mean and variance are a pair of equals.
• Normal areas: 68-95-99.7.
• Ratio 10:12:15 = 2/3, 4/5, 1 times 15.
• Origin at the middle makes $\sum u=0$ and halves the algebra.

Coverage checklist

• Theoretical Distribution: Discrete Distribution- Binomial Distribution: ten coins at least seven heads; 5 births and 960 families; derive mean and variance.
• Poisson Distribution: six coins 6400 tosses; areas and properties; limit of binomial.
• Continuous Distribution –Rectangular and Normal distribution: coffee and resistors; engines; 10:12:15 proof; Z = X - Y; income; $cx^2$; $6x(1-x)$; differentiate three distributions; define three distributions.
• Curve fitting and Methods of Least square: definition and normal equations.
• fitting a Straight line and a Parabola: sales trend line; parabola of 7 points; other data sets.