High Performance computing (AL-802 (B)) - Important Questions

Explain the Paxos algorithm and its role in achieving distributed consensus.

What is the Byzantine General Problem, and how does it relate to distributed consensus?

How does the Lamport-Shostak-Pease algorithm handle Byzantine faults in a distributed system?

Describe the Message Passing Interface (MPI) programming model. Explain point-to-point communication primitives (send/recv), their blocking and non-blocking variants, and typical use-cases.

Compare MPI point-to-point and collective communications; when would you prefer non-blocking communication over blocking communication? Illustrate with examples.

Apply Amdahl's law to compute theoretical speedup for a parallel program and discuss limits to scaling. Given the fraction $P$ of the program that is parallelizable and $N$ processors, derive the speedup formula and explain the scaling implications using $$S = \frac{1}{\left(1-P\right)+\frac{P}{N}}.$$

Describe OpenMP "parallel for" and the data scoping clauses (shared, private, reduction). Explain common pitfalls such as race conditions and false sharing, and show brief code snippets to illustrate each concept.

Perform a roofline / balance model calculation: given total floating-point operations (FLOPs) and memory traffic (Bytes), compute the operational intensity $$I = \frac{\text{FLOPs}}{\text{Bytes}}$$ and determine whether a kernel is compute-bound or memory-bound by comparing $I$ to the machine balance. Explain the steps and reasoning.