High Performance computing (AL-802 (B)) - Unit 1 Short Notes

UNIT 1: BLOCKCHAIN FUNDAMENTALS & DISTRIBUTED CONSENSUS

I. INTRODUCTION TO BLOCKCHAIN TECHNOLOGY

A. Core Concepts & Definitions

1. Public Ledger

• Definition: A decentralized, immutable digital record of all transactions shared across a network of nodes. It is "public" in the sense of being transparent and auditable by all participants within the network's permission model.

• Function in Financial Transactions: Eliminates the need for a central trusted authority (like a bank). Every participant has a copy, enabling direct peer-to-peer value transfer (e.g., Bitcoin). It provides a single source of truth for account balances and transaction history.

• Transparency vs. Privacy: Transaction data (amount, sender/receiver addresses/timestamps) is visible to all, offering transparency. However, pseudonymity is key—real-world identities are not directly attached to addresses, providing a layer of privacy. In permissioned blockchains, privacy is enhanced through channels and private data.

[!TIP] Exam Focus: Distinguish between public (permissionless, open) and permissioned (private, access-controlled) ledgers. The "public" in public ledger refers to shared visibility among authorized network participants, not necessarily the entire world.

2. Block Structure & Chaining

• Block Composition:

  • Block Header: Contains metadata: Version, Previous Block Hash (critical for chain), Merkle Root (hash of all transactions), Timestamp, Difficulty Target, Nonce.

  • Transaction Counter & List: The actual data/transactions (e.g., financial transfers, smart contract calls).

• "Block within a Block" Concept: Refers to the Merkle Tree structure within a block. Transactions are hashed in pairs, then those hashes are hashed again, recursively, until a single root hash (Merkle Root) is computed and stored in the header. This allows efficient verification of a single transaction without needing the entire block.

• Chaining Mechanism: Each block's header contains the cryptographic hash of the previous block's header. This creates a tamper-evident chain. Altering any transaction in a past block changes its hash, which would break the link to all subsequent blocks, requiring re-mining all subsequent blocks—computationally infeasible.

3. Smart Contracts

• Definition: Self-executing contracts with the terms of the agreement directly written into lines of code. They reside on the blockchain and automatically execute when predefined conditions are met.

• Self-Executing Nature: Once deployed, they run deterministically on all nodes. No intermediary is needed to enforce execution. They manage and transfer assets based on their internal logic.

• Potential Applications:

  • Finance: Automated loan disbursement, insurance payouts, decentralized exchanges.

  • Logistics: Automatic release of payment upon verified delivery (via IoT/sensor data).

  • Legal: Escrow services, royalty distribution, automated compliance.

  • Governance: Transparent voting, token-curated registries.

B. Cryptographic Foundations

1. Hash Functions

• Role: The backbone of blockchain integrity and immutability.

• Properties: Deterministic, Quick to compute, Pre-image resistant, Avalanche effect (small input change → large output change), Collision-resistant.

• Functions in Blockchain:

  • Generate block identifiers (block hash).

  • Create links between blocks (prev_block_hash).

  • Form the basis of the Merkle Tree.

  • Used in Proof-of-Work (PoW) puzzles.

2. Merkle Trees (Hash Trees)

• Structure: A binary tree where leaves are hashes of individual transactions. Each non-leaf node is the hash of its two child nodes.

  
  Level 3 (Root): Hash(H12)
  
                  /        \
  
  Level 2: Hash(H1)   Hash(H2)
  
            /   \       /   \
  
  Level 1: T1   T2   T3   T4  (Transactions)
  
  

• Construction: 1) Hash all transactions. 2) If odd number, duplicate last hash. 3) Pair and hash level-by-level until one root hash remains.

• Role in Efficient Verification:

  • Merkle Proof: To prove a specific transaction (e.g., T2) is in a block, only the hashes along the path from T2 to the root (Hash(H2), Hash(H12)) are needed, not the entire block. This is O(log n) data versus O(n) for full block.

  • Lightweight Clients (SPV): Can verify transaction inclusion by requesting a Merkle Proof from a full node, without storing the entire blockchain.

  • Data Integrity: Any change to a transaction changes its leaf hash, which propagates up, changing the Merkle Root stored in the block header, immediately breaking the chain's integrity.

[!TIP] Exam Focus: Be prepared to draw a simple Merkle Tree for 4 transactions and explain how a Merkle Proof works for a specific transaction. This is a very common 7-mark question.

C. Blockchain Types & Design Considerations

1. Permissionless vs. Permissioned

2. Design Considerations for Permissioned Blockchain

• Access Control & Membership Service: Rigorous identity management (MSP in Fabric), certificate authorities, node/application/user onboarding/offboarding.

• Governance Model: Clear rules for network changes, smart contract upgrades, dispute resolution. Who decides? (Consortium, foundation, company).

• Performance & Scalability: Choice of consensus algorithm (BFT vs. CFT), channel/partitioning design (like Fabric channels), database choice (CouchDB for rich queries), hardware sizing.

• Privacy & Confidentiality: Need-to-know data sharing. Mechanisms: Channels (Fabric), Private Transactions (Corda), Zero-Knowledge Proofs.

• Smart Contract (Chaincode) Lifecycle: Deployment, approval policy, upgrade strategy, versioning, and endorsement policy definition.

• Regulatory Compliance: Integration with existing legal/regulatory frameworks (KYC/AML, GDPR "right to be forgotten" challenges).

II. BITCOIN & CRYPTOCURRENCY ECOSYSTEM

A. Bitcoin Network Architecture

1. Peer-to-Peer (P2P) Network

• Structure: A decentralized, unstructured overlay network. Nodes (peers) connect to a random subset of other nodes (typically 8 outbound, many inbound connections).

• Node Roles:

  • Full Node: Validates all transactions and blocks, stores full blockchain. Enforces consensus rules.

  • Mining Node (Miner): A full node that also competes to solve the PoW puzzle.

  • SPV Client (Lightweight Node): Stores only block headers, uses Merkle Proofs for transaction verification.

• Facilitating Bitcoin Transfer:

  1. User A creates a transaction signing over BTC to User B's address.

  2. A's wallet broadcasts it to its connected P2P peers.

  3. Peers validate the transaction (signature, UTXO existence, no double-spend). If valid, they forward it to their peers (gossip protocol).

  4. Miners collect valid transactions into a candidate block, solve the PoW puzzle.

  5. Winning miner broadcasts the new block to the network.

  6. Other nodes validate the block (including all transactions and PoW). If valid, they add it to their local chain and propagate it. This is transaction propagation & validation.

B. Consensus Mechanism: Proof of Work (PoW)

1. HashCash Proof of Work

• Mechanism: A computational puzzle requiring significant CPU effort to solve but easy to verify.

• The Puzzle (Simplified): Find a nonce such that:

$$ \text{SHA256(SHA256(Block Header))} < \text{Target} $$

Where `Target` is a difficulty-adjusted value. The hash must have a certain number of leading zeros.

• Role in Security:

  • Sybil Attack Prevention: Creating network identity is cheap, but "voting" power (mining) is costly. An attacker needs >50% of global hash power to consistently control block creation.

  • Nakamoto Consensus: The "longest valid chain" is the truth. An attacker must re-mine the attacked block and all subsequent blocks faster than the honest network, which is economically irrational with >50% hash power.

  • Decentralized Leader Election: The miner who solves the puzzle first becomes the leader for that block interval.

2. Mining Process: Daily Challenges/Routines

• Hardware: Use of specialized, power-hungry ASICs (Application-Specific Integrated Circuits) for SHA256 hashing. CPU/GPU mining is obsolete.

• Energy: Massive electricity consumption is the primary cost, directly tied to the global hash rate and difficulty.

• Block Validation: Before starting PoW, miner must:

  1. Validate all transactions in the candidate block (no double-spends, valid signatures, sufficient fees).

  2. Ensure block reward + transaction fees are correctly assigned to their address.

• Reward: Block Subsidy (newly minted BTC, halves ~every 4 years) + Transaction Fees. This is the primary economic incentive.

• Routine: Continuously gather transactions from mempool, assemble candidate block, iterate nonce and other mutable fields (extra nonce) trillions of times per second until a valid hash is found. Immediately broadcast winning block.

3. Mining Pools & Difficulty Adjustment

• Mining Pools: Miners combine hash power to reduce variance in income. Pool operator assembles blocks, and rewards are split proportionally to contributed work (shares). This centralizes mining power but stabilizes miner income.

• Difficulty Adjustment: Every 2016 blocks (~2 weeks), the network recalculates the Target to maintain an average block time of 10 minutes.

$$ \text{New Target} = \text{Old Target} \times \frac{\text{Actual Time of last 2016 blocks}}{2016 \times 10 \text{ minutes}} $$

If blocks are found too quickly, difficulty increases; if too slowly, it decreases.

4. Economic & Security Aspects

• Incentive Structure: Miners are economically incentivized to act honestly because:

  1. Block reward/fees are only received for a valid block added to the main chain.

  2. Mining hardware and electricity are sunk costs; attacking the network (e.g., double-spending) risks the value of their BTC holdings and equipment.

• 51% Attack: If an entity controls >50% of the network's hash power, they can:

  • Censor transactions: Exclude specific transactions from blocks.

  • Double-spend: Spend coins, wait for confirmation, then secretly mine a longer chain that excludes the original payment, and release it. This is the fundamental security assumption of Bitcoin.

  • Cannot: Steal coins from other addresses (need private keys), create new coins beyond the consensus rules, or modify old, deeply confirmed blocks (practically impossible).

III. DISTRIBUTED CONSENSUS ALGORITHMS

A. The Byzantine General Problem

• Problem Statement: A group of Byzantine generals must agree on a common plan of attack (attack or retreat). Some generals may be traitors who will try to prevent agreement. Communication is via messengers who may be delayed, lost, or forged. The problem is to achieve reliable consensus despite faulty/malicious (Byzantine) nodes and unreliable communication.

• Implications: Models the worst-case scenario in distributed systems—nodes can fail in arbitrary ways (crash, send wrong messages, be malicious). Any system handling critical data (money, state) must solve this to be fault-tolerant.

B. Byzantine Fault Tolerance (BFT) Solutions

1. Lamport-Shostak-Pease (Practical Byzantine Fault Tolerance - PBFT)

• How it Handles Byzantine Faults: Operates in replicated state machine model. All replicas (nodes) execute the same state transition (transaction) in the same order. Tolerates up to f Byzantine nodes in a system of 3f + 1 total nodes.

• Message Passing & Voting Mechanism (3-Phase Protocol for a single request):

  1. Request: Client sends request to primary (leader) replica.

  2. Pre-Prepare: Primary assigns a sequence number n to request, broadcasts <<PRE-PREPARE, v, n, d>> to all replicas (v=view, d=request digest).

  3. Prepare: Each replica i that accepts the pre-prepare multicasts <<PREPARE, v, n, d, i>> to all others. A replica prepares a request if it receives 2f+1 matching prepares (including its own).

  4. Commit: After preparing, each replica multicasts <<COMMIT, v, n, d, i>>. A replica commits a request if it receives 2f+1 matching commits. It then executes the request and sends reply to client.

• Key Points:

  • View Change: If primary is faulty/slow, replicas trigger a view change to elect a new primary.

  • Safety & Liveness: Safety (no two non-faulty replicas decide differently) is guaranteed if ≤ f faults. Liveness (eventual progress) requires synchronous network bounds and ≤ f faults.

  • Complexity: O(n²) message complexity per request (where n = 3f+1). This limits scalability but provides fast, deterministic finality (seconds).

[!TIP] Exam Focus: Memorize the 3f+1 tolerance rule and the three phases (Pre-Prepare, Prepare, Commit). Understand that PBFT provides immediate finality unlike probabilistic PoW.

C. Crash Fault Tolerance (CFT) Solutions

1. Paxos Algorithm

• Role: The canonical algorithm for achieving consensus in the presence of crash faults only (nodes can fail-stop but not act maliciously). It's the foundation for many real-world systems (Google Chubby, Spanner, etc.).

• Core Idea: Elect a leader (proposer) to coordinate proposals. Consensus is reached on a single value through a two-phase voting process among a majority (quorum) of acceptors.

• Phases (Simplified Basic Paxos for one value):

  1. Prepare Phase (Leader Election & Proposal):

     • Leader chooses proposal number n, sends Prepare(n) to a quorum of acceptors.

     • Acceptor promises not to accept any proposal numbered < n. It may respond with the highest-numbered proposal it has already accepted ((n', v')).

  2. Accept Phase (Decision):

     • If leader receives promises from a quorum, it sends AcceptRequest(n, v) where v is the value from the highest (n', v') it learned, or its own value if none.

     • Acceptor accepts if it hasn't promised a higher number. When a quorum accepts, the value v is chosen.

• Key Concepts:

  • Quorums: Any two quorums must intersect (share at least one correct acceptor). This ensures consistency.

  • Multi-Paxos: Optimizes Basic Paxos for a sequence of values (log) by using a stable leader for multiple rounds, skipping prepare phases after leader election.

  • Leader Election: If prepare phase fails (no quorum), leader retries with higher n. This naturally handles leader failure.

[!TIP] Exam Focus: Differentiate Paxos (CFT, leader-based, for logs) from PBFT (BFT, replica-based, for state machine replication). Paxos is often described as "hard to understand"—focus on the two-phase quorum concept and the need for a leader.

IV. ENTERPRISE & PERMISSIONED BLOCKCHAIN PLATFORMS

A. Hyperledger Fabric

• Modular Architecture:

  • Peers: Host the ledger and chaincode (smart contracts). Endorsing Peers simulate transactions and check policies. Committing Peers validate and commit blocks.

  • Ordering Service (Orderers): Pluggable consensus service (Raft, Kafka, BFT-SMaRt). Does NOT execute chaincode. Its sole job is to order transactions into blocks and deliver them to peers. This separation of consensus (ordering) from execution (validation) is key to scalability.

  • Channels: Private subnets of communication between specific peers and orderers. Ledger data is partitioned by channel, enabling confidentiality and scalability (different business networks on same Fabric network).

  • Membership Service Provider (MSP): Manages identity (X.509 certificates) and access control.

• Consensus Mechanism: Pluggable & Not PoW. The ordering service determines the order of transactions (consensus on order). Peers then validate transactions against endorsement policies and world state (consensus on validity). Common: Raft (CFT, leader-based, crash fault tolerant) for most use cases.

• Smart Contract (Chaincode) Execution Model:

  1. Client sends transaction proposal to endorsing peers.

  2. Endorsers simulate execution, produce read/write sets (RWSet), and sign endorsement.

  3. Client collects sufficient endorsements (per policy), submits transaction to ordering service.

  4. Orderers order transactions into a block and broadcast to all peers.

  5. Committing peers validate RWSet against current world state and policies. If valid, commit block to ledger. Invalid transactions are still in the block but marked as such.

B. Alternative Enterprise Platforms

1. Ripple (XRP Ledger)

• Focus: Cross-border payments & settlement (real-time gross settlement system).

• Consensus Protocol (RPCA - Ripple Protocol Consensus Algorithm):

  • Unique Node List (UNL): Each server (validator) has a curated list of other validators it trusts. Consensus is reached only among servers in a server's UNL.

  • Process: In rounds, servers propose candidate transaction sets. They vote on the validity of transactions. A transaction is considered validated when >80% of a server's UNL agrees. This is a Byzantine agreement protocol within a trusted subset.

  • Finality: ~3-5 seconds. Deterministic.

• Use Cases: Bank-to-bank transfers, remittances, liquidity provision (using XRP as bridge currency).

2. Corda

• Notary-based Consensus: No global broadcast of all transactions. Consensus is achieved on a per-transaction basis among only the notary (for uniqueness) and the required counterparties.

• Privacy Model: Point-to-point communication. Transactions are shared only with necessary parties. No shared ledger. Each node maintains its own vault of relevant transactions. Confidentiality is achieved through transaction privacy and notary clustering.

• Legal Entity-Centric Design: States (data) represent legal agreements between specific, known identities (parties). Corda's model maps directly to real-world legal relationships and contracts.

• Use Cases: Trade finance, syndicated loans, insurance, inter-company reconciliation.

3. Comparative Analysis: Fabric vs. Ripple vs. Corda

V. APPLICATIONS & INDUSTRY IMPACT

A. Financial Services & Compliance

1. Know Your Customer (KYC) Process

• Key Components:

  1. Customer Identification: Collecting verified documents (passport, proof of address).

  2. Customer Due Diligence (CDD): Understanding nature/purpose of relationship, expected activity.

  3. Enhanced Due Diligence (EDD): For high-risk customers (PEPs, high-risk jurisdictions), deeper investigation.

  4. Ongoing Monitoring: Tracking transactions for suspicious activity (SARs).

  5. Risk Assessment: Periodic review and risk scoring.

• Importance for Financial Institutions: Mandatory regulatory requirement (anti-money laundering - AML, counter-terrorist financing - CTF). Prevents fraud, identity theft, financial crime, and reputational damage. Non-compliance results in massive fines.

• Blockchain's Role in KYC/AML:

  • Shared, Verified KYC Registry: A permissioned network where institutions can, with customer consent, query and verify KYC data. Reduces duplication, cost, and onboarding time.

  • Immutable Audit Trail: All KYC updates and verifications are immutably logged.

  • Improved AML Monitoring: Real-time, cross-institutional transaction monitoring on a shared ledger (with privacy controls) can detect complex money laundering patterns (smurfing, layering) more effectively.

  • Digital Identity: Self-sovereign identity (SSI) models on blockchain let users control and present verified credentials.

2. Supply Chain Financing

• Concept: Financing solutions (e.g., invoice discounting, PO financing) that bridge the gap between order placement and final payment in a supply chain. Traditionally, SMEs face cash flow issues due to long payment terms.

• How Blockchain Enables Transparency & Trust:

  • Single Source of Truth: All parties (supplier, buyer, financier, logistics) share an immutable record of purchase orders, invoices, shipments, and receipts.

  • Automated Triggering: Smart contracts can automatically release payment to the supplier upon verified delivery (IoT sensor data, document hash match) or invoice approval by the buyer.

  • Reduced Fraud: Tamper-proof records of asset ownership (e.g., bill of lading) prevent duplicate financing.

  • Lower Risk & Cost: Financiers can assess real-time risk based on verified supply chain events, leading to lower interest rates and faster approval.

B. Global Trade & Supply Chains

• Impact:

  • Traceability & Provenance: Immutable, step-by-step history of a product from raw material to end consumer. Combats counterfeiting (luxury goods, pharmaceuticals).

  • Transparency & Trust: All authorized participants see the same data, reducing disputes and the need for reconciliation.

  • Automation & Efficiency: Smart contracts automate customs clearance, letter of credit execution, and payments upon condition fulfillment, reducing paperwork and delays.

  • Compliance: Simplifies proof of ethical sourcing (conflict minerals, fair trade) and regulatory compliance (FDA, customs).

• Use Cases:

  • Provenance Tracking: IBM Food Trust (Walmart, Nestlé) tracking food sources for safety recalls.

  • Logistics: Maersk-TradeLens platform for shipping container tracking and documentation.

  • Automotive: BMW's PartChain for tracking automotive parts.

C. Cross-Industry Applications (Brief)

• Healthcare: Secure, patient-centric health records; drug traceability; clinical trial data integrity.

• Voting: Immutable, auditable, and potentially remote voting systems (challenges: identity, coercion, accessibility).

• Real Estate: Streamlined title transfers, reduced fraud, fractional ownership tokens.

VI. SHORT NOTE TOPICS (From Past Exams)

(Detailed explanations are provided within the relevant sections above. This is a summary index.)

• Merkle Tree: See I.B.2. Focus on structure, construction, and O(log n) verification efficiency via Merkle Proofs. High-frequency exam question.

• HashCash Proof of Work: See II.B.1. Explain the computational puzzle (hash < target), its role in Sybil resistance and leader election.

• Paxos Algorithm: See III.C.1. Explain the two-phase (Prepare, Accept) quorum-based consensus for crash faults. Emphasize leader role and quorum intersection.

• Lamport-Shostak-Pease (PBFT) Algorithm: See III.B.1. Explain the three-phase (Pre-Prepare, Prepare, Commit) protocol for Byzantine faults, 3f+1 tolerance, and deterministic finality.

• Hyperledger Fabric Architecture: See IV.A. Emphasize modularity (separation of ordering/execution), channels for privacy, pluggable consensus (Raft), and the endorsement-execution-commit flow.

• Ripple and Corda (Comparative): See IV.B.3 table. Contrast their domains (payments vs. legal agreements), consensus (RPCA vs. Notary), ledger model (shared vs. point-to-point), and privacy.

• Supply Chain Financing: See V.A.2. Explain how blockchain's transparency and smart contracts automate financing based on verified supply chain events.

• Public Ledger in Financial Transactions: See I.A.1. Define it, explain its function (disintermediation, single source of truth), and discuss the transparency-privacy (pseudonymity) balance.

VII. SYNTHESIS & COMPARATIVE ANALYSIS

A. Consensus Mechanism Comparison

Trade-offs Summary:

• Security (BFT): PoW & PBFT tolerate malicious nodes. CFT (Paxos, Raft) only tolerates crashes.

• Scalability (Throughput): CFT > BFT > PoW. PoW's block time and size limit TPS.

• Finality: CFT/BFT (deterministic) > PoW (probabilistic, requires waiting).

• Energy: PoW is orders of magnitude more wasteful.

• Decentralization: Permissionless PoW is most decentralized (open participation). Permissioned systems are more centralized by design for performance/control.

B. Blockchain Selection Framework

Choose based on:

1. Trust Model: Do you need to trust unknown entities? → Permissionless PoW/PoS. Do you know/trust participants? → Permissioned (PBFT/CFT).

2. Performance Needs: High TPS & low latency required? → Permissioned (Fabric, Ripple, Corda).

3. Data Privacy: Need confidential transactions between subsets? → Fabric (channels), Corda (point-to-point).

4. Finality Requirement: Need instant, irreversible settlement? → BFT/CFT systems. Can wait for confirmations? → PoW.

5. Regulatory Compliance: Need identity, access control, audit trails? → Permissioned.

6. Asset Type: Representing physical legal agreements? → Corda. General business logic? → Fabric. Native cryptocurrency? → Bitcoin/Ethereum.

C. Challenges & Limitations

• Scalability Trilemma (Decentralization-Security-Scalability): It's extremely difficult to maximize all three simultaneously. PoW maximizes decentralization/security at cost of scalability. Permissioned chains maximize scalability/security at cost of decentralization.

• Interoperability: Blockchains are siloed. Moving assets/data between chains (e.g., Bitcoin to Ethereum) requires complex bridges, which are often vulnerable to hacks.

• Regulatory & Legal Hurdles: Jurisdictional uncertainty, smart contract legal enforceability, data privacy laws (GDPR) vs. immutability, tax treatment of crypto-assets.

• Usability & Adoption: Poor user experience (private key management), lack of standards, integration with legacy systems.

• Energy Consumption (for PoW): Major environmental and economic criticism.

[!TIP] Exam Focus: Be ready to discuss the Scalability Trilemma and give examples of how different blockchains (Bitcoin vs. Fabric) make different trade-offs. This is a common higher-level analysis question.