Big Data Analytics (AL-802 (C)) - Unit 3 Short Notes

UNIT 3: BLOCKCHAIN TECHNOLOGIES & DISTRIBUTED SYSTEMS

I. BLOCKCHAIN FUNDAMENTALS & CORE ARCHITECTURE

A. Public Ledger

• Concept: A decentralized, append-only database (ledger) shared across all nodes in a network. Every transaction is recorded and visible to all participants.

• Function in Financial Transactions:

  • Eliminates need for a central trusted authority (e.g., bank).

  • Provides a single, immutable source of truth for all transaction history.

  • Enables peer-to-peer (P2P) value transfer.

• Immutability: Once data is written in a block and added to the chain, altering it requires changing all subsequent blocks and gaining majority network consensus, which is computationally infeasible in large networks.

  [!TIP] Exam Focus: Link immutability to cryptographic hashing and distributed consensus. A public ledger's trust comes from transparency and mathematical proof, not a central institution.

B. Blockchain Structure

1. Block:

   • Header: Contains metadata: Version, Previous Block Hash (link to parent), Merkle Root (hash of all transactions), Timestamp, Difficulty Target, Nonce.

   • Body: Contains the list of transactions.

   • Hash Pointer ("Block within a Block"): The Previous Block Hash field in a block's header is the cryptographic hash of the entire header of the preceding block. This creates a linked list structure where each block contains a reference (hash) to its parent.

     \boxed{\text{Block } N \xrightarrow{\text{contains hash of}} \text{Block } N-1}

   • Contribution to Structure: This chaining ensures that altering any historical block changes its hash, breaking the link for all subsequent blocks. This is the core mechanism enabling tamper-evidence.

2. Genesis Block: The first block in the blockchain (block #0). It has no Previous Block Hash (often set to 0). It is hardcoded into the software and serves as the root of trust for the entire chain.

C. Merkle Trees (Hash Trees)

1. Structure & Construction:

   • Transactions in a block are hashed (using SHA-256 in Bitcoin).

   • Hashes are paired and hashed again. This pairing and hashing continues in a tree-like fashion until a single root hash remains: the Merkle Root.

   • DiagramSEARCH: "Merkle Tree blockchain structure"

2. Role in Efficient Verification & Integrity:

   • Efficiency: To prove a specific transaction (T) is in a block, a Merkle Proof only needs the log₂(N) sibling hashes along the path from T to the root, not all transactions. This allows lightweight clients to verify transactions without downloading the entire block.

   • Data Integrity: Any change to a single transaction changes its hash, which propagates up, altering the Merkle Root. A node can quickly detect inconsistency by recalculating the root from the provided proof.

D. Cryptography Primitives

1. Cryptographic Hash Functions (e.g., SHA-256):

   • Properties: Deterministic, Fast to compute, Pre-image resistant, Second pre-image resistant, Collision resistant, Avalanche effect (small input change → large output change).

   • Role: Used to create block identifiers (hashes of headers), generate Merkle Trees, and in Proof-of-Work. Ensures data integrity and links blocks.

2. Digital Signatures (e.g., ECDSA - Elliptic Curve Digital Signature Algorithm):

   • Mechanism: Sender uses their private key to sign a transaction hash. Anyone can use the sender's public key to verify the signature.

   • Role in Transaction Authentication: Provides non-repudiation (proof of origin) and integrity (ensures transaction details weren't altered after signing). It authenticates the owner of the funds without revealing the private key.

II. DISTRIBUTED CONSENSUS MECHANISMS

A. The Byzantine General Problem & BFT

• Problem: A group of Byzantine generals must agree on a common plan of attack (attack/retreat). Some generals may be traitors who send conflicting messages. How to achieve reliable consensus despite faulty/malicious nodes?

• Byzantine Fault Tolerance (BFT): The property of a system that can tolerate up to f faulty/malicious nodes in a network of 3f + 1 total nodes and still reach correct consensus.

B. Consensus Algorithms

1. Proof-of-Work (PoW):

   • HashCash Mechanism: Miners compete to find a nonce such that Hash(Block Header) is less than a difficulty target (a number with leading zeros). This is a brute-force, probabilistic lottery.

   • Role in Security (Sybil Attack Resistance): The high computational cost (work) required to create a valid block makes it economically infeasible for an attacker to create multiple identities (Sybil attack) to dominate the network. Security is tied to physical hardware and energy.

   • Mining Process:

     • Difficulty Target: Adjusts periodically to maintain a constant block time (e.g., 10 mins in Bitcoin).

     • Nonce: A 32-bit number miners iterate through to find a valid block hash.

     \boxed{\text{Find } \text{nonce} \text{ such that } H(\text{header}) < \text{target}}

2. Practical Byzantine Fault Tolerance (PBFT): A state machine replication algorithm. Nodes exchange multiple rounds of voting (pre-prepare, prepare, commit) to agree on the next block. Efficient but communication overhead is O(n²). Tolerates <1/3 faulty nodes.

3. Paxos Algorithm: A family of protocols for achieving consensus in non-Byzantine (crash-fault) environments. Uses a proposer, acceptors, and learners. Role: Provides a foundational model for many consensus systems, but classic Paxos is complex and not directly used in blockchains; variants like Raft are more common in permissioned systems.

4. Lamport-Shostak-Pease (Oral Messages) Algorithm: A theoretical solution to the Byzantine Generals Problem for synchronous networks with digital signatures. It uses a recursive, message-passing scheme where each general sends their order to all others, who then forward it. With m traitors, requires m+1 rounds of messaging. Key Insight: Digital signatures prevent message forgery, limiting the power of traitors.

C. The Blockchain Trilemma

\boxed{\text{Decentralization} \quad \text{Security} \quad \text{Scalability}}

• Trade-off: It is extremely difficult to maximize all three simultaneously.

  • PoW (Bitcoin): High Security & Decentralization → Low Scalability (low TPS).

  • PBFT (Permissioned): High Security & Scalability → Low Decentralization (known, limited nodes).

  • Sharding/New Chains: Attempt to improve Scalability but may compromise Security or Decentralization.

III. BITCOIN NETWORK & MINING ECOSYSTEM

A. Bitcoin P2P Network Architecture

1. Node Types:

   • Full Node: Stores entire blockchain, validates all transactions/blocks. Backbone of the network.

   • Lightweight (SPV) Node: Stores only block headers. Uses Merkle proofs to verify transactions. Relies on full nodes.

   • Mining Node: A full node that also runs mining software to compete for block rewards.

2. Transaction & Block Propagation: Using a gossip protocol. A node broadcasts a new transaction/block to its peers, who validate it and rebroadcast to their peers, achieving network-wide propagation in seconds.

3. Facilitating Transfer: User creates & signs transaction → Broadcasts to P2P network → Nodes validate (UTXO check, signature) → Mempool → Miner includes in candidate block → After PoW, winning miner broadcasts new block → Nodes validate block & all transactions → If valid, add to local chain and update UTXO set. Confirmation count increases with subsequent blocks.

B. Bitcoin Mining

1. Miner's Role: Validate pending transactions (prevent double-spends), assemble them into a candidate block, perform PoW to find a valid nonce, broadcast the winning block. Secures the network.

2. Daily Routines & Challenges:

   • Hardware: Specialized ASICs (Application-Specific Integrated Circuits) are mandatory for competitive mining. High capital cost.

   • Energy Consumption: PoW is intentionally energy-intensive. Major operational cost and environmental concern.

   • Pool Mining: Miners combine hash power in pools to reduce variance of rewards. Pool operator distributes block rewards proportionally to contributed work.

   • Reward Structure: Block Subsidy (new BTC created, halves every 210,000 blocks) + Transaction Fees (sum of fees from transactions in the block). This is the economic incentive.

3. Mining Economics: Driven by revenue (block reward + fees) vs. costs (hardware depreciation + electricity). Mining difficulty adjusts to target 10-minute block intervals, making it a competitive, low-margin business.

C. Transaction Lifecycle

1. Creation: Sender creates transaction (inputs = UTXOs they own, outputs = recipient address + change, amount).

2. Signing: Sender signs transaction hash with private key.

3. Broadcasting: Transaction sent to P2P network, enters nodes' mempools.

4. Confirmation: Miner includes it in a block. After that block is mined, it has 1 confirmation. Each subsequent block adds a confirmation, making reversal exponentially harder.

IV. SMART CONTRACTS & DECENTRALIZED APPLICATIONS (DAPPS)

A. Concept

Self-executing code stored on a blockchain. It automatically enforces the rules and outcomes defined in its code when predefined conditions are met (e.g., "if X happens, transfer Y tokens to Z").

B. Key Properties

• Autonomy: Once deployed, it runs without any intermediary.

• Decentralization: Stored and executed across the network.

• Immutability: Code and execution history cannot be altered.

• Transparency: Code is publicly verifiable (on public chains).

C. Potential Applications

1. Finance (DeFi): Automated lending, borrowing, decentralized exchanges (DEXs), stablecoins.

2. Supply Chain: Automated payments upon IoT sensor confirmation of delivery, release of letters of credit.

3. Real Estate: Automated title transfer upon payment clearance.

4. Legal/Insurance: Parametric insurance (automatic payout for flight delays).

D. Platforms

Ethereum is the primary general-purpose smart contract platform. Others include Solana, Cardano, BNB Smart Chain. They provide a Virtual Machine (EVM) and environment for deploying and executing smart contract code.

V. PERMISSIONED VS. PERMISSIONLESS BLOCKCHAINS

B. Permissioned Blockchain Design Considerations

1. Identity Management & Access Control: Critical. Uses Membership Service Provider (MSP) to issue and manage cryptographic identities (certificates) for nodes/participants.

2. Privacy: Need for channels (Fabric) or private transactions (Corda) to hide data from non-participants in a specific business flow.

3. Performance: Trade-off between decentralization and speed. Fewer, trusted nodes enable faster consensus (PBFT vs. PoW).

4. Governance: Clear rules for membership, protocol changes, and dispute resolution defined by the consortium.

VI. ENTERPRISE BLOCKCHAIN PLATFORMS

A. Hyperledger Fabric

1. Modular Architecture: Separation of concerns:

   • Consensus (Ordering Service): Decides transaction order, does not execute contracts.

   • Execution (Peers/Chaincode): Peers (endorsing/committing) execute smart contracts (chaincode) and validate transactions.

2. Key Components:

   • Peers: Host ledgers and chaincode. Endorsing Peers simulate transactions for policy check. Committing Peers update ledger.

   • Ordering Service: Consensus cluster (Raft, Kafka) that orders transactions into blocks.

   • Channels: Private subnets. Ledger data is isolated per channel, enabling privacy.

   • Membership Service Provider (MSP): Manages identities and access control.

3. Scalability & Performance:

   • Execute-Order-Validate Paradigm: Transactions are executed (simulated) by endorsing peers before ordering. Only transactions that meet endorsement policy are ordered. This reduces wasted computation on invalid transactions.

   • Channels: Allow parallel execution of unrelated transactions on different channels, improving overall throughput and privacy.

B. Other Enterprise Platforms

1. Ripple (XRP Ledger):

   • Focus: Real-time, cross-border payment settlement for financial institutions.

   • Consensus Protocol (RPCA): A unique, low-latency consensus algorithm where trusted validators (run by banks/partners) agree on transaction order every 3-5 seconds. No mining. Uses UNL (Unique Node List).

   • Asset: XRP is the native digital asset used for liquidity.

2. Corda:

   • Designed for: Financial contracts (legal agreements). Not a general-purpose blockchain.

   • Privacy: UTXO model (like Bitcoin) but with confidential identities. Only parties to a transaction and necessary notaries see its details. No global broadcast.

   • Consensus: Notary service validates transaction uniqueness (prevents double-spend). Consensus is achieved between transaction participants, not the whole network.

   • Corda Network: A global network of Corda nodes with a network map service.

VII. BLOCKCHAIN APPLICATIONS IN FINANCE & SUPPLY CHAIN

A. Financial Applications

1. Know Your Customer (KYC):

   • Key Components:

     1. Identity Verification: Document (passport, license) authenticity check.

     2. Document Checks: Screening against sanctions, PEP lists.

     3. Ongoing Monitoring: Tracking transactions for suspicious activity.

   • Importance for Institutions: Mandatory regulatory requirement (AML - Anti-Money Laundering). Prevents fraud, terrorist financing, and regulatory penalties.

   • Blockchain's Potential: Create a shared, verifiable digital identity. A customer completes KYC once with a trusted entity. Their verified status (as a hash/claim) can be cryptographically shared with other institutions upon request, reducing duplication, cost, and onboarding time.

2. Trade Finance & Letters of Credit: Automates the paper-heavy, multi-party process. Smart contracts can release payment automatically when shipping documents (e.g., bill of lading) are verified on-chain, reducing settlement time from weeks to days.

B. Supply Chain Applications

1. Impact on International Trade:

   • Transparency & Traceability: All participants (manufacturer, shipper, customs, retailer) see the same immutable record of a product's journey.

   • Provenance: Verifies authenticity and origin (critical for food safety, luxury goods, pharmaceuticals).

   • Efficiency: Reduces paperwork, disputes, and delays through automated, shared data.

2. Supply Chain Financing:

   • Concept: Financing based on the value of inventory or accounts receivable (invoices) in the supply chain.

   • How Blockchain Enables It: Provides a single source of truth for order history, delivery proofs, and invoice authenticity. This allows lenders to:

     • Invoice Financing: Lend against verified, un-paid invoices with lower risk.

     • Dynamic Discounting: Offer early payment discounts automatically based on invoice age and risk.

3. Tracking Goods: From raw material origin (mine/farm) through manufacturing, shipping, and retail. Each step is recorded as a transaction on the blockchain, creating an auditable, tamper-proof history accessible to authorized parties.

[!TIP] Exam Strategy: For 7-mark questions, structure answers as: 1) Clear Definition, 2) 3-4 Key Points/Mechanisms, 3) 1-2 Relevant Examples/Applications, 4) Conclusion/Benefit. Always connect technical concepts (like hashing, consensus) to their real-world purpose (security, trust, efficiency).