UNIT 2: OS INTERNALS FOR SECURITY SUPPORT

Short Notes Based on RGPV Past Papers (Dec 2024, Nov 2023, Nov 2022)

I. CORE OPERATING SYSTEM INTERNALS & FILE SYSTEMS

UNIX/System Architecture & Components

• UNIX OS Components:

  • Kernel: Core OS, manages hardware resources (CPU, memory, I/O).

  • Shell: Command interpreter, user interface.

  • File System: Hierarchical, everything is a file.

  • Utilities/Tools: Standard programs (e.g., grep, awk).

  • Libraries: System call interfaces.

• Role of Kernel:

  • Acts as a supervisor and resource manager.

  • Provides abstraction (hides hardware complexity).

  • Implements protection (prevents unauthorized access).

  • Handles process scheduling, memory management, I/O operations.

• Windows System Architecture (Neat Sketch Required):

  • Hybrid Kernel (NT Kernel): Combines microkernel (minimal) and monolithic (performance) features.

  • Key Layers:

    1. User Mode: Applications, Subsystems (Win32, POSIX), Windows API.

    2. Kernel Mode: Executive (Object Manager, I/O Manager, etc.), Kernel, HAL (Hardware Abstraction Layer).

  • Object Manager: Central repository for kernel objects (processes, threads, files, events).

  • [!TIP] Exam often asks for a labeled sketch showing User Mode ↔ Kernel Mode transition via System Calls.

Buffer Management (High Frequency)

• Buffer Cache:

  • A region in main memory that holds copies of disk blocks.

  • Purpose: Reduces disk I/O by serving repeated read/write requests from memory (caching).

  • Trade-off: Memory vs. I/O speed.

• Buffer Pool / Buffer Header Structure (Neat Sketch Required):

  • Buffer Pool: Array of fixed-size buffer frames.

  • Buffer Header (per buffer): Metadata containing:

    • Device No., Block No. (identifies disk block)

    • Status Flags (e.g., VALID, DIRTY, LOCKED)

    • Device Queue Pointer (for async I/O)

    • Process Wait Queue (for blocked processes)

  • Free List: Linked list of available buffer headers.

  • Device Queues: Lists for pending I/O on specific devices.

  • [!TIP] Key concept: Double Buffering uses two buffers to overlap I/O and computation.

• Role of Super Block in UNIX:

  • File System Metadata Hub. Contains:

    • File system size, free block count, free inode count.

    • Pointer to free block list and free inode list.

    • Magic number (identifies file system type).

  • Loaded into memory at mount time; updated periodically.

File System Management

• Mounting & Unmounting:

  • Mounting: Attaching a file system (from disk partition) to the directory tree at a mount point.

    • Advantages: Seamless integration, unified namespace, access control via parent dir.

    • Disadvantages: Security risk if mount point is writable by all (e.g., /tmp), potential for circular mounts.

  • Unmounting: Detaching a file system; ensures all pending I/O is flushed, no open files.

• Data Structures in OS:

  • Queue (FIFO):

    • Application: Process Scheduling (Ready Queue), Buffer Management (Free List, Device Queues), Spooling (print queue).

    • Operations: enqueue(), dequeue().

  • Tree (Hierarchical):

    • Application: File System Directory Structure, Process Hierarchy (parent-child), Memory Management (page tables, segment trees).

    • Operations: Traversal (pre/in/post-order), search.

II. PROCESS MANAGEMENT & CONCURRENCY (High Frequency)

Process Life Cycle & PCB

• Process Life Cycle States:

  1. New: Process created.

  2. Ready: In main memory, waiting for CPU.

  3. Running: Executing on CPU.

  4. Waiting/Blocked: Waiting for I/O/event.

  5. Terminated: Execution finished.

  • Transitions: Admit (New→Ready), Dispatch (Ready→Running), Timeout (Running→Ready), I/O Request (Running→Waiting), I/O Complete (Waiting→Ready), Exit (Running→Terminated).

• Process Control Block (PCB): OS data structure representing a process. Contains:

  • Process State (Ready, Running, etc.)

  • Process ID (PID)

  • Program Counter (PC)

  • CPU Registers (saved during context switch)

  • CPU Scheduling Info (priority, scheduling queue pointers)

  • Memory Management Info (base/limit, page tables)

  • I/O Status Info (open files, allocated I/O devices)

  • Accounting Info (CPU time used, limits)

• Process Scheduling (Principles):

  • Goal: Maximize CPU utilization, throughput, fairness, minimize turnaround/waiting time, response time.

  • Types: Long-term (job), Mid-term (swap), Short-term (CPU scheduler).

  • Preemptive vs. Non-preemptive.

Principles of Concurrency

• Requirements for Concurrent Execution:

  • Speedup: Parallel tasks finish faster.

  • Resource Sharing: Multiple processes share CPUs, memory, I/O.

  • Modularity: Divide complex tasks.

  • Economy: Better resource utilization.

• Fundamental Principles:

  • Race Condition: Outcome depends on timing of concurrent accesses to shared resource.

  • Need for Synchronization: To enforce mutual exclusion, progress, and bounded wait.

  • Critical Section: Code segment accessing shared resource.

  • Deadlock: All processes wait indefinitely for resources held by each other.

Synchronization Mechanisms (High Frequency)

• Semaphore:

  • Definition: Integer variable used for signaling and synchronization, accessed only via atomic wait() (P) and signal() (V) operations.

  • Types:

    1. Binary Semaphore (Mutex): Value 0 or 1. Used for mutual exclusion.

       
       wait(S) { while(S<=0); S--; } // busy wait
       
       signal(S) { S++; }
       
       

    2. Counting Semaphore: Integer value ≥ 0. Manages pool of identical resources.

       
       wait(S) { while(S<=0); S--; }
       
       signal(S) { S++; }
       
       

  • [!TIP] Key Difference: Binary semaphore is for mutual exclusion (1 resource), counting for resource counting (multiple identical resources).

• Critical Section Problem & Solutions:

  • Requirements: Mutual Exclusion, Progress, Bounded Wait.

  • Peterson's Solution (for 2 processes): Uses shared flag[] and turn variables.

  • Hardware Support: Test-and-Set, Compare-and-Swap instructions (atomic).

  • Semaphore Solution: Use a binary semaphore (mutex) initialized to 1.

III. INTER-PROCESS COMMUNICATION (IPC) (High Frequency)

IPC Overview & Methods

• Definition: Mechanisms for processes to communicate and synchronize.

• Methods:

  1. Shared Memory: Fastest. Processes share a memory segment.

  2. Message Queues: Messages passed via kernel-managed queue.

  3. Pipes: Unidirectional byte stream (anonymous/named).

  4. Sockets: For network communication (see Unit VI).

  5. Signals: Event notifications (asynchronous).

Detailed IPC Mechanisms

• Shared Memory Method:

  • Implementation:

    1. Process requests shared memory segment from OS (shmget()).

    2. OS returns shared memory ID.

    3. Process attaches segment to its address space (shmat()).

    4. Processes read/write directly to shared memory.

    5. Synchronization required (semaphores) to avoid race conditions.

  • Advantage: Very high speed (no kernel copy).

  • Disadvantage: Requires explicit synchronization; processes must be on same machine.

• Message Queues (Client/Server Communication):

  • Kernel-managed linked list of messages.

  • Server:

    1. Create/Open queue (msgget()).

    2. Wait for message (msgrcv()).

    3. Process request.

    4. Send reply to client's queue.

  • Client:

    1. Create/Open queue.

    2. Send request message (msgsnd()).

    3. Wait for reply.

  • Advantage: Synchronization handled by kernel; messages have priorities.

  • Disadvantage: Slower than shared memory (kernel copy).

IV. SECURITY & PROTECTION FUNDAMENTALS (High Frequency)

OS-Level Security Implementation

• Authentication: Verifying user identity.

  • Methods: Passwords, biometrics, tokens, multi-factor.

  • System Access: Login prompts, secure attention sequences (e.g., Ctrl+Alt+Del in Windows).

• Access Control:

  • Discretionary Access Control (DAC): Owner decides permissions (e.g., UNIX rwx).

  • Mandatory Access Control (MAC): System-enforced policies (e.g., SELinux, military labels).

  • Role-Based Access Control (RBAC): Permissions based on roles.

  • Implementation: Access Control Lists (ACLs), Capabilities, Protection Rings (x86: Ring 0-3).

Malware & Threats (High Frequency)

• Definitions:

  • Malware: Malicious software (viruses, worms, trojans, ransomware).

  • Trojan: Disguised as legitimate software, creates backdoor.

  • Rootkit: Hides existence of other malware, gains admin/root privileges.

• Types of Viruses & Worms:

  • Viruses: Require host program to spread.

    • File Infector, Macro, Boot Sector, Polymorphic, Metamorphic.

  • Worms: Standalone, self-replicate over network.

    • Network worms, Email worms, Internet worms.

• Ransomware Attacks:

  • Risks: Data encryption (irreversible), financial loss, service disruption, reputational damage.

  • Protection: Regular offline backups, patch management, user training, email filtering, application whitelisting, network segmentation.

Vulnerabilities & Defense

• Common Vulnerabilities and Exposures (CVE):

  • Definition: Publicly known cybersecurity vulnerabilities listed with unique CVE IDs.

  • Purpose: Standardized reference for vulnerabilities (e.g., CVE-2021-44228 - Log4Shell).

  • Defense: Regular CVE scanning, patch management, vulnerability assessment.

• Honeypot:

  • Definition: Decoy system/network designed to attract and study attackers.

  • Use in Protection:

    • Detection: Early warning of attacks.

    • Diversion: Ties up attacker resources.

    • Intelligence: Learn attack tactics, techniques, procedures (TTPs).

    • Types: Low-interaction (simulated services), High-interaction (real OS).

Specialized Security Hardware (High Frequency)

• Secure Coprocessor:

  • Definition: Dedicated, tamper-resistant hardware chip (e.g., TPM - Trusted Platform Module).

  • Functions:

    • Secure Key Storage: Stores cryptographic keys.

    • Attestation: Proves system integrity (measured boot).

    • Sealed Storage: Data encrypted such that it can only be decrypted on same platform state.

    • Cryptographic Acceleration.

• Secure Wallet:

  • Definition: Hardware/software module for secure storage and management of cryptographic keys (especially for cryptocurrencies, digital identities).

  • Importance:

    • Protection against theft: Keys never leave secure element.

    • Transaction Authorization: Requires physical confirmation (button press).

    • Backup & Recovery: Seed phrase for wallet recovery.

    • Isolation: From internet-connected devices (cold wallet).

V. PLATFORM-SECIFIC SECURITY

Windows Internals & Security

• Windows Internals Overview:

  • Hybrid kernel (NT), object-based, preemptive multitasking.

  • Key components: Object Manager, Security Reference Monitor, I/O Manager.

• Windows-Specific Components:

  • System Worker Threads: Kernel threads that perform background tasks (e.g., memory management, cache flushing). Run in System (PID 4) process context.

  • Windows Global Flags (GFlags): Debugging/diagnostic tool to enable system-wide or image-specific flags (e.g., page heap, loader snaps).

  • Winsock Functions: API for network communication. Key functions: socket(), bind(), listen(), accept(), connect(), send(), recv(), closesocket().

Mobile Operating System Security (High Frequency)

• Android Security (Different Levels):

  1. Hardware Backed Security: Trusted Execution Environment (TEE), Secure Element, StrongBox.

  2. Linux Kernel Security: SELinux (enforcing mode), user/group separation, namespaces, seccomp.

  3. Application Sandbox: Each app runs as unique Linux UID, isolated by DAC and SELinux.

  4. Application Signing: APKs signed with developer key.

  5. Permission Model: Runtime permissions for dangerous operations.

  6. Verified Boot: Chain of trust from bootloader to system.

• Vulnerable Components in Mobile OS:

  • Baseband Processor (modem firmware): Often runs proprietary code, attack surface.

  • Drivers (GPU, Wi-Fi, NFC): Kernel-level, privilege escalation targets.

  • System Apps/Pre-installed Software: May have vulnerabilities, high privileges.

  • Bootloader: Unlocking bypasses security.

  • User Installed Apps: Malware, excessive permissions.

  • Radio Interface Layer (RIL): Communication with baseband.

VI. NETWORK SECURITY & SOCKET PROGRAMMING (High Frequency)

Sockets Fundamentals

• Socket Definition: Endpoint for bidirectional communication between two processes on a network. Identified by IP address + Port number.

• Types of Sockets:

  1. Stream Sockets (SOCK_STREAM): Reliable, connection-oriented (TCP). Guaranteed delivery, in-order, no duplicates.

  2. Datagram Sockets (SOCK_DGRAM): Unreliable, connectionless (UDP). No guarantee of delivery, order, or duplication. Faster, lower overhead.

  3. Raw Sockets: Allow direct access to lower-layer protocols (IP, ICMP). Used by network tools (ping, traceroute) and malicious tools (packet sniffers, spoofing). Requires root/admin privileges.

Socket Programming & Winsock

• Socket Programming Process (TCP Example):

  
  // Server:
  
  socket() -> bind() -> listen() -> accept() -> recv()/send() -> close()
  
  // Client:
  
  socket() -> connect() -> send()/recv() -> close()
  
  

• Winsock Functions Overview: Windows implementation of Berkeley sockets. Requires WSAStartup(), WSACleanup(). Functions prefixed with WSA (e.g., WSASocket, WSAConnect).

• Use & Characteristics of Datagram Sockets (UDP):

  • Use: DNS queries, VoIP, video streaming, gaming, broadcast/multicast.

  • Characteristics: Connectionless, no handshake, no flow control, possible packet loss/duplication/reordering. Message boundaries preserved (each send() is one recv()).

Virtualization for Security

• Virtualization Techniques for Security:

  • Sandboxing: Isolate untrusted applications/processes in a VM/container. Breakout is a critical vulnerability.

  • Honeypots/Honeynets: Deploy virtual decoy systems.

  • Malware Analysis: Run malware in isolated VM to study behavior.

  • Secure Multi-tenancy: Cloud providers use VMs to isolate different customers' data.

  • Live Migration Security: Encrypt VM state during migration.

  • Virtual Machine Monitors (VMM/Hypervisor): Must be secure; compromise of VMM compromises all VMs (e.g., VMware ESXi, Xen, Hyper-V).

\boxed{\text{Key Exam Focus: Buffer Cache, PCB, Semaphores, IPC (Shared Memory/Message Queues), Malware Types, CVE, Secure Coprocessor, Android Security, Sockets (TCP/UDP/Raw), Virtualization for Security}}