Simulation Lab (EC-406) - Unit 5 Short Notes

1.0 Introduction to Advanced Simulation Concepts

Recap of Fundamentals:

• System: A set of interacting entities with a purpose.

• Model: An abstract representation of a system (mathematical, logical, physical).

• Simulation: The process of experimenting with a model over time to understand system behavior.

Types of Simulation:

Steps in a Simulation Study:

1. Problem Formulation

2. Data Collection

3. Model Building

4. Verification & Validation (V&V)

5. Experimentation (Design & Execution)

6. Analysis of Output

7. Documentation & Reporting

[!TIP] Exam Focus: Be prepared to differentiate between Verification ("Are we building the model right?") and Validation ("Are we building the right model?").

Role in Engineering: Used when analytical solutions are infeasible, systems are too complex, or real-world testing is costly/dangerous. Common in network design, IoT protocol testing, cloud resource allocation, and VLSI performance analysis.

2.0 Simulation Software Tools and Environments

Popular Platforms:

Tool Selection Criteria:

• Problem domain compatibility

• Scalability & performance needs

• Licensing (open-source vs. commercial)

• Available model libraries & community support

• Learning curve & documentation quality

Installation & Configuration Principles:

• Check dependencies (compilers, libraries)

• Understand build process (e.g., ./waf in NS-3)

• Configure environment variables

• Verify installation with sample simulations

[!TIP] Common Pitfall: Not setting correct random seed → results not reproducible. Always document seed values.

3.0 Model Development and Implementation

Modeling Paradigms:

• Process-Oriented: Entities (e.g., packets) move through processes (queues, servers). Natural for queueing systems.

• Event-Oriented: Focus on events and their scheduling. Low-level control, efficient.

• Activity-Oriented: Entities perform activities with durations. Good for resource contention.

Building Network/System Models:

1. Topology: Define nodes (hosts, routers, switches) and links (point-to-point, bus, wireless).

2. Protocols: Implement or use built-in (TCP Reno, OLSR, 802.11).

3. Traffic Generation:

   • Poisson: Exponential inter-arrival times (memoryless).

   • Pareto: Heavy-tailed, self-similar traffic.

   • Custom: CBR, VBR, trace-based.

4. Mobility Models (Wireless):

   • Random Waypoint: Random destination, pause time.

   • Gauss-Markov: Smooth velocity changes, spatial/temporal correlation.

Parameterization:

• Use configuration files (.conf in NS-3, .ini in OMNeT++) for:

  • Packet size, buffer size, transmission range

  • Simulation duration, warm-up time

  • Number of nodes, traffic load

• Create reusable modules (C++ classes, NED compound modules).

4.0 Experiment Design and Execution

Designing an Experiment:

• Factors (Input Variables): What you change (e.g., packet size, number of flows, routing protocol).

• Responses (Output Metrics): What you measure (e.g., throughput, delay).

• Replications: Independent runs with different random seeds. Rule of thumb: ≥30 replications for reliable confidence intervals.

• Warm-up Period: Initial transient phase to discard. Method: delete-beginning or batch means.

Running Simulations:

• Batch Execution: Run multiple scenarios via scripts (preferred for large experiments).

• Interactive: Step-through debugging, animation.

• Wall-clock vs Simulated Time: Simulation may run faster/slower than real time.

• Parallel/Distributed: For large-scale models (e.g., using MPI in NS-3).

• Logging/Tracing:

  • Packet-level: Detailed (every packet), large files, used for debugging.

  • Flow-level: Aggregated statistics (e.g., per-flow throughput), smaller files.

[!TIP] Exam Tip: Always perform replications to account for stochasticity. Never base conclusions on a single run.

5.0 Performance Metrics and Data Collection

Key Performance Indicators (KPIs):

Data Collection Mechanisms:

• Built-in Statistics: Use trace callbacks (NS-3: TraceConnectWithoutContext).

• Post-Processing: Parse trace files (.tr, .pcap) with AWK, Python, or MATLAB.

• Standard Output Files:

  • .log: Text log.

  • .vec: Time-series vector data.

  • .sca: Scalar statistics (averages, sums).

6.0 Statistical Analysis of Simulation Output

Nature of Output: Simulation results are random variables due to stochastic inputs. Must use statistical methods.

Point Estimators:

• Sample Mean: $$\displaystyle \bar{x} = \frac{1}{n} \sum_{i=1}^{n} x_i $$

• Sample Standard Deviation: $$\displaystyle s = \sqrt{\frac{1}{n-1} \sum_{i=1}^{n} (x_i - \bar{x})^2} $$

Interval Estimators:

• Confidence Interval (CI) for Mean (assuming normality):

$$\bar{x} \pm t_{\alpha/2, \, n-1} \cdot \frac{s}{\sqrt{n}}$$

where $$\displaystyle t_{\alpha/2, \, n-1} $$ is the critical t-value for $(1-\alpha)\%$ CI and $n$ replications.

\boxed{\text{CI} = \bar{x} \pm t \cdot \frac{s}{\sqrt{n}}}

Comparing Systems:

• Paired-t Test (Two configurations, same random numbers):

  1. Compute differences $$\displaystyle d_i = x_{i}^{(A)} - x_{i}^{(B)} $$ for each replication $i$.

  2. Test statistic: $$\displaystyle t = \frac{\bar{d}}{s_d / \sqrt{n}} $$

  3. Compare with $$\displaystyle t_{\alpha/2, n-1} $$.

• ANOVA (Multiple configurations): Compares variances between groups vs. within groups. Conceptual knowledge required.

Graphical Analysis:

• Histogram: Distribution shape.

• Time-Series Plot: Behavior over simulation time (e.g., queue length).

• Box-and-Whisker Plot: Compare medians, spreads, outliers across scenarios.

[!TIP] Critical: Always report confidence intervals alongside means. Overlapping CIs suggest no significant difference.

7.0 Verification, Validation, and Credibility (V&V)

Verification (Build the model right):

• Debugging: Tracing, animation, breakpoints.

• Code Walkthroughs: Peer review of model logic.

• Modular Testing: Test components in isolation.

• Internal Consistency: Check invariants (e.g., queue length never negative).

Validation (Build the right model):

• Conceptual Validation: Face validation by domain experts.

• Operational Validation: Compare model output to real-world data or analytical results (if available).

• Sensitivity Analysis: Vary key parameters to see if model behaves realistically.

Building Credibility:

• Thorough V&V documentation.

• Transparent assumptions and limitations.

• Reproducible results (same seeds, configs).

[!TIP] Common Mistake: Skipping warm-up period → biased statistics. Always perform transient removal.

8.0 Advanced Topics and Current Trends (Lab-Focused)

Simulating Emerging Technologies:

• SDN: Model OpenFlow switches, controllers (e.g., using NS-3 openflow module).

• NFV: Virtual network functions (VNFs) as chainable modules.

• 5G/6G: Network slicing, mmWave propagation models, massive MIMO.

• IoT: Massive machine-type communications (mMTC), low-power models.

• VANETs: High-mobility models (e.g., IDM, SUMO integration).

Hybrid Simulation:

• Simulation-Emulation: Combine simulated networks with real applications/hosts (e.g., Mininet + real VMs).

• Hardware-in-the-Loop (HIL): Integrate real hardware components into simulation.

Simulation for Optimization:

• Parameter Tuning: Use design of experiments (DoE) to find optimal protocol parameters.

• Capacity Planning: Identify bottlenecks via resource utilization metrics.

9.0 Lab Report Writing and Presentation

Structure of Professional Report:

1. Introduction & Objectives: Problem statement, goals.

2. Model Description:

   • Assumptions (simplifications made)

   • Architecture (topology, protocols)

   • Parameters (table of key values)

3. Verification & Validation: Methods used, results.

4. Experiment Design: Factors, levels, replications, warm-up.

5. Results & Analysis:

   • Tables with means, CIs.

   • Graphs (properly labeled: title, axes, legend).

6. Discussion: Interpretation, comparison to expectations, anomalies.

7. Conclusions & Future Work: Summary, limitations, next steps.

Reproducibility Checklist:

• Tool version (e.g., NS-3.35)

• Configuration files (.conf, .ini)

• Random seeds used

• Scripts for analysis (Python/Matlab)

• System specs (if performance-critical)

Data Visualization Best Practices:

• Use bar charts with error bars (for CIs) instead of just lines.

• Avoid 3D plots unless necessary.

• Ensure readability: sufficient font size, distinct colors.

• Include raw data tables in appendix.

[!TIP] Exam Question Pattern: Often asks to design an experiment or interpret given results. Practice writing concise methods and results sections.