SIMULATION LAB (EX-804) - Unit 3 Short Notes

UNIT 3: SIMULATION LAB - DETAILED NOTES (Generic Blueprint-Based)

Disclaimer: These notes are based on the provided generic blueprint for UNIT 3 (EX-804). Without the official syllabus and past exam questions, this is a high-level, topic-coverage-focused guide. It emphasizes core concepts common to advanced simulation laboratory courses. Students must cross-reference with actual course materials.

3.0 Unit Overview & Learning Objectives

• Purpose: To move from theoretical understanding to practical implementation, experimentation, and critical analysis of complex simulation models.

• Integration: Builds upon Units 1 & 2 (Modeling Concepts, Input Data Analysis, Random Number Generation) by applying them in a full software lifecycle.

• Expected Competencies:

  • Develop hierarchical, modular models using professional software.

  • Design and execute rigorous simulation experiments.

  • Perform statistical analysis of output data.

  • Conduct thorough Verification & Validation (V&V).

  • Prepare professional technical reports and presentations.

3.1 Advanced Simulation Software & Environments

3.1.1 Industry-Standard Tools Deep Dive

• Discrete-Event Simulation (DES): Arena, Simio, FlexSim. Focus on flow of entities through resources.

• Agent-Based Simulation (ABS): AnyLogic (multi-method), NetLogo. Focus on autonomous agents with behaviors.

• System Dynamics (SD): Vensim, Stella, AnyLogic. Focus on feedback loops and stock/flow structures.

• General-Purpose/Continuous: MATLAB/Simulink (control systems, continuous dynamics).

3.1.2 Comparative Analysis: DES vs. ABS vs. SD

3.1.3 Software Architecture & Customization

• Modular Design: Building blocks (modules, objects, templates) for reusability.

• Libraries: Pre-built, validated components for common elements (e.g., queues, machines, transporters).

• Custom Coding: Embedding scripts (VBA in Arena, Java in AnyLogic/Simio, Python in Simulink) for complex logic not achievable with standard drag-and-drop.

3.1.4 Input/Output Interfaces

• Input: Reading from Excel/CSV, SQL databases, text files. Crucial for large, real-world datasets.

• Output: Writing results to files, connecting to BI tools (Power BI, Tableau), or APIs for real-time dashboards.

[!TIP] Exam Focus: Be prepared to compare/contrast simulation paradigms (DES/ABS/SD) for a given problem statement (e.g., "Which is best for modeling a taxi dispatch system?").

3.2 Complex Model Development & Implementation

3.2.1 Hierarchical and Modular Modeling

• Approach: Break a complex system into sub-models or modules (e.g., "Receiving," "Assembly," "Shipping").

• Benefit: Easier verification, reusability, and maintenance. Top-down design.

3.2.2 Advanced Entity & Resource Management

• Routing Logic: Conditional routing based on attributes (e.g., Entity.Type), expressions, or random selection.

• Resource Scheduling: Shift patterns, breaks, preemptive/non-preemptive priorities, resource failures (MTBF/MTTR).

• Dynamic Priorities: Changing queue discipline (FIFO, LIFO, Priority) based on simulation state.

3.2.3 Detailed Process Logic

• Sub-processes: Calling a defined sequence of actions from multiple points.

• Conditional Branching: IF-THEN-ELSE logic based on entity attributes, global variables, or resource states.

• Failure/Repair Cycles: Using failure distributions (e.g., exponential) and repair distributions (e.g., lognormal) on resources.

3.2.4 Custom Coding within Simulation

• Purpose: Implement algorithms, complex routing, custom statistics, or interface with external code.

• Common Scripting:

  • Arena: VBA (Visual Basic for Applications).

  • AnyLogic/Simio: Java.

  • FlexSim: C++ or FlexScript.

• Key Skill: Accessing and modifying entity attributes, global variables, and system variables via code.

3.2.5 Model Parameterization & Configuration

• Parameterization: Defining key inputs (e.g., Num_Machines, Processing_Time_Mean) as global variables or controls.

• Scenario Analysis: Easily changing these parameters to run "what-if" scenarios without altering core model logic.

• Configuration Files: Using external files (.csv, .txt) to store scenario parameters for batch runs.

[!TIP] Common Pitfall: Hard-coding values (like 10 for machine count) instead of using parameters. This makes scenario analysis impossible and is a major verification flaw.

3.3 Input Data Analysis & Modeling (Advanced)

3.3.1 Fitting Theoretical Distributions

• Process: 1) Identify distribution type (e.g., interarrival times ~ Exponential). 2) Estimate parameters (e.g., λ = 1/mean). 3) Test Goodness-of-Fit.

• Goodness-of-Fit Tests:

  • Chi-Square ($$\displaystyle \chi^2 $$) Test: Bins data, compares observed vs. expected frequencies. Sensitive to binning.

  • Kolmogorov-Smirnov (K-S) Test: Compares empirical CDF to theoretical CDF. More powerful for continuous distributions.

  • Anderson-Darling (A-D) Test: Weighted K-S, more sensitive to tails.

• Selection Criteria: Use p-value (>0.05 suggests fit is acceptable) and visual inspection (P-P or Q-Q plots).

3.3.2 Modeling Correlated & Time-Series Data

• Problem: Standard simulation assumes independent random variates. Real data often has autocorrelation (e.g., today's arrivals depend on yesterday's).

• Solution: Use time-series models (ARIMA, Exponential Smoothing) to generate correlated streams. Implement as custom code or use specialized modules.

3.3.3 Empirical Distributions & Data-Driven Models

• Empirical Distribution: Directly sample from the actual observed data (using RAND + lookup table). Use when no theoretical fit is good or data is scarce.

• Data-Driven Models: Use machine learning (regression, classification) to predict outputs based on inputs, then use predictions as simulation inputs.

3.3.4 Handling Data Scarcity & Uncertainty

• Bootstrapping: Resampling with replacement from the small dataset to create many "pseudo-datasets." Use to estimate confidence intervals for parameters.

• Bayesian Approaches: Incorporate prior knowledge (expert opinion) with limited data to form posterior distributions for inputs.

[!TIP] Exam Formula: The Chi-Square Test Statistic is:

$$\chi^2 = \sum_{i=1}^{k} \frac{(O_i - E_i)^2}{E_i}$$

where $$\displaystyle O_i $$ = observed frequency, $$\displaystyle E_i $$ = expected frequency in bin $i$, $k$ = number of bins.

\boxed{\text{Reject } H_0 \text{ (good fit) if } \chi^2_{calc} > \chi^2_{\alpha, df}} \text{ where } df = k - 1 - \text{#params estimated}.

3.4 Experimentation, Output Analysis & Optimization

3.4.1 Design of Experiments (DOE) for Simulation

• Goal: Systematically explore the effect of input factors (e.g., #_servers, shift_length) on output responses (e.g., Avg_Queue_Time, Throughput).

• Designs:

  • Full Factorial: Test all combinations of factor levels. Computationally expensive.

  • Fractional Factorial: Test a subset. Identifies main effects & some interactions.

  • Response Surface Methodology (RSM): Fits a polynomial model (e.g., quadratic) to find optimal settings.

3.4.2 Warm-up Period Determination

• Problem: Initial conditions (empty system) bias early output. Need to discard transient period.

• Welch's Method: Run multiple replications. Plot moving average of output metric (e.g., average queue length). The point where plots for different replications stabilize and overlap indicates end of warm-up.

• Alternative: Autocorrelation analysis; find lag where autocorrelation drops near zero.

3.4.3 Output Analysis: Steady-State vs. Terminating

3.4.4 Confidence Intervals & Comparative Statistics

• Confidence Interval (CI) for Mean (from n replications):

$$\bar{X} \pm t_{\alpha/2, n-1} \frac{S}{\sqrt{n}}$$

where $\bar{X}$ = average of replication means, $S$ = std. dev. of replication means.

\boxed{\text{CI} = \left( \bar{X} - t \cdot \frac{S}{\sqrt{n}}, \bar{X} + t \cdot \frac{S}{\sqrt{n}} \right)}

• Comparing Two Systems (t-test): Use paired-t if using same random numbers (common in simulation), or independent-t.

• Variance Reduction Techniques (VRTs): Methods like Common Random Numbers (CRN) to reduce variance when comparing systems. CRN is essential for valid paired comparisons.

3.4.5 Linking Simulation with Optimization

• Goal: Automate search for optimal input parameters.

• Methods:

  • Simulation-Optimization Software: OptQuest (in Arena/Simio), SimRunner (in Simul8).

  • Custom Metaheuristics: Implement Genetic Algorithms (GA), Simulated Annealing (SA), or Tabu Search in external code (Python/R) that calls the simulation model.

[!TIP] Critical Concept: For steady-state comparisons, always use multiple replications and CRN. A single long run's CI is invalid for comparing systems due to dependence between observations.

3.5 Model Verification, Validation, and Credibility (V&V)

3.5.1 Verification (Building the Model Right)

• Purpose: Ensure the model is coded correctly per the conceptual design.

• Techniques:

  • Debugging/Trace Debugging: Step through logic, watch variable changes.

  • Modular Testing: Test each sub-model in isolation.

  • "Extreme Condition" Tests: Feed in extreme inputs (e.g., zero arrivals, infinite resources) to check expected behavior.

  • Code Walkthroughs/Inspections.

3.5.2 Validation Frameworks (Building the Right Model)

• Conceptual Model Validity: Does the model's structure/assumptions represent the real system? (Face validation, expert review).

• Operational Validity: Does the model's output behavior match the real system's output? (Historical data validation, sensitivity analysis).

• Data Validity: Are input data and their distributions accurate?

3.5.3 Key Validation Techniques

• Face Validation: Show model animations/outputs to domain experts. Do they say "This looks realistic"?

• Sensitivity Analysis: Vary key inputs (within reasonable ranges). Does output change reasonably? (e.g., more servers → lower queue time).

• Historical Data Validation: Use past system data as input, compare model output to actual past performance (e.g., compare simulated throughput to last month's throughput).

3.5.4 Documentation for V&V

• Traceability Matrix: Links every model element (equation, logic block) to a requirement from the conceptual model.

• Validation Report: Documents all tests performed, results, discrepancies found, and how they were resolved.

[!TIP] Mnemonic: Verification = Viewing the code (Is it built right?). Validation = Verifying the results (Is it the right model?).

3.6 Specialized Application Domains (Case Study Focus)

• Manufacturing & Logistics: Cell design, push/pull systems, bullwhip effect in supply chains, warehouse picking strategies.

• Healthcare: Patient flow through ED/inpatient units, staff scheduling, capacity planning, impact of policies (e.g., triage).

• Transportation & Traffic: Port/airport operations, traffic signal timing, rideshare dispatch, pedestrian dynamics.

• Service Systems: Call center staffing (Erlang-C formulas vs. simulation), bank queue management, hospitality (check-in/out).

• Computer Systems & Networks: Data center cooling/power, cloud auto-scaling, network protocol performance, IoT device congestion.

[!TIP] Exam Strategy: For case study questions, always start by identifying the simulation paradigm (DES, ABS, SD) that best fits the domain's core problem.

3.7 Advanced Topics & Emerging Trends

3.7.1 Simulation-Optimization Hybrid Systems

• Concept: Simulation model acts as an expensive "black-box" objective function for an optimizer.

• Challenge: Simulation output is noisy (stochastic). Requires robust optimization algorithms (e.g., stochastic approximation, heuristics).

3.7.2 Digital Twins & Real-Time Data Streaming

• Digital Twin: A live, adaptive simulation model synchronized with a real physical system via IoT sensors.

• Application: Predictive maintenance, real-time decision support, "what-if" analysis on the digital copy before acting on the real system.

3.7.3 Parallel & Distributed Simulation

• Need: For very large, complex models (e.g., national infrastructure) that are too big/slow for one machine.

• Standard: High Level Architecture (HLA). Federates (sub-models) run on different machines, exchange data via Run-Time Infrastructure (RTI).

3.7.4 Sustainability Analysis

• New Metric: Extend traditional cost/time metrics to include energy consumption, carbon footprint, waste generation.

• Method: Model resource/energy use per process step, aggregate over simulation run.

3.8 Lab Report Writing & Presentation of Results

3.8.1 Structure of a Professional Simulation Report

1. Executive Summary: Problem, key methodology, most important findings, recommendations (1 page max).

2. Problem Statement & Objectives: Clear statement of the issue and specific goals.

3. Conceptual Model: Input/Output, assumptions, logic flow (often a high-level flowchart).

4. Input Data & Analysis: How data was collected, fitted, and validated. Include goodness-of-fit test results.

5. Verification & Validation Plan & Results: How you ensured the model was correct and credible.

6. Experimental Design: Scenarios tested, number of replications, warm-up period justification.

7. Results & Analysis: Tables and graphs of key outputs. Statistical comparisons (CIs, t-tests).

8. Conclusions & Recommendations: Directly answer objectives. What should the decision-maker do?

9. Appendices: Detailed code, raw data, full model screenshots.

3.8.2 Effective Visualization

• Use: Histograms for output distributions, time-series plots for performance over time, bar charts for scenario comparisons.

• Avoid: Overloading slides with tables. Use dashboards for key metrics.

• Animation: Useful for face validation and stakeholder communication, but not for final analysis.

3.8.3 Communicating Uncertainty

• Always report confidence intervals alongside point estimates (e.g., "Mean throughput is 95.2 units/hr ± 1.8 (95% CI)").

• Explain what the CI means: "We are 95% confident the true long-run mean lies in this interval."

• Discuss risk: "There is a 10% chance queue will exceed 50 units."

3.8.4 Ethical Considerations

• Transparency: Disclose all assumptions and model limitations.

• Avoid Overstatement: Do not claim precision beyond what the model/CI supports.

• Bias: Be aware of how model framing or output selection can lead to biased recommendations.

[!TIP] Golden Rule: Your report must tell a coherent story: Problem → How we modeled it → How we tested it → What we found → What it means. Results without context are meaningless.