Optimization Techniques (ME-604 (B)) - Unit 5 Short Notes

UNIT 5: OPTIMIZATION TECHNIQUES

I. FOUNDATIONS OF OPTIMIZATION

Definition and Core Concepts

• Optimization: The process of finding the best solution (optimal) from all feasible solutions, subject to given constraints. It involves minimizing or maximizing an objective function.

  • Example: Minimizing production cost while meeting demand and capacity limits.

• Significance in Engineering: Enables efficient resource use, cost reduction, performance enhancement, and robust design in product development, manufacturing, and systems engineering.

• Statement of an Optimization Problem:

  • Objective Function: $f(x)$ to be minimized or maximized.

  • Design Variables: $$\displaystyle x = [x_1, x_2, ..., x_n]^T $$ (decision variables).

  • Constraints:

    • Equality: $$\displaystyle h_i(x) = 0 $$

    • Inequality: $$\displaystyle g_j(x) \leq 0 $$ (or $\geq 0$)

  • General Form:

$$\boxed{\text{Minimize } f(x) \text{ subject to } g_j(x) \leq 0, \, h_i(x) = 0}$$

• Optimum Design Concept: A design that yields the best value of the objective function while satisfying all constraints. It can be:

  • Local Optimum: Best in a neighboring region.

  • Global Optimum: Best among all feasible solutions.

Problem Classification

Applications in Industrial Product Development:

• Design optimization (shape, size, material).

• Process optimization (manufacturing, scheduling).

• Supply chain and logistics optimization.

[!TIP] Exam Focus: Be ready to define optimization, state the problem mathematically, and classify problems with examples.

II. CLASSICAL OPTIMIZATION METHODS

A. Linear Programming (LPP)

• Fundamentals:

  • Objective Function & Constraints are linear.

  • Feasible Region: Convex polyhedron defined by linear constraints.

  • Optimal Solution lies at a corner point (vertex) of the feasible region.

• Role in Design & Manufacturing:

  • Resource allocation, blending problems, production planning, inventory control.

• Steps in Solving LPP:

  1. Formulate the problem (define variables, objective, constraints).

  2. Convert inequalities to equalities using slack/surplus variables.

  3. Solve using Graphical (2-variable) or Simplex (multi-variable) method.

  4. Interpret the solution (optimal values, shadow prices).

• Graphical Method for LPP (Frequent):

  • Procedure:

    1. Plot all constraint lines on graph.

    2. Identify the feasible region (common intersection satisfying all constraints).

    3. Evaluate objective function at all corner points of feasible region.

    4. The corner point giving optimal value (max/min) is the solution.

  [!CAUTION] If feasible region is unbounded, optimal solution may not exist (objective can go to ±∞).

  • Example:

    Maximize $$\displaystyle Z = 3x_1 + 2x_2 $$

    s.t. $$\displaystyle 2x_1 + x_2 \leq 100 $$

    $$\displaystyle x_1 + 2x_2 \leq 80 $$

    $$\displaystyle x_1, x_2 \geq 0 $$

    Solution: Corner points: (0,0), (50,0), (40,20), (0,40). Optimal at (40,20) with $$\displaystyle Z=160 $$.

B. Unconstrained Optimization

• Optimization Algorithms: Used when no constraints exist on variables.

  • Newton's Method (Frequent):

    • Role: Finds local minima/maxima using second-order (Hessian) information for quadratic convergence near optimum.

    • Iterative Procedure:

$$\boxed{x^{(k+1)} = x^{(k)} - [\nabla^2 f(x^{(k)})]^{-1} \nabla f(x^{(k)})}$$

    where $\nabla f$ = gradient (first derivative), $$\displaystyle \nabla^2 f $$ = Hessian matrix (second derivatives).

    - **Convergence**: Fast if starting point is close to optimum and Hessian is positive definite. May diverge otherwise.

    - **Limitation**: Computationally expensive for large $n$ (inverting Hessian).

C. Constrained Optimization

• Direct Methods (Frequent):

  • Solve constrained problem by transforming it into a sequence of unconstrained sub-problems.

  • Example: Method of Feasible Directions – moves along directions that keep iterates feasible.

• Penalty Function Methods (Frequent):

  • Concept: Convert constrained problem into unconstrained by adding a penalty term to objective for constraint violation.

  • Types:

    • Exterior (Penalty): Penalty applied outside feasible region. Solution approaches optimum from infeasible region.

$$\boxed{P(x, r) = f(x) + r \sum \max(0, g_j(x))^2}$$

    where $$\displaystyle r > 0 $$ is penalty parameter.

    - **Interior (Barrier)**: Barrier keeps iterates *inside* feasible region (for $$\displaystyle g_j(x) \leq 0 $$).

$$\boxed{B(x, r) = f(x) - r \sum \ln(-g_j(x))}$$

    $r$ decreases to zero; solution stays interior.

[!TIP] Common Pitfall: In penalty methods, large $r$ can cause ill-conditioning; in barrier methods, $$\displaystyle g_j(x) $$ must be strictly negative.

III. MODERN & HEURISTIC OPTIMIZATION METHODS

A. Genetic Algorithms (GA) (Frequent)

• What is GA?: A stochastic, population-based search algorithm inspired by natural selection and genetics. Used for global optimization of complex, non-linear, multimodal problems.

• Key Components:

  • Population: Set of candidate solutions (chromosomes).

  • Selection: Choosing parents based on fitness (e.g., roulette wheel, tournament).

  • Crossover: Recombining parent genes to produce offspring (e.g., single-point, uniform).

  • Mutation: Random alteration of genes to maintain diversity.

• Step-by-Step Procedure:

  1. Initialize random population.

  2. Evaluate fitness of each individual.

  3. Select parents.

  4. Apply crossover to produce offspring.

  5. Apply mutation.

  6. Form new population (replace old).

  7. Repeat from step 2 until termination criterion (generations, fitness).

• Advantages: Global search, handles discrete/continuous/non-differentiable functions, parallelizable.

• Limitations: Computationally intensive, no guarantee of global optimum, parameter tuning (population size, crossover/mutation rates) critical.

B. Fuzzy Optimization

• What is Fuzzy Optimization?: Optimization technique where objective, constraints, or parameters are expressed as fuzzy sets (with degrees of membership $\mu \in [0,1]$) instead of crisp values. Handles uncertainty and imprecision.

• Basic Principles:

  • Model vague goals (e.g., "cost should be low") as fuzzy objectives.

  • Model flexible constraints (e.g., "weight approximately 100 kg") as fuzzy constraints.

  • Solve by defuzzification (convert to crisp) or fuzzy ranking.

• Application Areas: Engineering design under uncertainty, decision-making, control systems, multi-objective problems with subjective preferences.

[!TIP] Exam Connection: GA is often contrasted with classical methods (deterministic vs. stochastic, local vs. global).

IV. COMPARATIVE ANALYSIS & IMPLEMENTATION

A. Traditional vs. Modern Methods

Steps in Modern Optimization Problem Solving:

1. Define objective and constraints (possibly fuzzy/stochastic).

2. Choose heuristic method (e.g., GA) and encode solution.

3. Initialize population.

4. Evaluate fitness.

5. Apply evolutionary operators (selection, crossover, mutation).

6. Iterate until stopping criterion.

7. Post-process and validate solution.

B. Engineering Applications

• Unconstrained Algorithms (Newton's): Parameter estimation, curve fitting, calibration.

• Constrained Algorithms (LPP, Penalty): Design optimization (truss, beam), production scheduling, portfolio optimization.

• GA Applications: Aerospace design, neural network training, scheduling, antenna design.

• Fuzzy Applications: Risk assessment, quality control, robust design.

C. Computational Tools: MATLAB

• Role: Provides built-in functions for rapid implementation and solving complex optimization problems.

• Key Built-in Functions:

  • linprog: Solves Linear Programming problems.

  • fmincon: Solves constrained nonlinear problems (uses interior-point, SQP, etc.).

  • ga: Solves problems using Genetic Algorithm.

  • fminsearch: Unconstrained derivative-free (Nelder-Mead simplex).

• Implementation Example:

  
  % LPP: Maximize Z = 3x1 + 2x2
  
  f = [-3; -2]; % Minimize -Z
  
  A = [2 1; 1 2]; b = [100; 80];
  
  lb = [0; 0];
  
  [x, fval] = linprog(f, A, b, [], [], lb);
  
  

[!TIP] Exam Strategy: For MATLAB questions, know syntax of linprog, fmincon, ga. Be able to formulate problem in standard form for these functions.

\boxed{\text{End of Unit 5 Notes}}