Optimization Techniques (ME-604 (B)) - Unit 2 Short Notes

I. FUNDAMENTALS OF OPTIMIZATION

Definition and Core Concept

• Optimization: The process of finding the best solution from all feasible alternatives, typically by maximizing or minimizing an objective function subject to constraints.

• Objective: To make decisions that yield the most desirable outcome (e.g., minimum cost, maximum efficiency).

• Decision Variables: Quantities that can be controlled (e.g., dimensions, material properties).

• Constraints: Equations or inequalities that limit the decision variables (e.g., physical laws, resource limits).

[!TIP]

Exam Focus: Always state the standard form:

Minimize $f(\mathbf{x})$

Subject to $$\displaystyle g_i(\mathbf{x}) \leq 0, \ i=1,\dots,m $$

$$\displaystyle \quad\quad h_j(\mathbf{x}) = 0, \ j=1,\dots,p $$

where $$\displaystyle \mathbf{x} \in \mathbb{R}^n $$ is the vector of decision variables.

Optimum Design Concept

• Optimum Design: A design where the objective function cannot be improved without violating at least one constraint.

• Local Optimum: Best solution within a neighboring region.

• Global Optimum: Best solution over the entire feasible region.

• Key Distinction:

  • Convex problems: Local optimum = Global optimum.

  • Non-convex problems: Multiple local optima; global optimum is the best among them.

II. LINEAR PROGRAMMING (LPP)

Role in Engineering

• Used for resource allocation, production planning, blending problems, and transportation optimization.

• Helps in design and manufacturing by maximizing profit or minimizing cost under linear constraints.

Solution Methodology (Step-by-Step)

1. Formulate the problem: Define decision variables, objective function (linear), and constraints (linear).

2. Convert inequalities to equalities using slack/surplus variables.

3. Solve using:

   • Graphical method (2 variables)

   • Simplex method (n variables)

4. Interpret the solution: Optimal values of variables and objective function.

Graphical Solution Method

• Feasible Region: Area satisfying all constraints (convex polygon for LPP).

• Optimal Point: Lies at a corner (extreme) point of the feasible region.

• Example:

  Maximize $$\displaystyle Z = 3x_1 + 2x_2 $$

  Subject to:

  $$\displaystyle 2x_1 + x_2 \leq 100 $$

  $$\displaystyle x_1 + 2x_2 \leq 100 $$

  $$\displaystyle x_1, x_2 \geq 0 $$

  → Plot lines, shade feasible region, evaluate $Z$ at corners: (0,0), (50,0), (33.33,33.33), (0,50).

  Optimum: $$\displaystyle x_1^* = 33.33, x_2^* = 33.33, Z^* = 166.67 $$.

[!TIP]

Common Pitfall: If the feasible region is unbounded, optimum may not exist (objective can increase indefinitely). Check direction of objective gradient.

III. UNCONSTRAINED OPTIMIZATION

Algorithms Overview

• Used when no constraints exist on decision variables.

• Goal: Find $$\displaystyle \mathbf{x}^* $$ such that $$\displaystyle \nabla f(\mathbf{x}^*) = 0 $$ and $$\displaystyle \nabla^2 f(\mathbf{x}^*) $$ is positive definite (for minimum).

Direct Search Methods

• Do not require gradient information.

• Examples:

  • Hill Climbing: Evaluate at nearby points, move toward improvement.

  • Pattern Search: Use geometric patterns (e.g., compass search) to explore.

• Advantage: Simple, derivative-free.

• Limitation: Slow convergence, may get stuck in local optima.

Newton's Method

• Role: Fast local convergence using second-order information.

• Iterative Procedure:

$$ \mathbf{x}_{k+1} = \mathbf{x}_k - \left[ \nabla^2 f(\mathbf{x}_k) \right]^{-1} \nabla f(\mathbf{x}_k) $$

• Convergence: Quadratic near optimum if Hessian is positive definite.

• Boxed Formula:

  \boxed{\mathbf{x}_{k+1} = \mathbf{x}_k - \mathbf{H}_k^{-1} \nabla f(\mathbf{x}_k)}

  where $$\displaystyle \mathbf{H}_k = \nabla^2 f(\mathbf{x}_k) $$.

[!TIP]

Caution: Newton's method requires invertible Hessian. If $$\displaystyle \mathbf{H}_k $$ is not positive definite, use modified Newton or quasi-Newton methods (e.g., BFGS).

IV. CONSTRAININED OPTIMIZATION

Penalty Function Method

• Concept: Transform constrained problem into unconstrained by adding a penalty term for constraint violation.

• Transformation:

  Minimize $$\displaystyle \Phi(\mathbf{x}) = f(\mathbf{x}) + P(\mathbf{x}) $$

  where $P(\mathbf{x})$ is penalty function.

• Types:

  • Exterior Penalty: Penalty applied outside feasible region (e.g., $$\displaystyle P = r \sum \max(0, g_i)^2 $$).

  • Interior Penalty: Penalty applied inside feasible region (e.g., barrier methods).

• Procedure:

  1. Start with small penalty parameter $r$.

  2. Solve unconstrained subproblem.

  3. Increase $r$ iteratively until solution approaches feasible region.

Engineering Applications

• Constrained Algorithms: Used when constraints are hard (must be satisfied exactly), e.g., stress limits, geometric bounds.

• Unconstrained Algorithms: Used for penalized formulations or when constraints are soft.

• Practical Implementation:

  • Design optimization: Size/shape optimization with stress, displacement constraints.

  • Process optimization: Operating within safety/environmental limits.

V. MODERN OPTIMIZATION METHODS

Genetic Algorithms (GA)

• Definition: Stochastic search inspired by natural selection, operating on a population of solutions.

• Key Operators:

  1. Selection: Choose parents based on fitness (e.g., roulette wheel).

  2. Crossover: Combine parents to produce offspring (e.g., single-point crossover).

  3. Mutation: Randomly alter genes to maintain diversity.

• Procedure:

  Initialize population → Evaluate fitness → Select → Crossover → Mutate → Repeat until convergence.

Fuzzy Optimization

• Basic Concept: Handles uncertainty in parameters by using fuzzy numbers (e.g., triangular, trapezoidal).

• Approach:

  • Fuzzy objective/constraints → Convert to crisp equivalents using expected value or ranking methods.

  • Solve as deterministic problem or use fuzzy ranking in evolutionary algorithms.

• Application: Problems with vague goals (e.g., "cost should be low") or imprecise data.

VI. TRADITIONAL VS. MODERN APPROACHES

Comparative Analysis

Modern Methodology Steps

1. Initialize population (GA) or fuzzy parameters.

2. Evaluate objective/constraints (handle fuzziness if needed).

3. Apply operators (selection, crossover, mutation).

4. Check convergence (e.g., max generations, fitness plateau).

5. Output best solution.

[!TIP]

Exam Tip: Traditional methods are efficient for smooth, convex problems. Modern methods excel in non-convex, discontinuous, or noisy problems but are computationally expensive.

VII. TOOLS AND APPLICATIONS

Software Utilization (MATLAB)

• MATLAB Optimization Toolbox provides functions:

  • fmincon: Constrained nonlinear optimization.

  • fminunc: Unconstrained optimization.

  • ga: Genetic algorithm.

  • patternsearch: Direct search method.

• Implementation Steps:

  1. Define objective function as .m file.

  2. Specify constraints (linear/nonlinear).

  3. Call solver with options (tolerance, max iterations).

  4. Extract solution and verify constraints.

Industrial Applications

• Product Development:

  • Design optimization: Minimize weight while meeting strength constraints (e.g., aerospace components).

  • Process optimization: Optimize manufacturing parameters (e.g., cutting speed, tool life).

• Case Study: Using GA to optimize topology of a bridge for minimum material usage under load constraints.

[!TIP]

Real-World Link: In practice, hybrid approaches are common (e.g., GA for global search, followed by Newton's method for local refinement). Always validate numerical solutions with engineering judgment.