Calculus of variations and optimization

About the course

Extrema of functions of one and two variables: gradient, Hessian, quadratic functions, gradient descent and Newton method. Constrained optimization: Lagrange multipliers, KKT conditions. Convex sets and functions, convex optimization. Linear programming: the graphical method, slack variables, the simplex tableau, the dual problem and shadow prices. The transportation problem: north-west corner and potentials. Functionals, the Euler–Lagrange equation and classic variational problems. Basics of operations research: the assignment problem, the Wilson formula, the M/M/1 queue. A continuation of linear algebra and multivariable calculus.

Course program: 33 lessons

Unconstrained optimization

  1. Extremum of a function of one variable
  2. Gradient and stationary points
  3. Hessian and the type of a point
  4. Quadratic function
  5. Gradient descent
  6. Newton's method
  7. Checkpoint: unconstrained optimization

Constrained optimization and convexity

  1. Equality constraint and the Lagrange multiplier
  2. The Lagrange function
  3. Inequality constraints and KKT conditions
  4. Convex sets and functions
  5. Testing convexity
  6. Convex optimization
  7. Checkpoint: constrained optimization and convexity

Linear programming

  1. Linear programming: the graphical method
  2. Standard form and basic solutions
  3. Simplex tableau: one step
  4. The simplex method to the end
  5. Checkpoint: linear programming

Duality and the transportation problem

  1. The dual problem
  2. Shadow prices
  3. Transportation problem: the first plan
  4. Potentials and optimality of a plan
  5. Checkpoint: duality and the transportation problem

Calculus of variations

  1. Functional
  2. The Euler–Lagrange equation
  3. Classic variational problems
  4. Checkpoint: calculus of variations

Basics of operations research

  1. The assignment problem
  2. Inventory: the Wilson formula
  3. The M/M/1 queue
  4. Checkpoint: operations research

Final test

  1. Final test in optimization