This course is an introduction to the basic concepts of numerical optimization. Two questions are examined: how to characterize a solution and how to conceive a numerical method for finding it.
The course focuses on the development and the study of numerical algorithms to solve unconstrained and constrained nonlinear optimization problems.
Considering continuous but not necessarily convex unconstrained and constrained nonlinear optimization problems, the first and most important part of the course is devoted to the unconstrained case. After studying the characterization of minima for a generic unconstrained optimization problem (optimality conditions), we develop the main ideas behind the so-called line-search and trust-region approaches to globalize first- and second-order methods such as the steepest descent method, the Newton method and quasi-Newton methods, with some insight on both convergence and numerical considerations. The second part of the course is devoted to the constrained case and derives the Karush-Kuhn-Tucker optimality conditions, together with the key ideas of some well-known methods, among which the sequential quadratic programming method, the augmented Lagrangian method and the interior point method.
Introduction
Partie I : Unconstrained Optimization
A. Optimality Conditions
B. Overview of Algorithms
C. Line Search Methods
D. Trust-Region Methods
E. Calculating Derivatives
F. Least-Squares Problems
G. Nonlinear Equations
Partie II : Constrained Optimization
A. Optimality Conditions
B. Overview of Algorithms
Exercise sessions are given for 1h30 per week.
Formula: Two exams per session: the first one on the theory is an oral exam and the second one for the exercises is either a written exam on computer or a work to be presented orally.
Modality: The teaching unit (TU) includes two learning activity assessments (LAA) per session: one on the theory covered in the course, the other on exercises. The TU will be considered as passed if the arithmetic average of the two marks obtained for each A.A. reaches at least 10/20. During the same academic year, the student is exempted from repeating the assessment of one of the two A.A. if it is passed (10/20) and provided that he/she presented both parts the first time.
Numerical Optimization (second edition), Jorge Nocedal and Stephen J. Wright Springer, New York, 2006.
Slides provided before the course.