NONLINEAR OPTIMIZATION (Ti5416300) LECTURES

Lecture material will be given mainly through links appearing in this page. Check the latest updates in the morning before the lecture. Some of the material will be distributed in class. The following contents of the course are tentative and may change during the course.

1. INTRODUCTION

1.1. DEFINITION OF OPTIMIZATION PROBLEMS

1.2. CLASSIFICATION OF OPTIMIZATION PROBLEMS

1.3. MATHEMATICAL DEFINITIONS AND PREREQUISITES

1.4. OPTIMALITY CONDITIONS

2. UNIVARIATE OPTIMIZATION

2.1. BRACKETING METHODS: 2.1.1. BISECTION METHOD. 2.1.2. FIBONACCI SEARCH. 2.1.3. GOLDEN SECTION SEARCH. 2.1.4. BOUNDING THE MINIMUM.

2.2. INTERPOLATION METHODS: 2.2.1. NEWTON’S METHOD. 2.2.2. OTHER POLYNOMIAL INTERPOLATION METHODS. 2.2.3. SAFEGUARDED METHODS. 

3. UNCONSTRAINED OPTIMIZATION

3.1. METHODS FOR NON-SMOOTH FUNCTIONS: 3.1.1. THE POLYTOPE METHOD (SIMPLEX METHOD OF NELDER AND MEAD). 3.1.2. OTHER DIRECT METHODS.

3.2. GRADIENT BASED METHODS: 3.2.1. METHOD OF STEEPEST DESCENT.

3.3. SECOND DERIVATIVE METHODS

3.3.1. NEWTON’S METHOD

3.3.2. QUASI-NEWTON METHODS

3.3.3. CONJUGATE GRADIENT METHODS

3.4. LEAST SQUARES PROBLEM: 3.4.1. THE GAUSS-NEWTON METHOD. 3.4.2. THE LEVENBERG-MARQUARDT METHOD.

4. CONSTRAINED OPTIMIZATION

4.1. OPTIMALITY CONDITIONS

4.2. METHODS FOR LINEAR CONSTRAINTS: 4.2.1. LINEAR EQUALITY CONSTRAINTS. 4.2.2. LINEAR INEQUALITY CONSTRAINTS.

4.3. QUADRATIC PROGRAMMING

4.4. CLASSIFICATION OF METHODS FOR NONLINEAR CONSTRAINTS

4.5. PENALTY AND BARRIER FUNCTION METHODS: 4.5.1. INTERIOR PENALTY FUNCTION METHOD OR BARRIER FUNCTION METHOD. 4.5.2. EXTERIOR PENALTY FUNCTION METHOD. 4.5.3. ABSOLUTE VALUE PENALTY FUNCTION METHOD. 4.5.4. PENALTY FUNCTIONS FOR GENERAL CONSTRAINTS.

4.6. AUGMENTED LAGRANGIAN METHODS   (Corrections to page 44 made 11.04.2006 at 09.10)

4.7. SEQUENTIAL QUADRATIC PROGRAMMING METHODS (SQP-METHODS)

4.8. CONVEX PROGRAMMING

5. GLOBAL OPTIMIZATION: 5.1. PROBLEM SETTING. 5.2. ON DETERMINISTIC METHODS. 5.3. ON STOCHASTIC METHIDS.

6. SPECIAL ALGORITHMS: 6.1. EVOLUTIONARY ALGORITHMS IN NONLINEAR OPTIMIZATION. 6.2. SIMULATED ANNEALING.