Modified quasi-Newton type methods using gradient flow system for solving unconstrained optimization
In this thesis, we are mainly concerned with finding the numerical solution of nonlinear unconstrained optimization problems via gradient ow system. First, we give some brief mathematical background and then we consider a famous class of optimization methods called the quasi-Newton methods. Specific...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Language: | English |
| Published: |
2016
|
| Subjects: | |
| Online Access: | http://psasir.upm.edu.my/id/eprint/85082/1/FS%202016%2092%20IR.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | In this thesis, we are mainly concerned with finding the numerical solution of nonlinear unconstrained optimization problems via gradient ow system. First, we give some brief mathematical background and then we consider a famous class of optimization methods called the quasi-Newton methods. Specifically, we focus on a class of quasi-Newton method named Broyden-Fletcher-Goldfarb-Shanno (BFGS) method. We investigate the possible use of control theory, particularly theory on gradient ow system to derive some modified line search and trust region methods for optimization. The implementation of these methods in line search algorithm in their original forms would generate a Newton-type matrix which require inversion of a non-sparse matrix or equivalently solving a linear system in every iteration. Thus, an approximation of the proposed methods via BFGS update is constructed. Numerical experiments are carried out to illustrate the numerical performance and efficiency of the proposed methods by comparing the number of iterations, the number of function evaluations and also the CPU time in second. Our computational results show that the proposed methods are comparable with the existing standard methods. Other than that, we also analyse the global convergence properties of the modified methods. It is shown that the modified methods converge globally and the rate of convergence is superlinear convergence. We also implement the Newton-type methods on trust region framework by using unit step length to adjust the radius of the region to obtain desired reduction in the objective function. We make an approximation to the proposed Newton-type matrix by using BFGS updating scheme and then apply this modified Newtontype matrix to generate new quadratic approximation subproblem. Numerical results are established to demonstrate the efficiency of our modified methods. Our proposed methods outperform the standard trust region method in term of lower number of function evaluations and much reduction in computational time. It is proved under appropriate assumptions that the modified trust region methods are globally convergent. |
|---|