A Free Line Search Steepest Descent Method for Solving Unconstrained Optimization Problems

Subject Areas : Statistics

Narges Bidabadi ¹ ( Assistant Professor, Department of Mathematical Sciences, Yazd University, Yazd, Iran )

Received: 2019-07-08 Accepted : 2020-08-02 Published : 2020-10-22

Keywords: جستجوی خطی, مسئله بهینه‌سازی نامقید, فرمول شبه نیوتن مقیاس‌بندی‌شده دو پارامتری, روش تندترین کاهش,

Abstract :

In this paper, we solve unconstrained optimization problem using a free line search steepest descent method. First, we propose a double parameter scaled quasi Newton formula for calculating an approximation of the Hessian matrix. The approximation obtained from this formula is a positive definite matrix that is satisfied in the standard secant relation. We also show that the largest eigen value of this matrix is not greater than the number of variables of the problem. Then, using this double parameter scaled quasi Newton formula, an explicit formula for calculating the step length in the steepest descent method is presented and therefore, this method does not require the use of approximate methods for calculating step length. The numerical results obtained from the implementation of the algorithm in MATLAB software environment are presented for some optimization problems. These results show the efficiency of the proposed method in comparison with other existing methods.

References:

[1] W. Cheng, D. Li. Spectral scaling bfgs method. Journal of Optimization Theory and Applications 146(2): 305–319 (2010)

 

[2] J. Barzilai, J.M. Borwein. Two-point step size gradient methods. IMA Journal of Numerical Analysis 8: 141–148 (1988)

 

[3] F. Biglari, M.A. Hassan, W.J.  Leong. New quasi-Newton methods via higher order tensor models. Journal of Computational and Applied Mathematics 235(8): 2412–2422 (2011)

 

[4] F. Biglari, M. Solimanpur. Scaling on the spectral gradient method. Journal of Optimization Theory and Applications 158(2): 626–635 (2013)

 

[5] Y.H. Xiao, Q.Y. Wang, D. Wang. Notes on the Dai-Yuan-Yuan modified spectral gradient method. Journal of omputational and Applied Mathematics 234(10): 2986–2992 (2010)

 

[6] B. Zhou, L. Gao, Y.H. Dai. Gradient methods with adaptive step sizes. Computational Optimization and Applications 35(1): 69–86 (2006)

 

[7] H. Liu, Z. Liu, X. Dong. A new adaptive Barzilai and Borwein method for unconstrained optimization. Optimization Letters 12: 845–873(2018)

 

[8] C.D. Maranas, C.A. Floudas. A deterministic global optimization approach for molecular structure determination, Journal of Chemical Physics 100(2): 1247–1261 (1994)