B-FORM OF THE DAVIDON–FLETCHER–POWELL METHOD
DOI:
https://doi.org/10.17721/2706-9699.2021.2.08Keywords:
quasi-Newtonian methods, DFP-method, space transformation, gradient method, r-algorithmAbstract
A special form (B-form) of methods of Quasi-Newton type is discussed, which makes it easy to interpret these methods as gradient in appropriately transformed argument space. B-form of the Davidon–Fletcher–Powell method is given and compared with r-algorithms. To minimize smooth convex functions, a gradient method with space transformation is built, combining properties of both quasi-Newtonian methods and r-algorithms. Possible schemes of this type of methods for minimizing non-smooth convex functions are discussed.
References
Polak E. Numerical optimization methods. A unified approach. Moscow: Mir, 1974. 376 p. (In Russian)
Himmelblau D. Applied nonlinear programming. Moscow: Mir, 1975. 535 p. (In Russian)
Pshenichny B. N., Danilin Yu. M. Numerical methods in extremal problems. Moscow: Nauka, 1975. 319 p. (In Russian)
Polyak B. T. Introduction to optimization. Moscow: Nauka, 1983. 384 p. (In Russian)
Gill F., Murray W., Wright M. Practical optimization. Moscow: Mir, 1985. 509 p. (In Russian)
Zhurbenko N. G. Quasi-Newtonian minimization algorithms based on the use of the operator of space dilation. Optimal decision theory. Kiev, 1999. P. 45-50. (In Russian)
Davidon W. C. Variable metric methods for minimization, AEC Research and Development Rept. ANL 5990 (Rev.), 1959.
Fletcher R., Powell M. J. D. A rapidly convergent descent method for minimization. Comput. J. 1963. №2(6). P. 163–168.
Byrd R. H., Lu P., Nocedal J. Limited Memory Algorithm for Bound Constrained Optimization. SIAM Journal on Scientific and Statistical Computing. 1995. No. 16(5). P. 1190–1208.
Stetsyuk P. I. Quasi-Newton Methods and r-Algorithms. Prepr. NAS of Ukraine. V. M. Glushkov Institute of Cybernetics. Kiev, 1996. 96-10. 21 p. (In Russian)
Shor N. Z. Methods of minimization of nondifferentiable functions and their applications. Кiev: Nauk. Dumka, 1979. 199 p. (In Russian)
Shor N. Z. Nondifferentiable optimization and polynomial problems. Boston; Dordrecht; London: Kluwer Academic Publishers, 1998. 412 p.
Stetsyuk P. I. Theory and Software Implementations of Shor’s r-Algorithms. Cybernetics and Systems Analysis. 2017. Vol. 53. P. 692–703. https://doi.org/10.1007/s10559-017-9971-1
Stetsyuk P. I. Methods of ellipsoids and r-algorithms. Chisinau: Eureka, 2014. 488 p. (In Russian)
Shor N. Z., Stetsenko S. I. Quadratic extremal problems and nondifferentiable optimization. Кiev: Nauk. Dumka, 1989. 208 p. (In Russian)
Lemarechal C. An extension of Davidon methods to nondifferentiable problems. Math. Progr. Study. 3, 1975. P. 95–109.
Stetsyuk P. I. Linear Operators in Quasi-Newtonian Methods. Theory and applications of optimization methods. Kiev, 1998. P. 3–8. (In Russian)
Izmailov A. F., Kurennoy A. S., Stetsyuk P. I. Levenberg–Marquardt method for unconstrained optimization. Russian Universities Reports. Mathematics. 2019. Vol. 24. No. 125. P. 60–74.
