Abstract
This paper addresses the problem of selecting the parameter in a family of algorithms for unconstrained minimization known as Self Scaling Variable Metric (SSVM) Algorithms. This family, that has some very attractive properties, is based on a two parameter formula for updating the inverse Hessian approximation, in which the parameters can take any values between zero and one. Earlier results obtained for SSVM algorithms apply to the entire family and give no indication of how the choice of parameter may affect the algorithm's performance. In this paper, we examine empirically the effect of varying the parameters and relaxing the line-search. Theoretical consideration also leads to a switching tule for these parameters. Numerical results obtained for the SSVM algorithm indicate that with proper parameter selection it is superior to the DFP algorithm, particularly for high-dimensional problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
M.C. Biggs, “Minimization algorithms making use of non-quadratic properties of the objective function”,Journal of the Institute of Mathematics and its Applications 8 (1971) 315–327.
A.R. Colville, “A comparative study on non-linear programming codes”, IBM Tech. Rept. No. 320-8949 (1968).
L.C.W. Dixon, “Variable metric algorithms necessary and sufficient conditions for identical behavior on non-quadratic functions”,Journal of Optimization Theory and Applications 10 (1972) 34–40.
L.C.W. Dixon, “The choice of step length, a crucial factor in performance of variable metric algorithms”, in: F.A. Lootsma, ed.,Numerical methods for non-linear optimization (Academic Press, London, 1972) pp. 149–170.
R. Fletcher and M.J.D. Powell, “A rapidly convergent descent method for minimization”,Computer Journal 6 (1963) 163–168.
R. Fletcher, “A new approach to variable metric algorithms”,Computer Journal 13 (1970) 317–322.
A.A. Goldstein, “On steepest descent”,SIAM Journal on Control 1 (1965) 147–151.
D.M. Himmelblau, “A uniform evaluation of unconstrained techniques”, in: F.A. Lootsma, ed.,Numerical methods for non-linear optimization (Academic Press, London, 1972) pp. 69–97.
H.Y. Huang, “Unified approach to quadratically convergent algorithms for function minimization”,Journal of Optimization Theory and Applications 5 (1970) 405–423.
S.S. Oren, “Self-scaling variable metric algorithms for unconstrained minimization”, Ph.D. thesis, Department of Engineering Economic Systems, Stanford University, Stanford, Calif. (1972).
S.S. Oren and D.G. Luenberger, “Self-scaling variable metric (SSVM) algorithms I: Criteria and sufficient conditions for scaling a class of algorithms”,Management Science 20 (1974) 845–862.
S.S. Oren, “Self-scaling variable metric (SSVM) algorithms II: Implementation and experiments”,Management Science 20 (1974) 863–874.
S.S. Oren, “Self-scaling variable metric algorithm without line-search for unconstrained minimization”,Mathematics of Computation 27 (1973) 873–885.
H.H. Rosenbrock, “An automatic method for finding the greatest or least value of a function”,Computer Journal 13 (1960) 175.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Oren, S.S. On the selection of parameters in Self Scaling Variable Metric Algorithms. Mathematical Programming 7, 351–367 (1974). https://doi.org/10.1007/BF01585530
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01585530