No Gradient? Use Quasi-Newton Algorithms for Optimization

In trust-region algorithms, the gradient of the objective function is required to construct a quadratic model that approximates the objective function locally. This quadratic model is then used to determine the direction and step size for each iteration.

If the gradient is not available or cannot be computed, a possible alternative is to use a quasi-Newton algorithm instead. Quasi-Newton algorithms approximate the Hessian matrix (or its inverse) of the objective function using the information from previous iterations. These algorithms do not require the explicit computation of the gradient.

One popular quasi-Newton algorithm is the Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm. It updates an approximation of the Hessian matrix based on the differences in gradients at different iterations. The BFGS algorithm is known for its good convergence properties and is widely used in practice.

Another alternative is the limited-memory BFGS (L-BFGS) algorithm, which is a memory-efficient version of BFGS. Instead of storing the whole Hessian approximation, L-BFGS only keeps a limited number of vectors to approximate the Hessian.

Both BFGS and L-BFGS algorithms are well-suited for unconstrained optimization problems and can be effective when the gradient is not available or expensive to compute. However, it is worth noting that these algorithms may not perform as well as trust-region algorithms when the objective function has strong nonlinearities or when the problem has constraints.

Therefore, if the gradient is not provided for a trust-region algorithm, using a quasi-Newton algorithm like BFGS or L-BFGS can be a viable alternative. However, the choice of algorithm depends on the specific problem and constraints at hand.