copt.minimize_saga¶
-
copt.
minimize_saga
(f_deriv, A, b, x0, step_size, prox=None, alpha=0, max_iter=500, tol=1e-06, verbose=1, callback=None)[source]¶ Stochastic average gradient augmented (SAGA) algorithm.
This algorithm can solve linearly-parametrized loss functions of the form
minimize_x sum_{i}^n_samples f(A_i^T x, b_i) + alpha ||x||_2^2 + g(x)
where g is a function for which we have access to its proximal operator.
Warning
This function is experimental, API is likely to change.
- Parameters
f – loss functions.
x0 (np.ndarray or None, optional) – Starting point for optimization.
step_size (float or None, optional) – Step size for the optimization. If None is given, this will be estimated from the function f.
max_iter (int) – Maximum number of passes through the data in the optimization.
tol (float) – Tolerance criterion. The algorithm will stop whenever the norm of the gradient mapping (generalization of the gradient for nonsmooth optimization) is below tol.
verbose (bool) – Verbosity level. True might print some messages.
trace (bool) – Whether to trace convergence of the function, useful for plotting and/or debugging. If ye, the result will have extra members trace_func, trace_time.
- Returns
opt – The optimization result represented as a
scipy.optimize.OptimizeResult
object. Important attributes are:x
the solution array,success
a Boolean flag indicating if the optimizer exited successfully andmessage
which describes the cause of the termination. See scipy.optimize.OptimizeResult for a description of other attributes.- Return type
OptimizeResult
References
This variant of the SAGA algorithm is described in:
“Breaking the Nonsmooth Barrier: A Scalable Parallel Method for Composite Optimization.”, Fabian Pedregosa, Remi Leblond, and Simon Lacoste-Julien. Advances in Neural Information Processing Systems (NIPS) 2017.