Source code¶
pySOT.auxiliary_problems module¶
Module: | auxiliary_problems |
---|---|
Author: | David Eriksson <dme65@cornell.edu>, |
-
pySOT.auxiliary_problems.
candidate_dycors
(num_pts, opt_prob, surrogate, X, fX, weights, prob_perturb, Xpend=None, sampling_radius=0.2, subset=None, dtol=0.001, num_cand=None)¶ Select new evaluations using DYCORS.
Parameters: - num_pts (int) – Number of points to generate
- opt_prob (object) – Optimization problem
- surrogate (object) – Surrogate model object
- X (numpy.array) – Previously evaluated points, of size n x dim
- fX (numpy.array) – Values at previously evaluated points, of size n x 1
- weights (list or numpy.array) – num_pts weights in [0, 1] for merit function
- prob_perturb (list or numpy.array) – Probability to perturb a given coordinate
- Xpend (numpy.array) – Pending evaluations
- sampling_radius (float) – Perturbation radius
- subset (list or numpy.array) – Coordinates that should be perturbed, use None for all
- dtol (float) – Minimum distance between evaluated and pending points
- num_cand (int) – Number of candidate points
Returns: The num_pts new points to evaluate
Return type: numpy.array of size num_pts x dim
-
pySOT.auxiliary_problems.
candidate_srbf
(num_pts, opt_prob, surrogate, X, fX, weights, Xpend=None, sampling_radius=0.2, subset=None, dtol=0.001, num_cand=None)¶ Select new evaluations using Stochastic RBF (SRBF).
Parameters: - num_pts (int) – Number of points to generate
- opt_prob (object) – Optimization problem
- surrogate (object) – Surrogate model object
- X (numpy.array) – Previously evaluated points, of size n x dim
- fX (numpy.array) – Values at previously evaluated points, of size n x 1
- weights (list or numpy.array) – num_pts weights in [0, 1] for merit function
- Xpend (numpy.array) – Pending evaluation, of size k x dim
- sampling_radius (float) – Perturbation radius
- subset (list or numpy.array) – Coordinates that should be perturbed, use None for all
- dtol (float) – Minimum distance between evaluated and pending points
- num_cand (int) – Number of candidate points
Returns: The num_pts new points to evaluate
Return type: numpy.array of size num_pts x dim
-
pySOT.auxiliary_problems.
candidate_uniform
(num_pts, opt_prob, surrogate, X, fX, weights, Xpend=None, subset=None, dtol=0.001, num_cand=None)¶ Select new evaluations from uniform candidate points.
Parameters: - num_pts (int) – Number of points to generate
- opt_prob (object) – Optimization problem
- surrogate (object) – Surrogate model object
- X (numpy.array) – Previously evaluated points, of size n x dim
- fX (numpy.array) – Values at previously evaluated points, of size n x 1
- weights (list or numpy.array) – num_pts weights in [0, 1] for merit function
- Xpend (numpy.array) – Pending evaluations
- subset (list or numpy.array) – Coordinates that should be perturbed, use None for all
- dtol (float) – Minimum distance between evaluated and pending points
- num_cand (int) – Number of candidate points
Returns: The num_pts new points to evaluate
Return type: numpy.array of size num_pts x dim
-
pySOT.auxiliary_problems.
ei_merit
(X, surrogate, fX, XX=None, dtol=0)¶ Compute the expected improvement merit function.
Parameters: - X (numpy.array) – Points where to compute EI, of size n x dim
- surrogate (object) – Surrogate model object, must implement predict_std
- fX (numpy.array) – Values at previously evaluated points, of size m x 1
- XX (numpy.array) – Previously evaluated points, of size m x 1
- dtol (float) – Minimum distance between evaluated and pending points
Returns: Evaluate the expected improvement for points X
Return type: numpy.array of length X.shape[0]
-
pySOT.auxiliary_problems.
expected_improvement_ga
(num_pts, opt_prob, surrogate, X, fX, Xpend=None, dtol=0.001, ei_tol=1e-06)¶ Maximize EI using a genetic algorithm.
Parameters: - num_pts (int) – Number of points to generate
- opt_prob (object) – Optimization problem
- surrogate (object) – Surrogate model object
- X (numpy.array) – Previously evaluated points, of size n x dim
- fX (numpy.array) – Values at previously evaluated points, of size n x 1
- Xpend (numpy.array) – Pending evaluations
- dtol (float) – Minimum distance between evaluated and pending points
- ei_tol (float) – Return None if we don’t find an EI of at least this value
Returns: num_pts new points to evaluate
Return type: numpy.array of size num_pts x dim
-
pySOT.auxiliary_problems.
expected_improvement_uniform
(num_pts, opt_prob, surrogate, X, fX, Xpend=None, dtol=0.001, ei_tol=1e-06, num_cand=None)¶ Maximize EI from a uniform set of points.
Parameters: - num_pts (int) – Number of points to generate
- opt_prob (object) – Optimization problem
- surrogate (object) – Surrogate model object
- X (numpy.array) – Previously evaluated points, of size n x dim
- fX (numpy.array) – Values at previously evaluated points, of size n x 1
- Xpend (numpy.array) – Pending evaluations
- dtol (float) – Minimum distance between evaluated and pending points
- ei_tol (float) – Return None if we can’t reach this threshold
- num_cand (int) – Number of candidate points
Returns: num_pts new points to evaluate
Return type: numpy.array of size num_pts x dim
-
pySOT.auxiliary_problems.
lcb_merit
(X, surrogate, fX, XX=None, dtol=0.0, kappa=2.0)¶ Compute the lcb merit function.
Parameters: - X (numpy.array) – Points where to compute LCB, of size n x dim
- surrogate (object) – Surrogate model object, must implement predict_std
- fX (numpy.array) – Values at previously evaluated points, of size m x 1
- XX (numpy.array) – Previously evaluated points, of size m x 1
- dtol (float) – Minimum distance between evaluated and pending points
- kappa (float) – Constant in front of standard deviation Default: 2.0
Returns: Evaluate the lower confidence bound for points X
Return type: numpy.array of length X.shape[0]
-
pySOT.auxiliary_problems.
lower_confidence_bound_ga
(num_pts, opt_prob, surrogate, X, fX, Xpend=None, kappa=2.0, dtol=0.001, lcb_target=None)¶ Minimize the LCB using a genetic algorithm.
Parameters: - num_pts (int) – Number of points to generate
- opt_prob (object) – Optimization problem
- surrogate (object) – Surrogate model object
- X (numpy.array) – Previously evaluated points, of size n x dim
- fX (numpy.array) – Values at previously evaluated points, of size n x 1
- Xpend (numpy.array) – Pending evaluations
- dtol (float) – Minimum distance between evaluated and pending points
- lcb_target (float) – Return None if we don’t find an LCB value <= lcb_target
Returns: num_pts new points to evaluate
Return type: numpy.array of size num_pts x dim
-
pySOT.auxiliary_problems.
weighted_distance_merit
(num_pts, surrogate, X, fX, cand, weights, Xpend=None, dtol=0.001)¶ Compute the weighted distance merit function.
Parameters: - num_pts (int) – Number of points to generate
- surrogate (object) – Surrogate model object
- X (numpy.array) – Previously evaluated points, of size n x dim
- fX (numpy.array) – Values at previously evaluated points, of size n x 1
- cand (numpy.array) – Candidate points to select from, of size m x dim
- weights (list or numpy.array) – num_pts weights in [0, 1] for merit function
- Xpend (numpy.array) – Pending evaluation, of size k x dim
- dtol (float) – Minimum distance between evaluated and pending points
Returns: The num_pts new points chosen from the candidate points
Return type: numpy.array of size num_pts x dim
pySOT.controller module¶
Module: | controller |
---|---|
Author: | David Eriksson <dme65@cornell.edu>, |
-
class
pySOT.controller.
CheckpointController
(controller, fname='checkpoint.pysot')¶ Checkpoint controller
Controller that uses dill to take snapshots of the strategy each time an evaluation is completed, killed, or the run is terminated. We assume that the strategy can be pickled, or this won’t work. We currently do not respect potential termination callbacks and failed evaluation callbacks. The strategy needs to implement a resume method that is called when a run is resumed. The strategy object can assume that all pending evaluations have been killed and that their respective callbacks won’t be executed
Parameters: - controller (Controller) – POAP controller
- fname (string) – Filename for checkpoint file (file cannot exist for new run)
Variables: - controller – POAP controller
- fname – Filename for snapshot
-
on_cancel
(record)¶ “Handle record cancelled.
Parameters: record (EvalRecord) – Evaluation record
-
on_complete
(record)¶ Handle feval completion.
Parameters: record (EvalRecord) – Evaluation record
-
on_kill
(record)¶ “Handle record killed.
Parameters: record (EvalRecord) – Evaluation record
-
on_new_feval
(record)¶ Handle new function evaluation request.
Parameters: record (EvalRecord) – Evaluation record
-
on_terminate
()¶ “Handle termination.
-
on_update
(record)¶ Handle feval update.
Parameters: record (EvalRecord) – Evaluation record
-
resume
()¶ Resume an optimization run.
Returns: The record corresponding to the best solution Return type: EvalRecord
-
run
()¶ Start the optimization run.
Make sure we do not overwrite any existing checkpointing files
Returns: The record corresponding to the best solution Return type: EvalRecord
pySOT.experimental_design module¶
Module: | experimental_design |
---|---|
Author: | David Eriksson <dme65@cornell.edu> Yi Shen <ys623@cornell.edu> |
-
class
pySOT.experimental_design.
ExperimentalDesign
¶ Base class for experimental designs.
Variables: - dim – Number of dimensions
- num_pts – Number of points in the experimental design
-
generate_points
()¶
-
class
pySOT.experimental_design.
LatinHypercube
(dim, num_pts, criterion='c')¶ Latin Hypercube experimental design.
Parameters: - dim (int) – Number of dimensions
- num_pts (int) – Number of desired sampling points
- criterion (string) –
Sampling criterion
- ”center” or “c”
- Center the points within the sampling intervals
- ”maximin” or “m”
- Maximize the minimum distance between points, but place the point in a randomized location within its interval
- ”centermaximin” or “cm”
- Same as “maximin”, but centered within the intervals
- ”correlation” or “corr”
- Minimize the maximum correlation coefficient
Variables: - dim – Number of dimensions
- num_pts – Number of points in the experimental design
- criterion – Criterion for generating the design
-
generate_points
()¶ Generate a Latin hypercube design in the unit hypercube.
Returns: Latin hypercube design in unit hypercube of size num_pts x dim Return type: numpy.array
-
class
pySOT.experimental_design.
SymmetricLatinHypercube
(dim, num_pts)¶ Symmetric Latin hypercube experimental design.
Parameters: - dim (int) – Number of dimensions
- num_pts (int) – Number of desired sampling points
Variables: - dim – Number of dimensions
- num_pts – Number of points in the experimental design
-
generate_points
()¶ Generate a symmetric Latin hypercube design in the unit hypercube.
Returns: Symmetric Latin hypercube design in the unit hypercube of size num_pts x dim Return type: numpy.array
-
class
pySOT.experimental_design.
TwoFactorial
(dim)¶ Two-factorial experimental design.
The two-factorial experimental design consists of the corners of the unit hypercube, and hence \(2^{dim}\) points.
Parameters: dim (int) – Number of dimensions
Variables: - dim – Number of dimensions
- num_pts – Number of points in the experimental design
Raises: ValueError – If dim >= 15
-
generate_points
()¶ Generate a two factorial design in the unit hypercube.
Returns: Two factorial design in unit hypercube of size num_pts x dim Return type: numpy.array
pySOT.optimization_problems module¶
Module: | optimization_problems |
---|---|
Author: | David Eriksson <dme65@cornell.edu>, David Bindel <bindel@cornell.edu> |
-
class
pySOT.optimization_problems.
Ackley
(dim=10)¶ Ackley function
\[f(x_1,\ldots,x_n) = -20\exp\left( -0.2 \sqrt{\frac{1}{n} \sum_{j=1}^n x_j^2} \right) -\exp \left( \frac{1}{n} \sum{j=1}^n \cos(2 \pi x_j) \right) + 20 - e\]subject to
\[-15 \leq x_i \leq 20\]Global optimum: \(f(0,0,...,0)=0\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Ackley function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
Branin
¶ Branin function
Details: http://www.sfu.ca/~ssurjano/branin.html
Global optimum: \(f(-\pi,12.275)=0.397887\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Branin function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
Exponential
(dim=10)¶ Exponential function
\[f(x_1,\ldots,x_n) = \sum_{j=1}^n e^{jx_j} - \sum_{j=1} e^{-5.12 j}\]subject to
\[-5.12 \leq x_i \leq 5.12\]Global optimum: \(f(0,0,...,0)=0\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Exponential function at x.
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
GoldsteinPrice
¶ -
eval
(x)¶ Evaluate the GoldStein Price function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
-
class
pySOT.optimization_problems.
Griewank
(dim=10)¶ Griewank function
\[f(x_1,\ldots,x_n) = 1 + \frac{1}{4000} \sum_{j=1}^n x_j^2 - \prod_{j=1}^n \cos \left( \frac{x_i}{\sqrt{i}} \right)\]subject to
\[-512 \leq x_i \leq 512\]Global optimum: \(f(0,0,...,0)=0\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Griewank function at x.
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
Hartman3
¶ Hartman 3 function
Details: http://www.sfu.ca/~ssurjano/hart3.html
Global optimum: \(f(0.114614,0.555649,0.852547)=-3.86278\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Hartman 3 function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
Hartman6
¶ Hartman 6 function
Details: http://www.sfu.ca/~ssurjano/hart6.html
Global optimum: \(f(0.201,0.150,0.476,0.275,0.311,0.657)=-3.322\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Hartman 6 function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
Himmelblau
(dim=10)¶ Himmelblau function
\[f(x_1,\ldots,x_n) = 10n - \frac{1}{2n} \sum_{i=1}^n (x_i^4 - 16x_i^2 + 5x_i)\]Global optimum: \(f(-2.903,...,-2.903)=-39.166\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Himmelblau function at x.
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
Levy
(dim=10)¶ Levy function
Details: https://www.sfu.ca/~ssurjano/levy.html
Global optimum: \(f(1,1,...,1)=0\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Levy function at x.
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
Michalewicz
(dim=10)¶ Michalewicz function
\[f(x_1,\ldots,x_n) = -\sum_{i=1}^n \sin(x_i) \sin^{20} \left( \frac{ix_i^2}{\pi} \right)\]subject to
\[0 \leq x_i \leq \pi\]Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Michalewicz function at x.
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
OptimizationProblem
¶ Base class for optimization problems.
-
eval
(record)¶
-
-
class
pySOT.optimization_problems.
Perm
(dim=10)¶ Perm function
Global optimum: \(f(1,1/2,1/3,...,1/n)=0\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Perm function at x.
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
Rastrigin
(dim=10)¶ Rastrigin function
\[f(x_1,\ldots,x_n)=10n-\sum_{i=1}^n (x_i^2 - 10 \cos(2 \pi x_i))\]subject to
\[-5.12 \leq x_i \leq 5.12\]Global optimum: \(f(0,0,...,0)=0\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Rastrigin function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
Rosenbrock
(dim=10)¶ Rosenbrock function
\[f(x_1,\ldots,x_n) = \sum_{j=1}^{n-1} \left( 100(x_j^2-x_{j+1})^2 + (1-x_j)^2 \right)\]subject to
\[-2.048 \leq x_i \leq 2.048\]Global optimum: \(f(1,1,...,1)=0\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Rosenbrock function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
Schwefel
(dim=10)¶ Schwefel function
\[f(x_1,\ldots,x_n) = \sum_{j=1}^{n} \left( -x_j \sin(\sqrt{|x_j|}) \right) + 418.982997 n\]subject to
\[-512 \leq x_i \leq 512\]Global optimum: \(f(420.968746,420.968746,...,420.968746)=0\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Schwefel function at x.
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
SixHumpCamel
¶ Six-hump camel function
Details: https://www.sfu.ca/~ssurjano/camel6.html
Global optimum: \(f(0.0898,-0.7126)=-1.0316\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Six Hump Camel function at x
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
Sphere
(dim=10)¶ Sphere function
\[f(x_1,\ldots,x_n) = \sum_{j=1}^n x_j^2\]subject to
\[-5.12 \leq x_i \leq 5.12\]Global optimum: \(f(0,0,...,0)=0\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Sphere function at x.
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
SumOfSquares
(dim=10)¶ Sum of squares function
\[f(x_1,\ldots,x_n)=\sum_{i=1}^n ix_i^2\]Global optimum: \(f(0,0,...,0)=0\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Sum of squares function at x.
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
class
pySOT.optimization_problems.
Weierstrass
(dim=10)¶ -
eval
(x)¶ Evaluate the Weierstrass function at x.
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
-
-
class
pySOT.optimization_problems.
Zakharov
(dim=10)¶ Zakharov function
Global optimum: \(f(0,0,...,0)=1\)
Variables: - dim – Number of dimensions
- lb – Lower variable bounds
- ub – Upper variable bounds
- int_var – Integer variables
- cont_var – Continuous variables
- min – Global minimum value
- minimum – Global minimizer
- info – String with problem info
-
eval
(x)¶ Evaluate the Zakharov function at x.
Parameters: x (numpy.array) – Data point Returns: Value at x Return type: float
pySOT.strategy module¶
Module: | strategy |
---|---|
Author: | David Eriksson <dme65@cornell.edu> David Bindel <bindel@cornell.edu>, |
-
class
pySOT.strategy.
DYCORSStrategy
(max_evals, opt_prob, exp_design, surrogate, asynchronous=True, batch_size=None, extra_points=None, extra_vals=None, weights=None, num_cand=None)¶ DYCORS optimization strategy.
This is an implementation of the DYCORS strategy by Regis and Shoemaker:
Rommel G Regis and Christine A Shoemaker. Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization. Engineering Optimization, 45(5): 529–555, 2013.
This is an extension of the SRBF strategy that changes how the candidate points are generated. The main idea is that many objective functions depend only on a few directions so it may be advantageous to perturb only a few directions. In particular, we use a perturbation probability to perturb a given coordinate and decrease this probability after each function evaluation so fewer coordinates are perturbed later in the optimization.
Parameters: - max_evals (int) – Evaluation budget
- opt_prob (OptimizationProblem) – Optimization problem object
- exp_design (ExperimentalDesign) – Experimental design object
- surrogate (Surrogate) – Surrogate object
- asynchronous (bool) – Whether or not to use asynchrony (True/False)
- batch_size (int) – Size of the batch (use 1 for serial, ignored if async)
- extra_points (numpy.array of size n x dim) – Extra points to add to the experimental design
- extra_vals (numpy.array of size n x 1) – Values for extra_points (np.nan/np.inf if unknown)
- reset_surrogate (bool) – Whether or not to reset surrogate model
- weights (list of np.array) – Weights for merit function, default = [0.3, 0.5, 0.8, 0.95]
- num_cand (int) – Number of candidate points, default = 100*dim
-
generate_evals
(num_pts)¶ Generate the next adaptive sample points.
-
class
pySOT.strategy.
EIStrategy
(max_evals, opt_prob, exp_design, surrogate, asynchronous=True, batch_size=None, extra_points=None, extra_vals=None, reset_surrogate=True, ei_tol=None, dtol=None)¶ Expected Improvement strategy.
This is an implementation of Expected Improvement (EI), arguably the most popular acquisition function in Bayesian optimization. Under a Gaussian process (GP) prior, the expected value of the improvement:
I(x) := max(f_best - f(x), 0) EI[x] := E[I(x)]
can be computed analytically, where f_best is the best observed function value.EI is one-step optimal in the sense that selecting the maximizer of EI is the optimal action if we have exactly one function value remaining and must return a solution with a known function value.
When using parallelism, we constrain each new evaluation to be a distance dtol away from previous and pending evaluations to avoid that the same point is being evaluated multiple times. We use a default value of dtol = 1e-3 * norm(ub - lb), but note that this value has not been tuned carefully and may be far from optimal.
The optimization strategy terminates when the evaluatio budget has been exceeded or when the EI of the next point falls below some threshold, where the default threshold is 1e-6 * (max(fX) - min(fX)).
Parameters: - max_evals (int) – Evaluation budget
- opt_prob (OptimizationProblem) – Optimization problem object
- exp_design (ExperimentalDesign) – Experimental design object
- surrogate (Surrogate) – Surrogate object
- asynchronous (bool) – Whether or not to use asynchrony (True/False)
- batch_size (int) – Size of the batch (use 1 for serial, ignored if async)
- extra_points (numpy.array of size n x dim) – Extra points to add to the experimental design
- extra_vals (numpy.array of size n x 1) – Values for extra_points (np.nan/np.inf if unknown)
- reset_surrogate (bool) – Whether or not to reset surrogate model
- ei_tol (float) – Terminate if the largest EI falls below this threshold Default: 1e-6 * (max(fX) - min(fX))
- dtol (float) – Minimum distance between new and pending/finished evaluations Default: 1e-3 * norm(ub - lb)
-
check_input
()¶ Check the inputs to the optimization strategt.
-
generate_evals
(num_pts)¶ Generate the next adaptive sample points.
-
class
pySOT.strategy.
LCBStrategy
(max_evals, opt_prob, exp_design, surrogate, asynchronous=True, batch_size=None, extra_points=None, extra_vals=None, reset_surrogate=True, kappa=2.0, dtol=None, lcb_tol=None)¶ Lower confidence bound strategy.
This is an implementation of Lower Confidence Bound (LCB), a popular acquisition function in Bayesian optimization. The main idea is to minimize:
LCB(x) := E[x] - kappa * sqrt(V[x])
where E[x] is the predicted function value, V[x] is the predicted variance, and kappa is a constant that balances exploration and exploitation. We use a default value of kappa = 2.
When using parallelism, we constrain each new evaluation to be a distance dtol away from previous and pending evaluations to avoid that the same point is being evaluated multiple times. We use a default value of dtol = 1e-3 * norm(ub - lb), but note that this value has not been tuned carefully and may be far from optimal.
The optimization strategy terminates when the evaluatio budget has been exceeded or when the LCB of the next point falls below some threshold, where the default threshold is 1e-6 * (max(fX) - min(fX)).
Parameters: - max_evals (int) – Evaluation budget
- opt_prob (OptimizationProblem) – Optimization problem object
- exp_design (ExperimentalDesign) – Experimental design object
- surrogate (Surrogate) – Surrogate object
- asynchronous (bool) – Whether or not to use asynchrony (True/False)
- batch_size (int) – Size of the batch (use 1 for serial, ignored if async)
- extra_points (numpy.array of size n x dim) – Extra points to add to the experimental design
- extra_vals (numpy.array of size n x 1) – Values for extra_points (np.nan/np.inf if unknown)
- reset_surrogate (bool) – Whether or not to reset surrogate model
- kappa (float) – Constant in the LCB merit function
- dtol (float) – Minimum distance between new and pending evaluations Default: 1e-3 * norm(ub - lb)
- lcb_tol (float) – Terminate if min(fX) - min(LCB(x)) < lcb_tol Default: 1e-6 * (max(fX) - min(fX))
-
check_input
()¶ Check the inputs to the optimization strategt.
-
generate_evals
(num_pts)¶ Generate the next adaptive sample points.
-
class
pySOT.strategy.
RandomSampling
(max_evals, opt_prob)¶ Random sampling strategy.
We generate and evaluate a fixed number of points using all resources. The optimization problem must implement OptimizationProblem and max_evals must be a positive integer.
Parameters: - max_evals (int) – Evaluation budget
- opt_prob (OptimizationProblem) – Optimization problem
-
propose_action
()¶ Propose an action based on outstanding points.
-
class
pySOT.strategy.
SRBFStrategy
(max_evals, opt_prob, exp_design, surrogate, asynchronous=True, batch_size=None, extra_points=None, extra_vals=None, reset_surrogate=True, weights=None, num_cand=None)¶ Stochastic RBF (SRBF) optimization strategy.
This is an implementation of the SRBF strategy by Regis and Shoemaker:
Rommel G Regis and Christine A Shoemaker. A stochastic radial basis function method for the global optimization of expensive functions. INFORMS Journal on Computing, 19(4): 497–509, 2007.
Rommel G Regis and Christine A Shoemaker. Parallel stochastic global optimization using radial basis functions. INFORMS Journal on Computing, 21(3):411–426, 2009.
The main idea is to pick the new evaluations from a set of candidate points where each candidate point is generated as an N(0, sigma^2) distributed perturbation from the current best solution. The value of sigma is modified based on progress and follows the same logic as in many trust region methods; we increase sigma if we make a lot of progress (the surrogate is accurate) and decrease sigma when we aren’t able to make progress (the surrogate model is inaccurate). More details about how sigma is updated is given in the original papers.
After generating the candidate points we predict their objective function value and compute the minimum distance to previously evaluated point. Let the candidate points be denoted by C and let the function value predictions be s(x_i) and the distance values be d(x_i), both rescaled through a linear transformation to the interval [0,1]. This is done to put the values on the same scale. The next point selected for evaluation is the candidate point x that minimizes the weighted-distance merit function:
merit(x) := w * s(x) + (1 - w) * (1 - d(x))
where 0 <= w <= 1. That is, we want a small function value prediction and a large minimum distance from previously evalauted points. The weight w is commonly cycled between a few values to achieve both exploitation and exploration. When w is close to zero we do pure exploration while w close to 1 corresponds to explotation.
This strategy has two additional arguments than the base class:
- weights: Specify a list of weights to cycle through
- Default = [0.3, 0.5, 0.8, 0.95]
- num_cand: Number of candidate to use when generating new evaluations
- Default = 100 * dim
Parameters: - max_evals (int) – Evaluation budget
- opt_prob (OptimizationProblem) – Optimization problem object
- exp_design (ExperimentalDesign) – Experimental design object
- surrogate (Surrogate) – Surrogate object
- asynchronous (bool) – Whether or not to use asynchrony (True/False)
- batch_size (int) – Size of the batch (use 1 for serial, ignored if async)
- extra_points (numpy.array of size n x dim) – Extra points to add to the experimental design
- extra_vals (numpy.array of size n x 1) – Values for extra_points (np.nan/np.inf if unknown)
- reset_surrogate (bool) – Whether or not to reset surrogate model
- weights (list of np.array) – Weights for merit function, default = [0.3, 0.5, 0.8, 0.95]
- num_cand (int) – Number of candidate points, default = 100*dim
-
adjust_step
()¶ Adjust the sampling radius sigma.
After succtol successful steps, we cut the sampling radius; after failtol failed steps, we double the sampling radius.
-
check_input
()¶ Check inputs.
-
generate_evals
(num_pts)¶ Generate the next adaptive sample points.
-
get_weights
(num_pts)¶ Generate the nextw weights.
-
on_adapt_completed
(record)¶ Handle completed evaluation.
-
class
pySOT.strategy.
SurrogateBaseStrategy
(max_evals, opt_prob, exp_design, surrogate, asynchronous=True, batch_size=None, extra_points=None, extra_vals=None, reset_surrogate=True)¶ -
adapt_proposal
()¶ Propose a point from the batch_queue.
-
check_input
()¶ Check the inputs to the optimization strategt.
-
generate_evals
(num_pts)¶
-
init_proposal
()¶ Propose a point from the initial experimental design.
-
log_completion
(record)¶ Record a completed evaluation to the log.
Parameters: record (EvalRecord) – Record of the function evaluation
-
make_proposal
(x)¶ Create proposal and update counters and budgets.
-
on_adapt_aborted
(record)¶ Handle aborted feval from sampling phase.
-
on_adapt_accept
(proposal)¶ Handle accepted proposal from sampling phase.
-
on_adapt_completed
(record)¶ Handle completion of feval from sampling phase.
-
on_adapt_proposal
(proposal)¶ Handle accept/reject of proposal from sampling phase.
-
on_adapt_reject
(proposal)¶ Handle rejected proposal from sampling phase.
-
on_adapt_update
(record)¶ Handle update of feval from sampling phase.
-
on_initial_aborted
(record)¶ Handle aborted feval from initial design.
-
on_initial_accepted
(proposal)¶ Handle proposal accept from initial design.
-
on_initial_completed
(record)¶ Handle successful completion of feval from initial design.
-
on_initial_proposal
(proposal)¶ Handle accept/reject of proposal from initial design.
-
on_initial_rejected
(proposal)¶ Handle proposal rejection from initial design.
-
on_initial_update
(record)¶ Handle update of feval from initial design.
-
propose_action
()¶ Propose an action.
NB: We allow workers to continue to the adaptive phase if the initial queue is empty. This implies that we need enough points in the experimental design for us to construct a surrogate.
-
remove_pending
(x)¶ Delete a pending point from self.Xpend.
-
resume
()¶ Resume a terminated run.
-
sample_initial
()¶ Generate and queue an initial experimental design.
-
save
(fname)¶ Save the state of the strategy.
- We do this in a 3-step procedure
- Save to temp file
- Move temp file to save file
- Remove temp file
Parameters: fname (string) – Filename
-
pySOT.surrogate module¶
Module: | surrogate |
---|---|
Author: | David Eriksson <dme65@cornell.edu> |
-
class
pySOT.surrogate.
ConstantTail
(dim)¶ Constant polynomial tail.
Constant polynomial in d-dimension, built from the basis \(\{ 1 \}\).
-
deriv
(x)¶ Evaluate the derivative of the constant polynomial tail.
Parameters: x (numpy.array) – Point to evaluate, of size (1, dim) or (dim,) Returns: A numpy.array of size dim_tail x dim Return type: numpy.array
-
eval
(X)¶ Evaluate the constant polynomial tail.
Parameters: X (numpy.array) – Points to evaluate, of size num_pts x dim Returns: A numpy.array of size num_pts x dim_tail(dim) Return type: numpy.array
-
-
class
pySOT.surrogate.
CubicKernel
¶ Cubic RBF kernel
This is a class for the Cubic RBF kernel: \(\varphi(r) = r^3\) which is conditionally positive definite of order 2.
-
deriv
(dists)¶ Evaluates the derivative of the Cubic kernel for a distance matrix.
Parameters: dists (numpy.array) – Distance input matrix Returns: a matrix where element \((i,j)\) is \(3 \| x_i - x_j \|^2\) Return type: numpy.array
-
eval
(dists)¶ Evaluates the Cubic kernel for a distance matrix
Parameters: dists (numpy.array) – Distance input matrix Returns: a matrix where element \((i,j)\) is \(\|x_i - x_j \|^3\) Return type: numpy.array
-
-
class
pySOT.surrogate.
GPRegressor
(dim, gp=None, n_restarts_optimizer=3)¶ Gaussian process (GP) regressor.
Wrapper around the GPRegressor in scikit-learn.
Parameters: - dim (int) – Number of dimensions
- gp (object) – GPRegressor model
- n_restarts_optimizer (int) – Number of restarts in hyperparam fitting
Variables: - dim – Number of dimensions
- num_pts – Number of points in surrogate model
- X – Point incorporated in surrogate model (num_pts x dim)
- fX – Function values in surrogate model (num_pts x 1)
- updated – True if model is up-to-date (no refit needed)
- model – GPRegressor object
-
predict
(xx)¶ Evaluate the GP regressor at the points xx.
Parameters: xx (numpy.ndarray) – Prediction points, must be of size num_pts x dim or (dim, ) Returns: Prediction of size num_pts x 1 Return type: numpy.ndarray
-
predict_deriv
(xx)¶ TODO: Not implemented
-
predict_std
(xx)¶ Predict standard deviation at points xx.
Parameters: xx (numpy.ndarray) – Prediction points, must be of size num_pts x dim or (dim, ) Returns: Predicted standard deviation, of size num_pts x 1 Return type: numpy.ndarray
-
class
pySOT.surrogate.
Kernel
¶ Base class for a radial kernel.
Variables: order – Order of the conditionally positive definite kernel -
deriv
(dists)¶ Evaluate derivatives of radial kernel wrt distance.
Parameters: dists (numpy.ndarray) – Array of size n x n with pairwise distances Returns: Array of size n x n with kernel derivatives Return type: numpy.ndarray
-
eval
(dists)¶ Evaluate the radial kernel.
Parameters: dists (numpy.ndarray) – Array of size n x n with pairwise distances Returns: Array of size n x n with kernel values Return type: numpy.ndarray
-
-
class
pySOT.surrogate.
LinearKernel
¶ Linear RBF kernel.
This is a basic class for the Linear RBF kernel: \(\varphi(r) = r\) which is conditionally positive definite of order 1.
-
deriv
(dists)¶ Evaluate the derivative of the Linear kernel.
Parameters: dists (numpy.array) – Distance input matrix Returns: a matrix where element \((i,j)\) is 1 Return type: numpy.array
-
eval
(dists)¶ Evaluate the Linear kernel.
Parameters: dists (numpy.array) – Distance input matrix Returns: a matrix where element \((i,j)\) is \(\|x_i - x_j \|\) Return type: numpy.array
-
-
class
pySOT.surrogate.
LinearTail
(dim)¶ Linear polynomial tail.
This is a standard linear polynomial in d-dimension, built from the basis \(\{1,x_1,x_2,\ldots,x_d\}\).
-
deriv
(x)¶ Evaluate the derivative of the linear polynomial tail
Parameters: x (numpy.array) – Point to evaluate, of size (1, dim) or (dim,) Returns: A numpy.array of size dim_tail x dim Return type: numpy.array
-
eval
(X)¶ Evaluate the linear polynomial tail.
Parameters: X (numpy.array) – Points to evaluate, of size num_pts x dim Returns: A numpy.array of size num_pts x dim_tail Return type: numpy.array
-
-
class
pySOT.surrogate.
MARSInterpolant
(dim)¶ Compute and evaluate a MARS interpolant
MARS builds a model of the form
\[\hat{f}(x) = \sum_{i=1}^{k} c_i B_i(x).\]The model is a weighted sum of basis functions \(B_i(x)\). Each basis function \(B_i(x)\) takes one of the following three forms:
- a constant 1.
- a hinge function of the form \(\max(0, x - const)\) or \(\max(0, const - x)\). MARS automatically selects variables and values of those variables for knots of the hinge functions.
- a product of two or more hinge functions. These basis functions c an model interaction between two or more variables.
Parameters: dim (int) – Number of dimensions
Variables: - dim – Number of dimensions
- num_pts – Number of points in surrogate model
- X – Point incorporated in surrogate model (num_pts x dim)
- fX – Function values in surrogate model (num_pts x 1)
- updated – True if model is up-to-date (no refit needed)
- model – Earth object
-
predict
(xx)¶ Evaluate the MARS interpolant at the points xx
Parameters: xx (numpy.ndarray) – Prediction points, must be of size num_pts x dim or (dim, ) Returns: Prediction of size num_pts x 1 Return type: numpy.ndarray
-
predict_deriv
(xx)¶ Evaluate the derivative of the MARS interpolant at points xx
Parameters: xx (numpy.array) – Prediction points, must be of size num_pts x dim or (dim, ) Returns: Derivative of the RBF interpolant at xx Return type: numpy.array
-
class
pySOT.surrogate.
PolyRegressor
(dim, degree=2)¶ Multi-variate polynomial regression with cross-terms
Parameters: - dim (int) – Number of dimensions
- degree (int) – Polynomial degree
Variables: - dim – Number of dimensions
- num_pts – Number of points in surrogate model
- X – Point incorporated in surrogate model (num_pts x dim)
- fX – Function values in surrogate model (num_pts x 1)
- updated – True if model is up-to-date (no refit needed)
- model – scikit-learn pipeline for polynomial regression
-
predict
(xx)¶ Evaluate the polynomial regressor at the points xx
Parameters: xx (numpy.ndarray) – Prediction points, must be of size num_pts x dim or (dim, ) Returns: Prediction of size num_pts x 1 Return type: numpy.ndarray
-
predict_deriv
(xx)¶ TODO: Not implemented
-
class
pySOT.surrogate.
RBFInterpolant
(dim, kernel=None, tail=None, eta=1e-06)¶ Compute and evaluate RBF interpolant.
Manages an expansion of the form
\[s(x) = \sum_j c_j \phi(\|x-x_j\|) + \sum_j \lambda_j p_j(x)\]where the functions \(p_j(x)\) are low-degree polynomials. The fitting equations are
\[\begin{split}\begin{bmatrix} \eta I & P^T \\ P & \Phi+\eta I \end{bmatrix} \begin{bmatrix} \lambda \\ c \end{bmatrix} = \begin{bmatrix} 0 \\ f \end{bmatrix}\end{split}\]where \(P_{ij} = p_j(x_i)\) and \(\Phi_{ij}=\phi(\|x_i-x_j\|)\) The regularization parameter \(\eta\) allows us to avoid problems with potential poor conditioning of the system. Consider using the SurrogateUnitBox wrapper or manually scaling the domain to the unit hypercube to avoid issues with the domain scaling.
We add k new points to the RBFInterpolant in \(O(kn^2)\) flops by updating the LU factorization of the old RBF system. This is better than computing the RBF coefficients from scratch, which costs \(O(n^3)\) flops.
Parameters: Variables: - dim – Number of dimensions
- num_pts – Number of points in surrogate model
- X – Point incorporated in surrogate model (num_pts x dim)
- fX – Function values in surrogate model (num_pts x 1)
- updated – True if model is up-to-date (no refit needed)
- kernel – RBF kernel
- tail – RBF tail
- eta – Regularization parameter
-
predict
(xx)¶ Evaluate the RBF interpolant at the points xx
Parameters: xx (numpy.ndarray) – Prediction points, must be of size num_pts x dim or (dim, ) Returns: Prediction of size num_pts x 1 Return type: numpy.ndarray
-
predict_deriv
(xx)¶ Evaluate the derivative of the RBF interpolant at a point xx
Parameters: xx (numpy.array) – Prediction points, must be of size num_pts x dim or (dim, ) Returns: Derivative of the RBF interpolant at xx Return type: numpy.array
-
reset
()¶ Reset the RBF interpolant.
-
class
pySOT.surrogate.
Surrogate
¶ Base class for a surrogate model.
Variables: - dim – Number of dimensions
- num_pts – Number of points in surrogate model
- X – Point incorporated in surrogate model (num_pts x dim)
- fX – Function values in surrogate model (num_pts x 1)
- updated – True if model is up-to-date (no refit needed)
-
add_points
(xx, fx)¶ Add new function evaluations.
This method SHOULD NOT trigger a new fit, it just updates X and fX but leaves the original surrogate object intact
Parameters: - xx (numpy.ndarray) – Points to add
- fx (numpy.array or float) – The function values of the point to add
-
predict
(xx)¶ Evaluate surroagte at points xx.
Parameters: xx (numpy.ndarray) – xx must be of size num_pts x dim or (dim, ) Returns: Surrogate predictions, of size num_pts x 1 Return type: numpy.ndarray
-
predict_deriv
(xx)¶ Evaluate derivative of interpolant at points xx.
Parameters: xx (numpy.ndarray) – xx must be of size num_pts x dim or (dim, ) Returns: Surrogate derivative predictions, of size num_pts x dim Return type: numpy.ndarray
-
reset
()¶ Reset the surrogate.
-
class
pySOT.surrogate.
SurrogateCapped
(model, transformation=None)¶ Wrapper for tranformation of function values.
This adapter takes an existing surrogate model and replaces it with a modified version where the function values are replaced according to some transformation. A common transformation is replacing all values above the median by the median to reduce the influence of large function values.
Parameters: - model (object) – Original surrogate model (must implement Surrogate)
- transformation (function) – Function that transforms the function values
Variables: - dim – Number of dimensions
- num_pts – Number of points in surrogate model
- X – Point incorporated in surrogate model (num_pts x dim)
- fX – Function values in surrogate model (num_pts x 1)
- updated – True if model is up-to-date (no refit needed)
- model – scikit-learn pipeline for polynomial regression
- transformation – Transformation function
-
add_points
(xx, fx)¶ Add new function evaluations.
This method SHOULD NOT trigger a new fit, it just updates X and fX but leaves the original surrogate object intact
Parameters: - xx (numpy.ndarray) – Points to add
- fx (numpy.array or float) – The function values of the point to add
-
predict
(xx)¶ Evaluate the surrogate model at the points xx
Parameters: xx (numpy.ndarray) – Prediction points, must be of size num_pts x dim or (dim, ) Returns: Prediction of size num_pts x 1 Return type: numpy.ndarray
-
predict_deriv
(xx)¶ Evaluate the derivative of the surrogate model at points xx.
Parameters: xx (numpy.array) – Prediction points, must be of size num_pts x dim or (dim, ) Returns: Derivative of the RBF interpolant at xx Return type: numpy.array
-
predict_std
(xx)¶ Predict standard deviation at points xx.
Parameters: xx (numpy.ndarray) – Prediction points, must be of size num_pts x dim or (dim, ) Returns: Predicted standard deviation, of size num_pts x 1 Return type: numpy.ndarray
-
reset
()¶ Reset the surrogate.
-
class
pySOT.surrogate.
SurrogateUnitBox
(model, lb, ub)¶ Unit box adapter for surrogate models.
This adapter takes an existing surrogate model and replaces it by a modified version where the domain is rescaled to the unit hypercube. This is useful for surrogate models that are sensitive to scaling, such as RBFs.
Parameters: - model (object) – Original surrogate model (must implement Surrogate)
- lb (function) – Lower variable bounds, of size 1 x dim
- ub (function) – Upper variable bounds, of size 1 x dim
Variables: - dim – Number of dimensions
- num_pts – Number of points in surrogate model
- X – Point incorporated in surrogate model (num_pts x dim)
- fX – Function values in surrogate model (num_pts x 1)
- updated – True if model is up-to-date (no refit needed)
- model – scikit-learn pipeline for polynomial regression
- lb – Lower variable bounds
- ub – Upper variable bounds
-
add_points
(xx, fx)¶ Add new function evaluations.
This method SHOULD NOT trigger a new fit, it just updates X and fX but leaves the original surrogate object intact
Parameters: - xx (numpy.ndarray) – Points to add
- fx (numpy.array or float) – The function values of the point to add
-
predict
(xx)¶ Evaluate the surrogate model at the points xx
Parameters: xx (numpy.ndarray) – Prediction points, must be of size num_pts x dim or (dim, ) Returns: Prediction of size num_pts x 1 Return type: numpy.ndarray
-
predict_deriv
(x)¶ Evaluate the derivative of the surrogate model at points xx
- Remember the chain rule:
- f’(x) = (d/dx) g((x-a)/(b-a)) = g’((x-a)/(b-a)) * 1/(b-a)
Parameters: xx (numpy.array) – Prediction points, must be of size num_pts x dim or (dim, ) Returns: Derivative of the RBF interpolant at xx Return type: numpy.array
-
predict_std
(x)¶ Predict standard deviation at points xx.
Parameters: xx (numpy.ndarray) – Prediction points, must be of size num_pts x dim or (dim, ) Returns: Predicted standard deviation, of size num_pts x 1 Return type: numpy.ndarray
-
reset
()¶ Reset the surrogate model.
-
class
pySOT.surrogate.
TPSKernel
¶ Thin-plate spline RBF kernel.
This is a basic class for the TPS RBF kernel: \(\varphi(r) = r^2 \log(r)\) which is conditionally positive definite of order 2.
-
deriv
(dists)¶ Evaluate the derivative of the TPS kernel.
Parameters: dists (numpy.array) – Distance input matrix Returns: a matrix where element \((i,j)\) is \(\|x_i - x_j \|(1 + 2 \log (\|x_i - x_j \|) )\) Return type: numpy.array
-
eval
(dists)¶ Evaluate the TPS kernel.
Parameters: dists (numpy.array) – Distance input matrix Returns: a matrix where element \((i,j)\) is \(\|x_i - x_j \|^2 \log (\|x_i - x_j \|)\) Return type: numpy.array
-
-
class
pySOT.surrogate.
Tail
¶ Base class for a polynomial tail.
“ivar dim: Dimensionality of the original space :ivar dim_tail: Dimensionality of the polynomial space (number of basis functions)
-
deriv
(x)¶ Evaluate derivative of the polynomial tail.
Parameters: x (numpy.ndarray) – Array of size 1 x dim or (dim,) Returns: Array of size dim_tail x dim Return type: numpy.ndarray
-
eval
(X)¶ Evaluate the polynomial tail.
Parameters: X (numpy.ndarray) – Array of size num_pts x dim Returns: Array of size num_pts x dim_tail Return type: numpy.ndarray
-
pySOT.utils module¶
Module: | utils |
---|---|
Author: | David Eriksson <dme65@cornell.edu> |
-
class
pySOT.utils.
GeneticAlgorithm
(function, dim, lb, ub, int_var=None, pop_size=100, num_gen=100, start='SLHD')¶ Genetic algorithm.
Implementation of the real-valued Genetic algorithm. The mutations are normally distributed perturbations, the selection mechanism is a tournament selection, and the crossover oepration is the standard linear combination taken at a randomly generated cutting point.
The total number of evaluations are popsize x ngen
Parameters: - function (Object) – Function that can be used to evaluate the entire population. It needs to take an input of size pop_size x dim and return a numpy.array of size pop_size x 1
- dim (int) – Number of dimensions
- lb (numpy.array) – Lower variable bounds, of length dim
- ub (numpy.array) – Lower variable bounds, of length dim
- int_var (list) – List of indices with the integer valued variables (e.g., [0, 1, 5])
- pop_size (int) – Population size
- num_gen (int) – Number of generations
- start (string) – Method for generating the initial population
Variables: - nvariables – Number of variables (dimensions)
- nindividuals – population size
- lower_boundary – lower bounds for the optimization problem
- upper_boundary – upper bounds for the optimization problem
- integer_variables – List of variables that are integer valued
- start – Method for generating the initial population
- sigma – Perturbation radius. Each pertubation is N(0, sigma)
- p_mutation – Mutation probability (1/dim)
- tournament_size – Size of the tournament (5)
- p_cross – Cross-over probability (0.9)
- ngenerations – Number of generations
- function – Object that can be used to evaluate the objective function
-
optimize
()¶ Method used to run the Genetic algorithm
Returns: Returns the best individual and its function value Return type: numpy.array, float
-
pySOT.utils.
check_opt_prob
(obj)¶ Check an implementation of the optimization problem.
This method checks everything, but can’t make sure that the objective function returns values of the correct type since this would involve actually evaluating the objective function, which isn’t feasible when the evaluations are expensive. If some test fails, an exception is raised.
Parameters: obj (Object) – Optimization problem :raise Appropriate error if object doesn’t follow the pySOT standard
-
pySOT.utils.
from_unit_box
(x, lb, ub)¶ Maps a set of points from the unit box to the original domain
Parameters: - x (numpy.array) – Points to be mapped from the unit box, of size npts x dim
- lb (numpy.array) – Lower bounds, of size 1 x dim
- ub (numpy.array) – Upper bounds, of size 1 x dim
Returns: Points mapped to the original domain
Return type: numpy.array
-
pySOT.utils.
progress_plot
(controller, title='', interactive=False)¶ Makes a progress plot from a POAP controller.
This method requires matplotlib and will terminate if matplotlib.pyplot is unavailable.
Parameters: - controller (Object) – POAP controller object
- title (string) – Title of the plot
- interactive (bool) – True if the plot should be interactive
-
pySOT.utils.
round_vars
(x, int_var, lb, ub)¶ Round integer variables to closest integer in the domain.
Parameters: - x (numpy.array) – Set of points, of size npts x dim
- int_var (numpy.array) – Set of indices of integer variables
- lb (numpy.array) – Lower bounds, of size 1 x dim
- ub (numpy.array) – Upper bounds, of size 1 x dim
Returns: The set of points with the integer variables rounded to the closest integer in the domain
Return type: numpy.array
-
pySOT.utils.
to_unit_box
(x, lb, ub)¶ Maps a set of points to the unit box
Parameters: - x (numpy.array) – Points to be mapped to the unit box, of size npts x dim
- lb (numpy.array) – Lower bounds, of size 1 x dim
- ub (numpy.array) – Upper bounds, of size 1 x dim
Returns: Points mapped to the unit box
Return type: numpy.array
-
pySOT.utils.
unit_rescale
(x)¶ Shift and rescale elements of a vector to the unit interval
Parameters: x (numpy.ndarray) – array that should be rescaled to the unit interval Returns: array scaled to the unit interval Return type: numpy.ndarray