EfficientGlobalOptimization

class EfficientGlobalOptimization(*args)

Efficient Global Optimization algorithm.

The EGO algorithm [Jones1998] is an adaptative optimization method based on kriging. An initial design of experiment is used to build a first metamodel. At each iteration a new point that maximizes a criterion is chosen as optimizer candidate. The criterion uses a tradeoff between the metamodel value and the conditional variance. Then the new point is evaluated using the original model and the metamodel is relearnt on the extended design of experiment.

Available constructors:
EfficientGlobalOptimization(problem, krigingResult)
Parameters:

problem : OptimizationProblem

The optimization problem to solve optionally, a 2nd objective marginal can be used as noise

krigingResult : KrigingResult

The result of the meta-model on the first design of experiment

Notes

Each point added to the metamodel design seeks to improve the current minimum. We chose the point so as to maximize an improvement criterion based on the metamodel.

I(x_{new}) = max(f_{min} - Y_{new}, 0)

The default criteria is called EI (Expected Improvement) and aims at maximizing the mean improvement:

\mathbb{E}\left[I(x_{new})\right] = \mathbb{E}\left[max(f_{min} - Y_{new}, 0)\right]

This criterion is explicited using the kriging mean and variance:

\mathbb{E}\left[I(x_{new})\right] = (f_{min} - m_K(x_{new})) \Phi\left( \frac{f_{min} - m_K(x_{new})}{s_K(x_{new})} \right) + s_K(x_{new}) \phi\left( \frac{f_{min} - m_K(x_{new})}{s_K(x_{new})} \right)

An observation noise variance can be provided thanks to a 2nd objective marginal.

Y_{obs} = Y(x) + \sigma_{\epsilon}(x) \epsilon

In that case the AEI (Augmented Expected Improvement) formulation is used. As we don’t have access to the real minimum of the function anymore a quantile of the kriging prediction is used, with the constant c:

u(x) = m_K(x) + c s_K(x)

This criterion is minimized over the design points:

x_{min} = \argmax_{x_i} (u(x_i))

The AEI criterion reads:

AEI(x_{new}) = \mathbb{E}\left[max(m_K(x_{min}) - Y_{new}, 0)\right] \times \left(1 - \frac{\sigma_{\epsilon}(x_{new})}{\sqrt{\sigma_{\epsilon}^2(x_{new})+s^2_K(x_{new})}} \right)

with

\mathbb{E}\left[max(m_K(x_{min}) - Y_{new}, 0)\right] = (m_K(x_{min}) - m_K(x_{new})) \Phi\left( \frac{m_K(x_{min}) - m_K(x_{new})}{s_K(x_{new})} \right) + s_K(x_{new}) \phi\left( \frac{m_K(x_{min}) - m_K(x_{new})}{s_K(x_{new})} \right)

A less computationally expensive noise function can be provided through setNoiseModel() to evaluate \sigma^2_{\epsilon}(x) for the improvement criterion optimization, the objective being only used to compute values and associated noise at design points.

By default the criteria is minimized using MultiStart with starting points uniformly sampled in the optimization problem bounds, see setMultiStartExperimentSize() and setMultiStartNumber(). This behavior can be overridden by using another solver with setOptimizationAlgorithm().

Examples

>>> import openturns as ot
>>> ot.RandomGenerator.SetSeed(0)
>>> dim = 4
>>> model = ot.SymbolicFunction(['x1', 'x2', 'x3', 'x4'],
...     ['x1*x1+x2^3*x1+x3+x4'])
>>> bounds = ot.Interval([-5.0] * dim, [5.0] * dim)
>>> problem = ot.OptimizationProblem()
>>> problem.setObjective(model)
>>> problem.setBounds(bounds)
>>> experiment = ot.Composite([0.0] * dim, [1.0, 2.0, 4.0])
>>> inputSample = experiment.generate()
>>> outputSample = model(inputSample)
>>> covarianceModel = ot.SquaredExponential([2.0] * dim, [0.1])
>>> basis = ot.ConstantBasisFactory(dim).build()
>>> kriging = ot.KrigingAlgorithm(inputSample, outputSample, covarianceModel, basis)
>>> kriging.run()
>>> algo = ot.EfficientGlobalOptimization(problem, kriging.getResult())
>>> algo.setMaximumIterationNumber(2)
>>> algo.run()
>>> result = algo.getResult()

Methods

computeLagrangeMultipliers(x) Compute the Lagrange multipliers of a problem at a given point.
getAIETradeoff() AEI tradeoff constant accessor.
getClassName() Accessor to the object’s name.
getCorrelationLengthFactor() Correlation length stopping criterion factor accessor.
getExpectedImprovement() Expected improvement values.
getId() Accessor to the object’s id.
getImprovementFactor() Improvement criterion factor accessor.
getMaximumAbsoluteError() Accessor to maximum allowed absolute error.
getMaximumConstraintError() Accessor to maximum allowed constraint error.
getMaximumEvaluationNumber() Accessor to maximum allowed number of evaluations.
getMaximumIterationNumber() Accessor to maximum allowed number of iterations.
getMaximumRelativeError() Accessor to maximum allowed relative error.
getMaximumResidualError() Accessor to maximum allowed residual error.
getMultiStartExperimentSize() Size of the design to draw starting points.
getMultiStartNumber() Number of starting points for the criterion optimization.
getName() Accessor to the object’s name.
getNoiseModel() Improvement noise model accessor.
getOptimizationAlgorithm() Expected improvement solver accessor.
getParameterEstimationPeriod() Parameter estimation period accessor.
getProblem() Accessor to optimization problem.
getResult() Accessor to optimization result.
getShadowedId() Accessor to the object’s shadowed id.
getStartingPoint() Accessor to starting point.
getVerbose() Accessor to the verbosity flag.
getVisibility() Accessor to the object’s visibility state.
hasName() Test if the object is named.
hasVisibleName() Test if the object has a distinguishable name.
run() Launch the optimization.
setAIETradeoff(c) AEI tradeoff constant accessor.
setCorrelationLengthFactor(b) Correlation length stopping criterion factor accessor.
setImprovementFactor(improvementFactor) Improvement criterion factor accessor.
setMaximumAbsoluteError(maximumAbsoluteError) Accessor to maximum allowed absolute error.
setMaximumConstraintError(maximumConstraintError) Accessor to maximum allowed constraint error.
setMaximumEvaluationNumber(…) Accessor to maximum allowed number of evaluations.
setMaximumIterationNumber(maximumIterationNumber) Accessor to maximum allowed number of iterations.
setMaximumRelativeError(maximumRelativeError) Accessor to maximum allowed relative error.
setMaximumResidualError(maximumResidualError) Accessor to maximum allowed residual error.
setMultiStartExperimentSize(…) Size of the design to draw starting points.
setMultiStartNumber(multiStartNumberSize) Number of starting points for the criterion optimization.
setName(name) Accessor to the object’s name.
setNoiseModel(noiseModel) Improvement noise model accessor.
setOptimizationAlgorithm(solver) Expected improvement solver accessor.
setParameterEstimationPeriod(…) Parameter estimation period accessor.
setProblem(problem) Accessor to optimization problem.
setProgressCallback(*args) Set up a progress callback.
setResult(result) Accessor to optimization result.
setShadowedId(id) Accessor to the object’s shadowed id.
setStartingPoint(startingPoint) Accessor to starting point.
setStopCallback(*args) Set up a stop callback.
setVerbose(verbose) Accessor to the verbosity flag.
setVisibility(visible) Accessor to the object’s visibility state.
__init__(*args)
computeLagrangeMultipliers(x)

Compute the Lagrange multipliers of a problem at a given point.

Parameters:

x : sequence of float

Point at which the Lagrange multipliers are computed.

Returns:

lagrangeMultiplier : sequence of float

Lagrange multipliers of the problem at the given point.

Notes

The Lagrange multipliers \vect{\lambda} are associated with the following Lagrangian formulation of the optimization problem:

\cL(\vect{x}, \vect{\lambda}_{eq}, \vect{\lambda}_{\ell}, \vect{\lambda}_{u}, \vect{\lambda}_{ineq}) = J(\vect{x}) + \Tr{\vect{\lambda}}_{eq} g(\vect{x}) + \Tr{\vect{\lambda}}_{\ell} (\vect{x}-\vect{\ell})^{+} + \Tr{\vect{\lambda}}_{u} (\vect{u}-\vect{x})^{+} + \Tr{\vect{\lambda}}_{ineq}  h^{+}(\vect{x})

where \vect{\alpha}^{+}=(\max(0,\alpha_1),\hdots,\max(0,\alpha_n)).

The Lagrange multipliers are stored as (\vect{\lambda}_{eq}, \vect{\lambda}_{\ell}, \vect{\lambda}_{u}, \vect{\lambda}_{ineq}), where:
  • \vect{\lambda}_{eq} is of dimension 0 if there is no equality constraint, else of dimension the dimension of g(\vect{x}) ie the number of scalar equality constraints
  • \vect{\lambda}_{\ell} and \vect{\lambda}_{u} are of dimension 0 if there is no bound constraint, else of dimension of \vect{x}
  • \vect{\lambda}_{eq} is of dimension 0 if there is no inequality constraint, else of dimension the dimension of h(\vect{x}) ie the number of scalar inequality constraints

The vector \vect{\lambda} is solution of the following linear system:

\Tr{\vect{\lambda}}_{eq}\left[\dfrac{\partial g}{\partial\vect{x}}(\vect{x})\right]+
\Tr{\vect{\lambda}}_{\ell}\left[\dfrac{\partial (\vect{x}-\vect{\ell})^{+}}{\partial\vect{x}}(\vect{x})\right]+
\Tr{\vect{\lambda}}_{u}\left[\dfrac{\partial (\vect{u}-\vect{x})^{+}}{\partial\vect{x}}(\vect{x})\right]+
\Tr{\vect{\lambda}}_{ineq}\left[\dfrac{\partial h}{\partial\vect{x}}(\vect{x})\right]=-\dfrac{\partial J}{\partial\vect{x}}(\vect{x})

If there is no constraint of any kind, \vect{\lambda} is of dimension 0, as well as if no constraint is active.

getAIETradeoff()

AEI tradeoff constant accessor.

Returns:

c : float

Used to define a quantile of the kriging prediction at the design points. u(x)=m_K(x)+c*s_K(x)

getClassName()

Accessor to the object’s name.

Returns:

class_name : str

The object class name (object.__class__.__name__).

getCorrelationLengthFactor()

Correlation length stopping criterion factor accessor.

Returns:

b : float

Used to define a stopping criterion on the minimum correlation length: \theta_i < \frac{\Delta_i^{min}}{b} with \Delta^{min} the minimum distance between design points.

getExpectedImprovement()

Expected improvement values.

Returns:

ei : Sample

The expected improvement optimal values.

getId()

Accessor to the object’s id.

Returns:

id : int

Internal unique identifier.

getImprovementFactor()

Improvement criterion factor accessor.

Returns:

a : float

Used to define a stopping criterion on the improvement criterion: I_{max} < \alpha |Y_{min}| with I_{max} the current maximum of the improvement and Y_{min} the current optimum.

getMaximumAbsoluteError()

Accessor to maximum allowed absolute error.

Returns:

maximumAbsoluteError : float

Maximum allowed absolute error.

getMaximumConstraintError()

Accessor to maximum allowed constraint error.

Returns:

maximumConstraintError : float

Maximum allowed constraint error.

getMaximumEvaluationNumber()

Accessor to maximum allowed number of evaluations.

Returns:

N : int

Maximum allowed number of evaluations.

getMaximumIterationNumber()

Accessor to maximum allowed number of iterations.

Returns:

N : int

Maximum allowed number of iterations.

getMaximumRelativeError()

Accessor to maximum allowed relative error.

Returns:

maximumRelativeError : float

Maximum allowed relative error.

getMaximumResidualError()

Accessor to maximum allowed residual error.

Returns:

maximumResidualError : float

Maximum allowed residual error.

getMultiStartExperimentSize()

Size of the design to draw starting points.

Returns:

multiStartExperimentSize : int

The size of the Monte Carlo design from which to select the best starting points.

getMultiStartNumber()

Number of starting points for the criterion optimization.

Returns:

multiStartNumber : int

The number of starting points for the criterion optimization.

getName()

Accessor to the object’s name.

Returns:

name : str

The name of the object.

getNoiseModel()

Improvement noise model accessor.

Returns:

noiseVariance : Function

The noise variance \sigma^2_{\epsilon}(x) used for the AEI criterion optimization only. Of same input dimension than the objective and 1-d output.

getOptimizationAlgorithm()

Expected improvement solver accessor.

Returns:

solver : OptimizationSolver

The solver used to optimize the expected improvement

getParameterEstimationPeriod()

Parameter estimation period accessor.

Returns:

period : int

The number of iterations between covariance parameters re-learn. Default is 1 (each iteration). Can be set to 0 (never).

getProblem()

Accessor to optimization problem.

Returns:

problem : OptimizationProblem

Optimization problem.

getResult()

Accessor to optimization result.

Returns:

result : OptimizationResult

Result class.

getShadowedId()

Accessor to the object’s shadowed id.

Returns:

id : int

Internal unique identifier.

getStartingPoint()

Accessor to starting point.

Returns:

startingPoint : Point

Starting point.

getVerbose()

Accessor to the verbosity flag.

Returns:

verbose : bool

Verbosity flag state.

getVisibility()

Accessor to the object’s visibility state.

Returns:

visible : bool

Visibility flag.

hasName()

Test if the object is named.

Returns:

hasName : bool

True if the name is not empty.

hasVisibleName()

Test if the object has a distinguishable name.

Returns:

hasVisibleName : bool

True if the name is not empty and not the default one.

run()

Launch the optimization.

setAIETradeoff(c)

AEI tradeoff constant accessor.

Parameters:

c : float

Used to define a quantile of the kriging prediction at the design points. u(x)=m_K(x)+c*s_K(x)

setCorrelationLengthFactor(b)

Correlation length stopping criterion factor accessor.

Parameters:

b : float

Used to define a stopping criterion on the minimum correlation length: \theta_i < \frac{\Delta_i^{min}}{b} with \Delta^{min} the minimum distance between design points.

setImprovementFactor(improvementFactor)

Improvement criterion factor accessor.

Parameters:

a : float

Used to define a stopping criterion on the improvement criterion: I_{max} < \alpha |Y_{min}| with I_{max} the current maximum of the improvement and Y_{min} the current optimum.

setMaximumAbsoluteError(maximumAbsoluteError)

Accessor to maximum allowed absolute error.

Parameters:

maximumAbsoluteError : float

Maximum allowed absolute error.

setMaximumConstraintError(maximumConstraintError)

Accessor to maximum allowed constraint error.

Parameters:

maximumConstraintError : float

Maximum allowed constraint error.

setMaximumEvaluationNumber(maximumEvaluationNumber)

Accessor to maximum allowed number of evaluations.

Parameters:

N : int

Maximum allowed number of evaluations.

setMaximumIterationNumber(maximumIterationNumber)

Accessor to maximum allowed number of iterations.

Parameters:

N : int

Maximum allowed number of iterations.

setMaximumRelativeError(maximumRelativeError)

Accessor to maximum allowed relative error.

Parameters:

maximumRelativeError : float

Maximum allowed relative error.

setMaximumResidualError(maximumResidualError)

Accessor to maximum allowed residual error.

Parameters:

maximumResidualError : float

Maximum allowed residual error.

setMultiStartExperimentSize(multiStartExperimentSize)

Size of the design to draw starting points.

Parameters:

multiStartExperimentSize : int

The size of the Monte Carlo design from which to select the best starting points. The default number can be tweaked with the EfficientGlobalOptimization-DefaultMultiStartExperimentSize key from ResourceMap.

setMultiStartNumber(multiStartNumberSize)

Number of starting points for the criterion optimization.

Parameters:

multiStartNumber : int

The number of starting points for the criterion optimization. The default number can be tweaked with the EfficientGlobalOptimization-DefaultMultiStartNumber key from ResourceMap.

setName(name)

Accessor to the object’s name.

Parameters:

name : str

The name of the object.

setNoiseModel(noiseModel)

Improvement noise model accessor.

Parameters:

noiseVariance : Function

The noise variance \sigma^2_{\epsilon}(x) used for the AEI criterion optimization only. Of same input dimension than the objective and 1-d output.

setOptimizationAlgorithm(solver)

Expected improvement solver accessor.

Parameters:

solver : OptimizationSolver

The solver used to optimize the expected improvement

setParameterEstimationPeriod(parameterEstimationPeriod)

Parameter estimation period accessor.

Parameters:

period : int

The number of iterations between covariance parameters re-learn. Default is 1 (each iteration). Can be set to 0 (never). The default number can be tweaked with the EfficientGlobalOptimization-DefaultParameterEstimationPeriod key from ResourceMap.

setProblem(problem)

Accessor to optimization problem.

Parameters:

problem : OptimizationProblem

Optimization problem.

setProgressCallback(*args)

Set up a progress callback.

Parameters:

callback : callable

Takes a float as argument as percentage of progress.

Notes

May not be implemented by all solvers, refer to the solver documentation.

setResult(result)

Accessor to optimization result.

Parameters:

result : OptimizationResult

Result class.

setShadowedId(id)

Accessor to the object’s shadowed id.

Parameters:

id : int

Internal unique identifier.

setStartingPoint(startingPoint)

Accessor to starting point.

Parameters:

startingPoint : Point

Starting point.

setStopCallback(*args)

Set up a stop callback.

Parameters:

callback : callable

Returns an int deciding whether to stop or continue.

Notes

May not be implemented by all solvers, refer to the solver documentation.

setVerbose(verbose)

Accessor to the verbosity flag.

Parameters:

verbose : bool

Verbosity flag state.

setVisibility(visible)

Accessor to the object’s visibility state.

Parameters:

visible : bool

Visibility flag.