EfficientGlobalOptimization

class EfficientGlobalOptimization(*args)

Efficient Global Optimization algorithm.

The EGO algorithm [jones1998] is an adaptative optimization method based on kriging. An initial design of experiment is used to build a first metamodel. At each iteration a new point that maximizes a criterion is chosen as optimizer candidate. The criterion uses a tradeoff between the metamodel value and the conditional variance. Then the new point is evaluated using the original model and the metamodel is relearnt on the extended design of experiment.

Parameters
problemOptimizationProblem

The optimization problem to solve

krigingResultKrigingResult

The result of the meta-model on the first design of experiment

noiseFunction

Noise model used on design experiment points, and also for improvement optimization

Notes

Each point added to the metamodel design seeks to improve the current minimum. We chose the point so as to maximize an improvement criterion based on the metamodel.

I(x_{new}) = max(f_{min} - Y_{new}, 0)

The default criteria is called EI (Expected Improvement) and aims at maximizing the mean improvement:

\mathbb{E}\left[I(x_{new})\right] = \mathbb{E}\left[max(f_{min} - Y_{new}, 0)\right]

This criterion is explicited using the kriging mean and variance:

\mathbb{E}\left[I(x_{new})\right] = (f_{min} - m_K(x_{new})) \Phi\left( \frac{f_{min} - m_K(x_{new})}{s_K(x_{new})} \right) + s_K(x_{new}) \phi\left( \frac{f_{min} - m_K(x_{new})}{s_K(x_{new})} \right)

An optional observation noise variance can be provided:

Y_{obs} = Y(x) + \sigma_{\epsilon}(x) \epsilon

In that case the AEI (Augmented Expected Improvement) formulation is used. As we don’t have access to the real minimum of the function anymore a quantile of the kriging prediction is used, with the constant c:

u(x) = m_K(x) + c s_K(x)

This criterion is minimized over the design points:

x_{min} = \argmax_{x_i} (u(x_i))

The AEI criterion reads:

AEI(x_{new}) = \mathbb{E}\left[max(m_K(x_{min}) - Y_{new}, 0)\right] \times \left(1 - \frac{\sigma_{\epsilon}(x_{new})}{\sqrt{\sigma_{\epsilon}^2(x_{new})+s^2_K(x_{new})}} \right)

with

\mathbb{E}\left[max(m_K(x_{min}) - Y_{new}, 0)\right] = (m_K(x_{min}) - m_K(x_{new})) \Phi\left( \frac{m_K(x_{min}) - m_K(x_{new})}{s_K(x_{new})} \right) + s_K(x_{new}) \phi\left( \frac{m_K(x_{min}) - m_K(x_{new})}{s_K(x_{new})} \right)

A less computationally expensive noise function can be provided through setNoiseModel() to evaluate \sigma^2_{\epsilon}(x) for the improvement criterion optimization, the objective being only used to compute values and associated noise at design points.

By default the criteria is minimized using MultiStart with starting points uniformly sampled in the optimization problem bounds, see setMultiStartExperimentSize() and setMultiStartNumber(). This behavior can be overridden by using another solver with setOptimizationAlgorithm().

Examples

>>> import openturns as ot
>>> ot.RandomGenerator.SetSeed(0)
>>> dim = 4
>>> model = ot.SymbolicFunction(['x1', 'x2', 'x3', 'x4'],
...     ['x1*x1+x2^3*x1+x3+x4'])
>>> model = ot.MemoizeFunction(model)
>>> bounds = ot.Interval([-5.0] * dim, [5.0] * dim)
>>> problem = ot.OptimizationProblem()
>>> problem.setObjective(model)
>>> problem.setBounds(bounds)
>>> experiment = ot.Composite([0.0] * dim, [1.0, 2.0, 4.0])
>>> inputSample = experiment.generate()
>>> outputSample = model(inputSample)
>>> print('Initial minimum output value: ', outputSample.getMin())
Initial minimum output value:  [-248]
>>> covarianceModel = ot.SquaredExponential([2.0] * dim, [0.1])
>>> basis = ot.ConstantBasisFactory(dim).build()
>>> kriging = ot.KrigingAlgorithm(inputSample, outputSample, covarianceModel, basis)
>>> kriging.run()
>>> algo = ot.EfficientGlobalOptimization(problem, kriging.getResult())
>>> algo.setMaximumEvaluationNumber(2)
>>> algo.run()
>>> result = algo.getResult()
>>> updatedKrigingResult = algo.getKrigingResult()
>>> updatedOutputSample = updatedKrigingResult.getOutputSample()
>>> print('Updated minimum output value: ', updatedOutputSample.getMin())
Updated minimum output value:  [-610]

Methods

getAEITradeoff()

AEI tradeoff constant accessor.

getClassName()

Accessor to the object's name.

getCorrelationLengthFactor()

Correlation length stopping criterion factor accessor.

getExpectedImprovement()

Expected improvement values.

getId()

Accessor to the object's id.

getImprovementFactor()

Improvement criterion factor accessor.

getKrigingResult()

Retrieve the Kriging result.

getMaximumAbsoluteError()

Accessor to maximum allowed absolute error.

getMaximumConstraintError()

Accessor to maximum allowed constraint error.

getMaximumEvaluationNumber()

Accessor to maximum allowed number of evaluations.

getMaximumIterationNumber()

Accessor to maximum allowed number of iterations.

getMaximumRelativeError()

Accessor to maximum allowed relative error.

getMaximumResidualError()

Accessor to maximum allowed residual error.

getMetamodelNoise()

Metamodel noise model accessor.

getMultiStartExperimentSize()

Size of the design to draw starting points.

getMultiStartNumber()

Number of starting points for the criterion optimization.

getName()

Accessor to the object's name.

getNoiseModel()

Improvement noise model accessor.

getOptimizationAlgorithm()

Expected improvement solver accessor.

getParameterEstimationPeriod()

Parameter estimation period accessor.

getProblem()

Accessor to optimization problem.

getResult()

Accessor to optimization result.

getShadowedId()

Accessor to the object's shadowed id.

getStartingPoint()

Accessor to starting point.

getVerbose()

Accessor to the verbosity flag.

getVisibility()

Accessor to the object's visibility state.

hasName()

Test if the object is named.

hasVisibleName()

Test if the object has a distinguishable name.

run()

Launch the optimization.

setAEITradeoff(c)

AEI tradeoff constant accessor.

setCorrelationLengthFactor(b)

Correlation length stopping criterion factor accessor.

setImprovementFactor(improvementFactor)

Improvement criterion factor accessor.

setMaximumAbsoluteError(maximumAbsoluteError)

Accessor to maximum allowed absolute error.

setMaximumConstraintError(maximumConstraintError)

Accessor to maximum allowed constraint error.

setMaximumEvaluationNumber(...)

Accessor to maximum allowed number of evaluations.

setMaximumIterationNumber(maximumIterationNumber)

Accessor to maximum allowed number of iterations.

setMaximumRelativeError(maximumRelativeError)

Accessor to maximum allowed relative error.

setMaximumResidualError(maximumResidualError)

Accessor to maximum allowed residual error.

setMetamodelNoise(metaModelNoise)

Metamodel noise model accessor.

setMultiStartExperimentSize(...)

Size of the design to draw starting points.

setMultiStartNumber(multiStartNumberSize)

Number of starting points for the criterion optimization.

setName(name)

Accessor to the object's name.

setNoiseModel(noiseModel)

Improvement noise model accessor.

setOptimizationAlgorithm(solver)

Expected improvement solver accessor.

setParameterEstimationPeriod(...)

Parameter estimation period accessor.

setProblem(problem)

Accessor to optimization problem.

setProgressCallback(*args)

Set up a progress callback.

setResult(result)

Accessor to optimization result.

setShadowedId(id)

Accessor to the object's shadowed id.

setStartingPoint(startingPoint)

Accessor to starting point.

setStopCallback(*args)

Set up a stop callback.

setVerbose(verbose)

Accessor to the verbosity flag.

setVisibility(visible)

Accessor to the object's visibility state.

__init__(*args)
getAEITradeoff()

AEI tradeoff constant accessor.

Returns
cfloat

Used to define a quantile of the kriging prediction at the design points. u(x)=m_K(x)+c*s_K(x)

getClassName()

Accessor to the object’s name.

Returns
class_namestr

The object class name (object.__class__.__name__).

getCorrelationLengthFactor()

Correlation length stopping criterion factor accessor.

When a correlation length becomes smaller than the minimal distance between design point for a single component that means the model tends to be noisy, and the EGO formulation is not adapted anymore.

Returns
bfloat

Used to define a stopping criterion on the minimum correlation length: \theta_i < \frac{\Delta_i^{min}}{b} with \Delta^{min} the minimum distance between design points.

getExpectedImprovement()

Expected improvement values.

Returns
eiSample

The expected improvement optimal values.

getId()

Accessor to the object’s id.

Returns
idint

Internal unique identifier.

getImprovementFactor()

Improvement criterion factor accessor.

Returns
alphapositive float, default=0.0 (disables the criterion)

Used to define a stopping criterion on the improvement criterion: I_{max} < \alpha |Y_{min}| with I_{max} the current maximum of the improvement and Y_{min} the current optimum.

getKrigingResult()

Retrieve the Kriging result.

Returns
krigingResultKrigingResult

Kriging result that takes all observations into account.

Notes

Before run() is called, this method returns the KrigingResult passed to the constructor. Once run() has been called, it returns an updated KrigingResult that takes new observations into account.

getMaximumAbsoluteError()

Accessor to maximum allowed absolute error.

Returns
maximumAbsoluteErrorfloat

Maximum allowed absolute error, where the absolute error is defined by \epsilon^a_n=\|\vect{x}_{n+1}-\vect{x}_n\|_{\infty} where \vect{x}_{n+1} and \vect{x}_n are two consecutive approximations of the optimum.

getMaximumConstraintError()

Accessor to maximum allowed constraint error.

Returns
maximumConstraintErrorfloat

Maximum allowed constraint error, where the constraint error is defined by \gamma_n=\|g(\vect{x}_n)\|_{\infty} where \vect{x}_n is the current approximation of the optimum and g is the function that gathers all the equality and inequality constraints (violated values only)

getMaximumEvaluationNumber()

Accessor to maximum allowed number of evaluations.

Returns
Nint

Maximum allowed number of evaluations.

getMaximumIterationNumber()

Accessor to maximum allowed number of iterations.

Returns
Nint

Maximum allowed number of iterations.

getMaximumRelativeError()

Accessor to maximum allowed relative error.

Returns
maximumRelativeErrorfloat

Maximum allowed relative error, where the relative error is defined by \epsilon^r_n=\epsilon^a_n/\|\vect{x}_{n+1}\|_{\infty} if \|\vect{x}_{n+1}\|_{\infty}\neq 0, else \epsilon^r_n=-1.

getMaximumResidualError()

Accessor to maximum allowed residual error.

Returns
maximumResidualErrorfloat

Maximum allowed residual error, where the residual error is defined by \epsilon^r_n=\frac{\|f(\vect{x}_{n+1})-f(\vect{x}_{n})\|}{\|f(\vect{x}_{n+1})\|} if \|f(\vect{x}_{n+1})\|\neq 0, else \epsilon^r_n=-1.

getMetamodelNoise()

Metamodel noise model accessor.

Returns
noiseModelFunction

The noise variance function \sigma^2_{\epsilon}(x) used for the AEI design update only. Of same input dimension as the objective and 1-d output.

getMultiStartExperimentSize()

Size of the design to draw starting points.

Returns
multiStartExperimentSizeint

The size of the Monte Carlo design from which to select the best starting points.

getMultiStartNumber()

Number of starting points for the criterion optimization.

Returns
multiStartNumberint

The number of starting points for the criterion optimization.

getName()

Accessor to the object’s name.

Returns
namestr

The name of the object.

getNoiseModel()

Improvement noise model accessor.

Returns
noiseModelFunction

The noise variance function \sigma^2_{\epsilon}(x) used for the AEI criterion optimization only. Of same input dimension as the objective and 1-d output.

getOptimizationAlgorithm()

Expected improvement solver accessor.

Returns
solverOptimizationSolver

The solver used to optimize the expected improvement

getParameterEstimationPeriod()

Parameter estimation period accessor.

Returns
periodint

The number of iterations between covariance parameters re-learn. Default is 1 (each iteration). Can be set to 0 (never).

getProblem()

Accessor to optimization problem.

Returns
problemOptimizationProblem

Optimization problem.

getResult()

Accessor to optimization result.

Returns
resultOptimizationResult

Result class.

getShadowedId()

Accessor to the object’s shadowed id.

Returns
idint

Internal unique identifier.

getStartingPoint()

Accessor to starting point.

Returns
startingPointPoint

Starting point.

getVerbose()

Accessor to the verbosity flag.

Returns
verbosebool

Verbosity flag state.

getVisibility()

Accessor to the object’s visibility state.

Returns
visiblebool

Visibility flag.

hasName()

Test if the object is named.

Returns
hasNamebool

True if the name is not empty.

hasVisibleName()

Test if the object has a distinguishable name.

Returns
hasVisibleNamebool

True if the name is not empty and not the default one.

run()

Launch the optimization.

setAEITradeoff(c)

AEI tradeoff constant accessor.

Parameters
cfloat

Used to define a quantile of the kriging prediction at the design points. u(x)=m_K(x)+c*s_K(x)

setCorrelationLengthFactor(b)

Correlation length stopping criterion factor accessor.

When a correlation length becomes smaller than the minimal distance between design point for a single component that means the model tends to be noisy, and the EGO formulation is not adapted anymore.

Parameters
bfloat

Used to define a stopping criterion on the minimum correlation length: \theta_i < \frac{\Delta_i^{min}}{b} with \Delta^{min} the minimum distance between design points.

setImprovementFactor(improvementFactor)

Improvement criterion factor accessor.

Parameters
alphapositive float, default=0.0 (disables the criterion)

Used to define a stopping criterion on the improvement criterion: I_{max} < \alpha |Y_{min}| with I_{max} the current maximum of the improvement and Y_{min} the current optimum.

setMaximumAbsoluteError(maximumAbsoluteError)

Accessor to maximum allowed absolute error.

Parameters
maximumAbsoluteErrorfloat

Maximum allowed absolute error, where the absolute error is defined by \epsilon^a_n=\|\vect{x}_{n+1}-\vect{x}_n\|_{\infty} where \vect{x}_{n+1} and \vect{x}_n are two consecutive approximations of the optimum.

setMaximumConstraintError(maximumConstraintError)

Accessor to maximum allowed constraint error.

Parameters
maximumConstraintErrorfloat

Maximum allowed constraint error, where the constraint error is defined by \gamma_n=\|g(\vect{x}_n)\|_{\infty} where \vect{x}_n is the current approximation of the optimum and g is the function that gathers all the equality and inequality constraints (violated values only)

setMaximumEvaluationNumber(maximumEvaluationNumber)

Accessor to maximum allowed number of evaluations.

Parameters
Nint

Maximum allowed number of evaluations.

setMaximumIterationNumber(maximumIterationNumber)

Accessor to maximum allowed number of iterations.

Parameters
Nint

Maximum allowed number of iterations.

setMaximumRelativeError(maximumRelativeError)

Accessor to maximum allowed relative error.

Parameters
maximumRelativeErrorfloat

Maximum allowed relative error, where the relative error is defined by \epsilon^r_n=\epsilon^a_n/\|\vect{x}_{n+1}\|_{\infty} if \|\vect{x}_{n+1}\|_{\infty}\neq 0, else \epsilon^r_n=-1.

setMaximumResidualError(maximumResidualError)

Accessor to maximum allowed residual error.

Parameters
Maximum allowed residual error, where the residual error is defined by

\epsilon^r_n=\frac{\|f(\vect{x}_{n+1})-f(\vect{x}_{n})\|}{\|f(\vect{x}_{n+1})\|} if \|f(\vect{x}_{n+1})\|\neq 0, else \epsilon^r_n=-1.

setMetamodelNoise(metaModelNoise)

Metamodel noise model accessor.

Parameters
noiseModelFunction

The noise variance function \sigma^2_{\epsilon}(x) used for the AEI design update only. Of same input dimension as the objective and 1-d output.

setMultiStartExperimentSize(multiStartExperimentSize)

Size of the design to draw starting points.

Parameters
multiStartExperimentSizeint

The size of the Monte Carlo design from which to select the best starting points. The default number can be tweaked with the EfficientGlobalOptimization-DefaultMultiStartExperimentSize key from ResourceMap.

setMultiStartNumber(multiStartNumberSize)

Number of starting points for the criterion optimization.

Parameters
multiStartNumberint

The number of starting points for the criterion optimization. The default number can be tweaked with the EfficientGlobalOptimization-DefaultMultiStartNumber key from ResourceMap.

setName(name)

Accessor to the object’s name.

Parameters
namestr

The name of the object.

setNoiseModel(noiseModel)

Improvement noise model accessor.

Parameters
noiseModelFunction

The noise variance function \sigma^2_{\epsilon}(x) used for the AEI criterion optimization only. Of same input dimension as the objective and 1-d output.

setOptimizationAlgorithm(solver)

Expected improvement solver accessor.

Parameters
solverOptimizationSolver

The solver used to optimize the expected improvement

setParameterEstimationPeriod(parameterEstimationPeriod)

Parameter estimation period accessor.

Parameters
periodint

The number of iterations between covariance parameters re-learn. Default is 1 (each iteration). Can be set to 0 (never). The default number can be tweaked with the EfficientGlobalOptimization-DefaultParameterEstimationPeriod key from ResourceMap.

setProblem(problem)

Accessor to optimization problem.

Parameters
problemOptimizationProblem

Optimization problem.

setProgressCallback(*args)

Set up a progress callback.

Can be used to programmatically report the progress of an optimization.

Parameters
callbackcallable

Takes a float as argument as percentage of progress.

Examples

>>> import sys
>>> import openturns as ot
>>> rosenbrock = ot.SymbolicFunction(['x1', 'x2'], ['(1-x1)^2+100*(x2-x1^2)^2'])
>>> problem = ot.OptimizationProblem(rosenbrock)
>>> solver = ot.OptimizationAlgorithm(problem)
>>> solver.setStartingPoint([0, 0])
>>> solver.setMaximumResidualError(1.e-3)
>>> solver.setMaximumEvaluationNumber(10000)
>>> def report_progress(progress):
...     sys.stderr.write('-- progress=' + str(progress) + '%\n')
>>> solver.setProgressCallback(report_progress)
>>> solver.run()
setResult(result)

Accessor to optimization result.

Parameters
resultOptimizationResult

Result class.

setShadowedId(id)

Accessor to the object’s shadowed id.

Parameters
idint

Internal unique identifier.

setStartingPoint(startingPoint)

Accessor to starting point.

Parameters
startingPointPoint

Starting point.

setStopCallback(*args)

Set up a stop callback.

Can be used to programmatically stop an optimization.

Parameters
callbackcallable

Returns an int deciding whether to stop or continue.

Examples

>>> import openturns as ot
>>> rosenbrock = ot.SymbolicFunction(['x1', 'x2'], ['(1-x1)^2+100*(x2-x1^2)^2'])
>>> problem = ot.OptimizationProblem(rosenbrock)
>>> solver = ot.OptimizationAlgorithm(problem)
>>> solver.setStartingPoint([0, 0])
>>> solver.setMaximumResidualError(1.e-3)
>>> solver.setMaximumEvaluationNumber(10000)
>>> def ask_stop():
...     return True
>>> solver.setStopCallback(ask_stop)
>>> solver.run()
setVerbose(verbose)

Accessor to the verbosity flag.

Parameters
verbosebool

Verbosity flag state.

setVisibility(visible)

Accessor to the object’s visibility state.

Parameters
visiblebool

Visibility flag.