EfficientGlobalOptimization¶

class EfficientGlobalOptimization(*args)¶

Efficient Global Optimization algorithm.

Warning

This class is experimental and likely to be modified in future releases. To use it, import the openturns.experimental submodule.

The EGO algorithm [jones1998] is an adaptative optimization method based on Gaussian process regression. An initial design of experiment is used to build a first metamodel. At each iteration a new point that maximizes a criterion is chosen as optimizer candidate. The criterion uses a tradeoff between the metamodel value and the conditional variance. Then the new point is evaluated using the original model and the metamodel is relearnt on the extended design of experiment.

Parameters:

problemOptimizationProblem: The optimization problem to solve
gprResultGaussianProcessRegressionResult: The result of the metamodel on the first design of experiment

Methods

`getAEITradeoff`()	AEI tradeoff constant accessor.
`getCheckStatus`()	Accessor to check status flag.
`getClassName`()	Accessor to the object's name.
`getCorrelationLengthFactor`()	Correlation length stopping criterion factor accessor.
`getExpectedImprovement`()	Expected improvement values.
`getGaussianProcessRegressionResult`()	Retrieve the GPR result.
`getMaximumAbsoluteError`()	Accessor to maximum allowed absolute error.
`getMaximumCallsNumber`()	Accessor to maximum allowed number of calls.
`getMaximumConstraintError`()	Accessor to maximum allowed constraint error.
`getMaximumIterationNumber`()	Accessor to maximum allowed number of iterations.
`getMaximumRelativeError`()	Accessor to maximum allowed relative error.
`getMaximumResidualError`()	Accessor to maximum allowed residual error.
`getMaximumTimeDuration`()	Accessor to the maximum duration.
`getMultiStartExperimentSize`()	Size of the design to draw starting points.
`getMultiStartNumber`()	Number of starting points for the criterion optimization.
`getName`()	Accessor to the object's name.
`getOptimizationAlgorithm`()	Expected improvement solver accessor.
`getParameterEstimationPeriod`()	Parameter estimation period accessor.
`getProblem`()	Accessor to optimization problem.
`getResult`()	Accessor to optimization result.
`getStartingPoint`()	Accessor to starting point.
`getStartingSample`()	Accessor to starting sample.
`hasName`()	Test if the object is named.
`run`()	Launch the optimization.
`setAEITradeoff`(c)	AEI tradeoff constant accessor.
`setCheckStatus`(checkStatus)	Accessor to check status flag.
`setCorrelationLengthFactor`(b)	Correlation length stopping criterion factor accessor.
`setMaximumAbsoluteError`(maximumAbsoluteError)	Accessor to maximum allowed absolute error.
`setMaximumCallsNumber`(maximumCallsNumber)	Accessor to maximum allowed number of calls
`setMaximumConstraintError`(maximumConstraintError)	Accessor to maximum allowed constraint error.
`setMaximumIterationNumber`(maximumIterationNumber)	Accessor to maximum allowed number of iterations.
`setMaximumRelativeError`(maximumRelativeError)	Accessor to maximum allowed relative error.
`setMaximumResidualError`(maximumResidualError)	Accessor to maximum allowed residual error.
`setMaximumTimeDuration`(maximumTime)	Accessor to the maximum duration.
`setMultiStartExperimentSize`(...)	Size of the design to draw starting points.
`setMultiStartNumber`(multiStartNumberSize)	Number of starting points for the criterion optimization.
`setName`(name)	Accessor to the object's name.
`setOptimizationAlgorithm`(solver)	Expected improvement solver accessor.
`setParameterEstimationPeriod`(...)	Parameter estimation period accessor.
`setProblem`(problem)	Accessor to optimization problem.
`setProgressCallback`(*args)	Set up a progress callback.
`setResult`(result)	Accessor to optimization result.
`setStartingPoint`(startingPoint)	Accessor to starting point.
`setStartingSample`(startingSample)	Accessor to starting sample.
`setStopCallback`(*args)	Set up a stop callback.

getKrigingResult

Notes

Each point added to the metamodel design seeks to improve the current minimum. We chose the point so as to maximize an improvement criterion based on the metamodel.

$I(x_{new}) = max(f_{min} - Y_{new}, 0)$

The default criteria is called EI (Expected Improvement) and aims at maximizing the mean improvement:

$EI(x_{new}) = \mathbb{E}\left[I(x_{new})\right] = \mathbb{E}\left[max(f_{min} - Y_{new}, 0)\right]$

This criterion is explicited using the Gaussian process mean and variance:

$\mathbb{E}\left[I(x_{new})\right] = (f_{min} - m_K(x_{new})) \Phi\left( \frac{f_{min} - m_K(x_{new})}{s_K(x_{new})} \right) + s_K(x_{new}) \phi\left( \frac{f_{min} - m_K(x_{new})}{s_K(x_{new})} \right)$

In case the covariance model of the Gaussian process model has a non-zero nugget factor $\sigma_{\epsilon}$ (i.e. the emulated function is noisy) the AEI (Augmented Expected Improvement) formulation is used. Because we do not have access to the real minimum of the function in this case, a quantile of the Gaussian process conditional mean is used, with the constant $c$ :

u(x) = m_K(x) + c s_K(x)

This criterion is minimized over the design points:

$x_{min} = \argmax_{x_i} (u(x_i))$

The AEI criterion is:

$AEI(x_{new}) = \mathbb{E}\left[max(m_K(x_{min}) - Y_{new}, 0)\right] \times \left(1 - \frac{\sigma_{\epsilon}(x_{new})}{\sqrt{\sigma_{\epsilon}^2(x_{new})+s^2_K(x_{new})}} \right)$

with

$\mathbb{E}\left[max(m_K(x_{min}) - Y_{new}, 0)\right] = (m_K(x_{min}) - m_K(x_{new})) \Phi\left( \frac{m_K(x_{min}) - m_K(x_{new})}{s_K(x_{new})} \right) + s_K(x_{new}) \phi\left( \frac{m_K(x_{min}) - m_K(x_{new})}{s_K(x_{new})} \right)$

By default the criteria is minimized using MultiStart with starting points uniformly sampled in the optimization problem bounds, see setMultiStartExperimentSize() and setMultiStartNumber(). This behavior can be overridden by using another solver with setOptimizationAlgorithm().

Examples

>>> import openturns as ot
>>> import openturns.experimental as otexp
>>> ot.RandomGenerator.SetSeed(0)
>>> dim = 4
>>> model = ot.SymbolicFunction(['x1', 'x2', 'x3', 'x4'], ['x1*x1+x2^3*x1+x3+x4'])
>>> model = ot.MemoizeFunction(model)
>>> bounds = ot.Interval([-5.0] * dim, [5.0] * dim)
>>> problem = ot.OptimizationProblem()
>>> problem.setObjective(model)
>>> problem.setBounds(bounds)
>>> inputSample = ot.JointDistribution([ot.Uniform(-5.0, 5.0)] * dim).getSample(30)
>>> outputSample = model(inputSample)
>>> yMin0 = outputSample.getMin()
>>> covarianceModel = ot.SquaredExponential([2.0] * dim, [0.1])
>>> basis = ot.ConstantBasisFactory(dim).build()
>>> fitter = otexp.GaussianProcessFitter(inputSample, outputSample, covarianceModel, basis)
>>> fitter.run()
>>> gpr = otexp.GaussianProcessRegression(fitter.getResult())
>>> gpr.run()
>>> algo = otexp.EfficientGlobalOptimization(problem, gpr.getResult())
>>> algo.setMaximumCallsNumber(2)
>>> algo.run()
>>> result = algo.getResult()
>>> updatedGPRResult = algo.getGaussianProcessRegressionResult()
>>> updatedOutputSample = updatedGPRResult.getOutputSample()
>>> yMin = updatedOutputSample.getMin()

__init__(*args)¶

getAEITradeoff()¶

AEI tradeoff constant accessor.

Returns:

cfloat: Used to define a quantile of the Gaussian process prediction at the design points. $u(x)=m_K(x)+c*s_K(x)$

getCheckStatus()¶

Accessor to check status flag.

Returns:

checkStatusbool: Whether to check the termination status. If set to False, run() will not throw an exception if the algorithm does not fully converge and will allow one to still find a feasible candidate.

getClassName()¶

Accessor to the object’s name.

Returns:

class_namestr: The object class name (object.__class__.__name__).

getCorrelationLengthFactor()¶

Correlation length stopping criterion factor accessor.

When a correlation length becomes smaller than the minimal distance between design point for a single component that means the model tends to be noisy, and the EGO formulation is not adapted anymore.

Returns:

bfloat: Used to define a stopping criterion on the minimum correlation length: $\theta_i < \frac{\Delta_i^{min}}{b}$ with $\Delta^{min}$ the minimum distance between design points.

getExpectedImprovement()¶

Expected improvement values.

Returns:

eiSample: The expected improvement optimal values.

getGaussianProcessRegressionResult()¶

Retrieve the GPR result.

Returns:

gprResultGaussianProcessRegressionResult: Result that takes all observations into account.

Notes

Before run() is called, this method returns the result passed to the constructor. Once run() has been called, it returns an updated result that takes new observations into account.

getMaximumAbsoluteError()¶

Accessor to maximum allowed absolute error.

Returns:

maximumAbsoluteErrorfloat: Maximum allowed absolute error, where the absolute error is defined by $\epsilon^a_n=\|\vect{x}_{n+1}-\vect{x}_n\|_{\infty}$ where $\vect{x}_{n+1}$ and $\vect{x}_n$ are two consecutive approximations of the optimum.

getMaximumCallsNumber()¶

Accessor to maximum allowed number of calls.

Returns:

maximumEvaluationNumberint: Maximum allowed number of direct objective function calls through the () operator. Does not take into account eventual indirect calls through finite difference gradient calls.

getMaximumConstraintError()¶

Accessor to maximum allowed constraint error.

Returns:

maximumConstraintErrorfloat: Maximum allowed constraint error, where the constraint error is defined by $\gamma_n=\|g(\vect{x}_n)\|_{\infty}$ where $\vect{x}_n$ is the current approximation of the optimum and $g$ is the function that gathers all the equality and inequality constraints (violated values only)

getMaximumIterationNumber()¶

Accessor to maximum allowed number of iterations.

Returns:

maximumIterationNumberint: Maximum allowed number of iterations.

getMaximumRelativeError()¶

Accessor to maximum allowed relative error.

Returns:

maximumRelativeErrorfloat: Maximum allowed relative error, where the relative error is defined by $\epsilon^r_n=\epsilon^a_n/\|\vect{x}_{n+1}\|_{\infty}$ if $\|\vect{x}_{n+1}\|_{\infty}\neq 0$ , else $\epsilon^r_n=-1$ .

getMaximumResidualError()¶

Accessor to maximum allowed residual error.

Returns:

maximumResidualErrorfloat: Maximum allowed residual error, where the residual error is defined by $\epsilon^r_n=\frac{\|f(\vect{x}_{n+1})-f(\vect{x}_{n})\|}{\|f(\vect{x}_{n+1})\|}$ if $\|f(\vect{x}_{n+1})\|\neq 0$ , else $\epsilon^r_n=-1$ .

getMaximumTimeDuration()¶

Accessor to the maximum duration.

Returns:

maximumTimefloat: Maximum optimization duration in seconds.

getMultiStartExperimentSize()¶

Size of the design to draw starting points.

Returns:

multiStartExperimentSizeint: The size of the Monte Carlo design from which to select the best starting points.

getMultiStartNumber()¶

Number of starting points for the criterion optimization.

Returns:

multiStartNumberint: The number of starting points for the criterion optimization.

getName()¶

Accessor to the object’s name.

Returns:

namestr: The name of the object.

getOptimizationAlgorithm()¶

Expected improvement solver accessor.

Returns:

solverOptimizationAlgorithm: The solver used to optimize the expected improvement

getParameterEstimationPeriod()¶

Parameter estimation period accessor.

Returns:

periodint: The number of iterations between covariance parameters re-learn. Default is 1 (each iteration). Can be set to 0 (never).

getProblem()¶

Accessor to optimization problem.

Returns:

problemOptimizationProblem: Optimization problem.

getResult()¶

Accessor to optimization result.

Returns:

resultOptimizationResult: Result class.

getStartingPoint()¶

Accessor to starting point.

Returns:

startingPointPoint: Starting point.

getStartingSample()¶

Accessor to starting sample.

Returns:

startingSampleSample: Starting sample.

hasName()¶

Test if the object is named.

Returns:

hasNamebool: True if the name is not empty.

run()¶: Launch the optimization.

setAEITradeoff(c)¶

AEI tradeoff constant accessor.

Parameters:

cfloat: Used to define a quantile of the Gaussian process prediction at the design points. $u(x)=m_K(x)+c*s_K(x)$

setCheckStatus(checkStatus)¶

Accessor to check status flag.

Parameters:

checkStatusbool: Whether to check the termination status. If set to False, run() will not throw an exception if the algorithm does not fully converge and will allow one to still find a feasible candidate.

setCorrelationLengthFactor(b)¶

Correlation length stopping criterion factor accessor.

When a correlation length becomes smaller than the minimal distance between design point for a single component that means the model tends to be noisy, and the EGO formulation is not adapted anymore.

Parameters:

bfloat: Used to define a stopping criterion on the minimum correlation length: $\theta_i < \frac{\Delta_i^{min}}{b}$ with $\Delta^{min}$ the minimum distance between design points.

setMaximumAbsoluteError(maximumAbsoluteError)¶

Accessor to maximum allowed absolute error.

Parameters:

maximumAbsoluteErrorfloat: Maximum allowed absolute error, where the absolute error is defined by $\epsilon^a_n=\|\vect{x}_{n+1}-\vect{x}_n\|_{\infty}$ where $\vect{x}_{n+1}$ and $\vect{x}_n$ are two consecutive approximations of the optimum.

setMaximumCallsNumber(maximumCallsNumber)¶

Accessor to maximum allowed number of calls

Parameters:

maximumEvaluationNumberint: Maximum allowed number of direct objective function calls through the () operator. Does not take into account eventual indirect calls through finite difference gradient calls.

setMaximumConstraintError(maximumConstraintError)¶

Accessor to maximum allowed constraint error.

Parameters:

maximumConstraintErrorfloat: Maximum allowed constraint error, where the constraint error is defined by $\gamma_n=\|g(\vect{x}_n)\|_{\infty}$ where $\vect{x}_n$ is the current approximation of the optimum and $g$ is the function that gathers all the equality and inequality constraints (violated values only)

setMaximumIterationNumber(maximumIterationNumber)¶

Accessor to maximum allowed number of iterations.

Parameters:

maximumIterationNumberint: Maximum allowed number of iterations.

setMaximumRelativeError(maximumRelativeError)¶

Accessor to maximum allowed relative error.

Parameters:

maximumRelativeErrorfloat: Maximum allowed relative error, where the relative error is defined by $\epsilon^r_n=\epsilon^a_n/\|\vect{x}_{n+1}\|_{\infty}$ if $\|\vect{x}_{n+1}\|_{\infty}\neq 0$ , else $\epsilon^r_n=-1$ .

setMaximumResidualError(maximumResidualError)¶

Accessor to maximum allowed residual error.

Parameters:

maximumResidualErrorfloat: Maximum allowed residual error, where the residual error is defined by $\epsilon^r_n=\frac{\|f(\vect{x}_{n+1})-f(\vect{x}_{n})\|}{\|f(\vect{x}_{n+1})\|}$ if $\|f(\vect{x}_{n+1})\|\neq 0$ , else $\epsilon^r_n=-1$ .

setMaximumTimeDuration(maximumTime)¶

Accessor to the maximum duration.

Parameters:

maximumTimefloat: Maximum optimization duration in seconds.

setMultiStartExperimentSize(multiStartExperimentSize)¶

Size of the design to draw starting points.

Parameters:

multiStartExperimentSizeint: The size of the Monte Carlo design from which to select the best starting points. The default number can be tweaked with the EfficientGlobalOptimization-DefaultMultiStartExperimentSize key from ResourceMap.

setMultiStartNumber(multiStartNumberSize)¶

Number of starting points for the criterion optimization.

Parameters:

multiStartNumberint: The number of starting points for the criterion optimization. The default number can be tweaked with the EfficientGlobalOptimization-DefaultMultiStartNumber key from ResourceMap.

setName(name)¶

Accessor to the object’s name.

Parameters:

namestr: The name of the object.

setOptimizationAlgorithm(solver)¶

Expected improvement solver accessor.

Parameters:

solverOptimizationAlgorithm: The solver used to optimize the expected improvement

setParameterEstimationPeriod(parameterEstimationPeriod)¶

Parameter estimation period accessor.

Parameters:

periodint: The number of iterations between covariance parameters re-learn. Default is 1 (each iteration). Can be set to 0 (never). The default number can be tweaked with the EfficientGlobalOptimization-DefaultParameterEstimationPeriod key from ResourceMap.

setProblem(problem)¶

Accessor to optimization problem.

Parameters:

problemOptimizationProblem: Optimization problem.

setProgressCallback(*args)¶

Set up a progress callback.

Can be used to programmatically report the progress of an optimization.

Parameters:

callbackcallable: Takes a float as argument as percentage of progress.

Examples

>>> import sys
>>> import openturns as ot
>>> rosenbrock = ot.SymbolicFunction(['x1', 'x2'], ['(1-x1)^2+100*(x2-x1^2)^2'])
>>> problem = ot.OptimizationProblem(rosenbrock)
>>> solver = ot.OptimizationAlgorithm(problem)
>>> solver.setStartingPoint([0, 0])
>>> solver.setMaximumResidualError(1.e-3)
>>> solver.setMaximumCallsNumber(10000)
>>> def report_progress(progress):
...     sys.stderr.write('-- progress=' + str(progress) + '%\n')
>>> solver.setProgressCallback(report_progress)
>>> solver.run()

setResult(result)¶

Accessor to optimization result.

Parameters:

resultOptimizationResult: Result class.

setStartingPoint(startingPoint)¶

Accessor to starting point.

Parameters:

startingPointPoint: Starting point.

setStartingSample(startingSample)¶

Accessor to starting sample.

Parameters:

startingSampleSample: Starting sample.

setStopCallback(*args)¶

Set up a stop callback.

Can be used to programmatically stop an optimization.

Parameters:

callbackcallable: Returns an int deciding whether to stop or continue.

Examples

>>> import openturns as ot
>>> rosenbrock = ot.SymbolicFunction(['x1', 'x2'], ['(1-x1)^2+100*(x2-x1^2)^2'])
>>> problem = ot.OptimizationProblem(rosenbrock)
>>> solver = ot.OptimizationAlgorithm(problem)
>>> solver.setStartingPoint([0, 0])
>>> solver.setMaximumResidualError(1.e-3)
>>> solver.setMaximumCallsNumber(10000)
>>> def ask_stop():
...     return True
>>> solver.setStopCallback(ask_stop)
>>> solver.run()

Examples using the class¶

EfficientGlobalOptimization examples

OpenTURNS

An Open source initiative for the Treatment of Uncertainties, Risks'N Statistics

Table of Contents

Previous topic

Next topic

This Page

EfficientGlobalOptimization¶

Examples using the class¶