.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_numerical_methods/optimization/plot_optimization_dlib.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_numerical_methods_optimization_plot_optimization_dlib.py: Optimization using dlib ======================= .. GENERATED FROM PYTHON SOURCE LINES 6-7 In this example we are going to explore optimization using OpenTURNS' `dlib `_ interface. .. GENERATED FROM PYTHON SOURCE LINES 9-16 .. code-block:: Python import numpy as np import openturns as ot import openturns.viewer as viewer from matplotlib import pylab as plt ot.Log.Show(ot.Log.NONE) .. GENERATED FROM PYTHON SOURCE LINES 17-18 List available algorithms .. GENERATED FROM PYTHON SOURCE LINES 18-21 .. code-block:: Python for algo in ot.Dlib.GetAlgorithmNames(): print(algo) .. rst-class:: sphx-glr-script-out .. code-block:: none cg bfgs lbfgs newton global least_squares least_squares_lm trust_region .. GENERATED FROM PYTHON SOURCE LINES 22-23 More details on dlib algorithms are available `here `_ . .. GENERATED FROM PYTHON SOURCE LINES 25-30 Solving an unconstrained problem with conjugate gradient algorithm ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The following example will demonstrate the use of dlib conjugate gradient algorithm to find the minimum of `Rosenbrock function `_. The optimal point can be computed analytically, and its value is [1.0, 1.0]. .. GENERATED FROM PYTHON SOURCE LINES 32-33 Define the problem based on Rosebrock function .. GENERATED FROM PYTHON SOURCE LINES 33-36 .. code-block:: Python rosenbrock = ot.SymbolicFunction(["x1", "x2"], ["(1-x1)^2+(x2-x1^2)^2"]) problem = ot.OptimizationProblem(rosenbrock) .. GENERATED FROM PYTHON SOURCE LINES 37-38 The optimization algorithm is instantiated from the problem to solve and the name of the algorithm .. GENERATED FROM PYTHON SOURCE LINES 38-47 .. code-block:: Python algo = ot.Dlib(problem, "cg") print("Dlib algorithm, type ", algo.getAlgorithmName()) print("Maximum iteration number: ", algo.getMaximumIterationNumber()) print("Maximum evaluation number: ", algo.getMaximumEvaluationNumber()) print("Maximum absolute error: ", algo.getMaximumAbsoluteError()) print("Maximum relative error: ", algo.getMaximumRelativeError()) print("Maximum residual error: ", algo.getMaximumResidualError()) print("Maximum constraint error: ", algo.getMaximumConstraintError()) .. rst-class:: sphx-glr-script-out .. code-block:: none Dlib algorithm, type cg Maximum iteration number: 100 Maximum evaluation number: 1000 Maximum absolute error: 1e-05 Maximum relative error: 1e-05 Maximum residual error: 1e-05 Maximum constraint error: 1e-05 .. GENERATED FROM PYTHON SOURCE LINES 48-52 When using conjugate gradient, BFGS/LBFGS, Newton, least squares or trust region methods, optimization proceeds until one of the following criteria is met: - the errors (absolute, relative, residual, constraint) are all below the limits set by the user ; - the process reaches the maximum number of iterations or function evaluations. .. GENERATED FROM PYTHON SOURCE LINES 54-55 Adjust number of iterations/evaluations .. GENERATED FROM PYTHON SOURCE LINES 55-61 .. code-block:: Python algo.setMaximumIterationNumber(1000) algo.setMaximumEvaluationNumber(10000) algo.setMaximumAbsoluteError(1e-3) algo.setMaximumRelativeError(1e-3) algo.setMaximumResidualError(1e-3) .. GENERATED FROM PYTHON SOURCE LINES 62-63 Solve the problem .. GENERATED FROM PYTHON SOURCE LINES 63-68 .. code-block:: Python startingPoint = [1.5, 0.5] algo.setStartingPoint(startingPoint) algo.run() .. GENERATED FROM PYTHON SOURCE LINES 69-70 Retrieve results .. GENERATED FROM PYTHON SOURCE LINES 70-80 .. code-block:: Python result = algo.getResult() print("x^ = ", result.getOptimalPoint()) print("f(x^) = ", result.getOptimalValue()) print("Iteration number: ", result.getIterationNumber()) print("Evaluation number: ", result.getEvaluationNumber()) print("Absolute error: ", result.getAbsoluteError()) print("Relative error: ", result.getRelativeError()) print("Residual error: ", result.getResidualError()) print("Constraint error: ", result.getConstraintError()) .. rst-class:: sphx-glr-script-out .. code-block:: none x^ = [0.995311,0.989195] f(x^) = [2.4084e-05] Iteration number: 41 Evaluation number: 85 Absolute error: 0.0009776096028751445 Relative error: 0.0006966679389276845 Residual error: 4.302851151659242e-06 Constraint error: 0.0 .. GENERATED FROM PYTHON SOURCE LINES 81-87 Solving problem with bounds, using LBFGS strategy ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In the following example, the input variables will be bounded so the function global optimal point is not included in the search interval. The problem will be solved using LBFGS strategy, which allows the user to limit the amount of memory used by the optimization process. .. GENERATED FROM PYTHON SOURCE LINES 89-90 Define the bounds and the problem .. GENERATED FROM PYTHON SOURCE LINES 90-95 .. code-block:: Python bounds = ot.Interval([0.0, 0.0], [0.8, 2.0]) boundedProblem = ot.OptimizationProblem( rosenbrock, ot.Function(), ot.Function(), bounds ) .. GENERATED FROM PYTHON SOURCE LINES 96-97 Define the Dlib algorithm .. GENERATED FROM PYTHON SOURCE LINES 97-105 .. code-block:: Python boundedAlgo = ot.Dlib(boundedProblem, "lbfgs") boundedAlgo.setMaxSize(15) # Default value for LBFGS' maxSize parameter is 10 startingPoint = [0.5, 1.5] boundedAlgo.setStartingPoint(startingPoint) boundedAlgo.run() .. GENERATED FROM PYTHON SOURCE LINES 106-107 Retrieve results .. GENERATED FROM PYTHON SOURCE LINES 107-117 .. code-block:: Python result = boundedAlgo.getResult() print("x^ = ", result.getOptimalPoint()) print("f(x^) = ", result.getOptimalValue()) print("Iteration number: ", result.getIterationNumber()) print("Evaluation number: ", result.getEvaluationNumber()) print("Absolute error: ", result.getAbsoluteError()) print("Relative error: ", result.getRelativeError()) print("Residual error: ", result.getResidualError()) print("Constraint error: ", result.getConstraintError()) .. rst-class:: sphx-glr-script-out .. code-block:: none x^ = [0.8,0.64] f(x^) = [0.04] Iteration number: 6 Evaluation number: 10 Absolute error: 0.03393533485971123 Relative error: 0.03312380566244159 Residual error: 0.0011516069520407304 Constraint error: 0.0 .. GENERATED FROM PYTHON SOURCE LINES 118-120 **Remark:** The bounds defined for input variables are always strictly respected when using dlib algorithms. Consequently, the constraint error is always 0. .. GENERATED FROM PYTHON SOURCE LINES 122-123 Draw optimal value history .. GENERATED FROM PYTHON SOURCE LINES 123-126 .. code-block:: Python graph = result.drawOptimalValueHistory() view = viewer.View(graph) .. image-sg:: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_dlib_001.png :alt: Optimal value history :srcset: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_dlib_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 127-134 Solving least squares problem ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In least squares problem, the user provides the residual function to minimize. Here the underlying OptimizationProblem is defined as a LeastSquaresProblem. dlib least squares algorithms use the same stop criteria as CG, BFGS/LBFGS and Newton algorithms. However, optimization will stop earlier if no significant improvement can be achieved during the process. .. GENERATED FROM PYTHON SOURCE LINES 136-137 Define residual function .. GENERATED FROM PYTHON SOURCE LINES 137-154 .. code-block:: Python n = 3 m = 20 x = [[0.5 + 0.1 * i] for i in range(m)] model = ot.SymbolicFunction(["a", "b", "c", "x"], ["a + b * exp(-c *x^2)"]) p_ref = [2.8, 1.2, 0.5] # Reference a, b, c modelx = ot.ParametricFunction(model, [0, 1, 2], p_ref) # Generate reference sample (with normal noise) y = np.multiply(modelx(x), np.random.normal(1.0, 0.05, m)) def residualFunction(params): modelx = ot.ParametricFunction(model, [0, 1, 2], params) return [modelx(x[i])[0] - y[i, 0] for i in range(m)] .. GENERATED FROM PYTHON SOURCE LINES 155-156 Definition of residual as ot.PythonFunction and optimization problem .. GENERATED FROM PYTHON SOURCE LINES 156-159 .. code-block:: Python residual = ot.PythonFunction(n, m, residualFunction) lsqProblem = ot.LeastSquaresProblem(residual) .. GENERATED FROM PYTHON SOURCE LINES 160-161 Definition of Dlib solver, setting starting point .. GENERATED FROM PYTHON SOURCE LINES 161-166 .. code-block:: Python lsqAlgo = ot.Dlib(lsqProblem, "least_squares") lsqAlgo.setStartingPoint([0.0, 0.0, 0.0]) lsqAlgo.run() .. GENERATED FROM PYTHON SOURCE LINES 167-168 Retrieve results .. GENERATED FROM PYTHON SOURCE LINES 168-178 .. code-block:: Python result = lsqAlgo.getResult() print("x^ = ", result.getOptimalPoint()) print("f(x^) = ", result.getOptimalValue()) print("Iteration number: ", result.getIterationNumber()) print("Evaluation number: ", result.getEvaluationNumber()) print("Absolute error: ", result.getAbsoluteError()) print("Relative error: ", result.getRelativeError()) print("Residual error: ", result.getResidualError()) print("Constraint error: ", result.getConstraintError()) .. rst-class:: sphx-glr-script-out .. code-block:: none x^ = [2.58248,1.10678,0.5] f(x^) = [5.91646e-31] Iteration number: 7 Evaluation number: 9 Absolute error: 8.380205055375393e-09 Relative error: 2.9365075790899105e-09 Residual error: 2.409849517864279e-17 Constraint error: 0.0 .. GENERATED FROM PYTHON SOURCE LINES 179-180 Draw errors history .. GENERATED FROM PYTHON SOURCE LINES 180-183 .. code-block:: Python graph = result.drawErrorHistory() view = viewer.View(graph) .. image-sg:: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_dlib_002.png :alt: Error history :srcset: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_dlib_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 184-185 Draw optimal value history .. GENERATED FROM PYTHON SOURCE LINES 185-189 .. code-block:: Python graph = result.drawOptimalValueHistory() view = viewer.View(graph) plt.show() .. image-sg:: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_dlib_003.png :alt: Optimal value history :srcset: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_dlib_003.png :class: sphx-glr-single-img .. _sphx_glr_download_auto_numerical_methods_optimization_plot_optimization_dlib.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_optimization_dlib.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_optimization_dlib.py `