.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_numerical_methods/optimization/plot_optimization_rosenbrock.py"
.. LINE NUMBERS ARE GIVEN BELOW.
.. only:: html
.. note::
:class: sphx-glr-download-link-note
:ref:`Go to the end <sphx_glr_download_auto_numerical_methods_optimization_plot_optimization_rosenbrock.py>`
to download the full example code
.. rst-class:: sphx-glr-example-title
.. _sphx_glr_auto_numerical_methods_optimization_plot_optimization_rosenbrock.py:
Quick start guide to optimization
=================================
.. GENERATED FROM PYTHON SOURCE LINES 6-26
In this example, we perform the optimization of the Rosenbrock test function.
Let :math:`a, b\in\mathbb{R}` be parameters. The Rosenbrock function is defined by
.. math::
f(x_1, x_2) = (a-x_1)^2 + b(x_2 - x_1^2)^2
for any :math:`\mathbf{x}\in\mathbb{R}^2`.
This function is often used with :math:`a=1` and :math:`b=100`. In this case, the function has a single global minimum at:
.. math::
\mathbf{x}^\star = (1, 1)^T.
This function has a nonlinear least squares structure.
*References*
* Rosenbrock, H.H. (1960). "An automatic method for finding the greatest or least value of a function". The Computer Journal. 3 (3): 175–184.
.. GENERATED FROM PYTHON SOURCE LINES 29-31
Definition of the function
--------------------------
.. GENERATED FROM PYTHON SOURCE LINES 33-39
.. code-block:: Python
import openturns as ot
import openturns.viewer as viewer
from matplotlib import pylab as plt
ot.Log.Show(ot.Log.NONE)
.. GENERATED FROM PYTHON SOURCE LINES 40-42
.. code-block:: Python
rosenbrock = ot.SymbolicFunction(["x1", "x2"], ["(1-x1)^2+100*(x2-x1^2)^2"])
.. GENERATED FROM PYTHON SOURCE LINES 43-45
.. code-block:: Python
x0 = [-1.0, 1.0]
.. GENERATED FROM PYTHON SOURCE LINES 46-48
.. code-block:: Python
xexact = ot.Point([1.0, 1.0])
.. GENERATED FROM PYTHON SOURCE LINES 49-52
.. code-block:: Python
lowerbound = [-2.0, -2.0]
upperbound = [2.0, 2.0]
.. GENERATED FROM PYTHON SOURCE LINES 53-55
Plot the iso-values of the objective function
---------------------------------------------
.. GENERATED FROM PYTHON SOURCE LINES 57-59
.. code-block:: Python
rosenbrock = ot.MemoizeFunction(rosenbrock)
.. GENERATED FROM PYTHON SOURCE LINES 60-64
.. code-block:: Python
graph = rosenbrock.draw(lowerbound, upperbound, [100] * 2)
graph.setTitle("Rosenbrock function")
view = viewer.View(graph)
.. image-sg:: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_001.png
:alt: Rosenbrock function
:srcset: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_001.png
:class: sphx-glr-single-img
.. GENERATED FROM PYTHON SOURCE LINES 65-67
We see that the minimum is on the top right of the picture and the starting point is on the top left of the picture.
Since the function has a long valley following the curve :math:`x_2 - x^2=0`, the algorithm generally have to follow the bottom of the valley.
.. GENERATED FROM PYTHON SOURCE LINES 69-71
Create and solve the optimization problem
-----------------------------------------
.. GENERATED FROM PYTHON SOURCE LINES 73-75
.. code-block:: Python
problem = ot.OptimizationProblem(rosenbrock)
.. GENERATED FROM PYTHON SOURCE LINES 76-82
.. code-block:: Python
algo = ot.Cobyla(problem)
algo.setMaximumRelativeError(1.0e-1) # on x
algo.setMaximumEvaluationNumber(50000)
algo.setStartingPoint(x0)
algo.run()
.. GENERATED FROM PYTHON SOURCE LINES 83-85
.. code-block:: Python
result = algo.getResult()
.. GENERATED FROM PYTHON SOURCE LINES 86-89
.. code-block:: Python
xoptim = result.getOptimalPoint()
xoptim
.. raw:: html
<div class="output_subarea output_html rendered_html output_result">
<p>[0.992501,0.985019]</p>
</div>
<br />
<br />
.. GENERATED FROM PYTHON SOURCE LINES 90-94
.. code-block:: Python
delta = xexact - xoptim
absoluteError = delta.norm()
absoluteError
.. rst-class:: sphx-glr-script-out
.. code-block:: none
0.016752861417341336
.. GENERATED FROM PYTHON SOURCE LINES 95-96
We see that the algorithm found an accurate approximation of the solution.
.. GENERATED FROM PYTHON SOURCE LINES 98-100
.. code-block:: Python
result.getOptimalValue() # f(x*)
.. raw:: html
<div class="output_subarea output_html rendered_html output_result">
<p>[5.63863e-05]</p>
</div>
<br />
<br />
.. GENERATED FROM PYTHON SOURCE LINES 101-103
.. code-block:: Python
result.getEvaluationNumber()
.. rst-class:: sphx-glr-script-out
.. code-block:: none
10520
.. GENERATED FROM PYTHON SOURCE LINES 104-112
.. code-block:: Python
graph = rosenbrock.draw(lowerbound, upperbound, [100] * 2)
cloud = ot.Cloud(ot.Sample([x0, xoptim]))
cloud.setColor("black")
cloud.setPointStyle("bullet")
graph.add(cloud)
graph.setTitle("Rosenbrock function")
view = viewer.View(graph)
.. image-sg:: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_002.png
:alt: Rosenbrock function
:srcset: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_002.png
:class: sphx-glr-single-img
.. GENERATED FROM PYTHON SOURCE LINES 113-114
We see that the algorithm had to start from the top left of the banana and go to the top right.
.. GENERATED FROM PYTHON SOURCE LINES 116-119
.. code-block:: Python
graph = result.drawOptimalValueHistory()
view = viewer.View(graph)
.. image-sg:: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_003.png
:alt: Optimal value history
:srcset: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_003.png
:class: sphx-glr-single-img
.. GENERATED FROM PYTHON SOURCE LINES 120-124
The function value history make the path of the algorithm clear.
In the first step, the algorithm went in the valley, which made the function value decrease rapidly.
Once there, the algorithm had to follow the bottom of the valley so that the function decreased but slowly.
In the final steps, the algorithm found the neighbourhood of the minimum so that the local convergence could take place.
.. GENERATED FROM PYTHON SOURCE LINES 126-127
In order to see where the function was evaluated, we use the `getInputSample` method.
.. GENERATED FROM PYTHON SOURCE LINES 129-131
.. code-block:: Python
inputSample = result.getInputSample()
.. GENERATED FROM PYTHON SOURCE LINES 132-138
.. code-block:: Python
graph = rosenbrock.draw(lowerbound, upperbound, [100] * 2)
graph.setTitle("Rosenbrock function solved with Cobyla")
cloud = ot.Cloud(inputSample)
graph.add(cloud)
view = viewer.View(graph)
.. image-sg:: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_004.png
:alt: Rosenbrock function solved with Cobyla
:srcset: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_004.png
:class: sphx-glr-single-img
.. GENERATED FROM PYTHON SOURCE LINES 139-140
We see that the algorithm made lots of evaluations in the bottom of the valley before getting in the neighbourhood of the minimum.
.. GENERATED FROM PYTHON SOURCE LINES 142-149
Solving the problem with NLopt
------------------------------
We see that the `Cobyla` algorithm required lots of function evaluations.
This is why we now use the `NLopt` class with the LBFGS algorithm.
However, the algorithm may use input points which are far away from the input domain we used so far.
This is why we had bounds to the problem, so that the algorithm never goes to far away from the valley.
.. GENERATED FROM PYTHON SOURCE LINES 151-153
.. code-block:: Python
bounds = ot.Interval(lowerbound, upperbound)
.. GENERATED FROM PYTHON SOURCE LINES 154-157
.. code-block:: Python
problem = ot.OptimizationProblem(rosenbrock)
problem.setBounds(bounds)
.. GENERATED FROM PYTHON SOURCE LINES 158-162
.. code-block:: Python
algo = ot.NLopt(problem, "LD_LBFGS")
algo.setStartingPoint(x0)
algo.run()
.. GENERATED FROM PYTHON SOURCE LINES 163-165
.. code-block:: Python
result = algo.getResult()
.. GENERATED FROM PYTHON SOURCE LINES 166-169
.. code-block:: Python
xoptim = result.getOptimalPoint()
xoptim
.. raw:: html
<div class="output_subarea output_html rendered_html output_result">
<p>[1,1]</p>
</div>
<br />
<br />
.. GENERATED FROM PYTHON SOURCE LINES 170-174
.. code-block:: Python
delta = xexact - xoptim
absoluteError = delta.norm()
absoluteError
.. rst-class:: sphx-glr-script-out
.. code-block:: none
7.740583643426769e-12
.. GENERATED FROM PYTHON SOURCE LINES 175-176
We see that the algorithm found an extremely accurate approximation of the solution.
.. GENERATED FROM PYTHON SOURCE LINES 178-180
.. code-block:: Python
result.getOptimalValue() # f(x*)
.. raw:: html
<div class="output_subarea output_html rendered_html output_result">
<p>[1.77616e-23]</p>
</div>
<br />
<br />
.. GENERATED FROM PYTHON SOURCE LINES 181-183
.. code-block:: Python
result.getEvaluationNumber()
.. rst-class:: sphx-glr-script-out
.. code-block:: none
44
.. GENERATED FROM PYTHON SOURCE LINES 184-185
This number of iterations is much less than the previous experiment.
.. GENERATED FROM PYTHON SOURCE LINES 187-189
.. code-block:: Python
inputSample = result.getInputSample()
.. GENERATED FROM PYTHON SOURCE LINES 190-197
.. code-block:: Python
graph = rosenbrock.draw(lowerbound, upperbound, [100] * 2)
graph.setTitle("Rosenbrock function solved with NLopt/LD_LBFGS")
cloud = ot.Cloud(inputSample)
graph.add(cloud)
view = viewer.View(graph)
plt.show()
.. image-sg:: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_005.png
:alt: Rosenbrock function solved with NLopt/LD_LBFGS
:srcset: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_005.png
:class: sphx-glr-single-img
.. _sphx_glr_download_auto_numerical_methods_optimization_plot_optimization_rosenbrock.py:
.. only:: html
.. container:: sphx-glr-footer sphx-glr-footer-example
.. container:: sphx-glr-download sphx-glr-download-jupyter
:download:`Download Jupyter notebook: plot_optimization_rosenbrock.ipynb <plot_optimization_rosenbrock.ipynb>`
.. container:: sphx-glr-download sphx-glr-download-python
:download:`Download Python source code: plot_optimization_rosenbrock.py <plot_optimization_rosenbrock.py>`