.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_numerical_methods/optimization/plot_optimization_rosenbrock.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_numerical_methods_optimization_plot_optimization_rosenbrock.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_numerical_methods_optimization_plot_optimization_rosenbrock.py:


Quick start guide to optimization
=================================

.. GENERATED FROM PYTHON SOURCE LINES 7-27

In this example, we perform the optimization of the Rosenbrock test function.

Let :math:`a, b\in\mathbb{R}` be parameters. The Rosenbrock function is defined by

.. math::
   f(x_1, x_2) = (a-x_1)^2 + b(x_2 - x_1^2)^2


for any :math:`\mathbf{x}\in\mathbb{R}^2`.
This function is often used with :math:`a=1` and :math:`b=100`. In this case, the function has a single global minimum at:

.. math::
   \mathbf{x}^\star = (1, 1)^T.


This function has a nonlinear least squares structure.

*References*

* Rosenbrock, H.H. (1960). "An automatic method for finding the greatest or least value of a function". The Computer Journal. 3 (3): 175–184.

.. GENERATED FROM PYTHON SOURCE LINES 30-32

Definition of the function
--------------------------

.. GENERATED FROM PYTHON SOURCE LINES 34-38

.. code-block:: Python

    import openturns as ot
    import openturns.viewer as otv









.. GENERATED FROM PYTHON SOURCE LINES 39-41

.. code-block:: Python

    rosenbrock = ot.SymbolicFunction(["x1", "x2"], ["(1-x1)^2+100*(x2-x1^2)^2"])








.. GENERATED FROM PYTHON SOURCE LINES 42-44

.. code-block:: Python

    x0 = [-1.0, 1.0]








.. GENERATED FROM PYTHON SOURCE LINES 45-47

.. code-block:: Python

    xexact = ot.Point([1.0, 1.0])








.. GENERATED FROM PYTHON SOURCE LINES 48-51

.. code-block:: Python

    lowerbound = [-2.0, -2.0]
    upperbound = [2.0, 2.0]








.. GENERATED FROM PYTHON SOURCE LINES 52-54

Plot the iso-values of the objective function
---------------------------------------------

.. GENERATED FROM PYTHON SOURCE LINES 56-58

.. code-block:: Python

    rosenbrock = ot.MemoizeFunction(rosenbrock)








.. GENERATED FROM PYTHON SOURCE LINES 59-63

.. code-block:: Python

    graph = rosenbrock.draw(lowerbound, upperbound, [100] * 2)
    graph.setTitle("Rosenbrock function")
    view = otv.View(graph)




.. image-sg:: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_001.svg
   :alt: Rosenbrock function
   :srcset: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_001.svg
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 64-66

We see that the minimum is on the top right of the picture and the starting point is on the top left of the picture.
Since the function has a long valley following the curve :math:`x_2 - x^2=0`, the algorithm generally have to follow the bottom of the valley.

.. GENERATED FROM PYTHON SOURCE LINES 68-70

Create and solve the optimization problem
-----------------------------------------

.. GENERATED FROM PYTHON SOURCE LINES 72-74

.. code-block:: Python

    problem = ot.OptimizationProblem(rosenbrock)








.. GENERATED FROM PYTHON SOURCE LINES 75-81

.. code-block:: Python

    algo = ot.Cobyla(problem)
    algo.setMaximumRelativeError(1.0e-1)  # on x
    algo.setMaximumCallsNumber(50000)
    algo.setStartingPoint(x0)
    algo.run()








.. GENERATED FROM PYTHON SOURCE LINES 82-84

.. code-block:: Python

    result = algo.getResult()








.. GENERATED FROM PYTHON SOURCE LINES 85-88

.. code-block:: Python

    xoptim = result.getOptimalPoint()
    xoptim






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    class=Point name=Unnamed dimension=2 values=[0.992501,0.985019]
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 89-93

.. code-block:: Python

    delta = xexact - xoptim
    absoluteError = delta.norm()
    absoluteError





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    0.016752861417341336



.. GENERATED FROM PYTHON SOURCE LINES 94-95

We see that the algorithm found an accurate approximation of the solution.

.. GENERATED FROM PYTHON SOURCE LINES 97-99

.. code-block:: Python

    result.getOptimalValue()  # f(x*)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    class=Point name=Unnamed dimension=1 values=[5.63863e-05]
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 100-102

.. code-block:: Python

    result.getCallsNumber()





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    10520



.. GENERATED FROM PYTHON SOURCE LINES 103-111

.. code-block:: Python

    graph = rosenbrock.draw(lowerbound, upperbound, [100] * 2)
    cloud = ot.Cloud(ot.Sample([x0, xoptim]))
    cloud.setColor("black")
    cloud.setPointStyle("bullet")
    graph.add(cloud)
    graph.setTitle("Rosenbrock function")
    view = otv.View(graph)




.. image-sg:: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_002.svg
   :alt: Rosenbrock function
   :srcset: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_002.svg
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 112-113

We see that the algorithm had to start from the top left of the banana and go to the top right.

.. GENERATED FROM PYTHON SOURCE LINES 115-118

.. code-block:: Python

    graph = result.drawOptimalValueHistory()
    view = otv.View(graph)




.. image-sg:: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_003.svg
   :alt: Optimal value history
   :srcset: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_003.svg
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 119-123

The function value history make the path of the algorithm clear.
In the first step, the algorithm went in the valley, which made the function value decrease rapidly.
Once there, the algorithm had to follow the bottom of the valley so that the function decreased but slowly.
In the final steps, the algorithm found the neighbourhood of the minimum so that the local convergence could take place.

.. GENERATED FROM PYTHON SOURCE LINES 125-126

In order to see where the function was evaluated, we use the `getInputSample` method.

.. GENERATED FROM PYTHON SOURCE LINES 128-130

.. code-block:: Python

    inputSample = result.getInputSample()








.. GENERATED FROM PYTHON SOURCE LINES 131-137

.. code-block:: Python

    graph = rosenbrock.draw(lowerbound, upperbound, [100] * 2)
    graph.setTitle("Rosenbrock function solved with Cobyla")
    cloud = ot.Cloud(inputSample)
    graph.add(cloud)
    view = otv.View(graph)




.. image-sg:: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_004.svg
   :alt: Rosenbrock function solved with Cobyla
   :srcset: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_004.svg
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 138-139

We see that the algorithm made lots of evaluations in the bottom of the valley before getting in the neighbourhood of the minimum.

.. GENERATED FROM PYTHON SOURCE LINES 141-148

Solving the problem with NLopt
------------------------------

We see that the `Cobyla` algorithm required lots of function evaluations.
This is why we now use the `NLopt` class with the LBFGS algorithm.
However, the algorithm may use input points which are far away from the input domain we used so far.
This is why we had bounds to the problem, so that the algorithm never goes to far away from the valley.

.. GENERATED FROM PYTHON SOURCE LINES 150-152

.. code-block:: Python

    bounds = ot.Interval(lowerbound, upperbound)








.. GENERATED FROM PYTHON SOURCE LINES 153-156

.. code-block:: Python

    problem = ot.OptimizationProblem(rosenbrock)
    problem.setBounds(bounds)








.. GENERATED FROM PYTHON SOURCE LINES 157-161

.. code-block:: Python

    algo = ot.NLopt(problem, "LD_LBFGS")
    algo.setStartingPoint(x0)
    algo.run()








.. GENERATED FROM PYTHON SOURCE LINES 162-164

.. code-block:: Python

    result = algo.getResult()








.. GENERATED FROM PYTHON SOURCE LINES 165-168

.. code-block:: Python

    xoptim = result.getOptimalPoint()
    xoptim






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    class=Point name=Unnamed dimension=2 values=[1,1]
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 169-173

.. code-block:: Python

    delta = xexact - xoptim
    absoluteError = delta.norm()
    absoluteError





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    7.740583643426769e-12



.. GENERATED FROM PYTHON SOURCE LINES 174-175

We see that the algorithm found an extremely accurate approximation of the solution.

.. GENERATED FROM PYTHON SOURCE LINES 177-179

.. code-block:: Python

    result.getOptimalValue()  # f(x*)






.. raw:: html

    <div class="output_subarea output_html rendered_html output_result">
    class=Point name=Unnamed dimension=1 values=[1.77616e-23]
    </div>
    <br />
    <br />

.. GENERATED FROM PYTHON SOURCE LINES 180-182

.. code-block:: Python

    result.getCallsNumber()





.. rst-class:: sphx-glr-script-out

 .. code-block:: none


    44



.. GENERATED FROM PYTHON SOURCE LINES 183-184

This number of iterations is much less than the previous experiment.

.. GENERATED FROM PYTHON SOURCE LINES 186-188

.. code-block:: Python

    inputSample = result.getInputSample()








.. GENERATED FROM PYTHON SOURCE LINES 189-195

.. code-block:: Python

    graph = rosenbrock.draw(lowerbound, upperbound, [100] * 2)
    graph.setTitle("Rosenbrock function solved with NLopt/LD_LBFGS")
    cloud = ot.Cloud(inputSample)
    graph.add(cloud)
    view = otv.View(graph)




.. image-sg:: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_005.svg
   :alt: Rosenbrock function solved with NLopt/LD_LBFGS
   :srcset: /auto_numerical_methods/optimization/images/sphx_glr_plot_optimization_rosenbrock_005.svg
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 196-197

Display all figures

.. GENERATED FROM PYTHON SOURCE LINES 197-198

.. code-block:: Python

    otv.View.ShowAll()








.. _sphx_glr_download_auto_numerical_methods_optimization_plot_optimization_rosenbrock.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_optimization_rosenbrock.ipynb <plot_optimization_rosenbrock.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_optimization_rosenbrock.py <plot_optimization_rosenbrock.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_optimization_rosenbrock.zip <plot_optimization_rosenbrock.zip>`