.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_data_analysis/sample_analysis/plot_draw_survival.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_data_analysis_sample_analysis_plot_draw_survival.py: Draw a survival function ======================== .. GENERATED FROM PYTHON SOURCE LINES 5-6 .. code-block:: default # sphinx_gallery_thumbnail_number = 9 .. GENERATED FROM PYTHON SOURCE LINES 7-63 Introduction ------------ The goal of this example is to show how to draw the survival function of a sample or a distribution, in linear and logarithmic scales. Let :math:`X` be a random variable with distribution function :math:`F`: .. math:: F(x) = P(X\leq x) for any :math:`x\in\mathbb{R}`. The survival function :math:`S` is: .. math:: S(x) = P(X>x) = 1 - P(X\leq x) = 1 - F(x) for any :math:`x\in\mathbb{R}`. Let us assume that :math:`\{x_1,...,x_N\}` is a sample from :math:`F`. Let :math:`\hat{F}_N` be the empirical cumulative distribution function: .. math:: \hat{F}_N(x) = \frac{1}{N} \sum_{i=1}^N \mathbf{1}_{x_i\leq x} for any :math:`x\in\mathbb{R}`. Let :math:`\hat{S}_n` be the empirical survival function: .. math:: \hat{S}_N(x) = \frac{1}{N} \sum_{i=1}^N \mathbf{1}_{x_i>x} for any :math:`x\in\mathbb{R}`. Motivations for the survival function ------------------------------------- For many probabilistic models associated with extreme events or lifetime models, the survival function has a simpler expression than the distribution function. * More specifically, several models (e.g. Pareto or Weibull) have a simple expression when we consider the logarithm of the survival function. In this situation, the :math:`(\log(x),\log(S(x)))` plot is often used. For some distributions, this plot is a straight line. * When we consider probabilities very close to 1 (e.g. with extreme events), a loss of precision can occur when we consider the :math:`1-F(x)` expression with floating point numbers. This loss of significant digits is known as "catastrophic cancellation" in the bibliography and happens when two close floating point numbers are subtracted. This is one of the reasons why we sometimes use directly the survival function instead of the complementary of the distribution. .. GENERATED FROM PYTHON SOURCE LINES 66-68 Define a distribution --------------------- .. GENERATED FROM PYTHON SOURCE LINES 70-76 .. code-block:: default import openturns as ot import openturns.viewer as viewer from matplotlib import pylab as plt ot.Log.Show(ot.Log.NONE) .. GENERATED FROM PYTHON SOURCE LINES 77-82 .. code-block:: default sigma = 1.4 xi = 0.5 u = 0.1 distribution = ot.GeneralizedPareto(sigma, xi, u) .. GENERATED FROM PYTHON SOURCE LINES 83-85 Draw the survival of a distribution ----------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 87-88 The `computeCDF` and `computeSurvivalFunction` computes the CDF :math:`F` and survival :math:`S` of a distribution. .. GENERATED FROM PYTHON SOURCE LINES 90-93 .. code-block:: default p1 = distribution.computeCDF(10.0) p1 .. rst-class:: sphx-glr-script-out .. code-block:: none 0.9513919027838056 .. GENERATED FROM PYTHON SOURCE LINES 94-97 .. code-block:: default p2 = distribution.computeSurvivalFunction(10.0) p2 .. rst-class:: sphx-glr-script-out .. code-block:: none 0.048608097216194426 .. GENERATED FROM PYTHON SOURCE LINES 98-100 .. code-block:: default p1 + p2 .. rst-class:: sphx-glr-script-out .. code-block:: none 1.0 .. GENERATED FROM PYTHON SOURCE LINES 101-102 The `drawCDF` and `drawSurvivalFunction` methods allows one to draw the functions :math:`F` and :math:`S`. .. GENERATED FROM PYTHON SOURCE LINES 104-108 .. code-block:: default graph = distribution.drawCDF() graph.setTitle("CDF of a distribution") view = viewer.View(graph) .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_001.png :alt: CDF of a distribution :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 109-113 .. code-block:: default graph = distribution.drawSurvivalFunction() graph.setTitle("Survival function of a distribution") view = viewer.View(graph) .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_002.png :alt: Survival function of a distribution :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 114-116 In order to get finite bounds for the next graphics, we compute the `xmin` and `xmax` bounds from the 0.01 and 0.99 quantiles of the distributions. .. GENERATED FROM PYTHON SOURCE LINES 118-121 .. code-block:: default xmin = distribution.computeQuantile(0.01)[0] xmin .. rst-class:: sphx-glr-script-out .. code-block:: none 0.11410588272579382 .. GENERATED FROM PYTHON SOURCE LINES 122-125 .. code-block:: default xmax = distribution.computeQuantile(0.99)[0] xmax .. rst-class:: sphx-glr-script-out .. code-block:: none 25.29999999999998 .. GENERATED FROM PYTHON SOURCE LINES 126-127 The `drawSurvivalFunction` methods also has an option to plot the survival with the X axis in logarithmic scale. .. GENERATED FROM PYTHON SOURCE LINES 129-136 .. code-block:: default npoints = 50 logScaleX = True graph = distribution.drawSurvivalFunction(xmin, xmax, npoints, logScaleX) graph.setTitle("Survival function of a distribution where X axis is in log scale") view = viewer.View(graph) # graph .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_003.png :alt: Survival function of a distribution where X axis is in log scale :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_003.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 137-138 In order to get both axes in logarithmic scale, we use the `LOGXY` option of the graph. .. GENERATED FROM PYTHON SOURCE LINES 140-150 .. code-block:: default npoints = 50 logScaleX = True graph = distribution.drawSurvivalFunction(xmin, xmax, npoints, logScaleX) graph.setLogScale(ot.GraphImplementation.LOGXY) graph.setTitle( "Survival function of a distribution where X and Y axes are in log scale" ) view = viewer.View(graph) # graph .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_004.png :alt: Survival function of a distribution where X and Y axes are in log scale :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_004.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 151-153 Compute the survival of a sample -------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 155-156 We now generate a sample that we are going to analyze. .. GENERATED FROM PYTHON SOURCE LINES 158-160 .. code-block:: default sample = distribution.getSample(1000) .. GENERATED FROM PYTHON SOURCE LINES 161-163 .. code-block:: default sample.getMin(), sample.getMax() .. rst-class:: sphx-glr-script-out .. code-block:: none (class=Point name=Unnamed dimension=1 values=[0.101087], class=Point name=Unnamed dimension=1 values=[198.743]) .. GENERATED FROM PYTHON SOURCE LINES 164-165 The `computeEmpiricalCDF` method of a `Sample` computes the empirical CDF. .. GENERATED FROM PYTHON SOURCE LINES 167-170 .. code-block:: default p1 = sample.computeEmpiricalCDF([10]) p1 .. rst-class:: sphx-glr-script-out .. code-block:: none 0.949 .. GENERATED FROM PYTHON SOURCE LINES 171-172 Activating the second optional argument allows one to compute the empirical survival function. .. GENERATED FROM PYTHON SOURCE LINES 174-177 .. code-block:: default p2 = sample.computeEmpiricalCDF([10], True) p2 .. rst-class:: sphx-glr-script-out .. code-block:: none 0.051 .. GENERATED FROM PYTHON SOURCE LINES 178-180 .. code-block:: default p1 + p2 .. rst-class:: sphx-glr-script-out .. code-block:: none 1.0 .. GENERATED FROM PYTHON SOURCE LINES 181-183 Draw the survival of a sample ----------------------------- .. GENERATED FROM PYTHON SOURCE LINES 185-189 In order to draw the empirical functions of a `Sample`, we use the `UserDefined` class. * The `drawCDF` method plots the CDF. * The `drawSurvivalFunction` method plots the survival function. .. GENERATED FROM PYTHON SOURCE LINES 191-197 .. code-block:: default userdefined = ot.UserDefined(sample) graph = userdefined.drawCDF() graph.setTitle("CDF of a sample") view = viewer.View(graph) # graph .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_005.png :alt: CDF of a sample :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_005.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 198-203 .. code-block:: default graph = userdefined.drawSurvivalFunction() graph.setTitle("Empirical survival function of a sample") view = viewer.View(graph) # graph .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_006.png :alt: Empirical survival function of a sample :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_006.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 204-205 As previously, the `drawSurvivalFunction` method of a distribution has an option to set the X axis in logarithmic scale. .. GENERATED FROM PYTHON SOURCE LINES 207-216 .. code-block:: default xmin = sample.getMin()[0] xmax = sample.getMax()[0] pointNumber = sample.getSize() logScaleX = True graph = userdefined.drawSurvivalFunction(xmin, xmax, pointNumber, logScaleX) graph.setTitle("Empirical survival function of a sample; X axis in log-scale") view = viewer.View(graph) # graph .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_007.png :alt: Empirical survival function of a sample; X axis in log-scale :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_007.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 217-220 We obviously have :math:`P(X>X_{max})=0`, where :math:`X_{max}` is the sample maximum. This prevents from using the sample maximum and have a logarithmic Y axis at the same time. This is why in the following example we restrict the interval where we draw the survival function. .. GENERATED FROM PYTHON SOURCE LINES 222-232 .. code-block:: default xmin = sample.getMin()[0] xmax = sample.getMax()[0] - 1 # To avoid log(0) because P(X>Xmax)=0 pointNumber = sample.getSize() logScaleX = True graph = userdefined.drawSurvivalFunction(xmin, xmax, pointNumber, logScaleX) graph.setLogScale(ot.GraphImplementation.LOGXY) graph.setTitle("Empirical survival function of a sample; X and Y axes in log-scale") view = viewer.View(graph) # graph .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_008.png :alt: Empirical survival function of a sample; X and Y axes in log-scale :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_008.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 233-235 Compare the distribution and the sample with respect to the survival -------------------------------------------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 237-238 In the final example, we compare the distribution and sample survival functions in the same graphics. .. GENERATED FROM PYTHON SOURCE LINES 240-256 .. code-block:: default xmin = sample.getMin()[0] xmax = sample.getMax()[0] - 1 # To avoid log(0) because P(X>Xmax)=0 npoints = 50 logScaleX = True graph = userdefined.drawSurvivalFunction(xmin, xmax, pointNumber, logScaleX) graph.setLogScale(ot.GraphImplementation.LOGXY) graph.setColors(["blue"]) graph.setLegends(["Sample"]) graphDistribution = distribution.drawSurvivalFunction(xmin, xmax, npoints, logScaleX) graphDistribution.setLegends(["GPD"]) graph.add(graphDistribution) graph.setLegendPosition("topright") graph.setTitle("GPD against the sample - n=%d" % (sample.getSize())) view = viewer.View(graph) # graph plt.show() .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_009.png :alt: GPD against the sample - n=1000 :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_009.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 1.170 seconds) .. _sphx_glr_download_auto_data_analysis_sample_analysis_plot_draw_survival.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_draw_survival.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_draw_survival.ipynb `