.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_data_analysis/sample_analysis/plot_draw_survival.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_data_analysis_sample_analysis_plot_draw_survival.py: Draw a survival function ======================== .. GENERATED FROM PYTHON SOURCE LINES 5-6 .. code-block:: Python # sphinx_gallery_thumbnail_number = 9 .. GENERATED FROM PYTHON SOURCE LINES 7-63 Introduction ------------ The goal of this example is to show how to draw the survival function of a sample or a distribution, in linear and logarithmic scales. Let :math:`X` be a random variable with distribution function :math:`F`: .. math:: F(x) = P(X\leq x) for any :math:`x\in\mathbb{R}`. The survival function :math:`S` is: .. math:: S(x) = P(X>x) = 1 - P(X\leq x) = 1 - F(x) for any :math:`x\in\mathbb{R}`. Let us assume that :math:`\{x_1,...,x_N\}` is a sample from :math:`F`. Let :math:`\hat{F}_N` be the empirical cumulative distribution function: .. math:: \hat{F}_N(x) = \frac{1}{N} \sum_{i=1}^N \mathbf{1}_{x_i\leq x} for any :math:`x\in\mathbb{R}`. Let :math:`\hat{S}_n` be the empirical survival function: .. math:: \hat{S}_N(x) = \frac{1}{N} \sum_{i=1}^N \mathbf{1}_{x_i>x} for any :math:`x\in\mathbb{R}`. Motivations for the survival function ------------------------------------- For many probabilistic models associated with extreme events or lifetime models, the survival function has a simpler expression than the distribution function. * More specifically, several models (e.g. Pareto or Weibull) have a simple expression when we consider the logarithm of the survival function. In this situation, the :math:`(\log(x),\log(S(x)))` plot is often used. For some distributions, this plot is a straight line. * When we consider probabilities very close to 1 (e.g. with extreme events), a loss of precision can occur when we consider the :math:`1-F(x)` expression with floating point numbers. This loss of significant digits is known as "catastrophic cancellation" in the bibliography and happens when two close floating point numbers are subtracted. This is one of the reasons why we sometimes use directly the survival function instead of the complementary of the distribution. .. GENERATED FROM PYTHON SOURCE LINES 66-68 Define a distribution --------------------- .. GENERATED FROM PYTHON SOURCE LINES 70-76 .. code-block:: Python import openturns as ot import openturns.viewer as viewer from matplotlib import pylab as plt ot.Log.Show(ot.Log.NONE) .. GENERATED FROM PYTHON SOURCE LINES 77-82 .. code-block:: Python sigma = 1.4 xi = 0.5 u = 0.1 distribution = ot.GeneralizedPareto(sigma, xi, u) .. GENERATED FROM PYTHON SOURCE LINES 83-85 Draw the survival of a distribution ----------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 87-88 The `computeCDF` and `computeSurvivalFunction` computes the CDF :math:`F` and survival :math:`S` of a distribution. .. GENERATED FROM PYTHON SOURCE LINES 90-93 .. code-block:: Python p1 = distribution.computeCDF(10.0) p1 .. rst-class:: sphx-glr-script-out .. code-block:: none 0.9513919027838056 .. GENERATED FROM PYTHON SOURCE LINES 94-97 .. code-block:: Python p2 = distribution.computeSurvivalFunction(10.0) p2 .. rst-class:: sphx-glr-script-out .. code-block:: none 0.048608097216194426 .. GENERATED FROM PYTHON SOURCE LINES 98-100 .. code-block:: Python p1 + p2 .. rst-class:: sphx-glr-script-out .. code-block:: none 1.0 .. GENERATED FROM PYTHON SOURCE LINES 101-102 The `drawCDF` and `drawSurvivalFunction` methods allows one to draw the functions :math:`F` and :math:`S`. .. GENERATED FROM PYTHON SOURCE LINES 104-108 .. code-block:: Python graph = distribution.drawCDF() graph.setTitle("CDF of a distribution") view = viewer.View(graph) .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_001.png :alt: CDF of a distribution :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_001.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 109-113 .. code-block:: Python graph = distribution.drawSurvivalFunction() graph.setTitle("Survival function of a distribution") view = viewer.View(graph) .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_002.png :alt: Survival function of a distribution :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_002.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 114-116 In order to get finite bounds for the next graphics, we compute the `xmin` and `xmax` bounds from the 0.01 and 0.99 quantiles of the distributions. .. GENERATED FROM PYTHON SOURCE LINES 118-121 .. code-block:: Python xmin = distribution.computeQuantile(0.01)[0] xmin .. rst-class:: sphx-glr-script-out .. code-block:: none 0.11410588272579382 .. GENERATED FROM PYTHON SOURCE LINES 122-125 .. code-block:: Python xmax = distribution.computeQuantile(0.99)[0] xmax .. rst-class:: sphx-glr-script-out .. code-block:: none 25.29999999999998 .. GENERATED FROM PYTHON SOURCE LINES 126-127 The `drawSurvivalFunction` methods also has an option to plot the survival with the X axis in logarithmic scale. .. GENERATED FROM PYTHON SOURCE LINES 129-136 .. code-block:: Python npoints = 50 logScaleX = True graph = distribution.drawSurvivalFunction(xmin, xmax, npoints, logScaleX) graph.setTitle("Survival function of a distribution where X axis is in log scale") view = viewer.View(graph) # graph .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_003.png :alt: Survival function of a distribution where X axis is in log scale :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_003.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 137-138 In order to get both axes in logarithmic scale, we use the `LOGXY` option of the graph. .. GENERATED FROM PYTHON SOURCE LINES 140-150 .. code-block:: Python npoints = 50 logScaleX = True graph = distribution.drawSurvivalFunction(xmin, xmax, npoints, logScaleX) graph.setLogScale(ot.GraphImplementation.LOGXY) graph.setTitle( "Survival function of a distribution where X and Y axes are in log scale" ) view = viewer.View(graph) # graph .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_004.png :alt: Survival function of a distribution where X and Y axes are in log scale :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_004.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 151-153 Compute the survival of a sample -------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 155-156 We now generate a sample that we are going to analyze. .. GENERATED FROM PYTHON SOURCE LINES 158-160 .. code-block:: Python sample = distribution.getSample(1000) .. GENERATED FROM PYTHON SOURCE LINES 161-163 .. code-block:: Python sample.getMin(), sample.getMax() .. rst-class:: sphx-glr-script-out .. code-block:: none (class=Point name=Unnamed dimension=1 values=[0.100135], class=Point name=Unnamed dimension=1 values=[45.8248]) .. GENERATED FROM PYTHON SOURCE LINES 164-165 The `computeEmpiricalCDF` method of a `Sample` computes the empirical CDF. .. GENERATED FROM PYTHON SOURCE LINES 167-170 .. code-block:: Python p1 = sample.computeEmpiricalCDF([10]) p1 .. rst-class:: sphx-glr-script-out .. code-block:: none 0.95 .. GENERATED FROM PYTHON SOURCE LINES 171-172 Activating the second optional argument allows one to compute the empirical survival function. .. GENERATED FROM PYTHON SOURCE LINES 174-177 .. code-block:: Python p2 = sample.computeEmpiricalCDF([10], True) p2 .. rst-class:: sphx-glr-script-out .. code-block:: none 0.05 .. GENERATED FROM PYTHON SOURCE LINES 178-180 .. code-block:: Python p1 + p2 .. rst-class:: sphx-glr-script-out .. code-block:: none 1.0 .. GENERATED FROM PYTHON SOURCE LINES 181-183 Draw the survival of a sample ----------------------------- .. GENERATED FROM PYTHON SOURCE LINES 185-189 In order to draw the empirical functions of a `Sample`, we use the `UserDefined` class. * The `drawCDF` method plots the CDF. * The `drawSurvivalFunction` method plots the survival function. .. GENERATED FROM PYTHON SOURCE LINES 191-197 .. code-block:: Python userdefined = ot.UserDefined(sample) graph = userdefined.drawCDF() graph.setTitle("CDF of a sample") view = viewer.View(graph) # graph .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_005.png :alt: CDF of a sample :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_005.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 198-203 .. code-block:: Python graph = userdefined.drawSurvivalFunction() graph.setTitle("Empirical survival function of a sample") view = viewer.View(graph) # graph .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_006.png :alt: Empirical survival function of a sample :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_006.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 204-205 As previously, the `drawSurvivalFunction` method of a distribution has an option to set the X axis in logarithmic scale. .. GENERATED FROM PYTHON SOURCE LINES 207-216 .. code-block:: Python xmin = sample.getMin()[0] xmax = sample.getMax()[0] pointNumber = sample.getSize() logScaleX = True graph = userdefined.drawSurvivalFunction(xmin, xmax, pointNumber, logScaleX) graph.setTitle("Empirical survival function of a sample; X axis in log-scale") view = viewer.View(graph) # graph .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_007.png :alt: Empirical survival function of a sample; X axis in log-scale :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_007.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 217-220 We obviously have :math:`P(X>X_{max})=0`, where :math:`X_{max}` is the sample maximum. This prevents from using the sample maximum and have a logarithmic Y axis at the same time. This is why in the following example we restrict the interval where we draw the survival function. .. GENERATED FROM PYTHON SOURCE LINES 222-232 .. code-block:: Python xmin = sample.getMin()[0] xmax = sample.getMax()[0] - 1 # To avoid log(0) because P(X>Xmax)=0 pointNumber = sample.getSize() logScaleX = True graph = userdefined.drawSurvivalFunction(xmin, xmax, pointNumber, logScaleX) graph.setLogScale(ot.GraphImplementation.LOGXY) graph.setTitle("Empirical survival function of a sample; X and Y axes in log-scale") view = viewer.View(graph) # graph .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_008.png :alt: Empirical survival function of a sample; X and Y axes in log-scale :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_008.png :class: sphx-glr-single-img .. GENERATED FROM PYTHON SOURCE LINES 233-235 Compare the distribution and the sample with respect to the survival -------------------------------------------------------------------- .. GENERATED FROM PYTHON SOURCE LINES 237-238 In the final example, we compare the distribution and sample survival functions in the same graphics. .. GENERATED FROM PYTHON SOURCE LINES 240-256 .. code-block:: Python xmin = sample.getMin()[0] xmax = sample.getMax()[0] - 1 # To avoid log(0) because P(X>Xmax)=0 npoints = 50 logScaleX = True graph = userdefined.drawSurvivalFunction(xmin, xmax, pointNumber, logScaleX) graph.setLogScale(ot.GraphImplementation.LOGXY) graph.setColors(["blue"]) graph.setLegends(["Sample"]) graphDistribution = distribution.drawSurvivalFunction(xmin, xmax, npoints, logScaleX) graphDistribution.setLegends(["GPD"]) graph.add(graphDistribution) graph.setLegendPosition("upper right") graph.setTitle("GPD against the sample - n=%d" % (sample.getSize())) view = viewer.View(graph) # graph plt.show() .. image-sg:: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_009.png :alt: GPD against the sample - n=1000 :srcset: /auto_data_analysis/sample_analysis/images/sphx_glr_plot_draw_survival_009.png :class: sphx-glr-single-img .. _sphx_glr_download_auto_data_analysis_sample_analysis_plot_draw_survival.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_draw_survival.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_draw_survival.py `