Note

Go to the end to download the full example code

Example of sensitivity analyses on the wing weight model¶

This example is a brief overview of the use of the most usual sensitivity analysis techniques and how to call them:

PCC: Partial Correlation Coefficients
PRCC: Partial Rank Correlation Coefficients
SRC: Standard Regression Coefficients
SRRC: Standard Rank Regression Coefficients
Pearson coefficients
Spearman coefficients
Taylor expansion importance factors
Sobol’ indices
HSIC : Hilbert-Schmidt Independence Criterion

We present the methods on the WingWeight function and use the same notations.

Definition of the model¶

We load the model from the usecases module.

import openturns as ot
import openturns.viewer as otv
from openturns.usecases.wingweight_function import WingWeightModel
from matplotlib import pylab as plt
import numpy as np

ot.Log.Show(ot.Log.NONE)
m = WingWeightModel()

Cross cuts of the function¶

Let’s have a look on 2D cross cuts of the wing weight function. For each 2D cross cut, the other variables are fixed to the input distribution mean values. This graph allows to have a first idea of the variations of the function in pair of dimensions. The colors of each contour plot are comparable. The number of contour levels are related to the amount of variation of the function in the corresponding coordinates.

fig = plt.figure(figsize=(12, 12))
lowerBound = m.distributionX.getRange().getLowerBound()
upperBound = m.distributionX.getRange().getUpperBound()

# Definition of number of meshes in x and y axes for the 2D cross cut plots
nX = 20
nY = 20
for i in range(m.dim):
    for j in range(i):
        crossCutIndices = []
        crossCutReferencePoint = []
        for k in range(m.dim):
            if k != i and k != j:
                crossCutIndices.append(k)
                # Definition of the reference point
                crossCutReferencePoint.append(m.distributionX.getMean()[k])
        # Definition of 2D cross cut function
        crossCutFunction = ot.ParametricFunction(
            m.model, crossCutIndices, crossCutReferencePoint
        )
        crossCutLowerBound = [lowerBound[j], lowerBound[i]]
        crossCutUpperBound = [upperBound[j], upperBound[i]]
        # Definition of the mesh
        inputData = ot.Box([nX, nY]).generate()
        inputData *= ot.Point(crossCutUpperBound) - ot.Point(crossCutLowerBound)
        inputData += ot.Point(crossCutLowerBound)
        meshX = np.array(inputData)[:, 0].reshape(nX + 2, nY + 2)
        meshY = np.array(inputData)[:, 1].reshape(nX + 2, nY + 2)
        data = crossCutFunction(inputData)
        meshZ = np.array(data).reshape(nX + 2, nY + 2)
        levels = [(150 + 3 * i) for i in range(101)]

        # Creation of the contour
        index = 1 + i * m.dim + j

        ax = fig.add_subplot(m.dim, m.dim, index)
        ax.pcolormesh(meshX, meshY, meshZ, cmap="hsv", vmin=176., vmax=363., shading='auto')
        ax.set_xticks([])
        ax.set_yticks([])

        # Creation of axes title
        if j == 0:
            ax.set_ylabel(m.distributionX.getDescription()[i])
        if i == 9:
            ax.set_xlabel(m.distributionX.getDescription()[j])

We can see that the variables $t_c, N_z, A, W_{dg}$ seem to be influent on the wing weight whereas $\Lambda, \ell, q, W_p, W_{fw}$ have less influence on the function.

Data generation¶

We create the input and output data for the estimation of the different sensitivity coefficients and we get the input variables description:

inputNames = m.distributionX.getDescription()

size = 500
inputDesign = m.distributionX.getSample(size)
outputDesign = m.model(inputDesign)

Let’s estimate the PCC, PRCC, SRC, SRRC, Pearson and Spearman coefficients, display and analyze them. We create a CorrelationAnalysis model.

corr_analysis = ot.CorrelationAnalysis(inputDesign, outputDesign)

PCC coefficients¶

We compute here PCC coefficients using the CorrelationAnalysis.

pcc_indices = corr_analysis.computePCC()
print(pcc_indices)

[0.940186,0.0882968,0.968989,0.0101513,0.115705,0.315289,-0.947166,0.981847,0.917402,0.44622]#10

graph = ot.SobolIndicesAlgorithm.DrawCorrelationCoefficients(
    pcc_indices, inputNames, "PCC coefficients - Wing weight"
)
view = otv.View(graph)

PRCC coefficients¶

We compute here PRCC coefficients using the CorrelationAnalysis.

prcc_indices = corr_analysis.computePRCC()
print(prcc_indices)

[0.8486,0.0649984,0.913677,-0.0206522,0.0858264,0.179864,-0.862092,0.949614,0.816437,0.340957]#10

graph = ot.SobolIndicesAlgorithm.DrawCorrelationCoefficients(
    prcc_indices, inputNames, "PRCC coefficients - Wing weight"
)
view = otv.View(graph)

SRC coefficients¶

We compute here SRC coefficients using the CorrelationAnalysis.

src_indices = corr_analysis.computeSRC()
print(src_indices)

[0.368479,0.0117622,0.519118,0.00135185,0.0153738,0.043904,-0.391804,0.692999,0.303627,0.0659533]#10

graph = ot.SobolIndicesAlgorithm.DrawCorrelationCoefficients(
    src_indices, inputNames, "SRC coefficients - Wing weight"
)
view = otv.View(graph)

Normalized squared SRC coefficients (coefficients are made to sum to 1) :

squared_src_indices = corr_analysis.computeSquaredSRC(True)
print(squared_src_indices)

[0.119327,0.000121588,0.236833,1.60608e-06,0.000207717,0.00169402,0.134911,0.422061,0.0810197,0.00382282]#10

And their associated graph:

graph = ot.SobolIndicesAlgorithm.DrawCorrelationCoefficients(
    squared_src_indices, inputNames, "Squared SRC coefficients - Wing weight"
)
view = otv.View(graph)

SRRC coefficients¶

We compute here SRRC coefficients using the CorrelationAnalysis.

srrc_indices = corr_analysis.computeSRRC()
print(srrc_indices)

[0.361267,0.0145646,0.501659,-0.00463828,0.0191614,0.0407509,-0.380531,0.683358,0.313877,0.0808765]#10

graph = ot.SobolIndicesAlgorithm.DrawCorrelationCoefficients(
    srrc_indices, inputNames, "SRRC coefficients - Wing weight"
)
view = otv.View(graph)

Pearson coefficients¶

We compute here the Pearson $\rho$ coefficients using the CorrelationAnalysis.

pearson_correlation = corr_analysis.computePearsonCorrelation()
print(pearson_correlation)

[0.235512,-0.0328824,0.419915,-0.0135446,-0.0692302,0.0434365,-0.379096,0.612647,0.335063,0.0419078]#10

title_pearson_graph = "Pearson correlation coefficients - Wing weight"
graph = ot.SobolIndicesAlgorithm.DrawCorrelationCoefficients(
    pearson_correlation, inputNames, title_pearson_graph
)
view = otv.View(graph)

Pearson correlation coefficients - Wing weight

Spearman coefficients¶

We compute here the Spearman $\rho_s$ coefficients using the CorrelationAnalysis.

spearman_correlation = corr_analysis.computeSpearmanCorrelation()
print(spearman_correlation)

[0.226962,-0.0274201,0.40528,-0.0187471,-0.0642766,0.0358186,-0.366801,0.605454,0.344385,0.0551515]#10

title_spearman_graph = "Spearman correlation coefficients - Wing weight"
graph = ot.SobolIndicesAlgorithm.DrawCorrelationCoefficients(
    spearman_correlation, inputNames, title_spearman_graph
)
view = otv.View(graph)
plt.show()

Spearman correlation coefficients - Wing weight

The different computed correlation estimators show that the variables $S_w, A, N_z, t_c$ seem to be the most correlated with the wing weight in absolute value. Pearson and Spearman coefficients do not reveal any linear nor monotonic correlation as no coefficients are equal to +/- 1. Coefficients about $t_c$ are negative revealing a negative correlation with the wing weight, that is consistent with the model expression.

Taylor expansion importance factors¶

We compute here the Taylor expansion importance factors using TaylorExpansionMoments.

We create a distribution-based RandomVector.

X = ot.RandomVector(m.distributionX)

We create a composite RandomVector Y from X and m.model.

Y = ot.CompositeRandomVector(m.model, X)

We create a Taylor expansion method to approximate moments.

taylor = ot.TaylorExpansionMoments(Y)

We get the importance factors.

print(taylor.getImportanceFactors())

[Sw : 0.130315, Wfw : 2.94004e-06, A : 0.228153, Lambda : 0, q : 8.25053e-05, l : 0.00180269, tc : 0.135002, Nz : 0.412794, Wdg : 0.0883317, Wp : 0.00351621]

We draw the importance factors

graph = taylor.drawImportanceFactors()
graph.setTitle("Taylor expansion imporfance factors - Wing weight")
view = otv.View(graph)

Taylor expansion imporfance factors - Wing weight

The Taylor expansion importance factors is consistent with the previous estimators as $S_w, A, N_z, t_c$ seem to be the most influent variables. To analyze the relevance of the previous indices, a Sobol’ analysis is now carried out.

Sobol’ indices¶

We compute the Sobol’ indices from both sampling approach and Polynomial Chaos Expansion.

sizeSobol = 1000
sie = ot.SobolIndicesExperiment(m.distributionX, sizeSobol)
inputDesignSobol = sie.generate()
inputNames = m.distributionX.getDescription()
inputDesignSobol.setDescription(inputNames)
inputDesignSobol.getSize()

We see that 12000 function evaluations are required to estimate the first order and total Sobol’ indices.

Then, we evaluate the outputs corresponding to this design of experiments.

outputDesignSobol = m.model(inputDesignSobol)

We estimate the Sobol’ indices with the SaltelliSensitivityAlgorithm.

sensitivityAnalysis = ot.SaltelliSensitivityAlgorithm(
    inputDesignSobol, outputDesignSobol, sizeSobol
)

The getFirstOrderIndices and getTotalOrderIndices methods respectively return estimates of all first order and total Sobol’ indices.

print("First order indices:", sensitivityAnalysis.getFirstOrderIndices())

First order indices: [0.0895403,-0.0324985,0.224239,-0.0324775,-0.0326605,-0.0297425,0.111533,0.459428,0.0692415,-0.0257065]#10

print("Total order indices:", sensitivityAnalysis.getTotalOrderIndices())

Total order indices: [0.132254,1.75663e-05,0.25098,0.000159035,0.000417434,0.000214447,0.144213,0.410061,0.101327,0.00225025]#10

The draw method produces the following graph. The vertical bars represent the 95% confidence intervals of the estimates.

graph = sensitivityAnalysis.draw()
graph.setTitle("Sobol indices with Saltelli - wing weight")
view = otv.View(graph)

Sobol indices with Saltelli - wing weight

We see that several Sobol’ indices are negative, that is inconsistent with the theory. Therefore, a larger number of samples is required to get consistent indices

sizeSobol = 10000
sie = ot.SobolIndicesExperiment(m.distributionX, sizeSobol)
inputDesignSobol = sie.generate()
inputNames = m.distributionX.getDescription()
inputDesignSobol.setDescription(inputNames)
inputDesignSobol.getSize()
outputDesignSobol = m.model(inputDesignSobol)

sensitivityAnalysis = ot.SaltelliSensitivityAlgorithm(
    inputDesignSobol, outputDesignSobol, sizeSobol
)

sensitivityAnalysis.getFirstOrderIndices()
sensitivityAnalysis.getTotalOrderIndices()

graph = sensitivityAnalysis.draw()
graph.setTitle("Sobol indices with Saltelli - wing weight")
view = otv.View(graph)

It improves the accuracy of the estimation but, for very low indices, Saltelli scheme is not accurate since several confidence intervals provide negative lower bounds.

Now, we estimate the Sobol’ indices using Polynomial Chaos Expansion. We create a Functional Chaos Expansion.

sizePCE = 800
inputDesignPCE = m.distributionX.getSample(sizePCE)
outputDesignPCE = m.model(inputDesignPCE)

algo = ot.FunctionalChaosAlgorithm(inputDesignPCE, outputDesignPCE, m.distributionX)

algo.run()
result = algo.getResult()
print(result.getResiduals())
print(result.getRelativeErrors())

[0.000633307]
[1.56991e-09]

The relative errors are low : this indicates that the PCE model has good accuracy. Then, we exploit the surrogate model to compute the Sobol’ indices.

sensitivityAnalysis = ot.FunctionalChaosSobolIndices(result)
print(sensitivityAnalysis.summary())
firstOrder = [sensitivityAnalysis.getSobolIndex(i) for i in range(m.dim)]
totalOrder = [sensitivityAnalysis.getSobolTotalIndex(i) for i in range(m.dim)]
graph = ot.SobolIndicesAlgorithm.DrawSobolIndices(inputNames, firstOrder, totalOrder)
graph.setTitle("Sobol indices by Polynomial Chaos Expansion - wing weight")
view = otv.View(graph)

Sobol indices by Polynomial Chaos Expansion - wing weight

 input dimension: 10
 output dimension: 1
 basis size: 299
 mean: [268.074]
 std-dev: [48.0827]
------------------------------------------------------------
Index   | Multi-indice                  | Part of variance
------------------------------------------------------------
      7 | [0,0,0,0,0,0,0,1,0,0]         | 0.410327
      3 | [0,0,1,0,0,0,0,0,0,0]         | 0.220054
      6 | [0,0,0,0,0,0,1,0,0,0]         | 0.13849
      1 | [1,0,0,0,0,0,0,0,0,0]         | 0.124432
      8 | [0,0,0,0,0,0,0,0,1,0]         | 0.0849403
------------------------------------------------------------


------------------------------------------------------------
Component | Sobol index            | Sobol total index
------------------------------------------------------------
        0 | 0.124441               | 0.127859
        1 | 2.77423e-06            | 2.93343e-06
        2 | 0.220203               | 0.225955
        3 | 0.000487693            | 0.000504128
        4 | 8.81411e-05            | 9.11378e-05
        5 | 0.00179913             | 0.00185859
        6 | 0.141038               | 0.145127
        7 | 0.411636               | 0.419666
        8 | 0.0849948              | 0.0876214
        9 | 0.00333265             | 0.00335527
------------------------------------------------------------

The Sobol’ indices confirm the previous analyses, in terms of ranking of the most influent variables. We also see that five variables have a quasi null total Sobol’ indices, that indicates almost no influence on the wing weight. There is no discrepancy between first order and total Sobol’ indices, that indicates no or very low interaction between the variables in the variance of the output. As the most important variables act only through decoupled first degree contributions, the hypothesis of a linear dependence between the input variables and the weight is legitimate. This explains why both squared SRC and Taylor give the exact same results even if the first one is based on a $\mathcal{L}^2$ linear approximation and the second one is based on a linear expansion around the mean value of the input variables.

HSIC indices¶

We then estimate the HSIC indices using a data-driven approach.

sizeHSIC = 250
inputDesignHSIC = m.distributionX.getSample(sizeHSIC)
outputDesignHSIC = m.model(inputDesignHSIC)

covarianceModelCollection = []

for i in range(m.dim):
    Xi = inputDesignHSIC.getMarginal(i)
    inputCovariance = ot.SquaredExponential(1)
    inputCovariance.setScale(Xi.computeStandardDeviation())
    covarianceModelCollection.append(inputCovariance)

We define a covariance kernel associated to the output variable.

outputCovariance = ot.SquaredExponential(1)
outputCovariance.setScale(outputDesignHSIC.computeStandardDeviation())
covarianceModelCollection.append(outputCovariance)

In this paragraph, we perform the analysis on the raw data: that is the global HSIC estimator.

estimatorType = ot.HSICUStat()

We now build the HSIC estimator:

globHSIC = ot.HSICEstimatorGlobalSensitivity(
    covarianceModelCollection, inputDesignHSIC, outputDesignHSIC, estimatorType
)

We get the R2-HSIC indices:

R2HSICIndices = globHSIC.getR2HSICIndices()
print("\n Global HSIC analysis")
print("R2-HSIC Indices: ", R2HSICIndices)

 Global HSIC analysis
R2-HSIC Indices:  [0.102709,-0.00336782,0.148905,0.000299579,-0.00488949,0.00479322,0.0845911,0.34874,0.0601762,-0.00321501]#10

and the HSIC indices:

HSICIndices = globHSIC.getHSICIndices()
print("HSIC Indices: ", HSICIndices)

HSIC Indices:  [0.00906902,-0.000293314,0.013635,2.72569e-05,-0.000444497,0.000436191,0.00777676,0.0309526,0.00545591,-0.000300297]#10

The p-value by permutation.

pvperm = globHSIC.getPValuesPermutation()
print("p-value (permutation): ", pvperm)

p-value (permutation):  [0,0.712871,0,0.376238,0.841584,0.128713,0,0,0,0.712871]#10

We have an asymptotic estimate of the value for this estimator.

pvas = globHSIC.getPValuesAsymptotic()
print("p-value (asymptotic): ", pvas)

p-value (asymptotic):  [3.02901e-12,0.680845,1.28369e-16,0.392815,0.82049,0.169798,1.63072e-09,6.32092e-35,4.2536e-07,0.675604]#10

We vizualise the results.

graph1 = globHSIC.drawHSICIndices()
view1 = otv.View(graph1)

graph2 = globHSIC.drawPValuesAsymptotic()
view2 = otv.View(graph2)

graph3 = globHSIC.drawR2HSICIndices()
view3 = otv.View(graph3)

graph4 = globHSIC.drawPValuesPermutation()
view4 = otv.View(graph4)

The HSIC indices go in the same way as the other estimators in terms the most influent variables. The variables $W_{fw}, q, l, W_p$ seem to be independent to the output as the corresponding p-values are high. We can also see that the asymptotic p-values and p-values estimated by permutation are quite similar.

Total running time of the script: ( 2 minutes 55.856 seconds)

OpenTURNS

An Open source initiative for the Treatment of Uncertainties, Risks'N Statistics

Table of Contents

Previous topic

Next topic

This Page