Note
Go to the end to download the full example code.
Conditional expectation of a polynomial chaos expansion¶
In this example, we compute the conditional expectation of a polynomial
chaos expansion of the Ishigami function using
the getConditionalExpectation()
method.
Introduction¶
Let be the dimension of the input random vector. Let be the mean of the input random vector . Let be the physical model:
Given a group of input variables, we want to create a new function :
where is the number of variables in the group.
In this example, we experiment two different ways to reduce the input dimension of a polynomial chaos expansion:
the parametric function,
the conditional expectation.
The goal of this page is to see how we can create these functions and the difference between them.
Parametric function¶
The simplest method to reduce the dimension of the input is to set some input variables to constant values. In this example, all marginal inputs, except those in the conditioning indices are set to the mean of the input random vector.
Let be the complementary set of input marginal indices such that and form a disjoint partition of the full set of variable indices:
The parametric function with reduced dimension is:
for any .
The previous function is a parametric function based on the function
where the parameter is .
Assuming that the input random vector has an independent copula,
computing
can be done by selecting the corresponding indices in .
This function can be created using the ParametricFunction
class.
Parametric PCE¶
If the physical model is a PCE, then the associated parametric model is also a PCE. Its coefficients and the associated functional basis can be computed from the original PCE. A significant fact, however, is that the coefficients of the parametric PCE are not the ones of the original PCE: the coefficients of the parametric PCE have to be multiplied by factors which depend on the value of the discarded basis functions on the parameter vector. This feature is not currently available in the library. However, we present it below as this derivation is interesting to understand why the conditional expectation may behave differently from the corresponding parametric PCE.
Let be the set of multi-indices corresponding to the truncated polynomial chaos expansion up to the -th coefficient. Let be the PCE in the standard space:
Let be a group of variables, let be its complementary set such that
i.e. the groups and create a disjoint partition of the set . Let be the number of elements in the group . Hence, we have .
Let be a given point. We are interested in the function :
for any . We assume that the polynomial basis are defined by the tensor product:
for any where is the polynomial of degree of the -th input standard variable.
Let denote the components of the group where is the number of elements in the group. Similarly, let denote the components of the complementary group . The components of which are in the group are and the complementary components are .
Let be the reduced polynomial:
(1)¶
where is the reduced multi-index defined from the multi-index by the equation:
for . The components of the reduced multi-index which corresponds to the components of the multi-index given by the complementary group .
We must then gather the reduced multi-indices. Let be the set of unique reduced multi-indices:
(2)¶
For any reduced multi-index of dimension , we note the set of corresponding (un-reduced) multi-indices of dimension :
(3)¶
Each aggregated coefficient is defined by the equation:
(4)¶
Finally:
(5)¶
for any .
The method is the following.
Create the reduced polynomial basis from equation (1).
Create the list of reduced multi-indices from the equation (2), and, for each reduced multi-index, the list of corresponding multi-indices from the equation (3).
Aggregate the coefficients from the equation (4).
The parametric PCE is defined by the equation (5).
Conditional expectation¶
One method to reduce the input dimension of a function is to consider its conditional expectation. The conditional expectation function is:
for any .
In general, there is no dedicated method to create such a conditional expectation
in the library.
We can, however, efficiently compute the conditional expectation of a polynomial
chaos expansion.
In turn, this conditional chaos expansion (PCE) is a polynomial chaos expansion
which can be computed using the getConditionalExpectation()
method from the FunctionalChaosResult
class.
Create the PCE¶
import openturns as ot
import openturns.viewer as otv
from openturns.usecases import ishigami_function
import matplotlib.pyplot as plt
The next function creates a parametric PCE based on a given PCE and a set of indices.
def meanParametricPCE(chaosResult, indices):
"""
Return the parametric PCE of Y with given input marginals set to the mean.
All marginal inputs, except those in the conditioning indices
are set to the mean of the input random vector.
The resulting function is :
g(xu) = PCE(xu, xnotu = E[Xnotu])
where xu is the input vector of conditioning indices,
xnotu is the input vector fixed indices and
E[Xnotu] is the expectation of the random vector of the components
not in u.
Parameters
----------
chaosResult: ot.FunctionalChaosResult(inputDimension)
The polynomial chaos expansion.
indices: ot.Indices()
The indices of the input variables which are set to constant values.
Returns
-------
parametricPCEFunction : ot.ParametricFunction(reducedInputDimension, outputDimension)
The parametric PCE.
The reducedInputDimension is equal to inputDimension - indices.getSize().
"""
inputDistribution = chaosResult.getDistribution()
if not inputDistribution.hasIndependentCopula():
raise ValueError(
"The input distribution has a copula" "which is not independent"
)
# Create the parametric function
pceFunction = chaosResult.getMetaModel()
xMean = inputDistribution.getMean()
referencePoint = xMean[indices]
parametricPCEFunction = ot.ParametricFunction(pceFunction, indices, referencePoint)
return parametricPCEFunction
The next function creates a sparse PCE using least squares.
def computeSparseLeastSquaresFunctionalChaos(
inputTrain,
outputTrain,
multivariateBasis,
basisSize,
distribution,
sparse=True,
):
"""
Create a sparse polynomial chaos based on least squares.
* Uses the enumerate rule in multivariateBasis.
* Uses the LeastSquaresStrategy to compute the coefficients based on
least squares.
* Uses LeastSquaresMetaModelSelectionFactory to use the LARS selection method.
* Uses FixedStrategy in order to keep all the coefficients that the
LARS method selected.
Parameters
----------
inputTrain : ot.Sample
The input design of experiments.
outputTrain : ot.Sample
The output design of experiments.
multivariateBasis : ot.Basis
The multivariate chaos basis.
basisSize : int
The size of the function basis.
distribution : ot.Distribution.
The distribution of the input variable.
sparse: bool
If True, create a sparse PCE.
Returns
-------
result : ot.PolynomialChaosResult
The estimated polynomial chaos.
"""
if sparse:
selectionAlgorithm = ot.LeastSquaresMetaModelSelectionFactory()
else:
selectionAlgorithm = ot.PenalizedLeastSquaresAlgorithmFactory()
projectionStrategy = ot.LeastSquaresStrategy(
inputTrain, outputTrain, selectionAlgorithm
)
adaptiveStrategy = ot.FixedStrategy(multivariateBasis, basisSize)
chaosAlgorithm = ot.FunctionalChaosAlgorithm(
inputTrain, outputTrain, distribution, adaptiveStrategy, projectionStrategy
)
chaosAlgorithm.run()
chaosResult = chaosAlgorithm.getResult()
return chaosResult
In the next cell, we create a training sample from the Ishigami test function. We choose a sample size equal to 1000.
ot.Log.Show(ot.Log.NONE)
ot.RandomGenerator.SetSeed(0)
im = ishigami_function.IshigamiModel()
input_names = im.inputDistribution.getDescription()
sampleSize = 1000
inputSample = im.inputDistribution.getSample(sampleSize)
outputSample = im.model(inputSample)
We then create a sparce PCE of the Ishigami function using a candidate basis up to the total degree equal to 12. This leads to 455 candidate coefficients. The coefficients are computed from least squares.
multivariateBasis = ot.OrthogonalProductPolynomialFactory([im.X1, im.X2, im.X3])
totalDegree = 12
enumerateFunction = multivariateBasis.getEnumerateFunction()
basisSize = enumerateFunction.getBasisSizeFromTotalDegree(totalDegree)
print("Basis size = ", basisSize)
Basis size = 455
Finally, we create the PCE.
Only 61 coefficients are selected by the LARS
algorithm.
chaosResult = computeSparseLeastSquaresFunctionalChaos(
inputSample,
outputSample,
multivariateBasis,
basisSize,
im.inputDistribution,
)
print("Selected basis size = ", chaosResult.getIndices().getSize())
chaosResult
Selected basis size = 61
In order to see the structure of the data, we create a grid of plots which shows all projections of versus for . We see that the Ishigami function is particularly non linear.
grid = ot.VisualTest.DrawPairsXY(inputSample, outputSample)
grid.setTitle(f"n = {sampleSize}")
view = otv.View(grid, figure_kw={"figsize": (8.0, 3.0)})
plt.subplots_adjust(wspace=0.4, bottom=0.25)
Parametric function¶
We now create the parametric function where is free and the other variables are set to their mean values. We can show that a parametric PCE is, again, a PCE. The library does not currently implement this feature. In the next cell, we create it from the meanParametricPCE we defined previously.
Create different parametric functions for the PCE. In the next cell, we create the parametric PCE function where is active while and are set to their mean values.
indices = [1, 2]
parametricPCEFunction = meanParametricPCE(chaosResult, indices)
print(parametricPCEFunction.getInputDimension())
1
Now that we know how the meanParametricPCE works, we loop over the input marginal indices and consider the three functions , and . For each marginal index i, we we plot the output against the input marginal of the sample. Then we plot the parametric function depending on .
inputDimension = im.inputDistribution.getDimension()
npPoints = 100
inputRange = im.inputDistribution.getRange()
inputLowerBound = inputRange.getLowerBound()
inputUpperBound = inputRange.getUpperBound()
# Create the palette with transparency
palette = ot.Drawable().BuildDefaultPalette(2)
firstColor = palette[0]
r, g, b, a = ot.Drawable.ConvertToRGBA(firstColor)
newAlpha = 64
newColor = ot.Drawable.ConvertFromRGBA(r, g, b, newAlpha)
palette[0] = newColor
grid = ot.VisualTest.DrawPairsXY(inputSample, outputSample)
reducedBasisSize = chaosResult.getCoefficients().getSize()
grid.setTitle(
f"n = {sampleSize}, total degree = {totalDegree}, "
f"basis = {basisSize}, selected = {reducedBasisSize}"
)
for i in range(inputDimension):
graph = grid.getGraph(0, i)
graph.setLegends(["Data"])
graph.setXTitle(f"$x_{1 + i}$")
# Set all indices except i
indices = list(range(inputDimension))
indices.pop(i)
parametricPCEFunction = meanParametricPCE(chaosResult, indices)
xiMin = inputLowerBound[i]
xiMax = inputUpperBound[i]
curve = parametricPCEFunction.draw(xiMin, xiMax, npPoints).getDrawable(0)
curve.setLineWidth(2.0)
curve.setLegend(r"$PCE(x_i, x_{-i} = \mathbb{E}[X_{-i}])$")
graph.add(curve)
if i < inputDimension - 1:
graph.setLegends([""])
graph.setColors(palette)
grid.setGraph(0, i, graph)
grid.setLegendPosition("topright")
view = otv.View(
grid,
figure_kw={"figsize": (8.0, 3.0)},
legend_kw={"bbox_to_anchor": (1.0, 1.0), "loc": "upper left"},
)
plt.subplots_adjust(wspace=0.4, right=0.7, bottom=0.25)
We see that the parametric function is located within each cloud, but sometimes seems a little vertically on the edges of the data. More precisely, the function represents well how depends on , but does not seem to represent well how depends on or .
Conditional expectation¶
In the next cell, we create the conditional expectation function .
conditionalPCE = chaosResult.getConditionalExpectation([0])
conditionalPCE
On output, we see that the result is, again, a PCE. Moreover, a subset of the previous coefficients are presented in this conditional expectation: only multi-indices which involve are presented (and the other marginal components are removed). We observe that the value of the coefficients are unchanged with respect to the previous PCE.
In the next cell, we create the conditional expectation function .
conditionalPCE = chaosResult.getConditionalExpectation([1, 2])
conditionalPCE
We see that the conditional PCE has input dimension 2.
In the next cell, we compare the parametric PCE and the conditional expectation of the PCE.
# sphinx_gallery_thumbnail_number = 3
inputDimension = im.inputDistribution.getDimension()
npPoints = 100
inputRange = im.inputDistribution.getRange()
inputLowerBound = inputRange.getLowerBound()
inputUpperBound = inputRange.getUpperBound()
# Create the palette with transparency
palette = ot.Drawable().BuildDefaultPalette(3)
firstColor = palette[0]
r, g, b, a = ot.Drawable.ConvertToRGBA(firstColor)
newAlpha = 64
newColor = ot.Drawable.ConvertFromRGBA(r, g, b, newAlpha)
palette[0] = newColor
grid = ot.VisualTest.DrawPairsXY(inputSample, outputSample)
grid.setTitle(f"n = {sampleSize}, total degree = {totalDegree}")
for i in range(inputDimension):
graph = grid.getGraph(0, i)
graph.setLegends(["Data"])
graph.setXTitle(f"$x_{1 + i}$")
xiMin = inputLowerBound[i]
xiMax = inputUpperBound[i]
# Set all indices except i to the mean
indices = list(range(inputDimension))
indices.pop(i)
parametricPCEFunction = meanParametricPCE(chaosResult, indices)
# Draw the parametric function
curve = parametricPCEFunction.draw(xiMin, xiMax, npPoints).getDrawable(0)
curve.setLineWidth(2.0)
curve.setLineStyle("dashed")
curve.setLegend(r"$PCE\left(x_i, x_{-i} = \mathbb{E}[X_{-i}]\right)$")
graph.add(curve)
# Compute conditional expectation given Xi
conditionalPCE = chaosResult.getConditionalExpectation([i])
print(f"i = {i}")
print(conditionalPCE)
conditionalPCEFunction = conditionalPCE.getMetaModel()
curve = conditionalPCEFunction.draw(xiMin, xiMax, npPoints).getDrawable(0)
curve.setLineWidth(2.0)
curve.setLegend(r"$\mathbb{E}\left[PCE | X_i = x_i\right]$")
graph.add(curve)
if i < inputDimension - 1:
graph.setLegends([""])
graph.setColors(palette)
# Set the graph into the grid
grid.setGraph(0, i, graph)
grid.setLegendPosition("topright")
view = otv.View(
grid,
figure_kw={"figsize": (8.0, 3.0)},
legend_kw={"bbox_to_anchor": (1.0, 1.0), "loc": "upper left"},
)
plt.subplots_adjust(wspace=0.4, right=0.7, bottom=0.25)
i = 0
FunctionalChaosResult
- input dimension=1
- output dimension=1
- distribution dimension=1
- transformation=1 -> 1
- inverse transformation=1 -> 1
- orthogonal basis dimension=1
- indices size=8
- relative errors=[4.89182e-12]
- residuals=[7.23589e-06]
- is least squares=true
- is model selection=false
| Index | Multi-index | Coefficient |
|-------|---------------|---------------|
| 0 | [0] | 3.5 |
| 1 | [1] | 1.6254 |
| 2 | [3] | -1.29066 |
| 3 | [5] | 0.194909 |
| 4 | [7] | -0.0126967 |
| 5 | [8] | -2.0694e-05 |
| 6 | [9] | 0.000433557 |
| 7 | [11] | -3.13448e-05 |
i = 1
FunctionalChaosResult
- input dimension=1
- output dimension=1
- distribution dimension=1
- transformation=1 -> 1
- inverse transformation=1 -> 1
- orthogonal basis dimension=1
- indices size=7
- relative errors=[4.89182e-12]
- residuals=[7.23589e-06]
- is least squares=true
- is model selection=false
| Index | Multi-index | Coefficient |
|-------|---------------|---------------|
| 0 | [0] | 3.5 |
| 1 | [2] | -0.594721 |
| 2 | [4] | -1.95229 |
| 3 | [6] | 1.35739 |
| 4 | [8] | -0.339403 |
| 5 | [10] | 0.0459064 |
| 6 | [12] | -0.00397025 |
i = 2
FunctionalChaosResult
- input dimension=1
- output dimension=1
- distribution dimension=1
- transformation=1 -> 1
- inverse transformation=1 -> 1
- orthogonal basis dimension=1
- indices size=4
- relative errors=[4.89182e-12]
- residuals=[7.23589e-06]
- is least squares=true
- is model selection=false
| Index | Multi-index | Coefficient |
|-------|---------------|---------------|
| 0 | [0] | 3.5 |
| 1 | [5] | -1.35667e-05 |
| 2 | [11] | -8.78431e-06 |
| 3 | [12] | -1.01054e-05 |
We see that the conditional expectation of the PCE is a better approximation of the data set than the parametric PCE.
Conclusion¶
In this example, we have seen how to compute the conditional expectation of a PCE. We have seen that this function is a good approximation of the Ishigami function when we reduce the input dimension. We have also seen that the parametric PCE might be a poor approximation of the Ishigami function. This is because the parametric PCE depends on the particular value that we have chosen to create the parametric function.
The fact that the conditional expectation of the PCE is a good approximation of the function when we reduce the input dimension is a consequence of a theorem which states that the conditional expectation is the best approximation of the function in the least squares sense (see [girardin2018] page 79).
otv.View.ShowAll()
Total running time of the script: (0 minutes 13.816 seconds)