Taylor expansion momentsΒΆ

In this page, we consider the Taylor expansion of a function. One way to evaluate the central dispersion (expectation and variance) of the variable \uY=\model(\uX) is to use the Taylor expansion of the function \model: \Rset^\inputDim \rightarrow \Rset^q at the mean point \vect{\mu} = \Expect{\uX}. Depending on the order of the Taylor expansion (classically first or second order), we get different approximations of the mean and variance of Y.

We use the notations introduced in Taylor Expansion.

In the remainder, let \Cov \uX be the covariance matrix of \uX, defined by:

\Cov \uX = \mat{C}

where \mat{C} \in \Rset^{\inputDim \times \inputDim} is the input covariance matrix:

c_{ij} = \Expect{\left(X_i - \Expect{X_i}\right)\left(X_j - \Expect{X_j} \right)}

for 1 \leq i, j \leq \inputDim. Notice that each diagonal element of the covariance matrix c_{ii} = \sigma_i^2, is equal to the variance of an input variable (X_i).

Case 1: \outputDim=1, Y = \model(\inputRV) \in \RsetΒΆ

In this section, we analyse the special case where q = 1 and Y = h(\vect{X}) \in \Rset.

The second-order Taylor expansion of \model at the point \ux = \vect{\mu} is:

y = \model (\vect{\mu}) + \sum_{i = 1}^\inputDim \left( \frac{\partial \model }{\partial x_i }\right)(\vect{\mu})(x_i-\mu_i) + \frac{1}{2} \sum_{i,j = 1}^\inputDim \left(\frac{\partial^2 \model }{\partial x_i \partial x_j}\right)(\vect{\mu}) (x_i-\mu_i)(x_j-\mu_j) + o\left(\|\vect{x}\|^2\right).

when \vect{x} \rightarrow \vect{\mu}. The expectation and variance of the first-order expansion are:

\Expect{Y} \approx \model (\vect{\mu})\\ \Var{Y} \approx \sum_{i=1}^\inputDim \sigma_i^2 \left(\left(\frac{\partial \model }{\partial x_i} \right)(\vect{\mu}) \right)^2

In the special case where the inputs are independent, then the variance expression is simplified and we get:

\Expect{Y} \approx \model (\vect{\mu})\\ \Var{Y} \approx \sum_{i=1}^\inputDim \sigma_i^2 \left( \left(\frac{\partial \model }{\partial x_i} \right)(\vect{\mu}) \right)^2

The expectation of the second-order expansion is:

\Expect{Y} \approx \model (\vect{\mu}) + \frac{1}{2} \sum_{i,j=1}^\inputDim c_{ij}\left(\frac{\partial^2 \model} {\partial x_i \partial x_j}\right)(\vect{\mu}).

The second-order approximation of the variance is not implemented because it requires both the knowledge of higher order derivatives of \model and the knowledge of moments of order strictly greater than 2 of the distribution of \uX.

Case 2: \outputDim>1, Y =(Y_1, \dots, Y_{\outputDim}) \in \Rset^{\outputDim}ΒΆ

In this section, we present the general case where \outputDim > 1 and Y =(Y_1, \dots, Y_{\outputDim}) \in \Rset^{\outputDim}.

The second-order Taylor expansion of \model = (\model_1, \dots, \model_{\outputDim}) at the point \ux = \vect{\mu} for each marginal function \model_k is:

y_k = \model_k(\vect{\mu}) + \sum_{i = 1}^\inputDim \left( \frac{\partial \model_k}{\partial x_i }\right)(\vect{\mu}) (x_i-\mu_i)+ \frac{1}{2} \sum_{i,j = 1}^\inputDim \left( \frac{\partial^2 \model_k}{\partial x_i \partial x_j}\right)(\vect{\mu})(x_i-\mu_i)(x_j-\mu_j) + o(\|\vect{x}\|^2).

where 1\leq k \leq \outputDim.

The expectation and covariance matrix of the first-order expansion are:

\Expect{\uY} & \approx \model(\vect{\mu})\\ \Cov \uY & \approx \left( \sum_{i,j=1}^\inputDim c_{ij} \left( \frac{\partial \model_k}{\partial x_i } \right)(\vect{\mu})\left( \frac{\partial \model_\ell}{\partial x_j }\right)(\vect{\mu})\right)_{k, \ell}

for 1\leq k \leq \outputDim.

The expectation of the second-order expansion is:

(\Expect{\uY})_k = \Expect{Y_k} \approx \model_k(\vect{\mu}) + \frac{1}{2} \sum_{i,j=1}^\inputDim c_{ij}\left( \frac{\partial^2 \model_k}{\partial x_i \partial x_j}\right)(\vect{\mu})

for 1\leq k \leq \outputDim.

The second-order approximation of the variance is not implemented because it requires both the knowledge of higher order derivatives of \model and the knowledge of moments of order strictly greater than 2 of the probability density function.