Gaussian calibration¶
Introduction¶
In this page, we present the method used in the GaussianLinearCalibration
and GaussianNonLinearCalibration
classes.
Gaussian calibration is a particular case of Bayesian calibration
(see Bayesian calibration) where
the prior has a Gaussian distribution and the observation error has a Gaussian
distribution.
In this case, simplifications occur which lead to an efficient
algorithm that requires fewer model evaluations compared to Monte-Carlo
Markov Chains (MCMC).
We consider a computer model (i.e. a deterministic function)
to calibrate:
where
is the input vector;
is the output vector;
are the unknown parameters of
to calibrate.
Let be the number of observations.
The standard hypothesis of the probabilistic calibration is:
for where
is a random measurement error.
The goal of Gaussian calibration is to estimate , based on
observations of
inputs
and the associated
observations of the output
.
In other words, the calibration process reduces the discrepancy between
the observations
and the
predictions
.
Given that
are realizations of a
random variable, the estimate of
, denoted by
, is also a random variable.
Hence, the secondary goal of calibration is to estimate the distribution of
representing the uncertainty of the calibration
process.
In the remaining of this section, the input is not involved
anymore in the equations.
This is why we simplify the equation into:
Bayesian calibration¶
The Bayesian calibration framework is based on two hypotheses.
The first hypothesis is that the parameter has
a known distribution, called the prior distribution, and denoted by
.
The second hypothesis is that the output observations
are sampled from a known conditional distribution denoted by
.
For any
such that
, the Bayes theorem implies
that the conditional distribution of
given
is:
for any .
The denominator of the previous Bayes fraction is independent of
, so that
the posterior distribution is proportional to the numerator:
for any .
In Gaussian calibration, the two previous distributions are assumed to be gaussian.
More precisely, we make the hypothesis that the parameter
follows the normal distribution:
where is the mean of the gaussian prior distribution,
which is named the background and
is the covariance
matrix of the parameter.
Secondly, we make the hypothesis that the output observations follow the conditional normal distribution:
where is the covariance
matrix of the output observations.
Posterior distribution¶
Let be the Mahalanobis distance associated with the matrix
:
for any .
Denote by
the Mahalanobis distance associated with the matrix
:
for any and any
.
Therefore, the posterior distribution of
given the observations
is:
for any .
MAP estimator¶
The maximum of the posterior distribution of given the observations
is
reached at :
It is called the maximum a posteriori posterior estimator or MAP estimator.
Regularity of solutions of the Gaussian Calibration¶
The gaussian calibration is a tradeoff, so that the
second expression acts as a spring which pulls the parameter
closer to the background
(depending on the “spring constant”
)
meanwhile getting as close a possible to the observations.
Depending on the matrix
, the computation may have
better regularity properties than the plain non linear least squares problem.
Non Linear Gaussian Calibration : 3DVAR¶
The cost function of the gaussian nonlinear calibration problem is :
for any .
The goal of non-linear Gaussian calibration is to find the
value of
which minimizes the cost function
.
In general, this involves using a nonlinear unconstrained optimization solver.
Let
be the Jacobian matrix made of the
partial derivatives of
with respect to
:
The Jacobian matrix of the cost function can be expressed
depending on the matrices
and
and the Jacobian matrix
of the function
:
for any .
The Hessian matrix of the cost function is:
for any .
If the covariance matrix is positive definite,
then the Hessian matrix of the cost function is positive definite.
Under this hypothesis, the solution of the nonlinear Gaussian calibration is unique.
Solving the Non Linear Gaussian Calibration Problem¶
The implementation of the resolution of the Gaussian non linear calibration
problem involves the Cholesky decomposition of the covariance matrices
and
.
This allows one to transform the sum of two Mahalanobis distances into a single
Euclidean norm.
This leads to a classical non linear least squares problem.
Linear Gaussian Calibration¶
In Gaussian linear calibration, we assume that the parametric
function is linear with respect to the parameters.
This method is known as the Kalman filter in data assimilation.
We make the hypothesis that is linear with respect to
,
i.e., for any
, we have:
where is the constant Jacobian matrix of
.
Let be the matrix:
We denote by the Kalman matrix:
The maximum of the posterior distribution of given the
observations
is:
It can be proved that:
for any .
This implies:
Bias of Linear Gaussian Calibration¶
Assume that there is a true value of the parameter
that generated the output observations
.
Consider the MAP estimator
as an
estimator of
.
This estimator is a random variable which depends on the random sample of
output observations.
The MAP estimator is biased and the bias is:
This bias is introduced as a consequence of the regularization properties of Gaussian calibration. Due to the trade-off between bias and variance of the estimator of the parameters, reducing the variance comes at the price of an increased bias.