Least squares polynomial response surface¶
Instead of replacing the model response with a local approximation around a given set of input parameters as in Taylor expansion, one may seek a global approximation of over its whole domain of definition. A common choice to this end is global polynomial approximation. For the sake of simplicity, a scalar model response will be considered from now on. Nonetheless, the following derivations hold for a vector-valued response.
In the following, one considers global approximations of the model response using:
a linear function, i.e. a polynomial of degree one;
where is a set of unknown coefficients.
a quadratic function, i.e. a polynomial of degree two.
The two previous equations may be recast as:
where denotes the number of terms, which is equal to (resp. to ) when using a linear (resp. a quadratic) approximation, and the family gathers the constant monomial , the monomials of degree one and possibly the cross-terms as well as the monomials of degree two . Using the vector notation and , this can be rewritten:
A global approximation of the model response over its whole definition domain is sought. To this end, the coefficients may be computed using a least squares regression approach. In this context, an experimental design, that is, a set of observations of input parameters, is required:
as well as the corresponding model evaluations:
The least squares problem is to solve:
where is the cost function, defined as:
Let be the vector of output observations. If the design matrix has full rank, then the solution is given by the normal equations:
(1)¶
where:
for and . A necessary condition for having a solution is that the size of the experimental design is not less than the number of coefficients to estimate. The Gram matrix can be ill-conditionned. Hence, the best method is not necessarily to invert the Gram matrix, because the solution may be particularly sensitive to rounding errors. The least-squares problem is rather solved using more robust numerical methods such as the singular value decomposition (SVD) or the QR-decomposition.