# Least squares polynomial response surface¶

Instead of replacing the model response with a
*local* approximation around a given set of
input parameters as in Taylor expansion, one may seek a *global* approximation of
over its whole domain of definition. A common
choice to this end is global polynomial approximation. For the sake of
simplicity, a *scalar* model response will
be considered from now on. Nonetheless, the following derivations hold
for a vector-valued response.

In the following, one considers global approximations of the model response using:

a linear function, i.e. a polynomial of degree one;

where is a set of unknown coefficients.

a quadratic function, i.e. a polynomial of degree two.

The two previous equations may be recast as:

where denotes the number of terms, which is equal to (resp. to ) when using a linear (resp. a quadratic) approximation, and the family gathers the constant monomial , the monomials of degree one and possibly the cross-terms as well as the monomials of degree two . Using the vector notation and , this can be rewritten:

A *global* approximation of the model response over its whole
definition domain is sought. To this end, the coefficients
may be computed using a least squares regression approach. In this
context, an experimental design, that is, a set of observations of
input parameters, is required:

as well as the corresponding model evaluations:

The least squares problem is to solve:

where is the cost function, defined as:

Let be the vector of output observations. If the design matrix has full rank, then the solution is given by the normal equations:

(1)¶

where:

for and .
A necessary condition for having a solution is that the size
of the experimental design is not less than the number of
coefficients to estimate.
The Gram matrix can be
ill-conditionned.
Hence, the best method is not necessarily to invert the Gram matrix,
because the solution may be particularly sensitive to rounding errors.
The least-squares problem is rather solved using more robust numerical methods
such as the *singular value decomposition* (SVD) or the *QR-decomposition*.