# Kernel smoothing¶

Kernel smoothing is a non parametric estimation method of the probability density function of a distribution.

In dimension 1, the kernel smoothed probability density function has the following expression, where K is the univariate kernel, n the sample size and the univariate random sample with :

(1)

The kernel K is a function satisfying . Usually K is chosen to be a unimodal probability density function that is symmetric about 0. The parameter h is called the bandwidth.

In dimension , the kernel may be defined as a product kernel , as follows where :

which leads to the kernel smoothed probability density function in dimension d, where is the d-variate random sample which components are denoted :

Let’s note that the bandwidth is the vector .

The quality of the approximation may be controlled by the AMISE (Asymptotic Mean Integrated Square error) criteria defined as:

The quality of the estimation essentially depends on the value of the bandwidth h. The bandwidth that minimizes the AMISE criteria has the expression (given in dimension 1):

(2)

where and .

If we note that with , then relation writes:

(3)

Several methods exist to evaluate the optimal bandwidth based on different approximations of :

• Silverman’s rule in dimension 1,

• the plug-in bandwidth selection,

• Scott’s rule in dimension d.

## Silverman’s rule (dimension 1)¶

In the case where the density p is normal with standard deviation , then the term can be exactly evaluated. In that particular case, the optimal bandwidth of relation (3) with respect to the AMISE criteria writes as follows:

(4)

An estimator of is obtained by replacing by its estimator , evaluated from the sample :

(5)

The Silverman rule consists in considering of relation (5) even if the density p is not normal:

(6)

Relation (6) is empirical and gives good results when the density is not far from a normal one.

## Plug-in bandwidth selection method (dimension 1)¶

The plug-in bandwidth selection method improves the estimation of the rugosity of the second derivative of the density. Instead of making the gaussian assumption, the method uses a kernel smoothing method in order to make an approximation of higher derivatives of the density.

Relation (3) requires the evaluation of the quantity . As a general rule, we use the estimator of defined by:

(7)

Deriving relation (1) leads to:

(8)

and then the estimator is defined as:

(9)

We note that depends of the parameter h which can be taken in order to minimize the AMSE (Asymptotic Mean Square Error) criteria evaluated between and . The optimal parameter h is:

(10)

Given that preliminary results, the solve-the-equation plug-in method proceeds as follows:

• Relation (3) defines as a function of we denote here as:

(11)

• The term is approximated by its estimator defined in (9) evaluated with its optimal parameter defined in (10):

(12)

which leads to a relation of type:

(13)

• Relations (3) and (12) lead to the new one:

(14)

which rewrites:

(15)

• Relation (14) depends on both terms and which are evaluated with their estimators defined in (9) respectively with their AMSE optimal parameters and (see relation (10)). It leads to the expressions:

(16)

• In order to evaluate and , we suppose that the density p is normal with a variance which is approximated by the empirical variance of the sample, which leads to:

(17)

Then, to summarize, thanks to relations (11), (13), (15), (16) and (17), the optimal bandwidth is solution of the equation:

(18)