# Compare samples using Komogorov-Smirnov test, QQ-plot¶

In this example we are going to estimate whether two samples follow the same distribution using the two samples Kolmogorov-Smirnov test and the graphical QQ-plot test.

The Smirnov test relies on the maximum distance between the cumulative distribution function. If and are the empirical cumulative density functions of both samples of size and , the Smirnov test evaluates the decision variable: which tends towards the Kolmogorov distribution. The hypothesis of same distribution is rejected if is too high (depending on the p-value threshold).

The QQ-plot graph plots empirical quantiles levels from two samples. If both samples correspond to the same probability distribution the curve should be close to the diagonal.

:

from __future__ import print_function
import openturns as ot

:

# Generate 3 samples, sample1 and sample2 arise from the same distribution
distribution1 = ot.Gumbel(0.2, 0.5)
distribution2 = ot.Uniform()

ot.RandomGenerator.SetSeed(5)
sample1 = distribution1.getSample(100)
sample2 = distribution1.getSample(100)
sample3 = distribution2.getSample(100)

:

# Visually compare sample1 and sample2 using QQ-plot
ot.VisualTest.DrawQQplot(sample1, sample2)

: :

# Visually compare sample1 and sample3 using QQ-plot
ot.VisualTest.DrawQQplot(sample1, sample3)

: :

# Numerically test sample1 against sample2
test_result = ot.HypothesisTest.TwoSamplesKolmogorov(sample1, sample2)
print('Samples follow the same distribution?', test_result.getBinaryQualityMeasure(),
'p-value=%.6g' % test_result.getPValue(),
'threshold=%.6g' % test_result.getThreshold())

Samples follow the same distribution? True p-value=0.190264 threshold=0.05

:

# Numerically test sample1 against sample3
test_result = ot.HypothesisTest.TwoSamplesKolmogorov(sample1, sample3)
print('Samples follow the same distribution?', test_result.getBinaryQualityMeasure(),
'p-value=%.6g' % test_result.getPValue(),
'threshold=%.6g' % test_result.getThreshold())

Samples follow the same distribution? False p-value=6.85623e-16 threshold=0.05