DiscreteMarkovChain¶

class DiscreteMarkovChain(*args)¶

Discrete Markov chain process.

Parameters

originDistribution or $int$ , optional: Probability distribution of the Markov chain origin, i.e. state of the process at $t_0$ . By default, the origin is set to a Dirac distribution of value 0.0.
transitionMatrixSquareMatrix, optional: Transition matrix of the process. The matrix must be square, of dimension equal to the number of possible states of the process. By default, the transition matrix of the process is set to the 1x1 matrix [1].
timeGridTimeSeries, optional: The time grid of the process. By default, the time grid is reduced to one time stamp equal to 0.

Notes

A discrete Markov chain is a process $X: \Omega \times \cD \rightarrow E$ , where $\cD=\Rset$ discretized on the time grid $(t_i)_{i \geq 0}$ , and $E = [\![ 0,...,p-1]\!]$ is the space of states, such that:

$\forall n>0,\: \Prob ( X_{t_n} \> | \> X_{t_0},...X_{t_{n-1}} ) = \Prob ( X_{t_n} \> | \> X_{t_{n-1}} )$

The transition matrix of the process $\cM = (m_{i,j})$ can be defined such that:

$\forall t_n \in \cD, \forall i,j \in [\![ 0,...,p-1]\!], m_{i+1 , j+1} = \Prob (X_{t_{n+1}} = j \> | \> X_{t_{n}} = i)$

The transition matrix $\cM$ of the process is square, and its dimension $p$ is equal to the number of states of the process. Besides, $\cM$ is a stochastic matrix, i.e.:

$\forall i,j \leq p , \: m_{i,j} \geq 0 \forall i \leq p, \: \sum_{j=1}^{p}{m_{i,j}} = 1$

The origin of the process must be provided either as a deterministic value $x_0 \in [\![ 0,...,p-1]\!]$ , or as a probability distribution. In this case, the distribution of $X_0$ must be 1D, and its support must be a part of $[\![ 0,...,p-1]\!]$ .

Examples

Create a Markov chain:

>>> import openturns as ot
>>> timeGrid = ot.RegularGrid(0, 0.1, 10)
>>> transitionMatrix = ot.SquareMatrix([[0.9,0.05,0.05],[0.7,0.0,0.3],[0.8,0.0,0.2]])
>>> origin = 0
>>> myMarkovChain = ot.DiscreteMarkovChain(origin, transitionMatrix, timeGrid)

Get a realization:

>>> myReal = myMarkovChain.getRealization()

Methods

`computeStationaryDistribution`()	Compute the stationary distribution of the Markov chain.
`exportToDOTFile`(filename)	Export to DOT graph.
`getClassName`()	Accessor to the object’s name.
`getContinuousRealization`()	Get a continuous realization.
`getCovarianceModel`()	Accessor to the covariance model.
`getDescription`()	Get the description of the process.
`getFuture`(*args)	Prediction of the $N$ future iterations of the process.
`getId`()	Accessor to the object’s id.
`getInputDimension`()	Get the dimension of the domain $\cD$ .
`getMarginal`(*args)	Get the $k^{th}$ marginal of the random process.
`getMesh`()	Get the mesh.
`getName`()	Accessor to the object’s name.
`getOrigin`()	Accessor to the origin.
`getOutputDimension`()	Get the dimension of the domain $\cD$ .
`getRealization`()	Get a realization of the process.
`getSample`(size)	Get $n$ realizations of the process.
`getShadowedId`()	Accessor to the object’s shadowed id.
`getTimeGrid`()	Get the time grid of observation of the process.
`getTransitionMatrix`()	Accessor to the transition matrix.
`getTrend`()	Accessor to the trend.
`getVisibility`()	Accessor to the object’s visibility state.
`hasName`()	Test if the object is named.
`hasVisibleName`()	Test if the object has a distinguishable name.
`isComposite`()	Test whether the process is composite or not.
`isNormal`()	Test whether the process is normal or not.
`isStationary`()	Test whether the process is stationary or not.
`setDescription`(description)	Set the description of the process.
`setMesh`(mesh)	Set the mesh.
`setName`(name)	Accessor to the object’s name.
`setOrigin`(*args)	Accessor to the origin.
`setShadowedId`(id)	Accessor to the object’s shadowed id.
`setTimeGrid`(timeGrid)	Set the time grid of observation of the process.
`setTransitionMatrix`(transitionMatrix)	Accessor to the transition matrix.
`setVisibility`(visible)	Accessor to the object’s visibility state.

__init__(*args)¶: Initialize self. See help(type(self)) for accurate signature.

computeStationaryDistribution()¶

Compute the stationary distribution of the Markov chain.

Returns

distributionUserDefined: The stationary probability distribution of the Markov chain: its probability table is the left eigenvector of the transition matrix associated to the eigenvalue $1$ .

Examples

Compute the stationary distribution of a Markov chain:

>>> import openturns as ot
>>> timeGrid = ot.RegularGrid(0, 0.1, 10)
>>> transitionMatrix = ot.SquareMatrix([[0.9,0.05,0.05],[0.7,0.0,0.3],[0.8,0.0,0.2]])
>>> origin = 0
>>> myMarkovChain = ot.DiscreteMarkovChain(origin, transitionMatrix, timeGrid)
>>> distribution = myMarkovChain.computeStationaryDistribution()
>>> print(distribution)
UserDefined({x = [0], p = 0.333333}, {x = [1], p = 0.333333}, {x = [2], p = 0.333333})

exportToDOTFile(filename)¶

Export to DOT graph.

DOT is a graph description language.

Parameters

filenamestr: The name of the file to be written.

Notes

The graph can be customized using the following ResourceMap string entries:

DiscreteMarkovChain-DOTArcColor
DiscreteMarkovChain-DOTLayout
DiscreteMarkovChain-DOTNodeColor
DiscreteMarkovChain-DOTNodeShape

getClassName()¶

Accessor to the object’s name.

Returns

class_namestr: The object class name (object.__class__.__name__).

getContinuousRealization()¶

Get a continuous realization.

Returns

realizationFunction

According to the process, the continuous realizations are built:

either using a dedicated functional model if it exists: e.g. a functional basis process.
or using an interpolation from a discrete realization of the process on $\cM$ : in dimension $d=1$ , a linear interpolation and in dimension $d \geq 2$ , a piecewise constant function (the value at a given position is equal to the value at the nearest vertex of the mesh of the process).

getCovarianceModel()¶

Accessor to the covariance model.

Returns

cov_modelCovarianceModel: Covariance model, if any.

getDescription()¶

Get the description of the process.

Returns

descriptionDescription: Description of the process.

getFuture(*args)¶

Prediction of the $N$ future iterations of the process.

Parameters

stepNumberint, $N \geq 0$: Number of future steps.
sizeint, $size \geq 0$ , optional: Number of futures needed. Default is 1.

Returns

predictionProcessSample or TimeSeries: $N$ future iterations of the process. If $size = 1$ , prediction is a TimeSeries. Otherwise, it is a ProcessSample.

getId()¶

Accessor to the object’s id.

Returns

idint: Internal unique identifier.

getInputDimension()¶

Get the dimension of the domain $\cD$ .

Returns

nint: Dimension of the domain $\cD$ : $n$ .

getMarginal(*args)¶

Get the $k^{th}$ marginal of the random process.

Parameters

kint or list of ints $0 \leq k < d$: Index of the marginal(s) needed.

Returns

marginalsProcess: Process defined with marginal(s) of the random process.

getMesh()¶

Get the mesh.

Returns

meshMesh: Mesh over which the domain $\cD$ is discretized.

getName()¶

Accessor to the object’s name.

Returns

namestr: The name of the object.

getOrigin()¶

Accessor to the origin.

Returns

originDistribution: The probability distribution of the origin of the Markov chain.

getOutputDimension()¶

Get the dimension of the domain $\cD$ .

Returns

dint: Dimension of the domain $\cD$ .

getRealization()¶

Get a realization of the process.

Returns

realizationField: Contains a mesh over which the process is discretized and the values of the process at the vertices of the mesh.

getSample(size)¶

Get $n$ realizations of the process.

Parameters

nint, $n \geq 0$: Number of realizations of the process needed.

Returns

processSampleProcessSample: $n$ realizations of the random process. A process sample is a collection of fields which share the same mesh $\cM \in \Rset^n$ .

getShadowedId()¶

Accessor to the object’s shadowed id.

Returns

idint: Internal unique identifier.

getTimeGrid()¶

Get the time grid of observation of the process.

Returns

timeGridRegularGrid: Time grid of a process when the mesh associated to the process can be interpreted as a RegularGrid. We check if the vertices of the mesh are scalar and are regularly spaced in $\Rset$ but we don’t check if the connectivity of the mesh is conform to the one of a regular grid (without any hole and composed of ordered instants).

getTransitionMatrix()¶

Accessor to the transition matrix.

Returns

matrixSquareMatrix: The transition matrix of the process, of dimension $n$ .

getTrend()¶

Accessor to the trend.

Returns

trendTrendTransform: Trend, if any.

getVisibility()¶

Accessor to the object’s visibility state.

Returns

visiblebool: Visibility flag.

hasName()¶

Test if the object is named.

Returns

hasNamebool: True if the name is not empty.

hasVisibleName()¶

Test if the object has a distinguishable name.

Returns

hasVisibleNamebool: True if the name is not empty and not the default one.

isComposite()¶

Test whether the process is composite or not.

Returns

isCompositebool: True if the process is composite (built upon a function and a process).

isNormal()¶

Test whether the process is normal or not.

Returns

isNormalbool: True if the process is normal.

Notes

A stochastic process is normal if all its finite dimensional joint distributions are normal, which means that for all $k \in \Nset$ and $I_k \in \Nset^*$ , with $cardI_k=k$ , there is $\vect{m}_1, \dots, \vect{m}_k \in \Rset^d$ and $\mat{C}_{1,\dots,k}\in\mathcal{M}_{kd,kd}(\Rset)$ such that:

$\Expect{\exp\left\{i\Tr{\vect{X}}_{I_k} \vect{U}_{k} \right\}} = \exp{\left\{i\Tr{\vect{U}}_{k}\vect{M}_{k}-\frac{1}{2}\Tr{\vect{U}}_{k}\mat{C}_{1,\dots,k}\vect{U}_{k}\right\}}$

where $\Tr{\vect{X}}_{I_k} = (\Tr{X}_{\vect{t}_1}, \hdots, \Tr{X}_{\vect{t}_k})$ , $\\Tr{vect{U}}_{k} = (\Tr{\vect{u}}_{1}, \hdots, \Tr{\vect{u}}_{k})$ and $\Tr{\vect{M}}_{k} = (\Tr{\vect{m}}_{1}, \hdots, \Tr{\vect{m}}_{k})$ and $\mat{C}_{1,\dots,k}$ is the symmetric matrix:

$\mat{C}_{1,\dots,k} = \left( \begin{array}{cccc} C(\vect{t}_1, \vect{t}_1) &C(\vect{t}_1, \vect{t}_2) & \hdots & C(\vect{t}_1, \vect{t}_{k}) \\ \hdots & C(\vect{t}_2, \vect{t}_2) & \hdots & C(\vect{t}_2, \vect{t}_{k}) \\ \hdots & \hdots & \hdots & \hdots \\ \hdots & \hdots & \hdots & C(\vect{t}_{k}, \vect{t}_{k}) \end{array} \right)$

A Gaussian process is entirely defined by its mean function $m$ and its covariance function $C$ (or correlation function $R$ ).

isStationary()¶

Test whether the process is stationary or not.

Returns

isStationarybool: True if the process is stationary.

Notes

A process $X$ is stationary if its distribution is invariant by translation: $\forall k \in \Nset$ , $\forall (\vect{t}_1, \dots, \vect{t}_k) \in \cD$ , $\forall \vect{h}\in \Rset^n$ , we have:

$(X_{\vect{t}_1}, \dots, X_{\vect{t}_k}) \stackrel{\mathcal{D}}{=} (X_{\vect{t}_1+\vect{h}}, \dots, X_{\vect{t}_k+\vect{h}})$

setDescription(description)¶

Set the description of the process.

Parameters

descriptionsequence of str: Description of the process.

setMesh(mesh)¶

Set the mesh.

Parameters

meshMesh: Mesh over which the domain $\cD$ is discretized.

setName(name)¶

Accessor to the object’s name.

Parameters

namestr: The name of the object.

setOrigin(*args)¶

Accessor to the origin.

Parameters

originDistribution or $int$: The probability distribution of the origin of the Markov chain.

setShadowedId(id)¶

Accessor to the object’s shadowed id.

Parameters

idint: Internal unique identifier.

setTimeGrid(timeGrid)¶

Set the time grid of observation of the process.

Returns

timeGridRegularGrid: Time grid of observation of the process when the mesh associated to the process can be interpreted as a RegularGrid. We check if the vertices of the mesh are scalar and are regularly spaced in $\Rset$ but we don’t check if the connectivity of the mesh is conform to the one of a regular grid (without any hole and composed of ordered instants).

setTransitionMatrix(transitionMatrix)¶

Accessor to the transition matrix.

Parameters

matrixSquareMatrix: The transition matrix of the process, of dimension $n$ .

setVisibility(visible)¶

Accessor to the object’s visibility state.

Parameters

visiblebool: Visibility flag.

OpenTURNS

An Open source initiative for the Treatment of Uncertainties, Risks'N Statistics

Previous topic

Next topic

This Page

DiscreteMarkovChain¶