Markov kernel

In probability theory, a Markov kernel (or stochastic kernel) is a map that plays the role, in the general theory of Markov processes, that the transition matrix does in the theory of Markov processes with a finite state space.

Formal definition

Let $(X,\mathcal A)$, $(Y,\mathcal B)$ be measurable spaces. A Markov kernel with source $(X,\mathcal A)$ and target $(Y,\mathcal B)$ is a map $\kappa \colon X \times \mathcal B \to [0,1]$ with the following properties:

1. The map $x \mapsto \kappa(x,B)$ is $\mathcal A$ - measureable for every $B \in \mathcal B$.
2. The map $B \mapsto \kappa(x,B)$ is a probability measure on $(Y, \mathcal B)$ for every $x \in X$.

(i.e. It associates to each point $x \in X$ a probability measure $\kappa(x,.)$ on $(Y,\mathcal B)$ such that, for every measurable set $B\in\mathcal B$, the map $x\mapsto \kappa(x,B)$ is measurable with respect to the $\sigma$-algebra $\mathcal A$.)

Examples

• Simple random walk: Take $X=Y=\Z$ and $\mathcal A = \mathcal B = \mathcal P(\Z)$, then the Markov kernel $\kappa$ with

$\kappa(x,B)=\frac{1}{2}\mathbf{1}_{B}(x-1)+\frac{1}{2}\mathbf{1}_{B}(x+1), \quad \forall x \in \Z, \quad \forall B \in \mathcal P(\Z)$,

describes the transition rule for the random walk on $\Z$. Where $\mathbf{1}$ is the indicator function.

• Galton-Watson process: Take $X=Y=\N$, $\mathcal A = \mathcal B = \mathcal P(\N)$, then

$\kappa(x,B)=\begin{cases} \mathbf{1}_B(0) & \quad x=0,\\ P[\xi_1 + \dots + \xi_x \in B] & \quad \text{else,}\\ \end{cases}$

with i.i.d. random variables $\xi_i$.

• General Markov processes with finite state space: Take $X=Y$, $\mathcal A = \mathcal B = \mathcal P(X) = \mathcal P(Y)$ and $|X|=|Y|=n$, then the transition rule can be represented as a stochastic matrix $(K_{ij})_{1 \leq i,j \leq n}$ with $\Sigma_{j \in Y}K_{ij}=1$ for every $i \in X$. In the convention of Markov kernels we write

$\kappa(i,B)=\Sigma_{j \in B}K_{ij}, \quad \forall i \in X, \quad \forall B \in \mathcal B$.

• Construction of a Markov kernel: If $\nu$ is a finite measure on $(Y, \mathcal B)$ and $k:X\times Y \to \mathbb{R}_{+}$ is a measurable function with respect to the product $\sigma$-algebra $\mathcal A \otimes \mathcal B$ and has the property

$\int_X k(x,y)\nu(\mathrm{d} y) = 1,$

for all $x\in X$, then the mapping $\kappa:X\times \mathcal B \to [0,1]$

$\kappa(x,B)=\int_{B}k(x,y)\nu(\mathrm{d} y),$

defines a Markov kernel.

Properties

Semidirect product

Let $(X, \mathcal A, P)$ be a probability space and $\kappa$ a Markov kernel from $(X, \mathcal A)$ to some $(Y, \mathcal B)$.

Then there exists a unique measure $Q$ on $(X \times Y, \mathcal A \otimes \mathcal B)$, such that

$Q(A \times B) = \int_A \kappa(x,B)dP(x), \quad \forall A \in \mathcal A, \quad \forall B \in \mathcal B$.

Regular conditional distribution

Let $(S,Y)$ be a Borel space, $X$ a $(S,Y)$ - valued random variable on the measure space $(\Omega, \mathcal F,P)$ and $\mathcal G \subseteq \mathcal F$ a sub-$\sigma$-algebra.

Then there exists a Markov kernel $\kappa$ from $(\Omega, \mathcal G)$ to $(S,Y)$, such that $\kappa(.,B)$ is a version of the conditional expectation $E[\mathbf 1_{\{X \in B\}}| \mathcal G]$ for every $B \in Y$, i.e.

$P[X \in B|\mathcal G]=E[\mathbf 1_{\{X \in B\}}|\mathcal G]=\kappa(\omega,B), \quad P-a.s. \forall B \in \mathcal G$.

It is called regular conditional distribution of $X$ given $\mathcal G$ and is not uniquely defined.

References

§36. Kernels and semigroups of kernels