5.1. Multivariate Probability Generating Functions

We now consider PGFs that depend on multiple variables

Let us assume that we have some distribution joint distribution of two non-negative integer random variables \(p_{j,k} = \mathbb{P}[X=j, Y=k]\). Then we define a two-variable PGF

\[ \mu(x,y) = \sum_{j=0}^\infty \sum_{k=0}^\infty p_{j,k} x^j y^k \]

We make a few observations. First, if we set \(x=y=1\), we get

\[ \mu(1,1) = \sum_{j} \sum_k p_{j,k} = 1 \]

Next, if we just set \(x=1\) or \(y=1\) we get the PGFs of the marginal distributions of \(k\) and \(j\) respectively

\[\begin{align*} \mu(1,y) &= \sum_{k} \left(\sum_j p_{j,k}\right) y^k = \sum_k \mathbb{P}[Y=k] y^k\\ \mu(x,1) &= \sum_j \mathbb{P}[X=j] x^j \end{align*}\]

We can find the expected values of \(j\) and \(k\) through differentiation:

\[\begin{align*} \left[\frac{\partial}{\partial x} \mu(x,y) \right|_{(x,y)=(1,1)} &= \sum_j \sum_k j p_{j,k}1^{j-1}1^k = \mathbb{E}[j]\\ \left[\frac{\partial}{\partial y} \mu(x,y) \right|_{(x,y)=(1,1)} &= \mathbb{E}[k] \end{align*}\]

and the expected value of the product is

\[ \left[\frac{\partial^2}{\partial x \partial y} \mu(x,y) \right]_{(x,y)=(1,1)} = \sum_j \sum_k jk p_{j,k}1^{j-1}1^{k-1} = \mathbb{E}[jk] \]

The expected value of the sum \(X+Y\) is found by taking \(y=x\) and taking \(\frac{d}{dx}\) of the result:

\[\begin{align*} \left[\frac{d}{dx} \mu(x,x) \right|_{x=1} &= \left[ \frac{d}{dx} \sum_j\sum_k p_{j,k} x^{j}x^k\right|_{x=1}\\ &= \left[\sum_{j} \sum_k p_{j,k} (j+k) x^{j+k-1} \right|_{x=1}\\ & = \sum_{j,k} p_{j,k} (j+k)\\ &= \mathbb{E}[X+Y] \end{align*}\]

These expressions generalize to joint distributions of more than two random variables.

5.1.1. Sums of random variables

We saw earlier that given a probability distribution of non-negative integers \(j\) with PGF \(\mu(x) = \sum_j p_j x^j\), the PGF for the sum of sum exactly \(k\) numbers \(\sum_{i=1}^k X_i\) chosen independently from that distribution is \(\mu(x)^k\). Further, if the number of terms \(k\) is itself a random variable, then the PGF for the “randomly-stopped sum” \(\sum_{i=1}^k X_i\) is \(\psi(\mu(x))\) where \(\psi(x)\) is the PGF for \(k\).

This generalizes to pairs of random non-negative integers. Let us assume that we have a PGF \(\mu(x,y)\) and we want to find the sum \(\sum_{i=1}^k (X_i,Y_i)\) we get

Theorem 5.1 (Composition of PGFs and randomly-stopped sums)

Given a PGF \(\mu(x,y)\) for the joint distribution of a pair of non-negative integers \((X,Y)\).

The PGF of the joint distribution of their sums \(\sum_{i=1}^\ell (X_i, Y_i)\) is \(\xi(x,y)^\ell\).

If \(\ell\) itself is a random variable with PGF \(\psi(x)\), then the PGF for the joint distribution of the sums is \(\psi(\xi(x,y))\).

This generalizes to the case where \(\psi\) is a function of two variables, corresponding to taking the sum of \(\ell\) pairs \((X_i,Y_i)\), \(i=1,\ldots, \ell\) from one distribution and another \(m\) pairs \((X_{i}, Y_{i})\), \(i=\ell+1,\ldots, \ell+m\) from another distribution where \(\ell\) and \(m\) are random values having some joint distribution.

Theorem 5.2 (Composition of PGFs and sums of randomly-stopped sums)

Given two joint distributions of non-negative integers \(p_{j,k}\) and \(q_{j,k}\) with PGFs \(\xi_1(x,y)\), \(\xi_2(x,y)\).

We take \(\ell\) pairs \((X_i,Y_i)\), \(i=1,\ldots, \ell\) from the first distribution and another \(m\) pairs \((X_{i}, Y_{i})\), \(i=\ell+1,\ldots, \ell+m\) from the second distribution. The PGF of the sum \(\sum_{i=1}^{\ell+m} (X_i,Y_i)\) of all of these pairs is \(\xi_1(x,y)^\ell \xi_2(x,y)^m\).

If in turn \(\ell\) and \(m\) are random variables whose joint distribution has PGF \(\psi(x,y)\), then the PGF for the randomly-stopped sum is \(\psi(\xi_1(x,y),\xi_2(x,y))\).