Let's have fun on probability theory, here is my first problem set in the said subject.
Problems
-
It was noted that statisticians who follow the deFinetti school do not accept the Axiom of Countable Additivity, instead adhering to the Axiom of Finite Additivity.
- Show that the Axiom of Countable Additivity implies Finite Additivity.
- Although, by itself, the Axiom of Finite Additivity does not imply Countable Additivity, suppose we supplement it with the following. Let $A_1\supset A_2\supset\cdots\supset A_n\supset \cdots$ be an infinite sequence of nested sets whose limit is the empty set, which we denote by $A_n\downarrow\emptyset$. Consider the following:
Axiom of Continuity: If $A_n\downarrow\emptyset$, then $P(A_n)\rightarrow 0$
Prove that the Axiom of Continuity and the Axiom of Finite Additivity imply Countable Additivity.
- Prove each of the following statements. (Assume that any conditioning event has positive probability.)
- If $P(B)=1$, then $P(A|B)=P(A)$ for any $A$.
- If $A\subset B$, then $P(B|A)=1$ and $P(A|B)=P(A)/P(B)$.
- If $A$ and $B$ are mutually exclusive, then \begin{equation}\nonumber P(A|A\cup B) = \displaystyle\frac{P(A)}{P(A)+P(B)}. \end{equation}
- $P(A\cap B\cap C)=P(A|B\cap C)P(B|C)P(C)$.
- Prove that the following functions are cdfs.
- $\frac{1}{2}+\frac{1}{\pi}\arctan(x), x\in (-\infty, \infty)$
- $(1+e^{-x})^{-1},x\in (-\infty,\infty)$
- $e^{-e^{-x}}, x\in (-\infty, \infty)$
- $1-e^{-x}, x\in (0,\infty)$
- the function defined in (1.5.6), (Check in the reference below.)
- A cdf $F_X$ is stochastically greater than a cdf $F_{Y}$ if $F_{X}(t)\leq F_{Y}(t)$ for all $t$ and $F_{X}(t) < F_{Y}(t)$ for some $t$. Prove that if $X\sim F_X$ and $Y\sim F_Y$, then \begin{equation}\nonumber P(X>t) \geq P(Y>t)\;\text{for every}\;t \end{equation} and \begin{equation}\nonumber P(X>t)>P(Y>t),\;\text{for some}\; t \end{equation} that is, $X$ tends to be bigger than $Y$.
- Let $X$ be a continuous random variable with pdf $f(x)$ and cdf $F(x)$. For a fixed number $x_0$, define the function \begin{equation}\nonumber g(x) = \begin{cases} f(x) / [1-F(x_0)]& x \geq x_0\\ 0 & x < x_0. \end{cases} \end{equation} Prove that $g(x)$ is a pdf. (Assume that $F(x_0)<1$.)
- For each of the following, determine the value of $c$ that makes $f(x)$ a pdf.
- $f(x)=\mathrm{c}\sin x, 0 < x < \pi/2$
- $f(x)=\mathrm{c}e^{-|x|},-\infty < x < \infty$
Solutions
-
- Proof. Let $\mathscr{B}$ be a $\sigma$-algebra and consider $A_1,A_2,\cdots\in \mathscr{B}$ are pairwise disjoint, then by countable additivity
\begin{equation}\nonumber
P\left(\displaystyle\bigcup_{i=1}^{\infty}A_i\right)=\displaystyle\sum_{i=1}^{\infty}P(A_i).
\end{equation}
Now,
\begin{equation}
\begin{aligned}
P\left(\displaystyle\bigcup_{i=1}^{\infty}A_i\right)&=
P\left(\displaystyle\bigcup_{i=1}^{n}A_i\cup\displaystyle
\bigcup_{i=n+1}^{\infty}A_i\right)\\
&=
P\left(\displaystyle\bigcup_{i=1}^{n}A_i\right)+P\left(\displaystyle
\bigcup_{i=n+1}^{\infty}A_i\right),\;(\text{since}\;A_i's\;\text{are disjoints})\\
&=P(A_1)+\cdots+P(A_n)+P\left(\displaystyle
\bigcup_{i=n+1}^{\infty}A_i\right),\\
&\quad(\text{by finite additivity})\\
&=\displaystyle\sum_{i=1}^{n}P(A_i)+P\left(\displaystyle
\bigcup_{i=n+1}^{\infty}A_i\right)
\end{aligned}\nonumber
\end{equation}
Notice that for any $n$, we can consider $P(A_i),\;i>n$ to be empty. Implying
\begin{equation}\nonumber
P\left(\displaystyle\bigcup_{i=n+1}^{\infty}A_i\right)=\displaystyle
\sum_{i=n+1}^{\infty}P(A_i)=P(\emptyset)+P(\emptyset)+\cdots,
\end{equation}
that is,
\begin{equation}\nonumber
\begin{aligned}
P\left(\displaystyle\bigcup_{i=1}^{\infty}A_i\right)&=
\displaystyle\sum_{i=1}^{n}P(A_i)+\sum_{i=n+1}^{\infty}P(A_i)\\
&=\displaystyle\sum_{i=1}^{n}P(A_i)+P(\emptyset)+P(\emptyset)+\cdots
\end{aligned}
\end{equation}
$\therefore$ countable additivity implies finite additivity.
$\hspace{12.5cm}\blacksquare$ - From (a), we have shown that countable additivity implies finite additivity, i.e.,
\begin{equation}
P\left(\displaystyle\bigcup_{i=1}^{\infty}A_i\right)=\displaystyle\sum_{i=1}^{n}P(A_i)+P\left(\displaystyle
\bigcup_{i=n+1}^{\infty}A_i\right)
\nonumber
\end{equation}
If we supplement this with the following condition, that $A_1\supset A_2\supset A_3\supset\cdots$. By Axiom of Continuity, $\displaystyle\lim_{n\to \infty}A_n=\emptyset$, and by Monotone Sequential Continuity, $P\left(\displaystyle\lim_{n\to\infty}A_n\right)=
\displaystyle\lim_{n\to\infty}P(A_n)=0$.
Now we can write $A_1\supset A_2\supset A_3\supset\cdots$ as
\begin{equation}\nonumber
B_k=\bigcup_{i=k}^{\infty}A_i,\;\text{such that}\;B_{k+1}\subset B_k, \text{implying}\; \lim_{k\to\infty}B_k=\emptyset
\end{equation}
Thus, finite additivity plus axiom of continuity, we have
\begin{equation}\nonumber
\begin{aligned}
P\left(\bigcup_{i=1}^{\infty}A_i\right)&=\lim_{n\to\infty}\left(
\sum_{i=1}^{n}P(A_i)+P(B_{n+1})\right)\\
&=\lim_{n\to\infty}\left(\sum_{i=1}^{n}P(A_i)\right)+\lim_{n\to\infty}
P(B_{n+1})\\
&=\sum_{i=1}^{\infty}P(A_i)+0,\;(\text{by axiom of continuity}).
\end{aligned}
\end{equation}
Implying countable additivity.
$\hspace{12.5cm}\blacksquare$
- Proof. Let $\mathscr{B}$ be a $\sigma$-algebra and consider $A_1,A_2,\cdots\in \mathscr{B}$ are pairwise disjoint, then by countable additivity
\begin{equation}\nonumber
P\left(\displaystyle\bigcup_{i=1}^{\infty}A_i\right)=\displaystyle\sum_{i=1}^{\infty}P(A_i).
\end{equation}
Now,
\begin{equation}
\begin{aligned}
P\left(\displaystyle\bigcup_{i=1}^{\infty}A_i\right)&=
P\left(\displaystyle\bigcup_{i=1}^{n}A_i\cup\displaystyle
\bigcup_{i=n+1}^{\infty}A_i\right)\\
&=
P\left(\displaystyle\bigcup_{i=1}^{n}A_i\right)+P\left(\displaystyle
\bigcup_{i=n+1}^{\infty}A_i\right),\;(\text{since}\;A_i's\;\text{are disjoints})\\
&=P(A_1)+\cdots+P(A_n)+P\left(\displaystyle
\bigcup_{i=n+1}^{\infty}A_i\right),\\
&\quad(\text{by finite additivity})\\
&=\displaystyle\sum_{i=1}^{n}P(A_i)+P\left(\displaystyle
\bigcup_{i=n+1}^{\infty}A_i\right)
\end{aligned}\nonumber
\end{equation}
Notice that for any $n$, we can consider $P(A_i),\;i>n$ to be empty. Implying
\begin{equation}\nonumber
P\left(\displaystyle\bigcup_{i=n+1}^{\infty}A_i\right)=\displaystyle
\sum_{i=n+1}^{\infty}P(A_i)=P(\emptyset)+P(\emptyset)+\cdots,
\end{equation}
that is,
\begin{equation}\nonumber
\begin{aligned}
P\left(\displaystyle\bigcup_{i=1}^{\infty}A_i\right)&=
\displaystyle\sum_{i=1}^{n}P(A_i)+\sum_{i=n+1}^{\infty}P(A_i)\\
&=\displaystyle\sum_{i=1}^{n}P(A_i)+P(\emptyset)+P(\emptyset)+\cdots
\end{aligned}
\end{equation}
$\therefore$ countable additivity implies finite additivity.
-
- Proof. If $P(B)=1$, then $P(S)=P(B)=1$. Because $A\subseteq S$, implies $A\subseteq B$. Thus, $A\cap B = A$, and therefore \begin{equation}\nonumber P(A|B)=\displaystyle\frac{P(A\cap B)}{P(B)}=\displaystyle\frac{P(A)}{P(B)}=P(A) \end{equation} $\hspace{12.5cm}\blacksquare$
- Proof. If $A\subseteq B$ then \begin{equation}\nonumber P(B|A)=\displaystyle\frac{P(A\cap B)}{P(A)}=\displaystyle\frac{P(A)}{P(A)}=1 \end{equation} and, \begin{equation}\nonumber P(A|B)=\displaystyle\frac{P(A\cap B)}{P(B)}=\displaystyle\frac{P(A)}{P(B)} \end{equation} $\hspace{12.5cm}\blacksquare$
- Proof. If $A$ and $B$ are mutually exclusive, then \begin{equation} \nonumber \begin{aligned} P(A|A\cup B)&=\displaystyle\frac{P(A\cap (A\cup B))}{P(A\cup B)}\\ &=\displaystyle\frac{P(A)\cup [P(A\cap B)]}{P(A)+ P(B)}\\ &=\displaystyle\frac{P(A)}{P(A)+ P(B)} \end{aligned} \end{equation}$\hspace{12.5cm}\blacksquare$
- Proof. Consider, \begin{equation}\nonumber P(A|B\cap C)=\displaystyle\frac{P(A\cap B\cap C)}{P(B\cap C)} \end{equation} Hence, \begin{equation}\nonumber P(A\cap B\cap C) = P(A|B\cap C)P(B\cap C) \end{equation} Now $P(B\cap C)=P(B|C)P(C)$, therefore \begin{equation}\nonumber P(A\cap B\cap C) = P(A|B\cap C)P(B|C)P(C) \end{equation}$\hspace{12.5cm}\blacksquare$
- $F(x)$ is a cdf if it satisfies the following conditions:
- $\displaystyle\lim_{x\to-\infty}F(x)=0$ and $\displaystyle\lim_{x\to\infty}F(x)=1$
- $F(x)$ is nondecreasing.
- $F(x)$ is right-continuous.
- Proof.
- $F(x)=\frac{1}{2}+\frac{1}{\pi}\arctan(x), x\in (-\infty, \infty)$
\begin{equation}\nonumber \begin{aligned} \displaystyle\lim_{x\to-\infty}F(x)&=\displaystyle\lim_{x\to-\infty} \left(\frac{1}{2}+\frac{1}{\pi}\arctan(x)\right)\\ &=\frac{1}{2}+\frac{1}{\pi}\displaystyle\lim_{x\to-\infty}\left(\arctan(x)\right)\\ &=\frac{1}{2}+\frac{1}{\pi} \left(\frac{-\pi}{2}\right),\;\text{since}\;\displaystyle\lim_{x\to-\frac{\pi}{2}}\frac{\sin(x)}{\cos(x)}=-\infty\\ &=0\\[0.5cm] \displaystyle\lim_{x\to\infty}F(x)&=\displaystyle\lim_{x\to\infty} \left(\frac{1}{2}+\frac{1}{\pi}\arctan(x)\right)\\ &=\frac{1}{2}+\frac{1}{\pi}\displaystyle\lim_{x\to\infty}\left(\arctan(x)\right)\\ &=\frac{1}{2}+\frac{1}{\pi} \left(\frac{\pi}{2}\right),\;\text{since}\;\displaystyle\lim_{x\to\frac{\pi}{2}}\frac{\sin(x)}{\cos(x)}=\infty\\ &=1 \end{aligned} \end{equation} - To test if $F(x)$ is nondecreasing, recall in Calculus that, first differentiation of the function tells us if it is decreasing or increasing. In particular, $\frac{dF(x)}{dx}>0$ tells us that the function is increasing in a given interval of $x$. Thus,
\begin{equation}
\nonumber
\frac{dF(x)}{dx}=\frac{d}{dx}\left(\frac{1}{2}+\frac{1}{\pi}\arctan(x)\right)=\frac{1}{\pi(1+x^2)}
\end{equation}
Confirm the above differentiation with Python using sympy module.
Since $x^2$ is always positive for all $x$, thus $\frac{dF(x)}{dx}>0$, implying $F(x)$ is increasing. - $F(x)$ is continuous, implies that $F(x)$ is right-continuous.
- $F(x)=\frac{1}{2}+\frac{1}{\pi}\arctan(x), x\in (-\infty, \infty)$
- Proof.
- $
F(x)=\displaystyle\frac{1}{1+e^{-x}}, x\in(-\infty,\infty)
$
- Using the same method we did in (a), we have \begin{equation} \nonumber \begin{aligned} \frac{dF(x)}{dx}&=\frac{d}{dx}\left(\displaystyle\frac{1}{1+e^{-x}}\right)\\ &=\frac{e^{-x}}{(1+e^{-x})^2} \end{aligned} \end{equation} $\frac{dF(x)}{dx}=\frac{e^{-x}}{(1+e^{-x})^2}>0,\;\forall\;x\in(-\infty,\infty)$. Thus the function is increasing in the interval of $x$.
- $F(x)$ is continuous, implies the function is right-continuous.
- $
F(x)=\displaystyle\frac{1}{1+e^{-x}}, x\in(-\infty,\infty)
$
- Proof.
- $F(x)=e^{-e^{-x}}, x\in (-\infty, \infty)$
- Like what we did above, $\frac{dF(x)}{dx}$ is, \begin{equation} \nonumber \frac{dF(x)}{dx}=\frac{d}{dx}\left(e^{-e^{-x}}\right)=e^{-x}e^{-e^{-x}}>0 \end{equation} Because $e^{-x}e^{-e^{-x}}>0,\;\forall\; x\in(-\infty,\infty)$. Then we say $F(x)$ is an increasing function in the interval of $x$.
- $F(x)$ is continuous, implies that $F(x)$ is right-continuous.
- $F(x)=e^{-e^{-x}}, x\in (-\infty, \infty)$
- Proof.
- $F(x)=1-\displaystyle\frac{1}{e^{x}}, x\in(0,\infty)$
- \begin{equation}\nonumber \frac{dF(x)}{dx}=\frac{d}{dx}\left(1-\frac{1}{e^{x}}\right)=0-(-e^{-x})=\frac{1}{e^{x}} \end{equation} $F(x)$ is an increasing function since $\frac{1}{e^{x}}>0,\;\forall\;x\in(0,\infty)$.
- $F(x)$ is right-continuous, since it is continuous.
- $F(x)=1-\displaystyle\frac{1}{e^{x}}, x\in(0,\infty)$
- Proof.
The function in Equation (1.5.6) is given by,
\begin{equation}
F_Y(y)=\begin{cases}
\displaystyle\frac{1-\varepsilon}{1+e^{-y}}&\text{if}\;y<0,\; \text{for some}\;\varepsilon, 1>\varepsilon>0\\
\varepsilon+\displaystyle\frac{1-\varepsilon}{1+e^{-y}}&\text{if}\;y\geq 0,\;\text{for some}\;\varepsilon, 1>\varepsilon>0
\end{cases}\nonumber
\end{equation}
- \begin{equation}\nonumber \begin{aligned} \displaystyle\lim_{y\to-\infty}F_Y(y)&=\displaystyle\lim_{y\to-\infty} \left(\displaystyle\frac{1-\varepsilon}{1+e^{-y}}\right)=\displaystyle\lim_{y\to-\infty} \left(\displaystyle\frac{1-\varepsilon}{1+\frac{1}{e^{y}}}\right)=0\\[0.5cm] \displaystyle\lim_{y\to\infty}F(y)&=\displaystyle\lim_{y\to\infty} \left(\varepsilon+\displaystyle\frac{1-\varepsilon}{1+e^{-y}}\right)=\varepsilon + \displaystyle\lim_{y\to\infty} \left(\displaystyle\frac{1-\varepsilon}{1+\frac{1}{e^{y}}}\right)=1 \end{aligned} \end{equation}
- For $y<0$, we have
\begin{equation}
\begin{aligned}
\frac{d}{dy}\left(\frac{1-\varepsilon}{1+e^{-y}}\right)&=(1-\varepsilon)\frac{d}{dy}\left(\frac{1}{1+e^{-y}}\right)\\
&=(1-\varepsilon)\frac{(1+\varepsilon^{-y})\cdot 0 - 1\cdot e^{-y}(-1)}{(1+e^{-y})^2}\\
&=\frac{(1-\varepsilon)e^{-y}}{(1+e^{-y})^2}
\end{aligned}\nonumber
\end{equation}
$(1-\varepsilon)>0$ since $0<\varepsilon<1$. Thus for all $y < 0$, $\frac{(1-\varepsilon)e^{-y}}{(1+e^{-y})^2}>0$, implying that the function is increasing.
For $y\geq 0$, \begin{equation} \begin{aligned} \frac{d}{dy}\left(\varepsilon+\frac{1-\varepsilon}{1+e^{-y}}\right)&=\varepsilon+\frac{(1-\varepsilon)e^{-y}}{(1+e^{-y})^2} \end{aligned}\nonumber \end{equation} The function is increasing since $\varepsilon + \frac{(1-\varepsilon)e^{-y}}{(1+e^{-y})^2}>0$ for all $y\geq 0$. - Since the function is continuous, then the function is right-continuous.
- Proof. We know that,
\begin{equation}\nonumber
P(X>t)=1-P(X\leq t)=1-F_X(t)
\end{equation}
and
\begin{equation}\nonumber
P(Y>t)=1-P(Y\leq t)=1-F_Y(t)
\end{equation}
Hence we have,
\begin{equation}\nonumber
\begin{aligned}
P(X>t)=1-F_X(t)\;&\overset{?}{\geq}\;1-F_Y(t)=P(Y>t)\\
\end{aligned}
\end{equation}
Since $F_X(t)\leq F_Y(t)$, then the difference $1-F_X(t)$ tends to get bigger than $1-F_Y(t)$. Thus for all $t$, $P(X>t)\geq P(X>t)$.
Now if $F_X(t) < F_Y(t)$ for some $t$, then using the same argument above, $P(X>t) > P(X>t)$ for some $t$.
$\hspace{13.5cm}\blacksquare$ - Proof. For a function to be a pdf, it has to satisfy the following:
- $g(x)\geq 0$ for all $x$; and,
- $\displaystyle\int_{-\infty}^{\infty}g(x)\,dx=1$.
- In order for $f(x)$ to be a pdf, it has to integrate to 1.
- \begin{equation}
\begin{aligned}
\int_{-\infty}^{\infty}f(x)&=\int_{0}^{\frac{\pi}{2}}\mathrm{c}\sin x=\displaystyle\left.-(\mathrm{c})\cos x\displaystyle\right\rvert_{0}^{\frac{\pi}{2}}\\
&=-\mathrm{c}\left(\cos\left(\frac{\pi}{2}\right)-\cos(0)\right)\\
&=-\mathrm{c}(0-1)=1\mathrm{c}
\end{aligned}\nonumber
\end{equation}
Hence, $\mathrm{c}$ is 1. Confirm this with python,
-
\begin{equation}
\begin{aligned}
\int_{-\infty}^{\infty}f(x)&=\int_{-\infty}^{\infty}
\mathrm{c}\,e^{-|x|}\\
&=\mathrm{c}\left(\int_{-\infty}^{0}
\,e^{x}\,dx+\int_{0}^{\infty}
e^{-x}\,dx\right)\\
&=\mathrm{c}\left[(e^{0}-e^{-\infty})-(e^{-\infty}-e^{0})\right]\\
&=\mathrm{c}(1+1) = 2\mathrm{c}
\end{aligned}\nonumber
\end{equation}
Hence, c is $\frac{1}{2}$. Confirm this with Python,
- \begin{equation}
\begin{aligned}
\int_{-\infty}^{\infty}f(x)&=\int_{0}^{\frac{\pi}{2}}\mathrm{c}\sin x=\displaystyle\left.-(\mathrm{c})\cos x\displaystyle\right\rvert_{0}^{\frac{\pi}{2}}\\
&=-\mathrm{c}\left(\cos\left(\frac{\pi}{2}\right)-\cos(0)\right)\\
&=-\mathrm{c}(0-1)=1\mathrm{c}
\end{aligned}\nonumber
\end{equation}
Hence, $\mathrm{c}$ is 1. Confirm this with python,
Comments
Post a Comment