The Living Thing / Notebooks :

The Gaussian distribution

and Erf and normality and such

Stunts with Gaussian distributions.

Let’s start here with the basic thing. The (univariate) standard Gaussian pdf

\begin{equation*} \psi:x\mapsto \frac{1}{\sqrt{2\pi}}\text{exp}\left(-\frac{x^2}{2}\right) \end{equation*}

We define

\begin{equation*} \Psi:x\mapsto \int_{-\infty}^x\psi(t) dt \end{equation*}

Left tail of icdf

For small \(p\), the quantile function has the useful asymptotic expansion

\begin{equation*} \Phi^{-1}(p) = -\sqrt{\ln\frac{1}{p^2} - \ln\ln\frac{1}{p^2} - \ln(2\pi)} + \mathcal{o}(1). \end{equation*}

What is Erf again?

This erf function is popular, isn’t it? Unavoidable if you do computer algebra. But I can never remember what it is. There’s these two scaling factors tacked on.


\begin{equation*} \operatorname{erf}(x)\; =\; \frac{1}{\sqrt{\pi}} \int_{-x}^x e^{-t^2} \, dt \end{equation*}


\begin{equation*} \sqrt{\frac{\pi }{2}} \left(\text{erf}\left(\frac{x}{\sqrt{2}}\right)+1\right) \end{equation*}



Rational approximations


ODE representation for the univariate density

\begin{align*} \sigma ^2 f'(x)+f(x) (x-\mu )&=0\\ f(0) &=\frac{e^{-\mu ^2/(2\sigma ^2)}}{\sqrt{2 \sigma^2\pi } }\\ L(x) &=(\sigma^2 D+x-\mu) \end{align*}

TODO: note where I learned this.

ODE representation for the icdf

From StSh08 via Wikipedia.

\begin{equation*} \begin{aligned} {\frac {d^{2}w}{dp^{2}}} &=w\left({\frac {dw}{dp}}\right)^{2}\\ w\left(1/2\right)&=0,\\ w'\left(1/2\right)&={\sqrt {2\pi }}. \end{aligned} \end{equation*}

Density PDE representation as a diffusion equation

(see, e.g. BoGK10)

\begin{align*} \frac{\partial}{\partial t}f(x;t) &=\frac{1}{2}\frac{\partial^2}{\partial x^2}f(x;t)\\ f(x;0)&=\delta(x-\mu) \end{align*}

Look, it’s the diffusion equation of Wiener process. Surprise.


Univariate -

\begin{align*} \left\| \frac{d}{dx}\phi_\sigma \right\|_2 &= \frac{1}{4\sqrt{\pi}\sigma^3}\\ \left\| \left(\frac{d}{dx}\right)^n \phi_\sigma \right\|_2 &= \frac{\prod_{i<n}2n-1}{2^{n+1}\sqrt{\pi}\sigma^{2n+1}} \end{align*}

Multidimensional marginals

As made famous by Wiener processes in finance and Gaussian processes in Bayesian nonparametrics.

See, e.g. these lectures, or Michael I Jordan’s backgrounders.

Transformed variables

\begin{equation*} Y \sim N(X\beta, I) \end{equation*}


\begin{equation*} W^{1/2}Y \sim N(W^{1/2}X\beta, W) \end{equation*}


Botev, Z. I.(2017) The Normal Law Under Linear Restrictions: Simulation and Estimation via Minimax Tilting. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 79(1), 125–148. DOI.
Botev, Z. I., Grotowski, J. F., & Kroese, D. P.(2010) Kernel density estimation via diffusion. The Annals of Statistics, 38(5), 2916–2957. DOI.
Steinbrecher, G., & Shaw, W. T.(2008) Quantile mechanics. European Journal of Applied Mathematics, 19(2), 87–112. DOI.
Wichura, M. J.(1988) Algorithm AS 241: The Percentage Points of the Normal Distribution. Journal of the Royal Statistical Society. Series C (Applied Statistics), 37(3), 477–484. DOI.