Propagation of uncertainty

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Propagation_of_uncertainty

In statistics, propagation of uncertainty (or propagation of error) is the effect of variables' uncertainties (or errors, more specifically random errors) on the uncertainty of a function based on them. When the variables are the values of experimental measurements they have uncertainties due to measurement limitations (e.g., instrument precision) which propagate due to the combination of variables in the function.

The uncertainty u can be expressed in a number of ways. It may be defined by the absolute error Δx. Uncertainties can also be defined by the relative error (Δx)/x, which is usually written as a percentage. Most commonly, the uncertainty on a quantity is quantified in terms of the standard deviation, σ, which is the positive square root of the variance. The value of a quantity and its error are then expressed as an interval x ± u. However, the most general way of characterizing uncertainty is by specifying its probability distribution. If the probability distribution of the variable is known or can be assumed, in theory it is possible to get any of its statistics. In particular, it is possible to derive confidence limits to describe the region within which the true value of the variable may be found. For example, the 68% confidence limits for a one-dimensional variable belonging to a normal distribution are approximately ± one standard deviation σ from the central value x, which means that the region x ± σ will cover the true value in roughly 68% of cases.

If the uncertainties are correlated then covariance must be taken into account. Correlation can arise from two different sources. First, the measurement errors may be correlated. Second, when the underlying values are correlated across a population, the uncertainties in the group averages will be correlated.

In a general context where a nonlinear function modifies the uncertain parameters (correlated or not), the standard tools to propagate uncertainty, and infer resulting quantity probability distribution/statistics, are sampling techniques from the Monte Carlo method family. For very expansive data or complex functions, the calculation of the error propagation may be very expansive so that a surrogate model or a parallel computing strategy may be necessary.

In some particular cases, the uncertainty propagation calculation can be done through simplistic algebraic procedures. Some of these scenarios are described below.

Linear combinations

Let ${\displaystyle \{f_k(x_1,x_2,\dots ,x_n)\}}$ be a set of m functions, which are linear combinations of ${\displaystyle n}$ variables ${\displaystyle x_1,x_2,\dots ,x_n}$ with combination coefficients ${\displaystyle A_{k1},A_{k2},\dots ,A_{kn},(k=1,\dots ,m)}$:

${\displaystyle f_k=\sum _{i=1}^n{A}_{ki}{x}_i,}$

or in matrix notation,

${\displaystyle \mathbf{f}=\mathbf{A}\mathbf{x}.}$

Also let the variance–covariance matrix of x = (x₁, ..., x_n) be denoted by ${\displaystyle {\mathbf{\Sigma }}^x}$ and let the mean value be denoted by ${\displaystyle \mu }$:

${\displaystyle {\mathbf{\Sigma }}^x=E[\mathbf{(}\mathbf{x}\mathbf{-}\mu \mathbf{)}\otimes \mathbf{(}\mathbf{x}\mathbf{-}\mu \mathbf{)}]=\left(\begin{array}{cccc}{\sigma }_1^2& {\sigma }_{12}& {\sigma }_{13}& \cdots \\ {\sigma }_{21}& {\sigma }_2^2& {\sigma }_{23}& \cdots \\ {\sigma }_{31}& {\sigma }_{32}& {\sigma }_3^2& \cdots \\ ⋮& ⋮& ⋮& \ddots \end{array}\right)=\left(\begin{array}{cccc}{\mathrm{\Sigma }}_{11}^x& {\mathrm{\Sigma }}_{12}^x& {\mathrm{\Sigma }}_{13}^x& \cdots \\ {\mathrm{\Sigma }}_{21}^x& {\mathrm{\Sigma }}_{22}^x& {\mathrm{\Sigma }}_{23}^x& \cdots \\ {\mathrm{\Sigma }}_{31}^x& {\mathrm{\Sigma }}_{32}^x& {\mathrm{\Sigma }}_{33}^x& \cdots \\ ⋮& ⋮& ⋮& \ddots \end{array}\right).}$

${\displaystyle \otimes }$ is the outer product.

Then, the variance–covariance matrix ${\displaystyle {\mathbf{\Sigma }}^f}$ of f is given by

${\displaystyle {\mathbf{\Sigma }}^f=E[(\mathbf{f}-E[\mathbf{f}])\otimes (\mathbf{f}-E[\mathbf{f}])]=E[\mathbf{A}\mathbf{(}\mathbf{x}\mathbf{-}\mu \mathbf{)}\otimes \mathbf{A}\mathbf{(}\mathbf{x}\mathbf{-}\mu \mathbf{)}]=\mathbf{A}E[\mathbf{(}\mathbf{x}\mathbf{-}\mu \mathbf{)}\otimes \mathbf{(}\mathbf{x}\mathbf{-}\mu \mathbf{)}]{\mathbf{A}}^{\mathrm{T}}=\mathbf{A}{\mathbf{\Sigma }}^x{\mathbf{A}}^{\mathrm{T}}}$

In component notation, the equation

${\displaystyle {\mathbf{\Sigma }}^f=\mathbf{A}{\mathbf{\Sigma }}^x{\mathbf{A}}^{\mathrm{T}}.}$

reads

${\displaystyle {\mathrm{\Sigma }}_{ij}^f=\sum _k^n\sum _l^n{A}_{ik}{\mathrm{\Sigma }}_{kl}^x{A}_{jl}.}$

This is the most general expression for the propagation of error from one set of variables onto another. When the errors on x are uncorrelated, the general expression simplifies to

${\displaystyle {\mathrm{\Sigma }}_{ij}^f=\sum _k^n{A}_{ik}{\mathrm{\Sigma }}_k^x{A}_{jk},}$

where ${\displaystyle {\mathrm{\Sigma }}_k^x={\sigma }_{x_k}^2}$ is the variance of k-th element of the x vector. Note that even though the errors on x may be uncorrelated, the errors on f are in general correlated; in other words, even if ${\displaystyle {\mathbf{\Sigma }}^x}$ is a diagonal matrix, ${\displaystyle {\mathbf{\Sigma }}^f}$ is in general a full matrix.

The general expressions for a scalar-valued function f are a little simpler (here a is a row vector):

${\displaystyle f=\sum _i^n{a}_i{x}_i=\mathbf{a}\mathbf{x},}$

${\displaystyle {\sigma }_f^2=\sum _i^n\sum _j^n{a}_i{\mathrm{\Sigma }}_{ij}^x{a}_j=\mathbf{a}{\mathbf{\Sigma }}^x{\mathbf{a}}^{\mathrm{T}}.}$

Each covariance term ${\displaystyle {\sigma }_{ij}}$ can be expressed in terms of the correlation coefficient ${\displaystyle {\rho }_{ij}}$ by ${\displaystyle {\sigma }_{ij}={\rho }_{ij}{\sigma }_i{\sigma }_j}$, so that an alternative expression for the variance of f is

${\displaystyle {\sigma }_f^2=\sum _i^n{a}_i^2{\sigma }_i^2+\sum _i^n\sum _{j(j\ne i)}^n{a}_i{a}_j{\rho }_{ij}{\sigma }_i{\sigma }_j.}$

In the case that the variables in x are uncorrelated, this simplifies further to

${\displaystyle {\sigma }_f^2=\sum _i^n{a}_i^2{\sigma }_i^2.}$

In the simple case of identical coefficients and variances, we find

${\displaystyle {\sigma }_f=\sqrt{n}|a|\sigma .}$

For the arithmetic mean, ${\displaystyle a=1/n}$, the result is the standard error of the mean:

${\displaystyle {\sigma }_f=\sigma /\sqrt{n}.}$

Non-linear combinations

See also: Taylor expansions for the moments of functions of random variables

When f is a set of non-linear combination of the variables x, an interval propagation could be performed in order to compute intervals which contain all consistent values for the variables. In a probabilistic approach, the function f must usually be linearised by approximation to a first-order Taylor series expansion, though in some cases, exact formulae can be derived that do not depend on the expansion as is the case for the exact variance of products.^[7] The Taylor expansion would be:

${\displaystyle f_k\approx f_k^0+\sum _i^n\frac{\mathrm{\partial }{f}_k}{\mathrm{\partial }{x}_i}{x}_i}$

where ${\displaystyle \mathrm{\partial }{f}_k/\mathrm{\partial }{x}_i}$ denotes the partial derivative of f_k with respect to the i-th variable, evaluated at the mean value of all components of vector x. Or in matrix notation,

${\displaystyle \mathrm{f}\approx {\mathrm{f}}^0+\mathrm{J}\mathrm{x}}$

where J is the Jacobian matrix. Since f⁰ is a constant it does not contribute to the error on f. Therefore, the propagation of error follows the linear case, above, but replacing the linear coefficients, A_ki and A_kj by the partial derivatives, ${\displaystyle \frac{\mathrm{\partial }{f}_k}{\mathrm{\partial }{x}_i}}$ and ${\displaystyle \frac{\mathrm{\partial }{f}_k}{\mathrm{\partial }{x}_j}}$. In matrix notation,^[8]

${\displaystyle {\mathrm{\Sigma }}^{\mathrm{f}}=\mathrm{J}{\mathrm{\Sigma }}^{\mathrm{x}}{\mathrm{J}}^{\mathrm{\top }}.}$

That is, the Jacobian of the function is used to transform the rows and columns of the variance-covariance matrix of the argument. Note this is equivalent to the matrix expression for the linear case with ${\displaystyle \mathrm{J}=\mathrm{A}}$.

Simplification

Neglecting correlations or assuming independent variables yields a common formula among engineers and experimental scientists to calculate error propagation, the variance formula:^[9]

${\displaystyle s_f=\sqrt{{\left(\frac{\mathrm{\partial }f}{\mathrm{\partial }x}\right)}^2{s}_x^2+{\left(\frac{\mathrm{\partial }f}{\mathrm{\partial }y}\right)}^2{s}_y^2+{\left(\frac{\mathrm{\partial }f}{\mathrm{\partial }z}\right)}^2{s}_z^2+\cdots }}$

where ${\displaystyle s_f}$ represents the standard deviation of the function ${\displaystyle f}$, ${\displaystyle s_x}$ represents the standard deviation of ${\displaystyle x}$, ${\displaystyle s_y}$ represents the standard deviation of ${\displaystyle y}$, and so forth.

It is important to note that this formula is based on the linear characteristics of the gradient of ${\displaystyle f}$ and therefore it is a good estimation for the standard deviation of ${\displaystyle f}$ as long as ${\displaystyle s_x,s_y,s_z,\dots }$ are small enough. Specifically, the linear approximation of ${\displaystyle f}$ has to be close to ${\displaystyle f}$ inside a neighbourhood of radius ${\displaystyle s_x,s_y,s_z,\dots }$.^[10]

Example

Any non-linear differentiable function, ${\displaystyle f(a,b)}$, of two variables, ${\displaystyle a}$ and ${\displaystyle b}$, can be expanded as

${\displaystyle f\approx f^0+\frac{\mathrm{\partial }f}{\mathrm{\partial }a}a+\frac{\mathrm{\partial }f}{\mathrm{\partial }b}b}$

now, taking variance on both sides, and using the formula^[11] for variance of a linear combination of variables:

${\displaystyle Var(aX+bY)=a^{2}Var(X)+b^{2}Var(Y)+2ab\ast Cov(X,Y)}$

hence:

${\displaystyle {\sigma }_f^2\approx {\left|\frac{\mathrm{\partial }f}{\mathrm{\partial }a}\right|}^2{\sigma }_a^2+{\left|\frac{\mathrm{\partial }f}{\mathrm{\partial }b}\right|}^2{\sigma }_b^2+2\frac{\mathrm{\partial }f}{\mathrm{\partial }a}\frac{\mathrm{\partial }f}{\mathrm{\partial }b}{\sigma }_{ab}}$

where ${\displaystyle {\sigma }_f}$ is the standard deviation of the function ${\displaystyle f}$, ${\displaystyle {\sigma }_a}$ is the standard deviation of ${\displaystyle a}$, ${\displaystyle {\sigma }_b}$ is the standard deviation of ${\displaystyle b}$ and ${\displaystyle {\sigma }_{ab}={\sigma }_a{\sigma }_b{\rho }_{ab}}$ is the covariance between ${\displaystyle a}$ and ${\displaystyle b}$.

In the particular case that ${\displaystyle f=ab}$, ${\displaystyle \frac{\mathrm{\partial }f}{\mathrm{\partial }a}=b,\frac{\mathrm{\partial }f}{\mathrm{\partial }b}=a}$. Then

${\displaystyle {\sigma }_f^2\approx b^2{\sigma }_a^2+a^2{\sigma }_b^2+2ab{\sigma }_{ab}}$

or

${\displaystyle {\left(\frac{{\sigma }_f}{f}\right)}^2\approx {\left(\frac{{\sigma }_a}{a}\right)}^2+{\left(\frac{{\sigma }_b}{b}\right)}^2+2\left(\frac{{\sigma }_a}{a}\right)\left(\frac{{\sigma }_b}{b}\right){\rho }_{ab}}$

where ${\displaystyle {\rho }_{ab}}$ is the correlation between ${\displaystyle a}$ and ${\displaystyle b}$.

When the variables ${\displaystyle a}$ and ${\displaystyle b}$ are uncorrelated, ${\displaystyle {\rho }_{ab}=0}$. Then

${\displaystyle {\left(\frac{{\sigma }_f}{f}\right)}^2\approx {\left(\frac{{\sigma }_a}{a}\right)}^2+{\left(\frac{{\sigma }_b}{b}\right)}^2.}$

Caveats and warnings

Error estimates for non-linear functions are biased on account of using a truncated series expansion. The extent of this bias depends on the nature of the function. For example, the bias on the error calculated for log(1+x) increases as x increases, since the expansion to x is a good approximation only when x is near zero.

For highly non-linear functions, there exist five categories of probabilistic approaches for uncertainty propagation; see Uncertainty quantification for details.

Reciprocal and shifted reciprocal

Main article: Reciprocal normal distribution

In the special case of the inverse or reciprocal ${\displaystyle 1/B}$, where ${\displaystyle B=N(0,1)}$ follows a standard normal distribution, the resulting distribution is a reciprocal standard normal distribution, and there is no definable variance.

However, in the slightly more general case of a shifted reciprocal function ${\displaystyle 1/(p-B)}$ for ${\displaystyle B=N(\mu ,\sigma )}$ following a general normal distribution, then mean and variance statistics do exist in a principal value sense, if the difference between the pole ${\displaystyle p}$ and the mean ${\displaystyle \mu }$ is real-valued.

Ratios

Main article: Normal ratio distribution

Ratios are also problematic; normal approximations exist under certain conditions.

Example formulae

This table shows the variances and standard deviations of simple functions of the real variables ${\displaystyle A,B}$, with standard deviations ${\displaystyle {\sigma }_A,{\sigma }_B,}$ covariance ${\displaystyle {\sigma }_{AB}={\rho }_{AB}{\sigma }_A{\sigma }_B}$, and correlation ${\displaystyle {\rho }_{AB}}$. The real-valued coefficients ${\displaystyle a}$ and ${\displaystyle b}$ are assumed exactly known (deterministic), i.e., ${\displaystyle {\sigma }_a={\sigma }_b=0}$.

In the columns "Variance" and "Standard Deviation", A and B should be understood as expectation values (i.e. values around which we're estimating the uncertainty), and ${\displaystyle f}$ should be understood as the value of the function calculated at the expectation value of ${\displaystyle A,B}$.

Function	Variance	Standard Deviation
${\displaystyle f=aA}$	${\displaystyle {\sigma }_f^2=a^2{\sigma }_A^2}$	${\displaystyle {\sigma }_f=|a|{\sigma }_A}$
${\displaystyle f=aA+bB}$	${\displaystyle {\sigma }_f^2=a^2{\sigma }_A^2+b^2{\sigma }_B^2+2ab{\sigma }_{AB}}$	${\displaystyle {\sigma }_f=\sqrt{a^2{\sigma }_A^2+b^2{\sigma }_B^2+2ab{\sigma }_{AB}}}$
${\displaystyle f=aA-bB}$	${\displaystyle {\sigma }_f^2=a^2{\sigma }_A^2+b^2{\sigma }_B^2-2ab{\sigma }_{AB}}$	${\displaystyle {\sigma }_f=\sqrt{a^2{\sigma }_A^2+b^2{\sigma }_B^2-2ab{\sigma }_{AB}}}$
${\displaystyle f=A-B,}$	${\displaystyle {\sigma }_f^2={\sigma }_A^2+{\sigma }_B^2-2{\sigma }_{AB}}$	${\displaystyle {\sigma }_f=\sqrt{{\sigma }_A^2+{\sigma }_B^2-2{\sigma }_{AB}}}$
${\displaystyle f=AB}$	${\displaystyle {\sigma }_f^2\approx f^2\left[{\left(\frac{{\sigma }_A}{A}\right)}^2+{\left(\frac{{\sigma }_B}{B}\right)}^2+2\frac{{\sigma }_{AB}}{AB}\right]}$	${\displaystyle {\sigma }_f\approx \left|f\right|\sqrt{{\left(\frac{{\sigma }_A}{A}\right)}^2+{\left(\frac{{\sigma }_B}{B}\right)}^2+2\frac{{\sigma }_{AB}}{AB}}}$
${\displaystyle f=\frac{A}{B}}$	${\displaystyle {\sigma }_f^2\approx f^2\left[{\left(\frac{{\sigma }_A}{A}\right)}^2+{\left(\frac{{\sigma }_B}{B}\right)}^2-2\frac{{\sigma }_{AB}}{AB}\right]}$	${\displaystyle {\sigma }_f\approx \left|f\right|\sqrt{{\left(\frac{{\sigma }_A}{A}\right)}^2+{\left(\frac{{\sigma }_B}{B}\right)}^2-2\frac{{\sigma }_{AB}}{AB}}}$
${\displaystyle f=a{A}^b}$	${\displaystyle {\sigma }_f^2\approx {\left(ab{A}^{b-1}{\sigma }_A\right)}^2={\left(\frac{fb{\sigma }_A}{A}\right)}^2}$	${\displaystyle {\sigma }_f\approx \left|ab{A}^{b-1}{\sigma }_A\right|=\left|\frac{fb{\sigma }_A}{A}\right|}$
${\displaystyle f=a\ln (bA)}$	${\displaystyle {\sigma }_f^2\approx {\left(a\frac{{\sigma }_A}{A}\right)}^2}$	${\displaystyle {\sigma }_f\approx \left|a\frac{{\sigma }_A}{A}\right|}$
${\displaystyle f=a{\log }_{10}(bA)}$	${\displaystyle {\sigma }_f^2\approx {\left(a\frac{{\sigma }_A}{A\ln (10)}\right)}^2}$	${\displaystyle {\sigma }_f\approx \left|a\frac{{\sigma }_A}{A\ln (10)}\right|}$
${\displaystyle f=a{e}^{bA}}$	${\displaystyle {\sigma }_f^2\approx f^2{\left(b{\sigma }_A\right)}^2}$[19]	${\displaystyle {\sigma }_f\approx \left|f\right|\left|\left(b{\sigma }_A\right)\right|}$
${\displaystyle f=a^{bA}}$	${\displaystyle {\sigma }_f^2\approx f^2(b\ln (a){\sigma }_A{)}^2}$	${\displaystyle {\sigma }_f\approx \left|f\right|\left|(b\ln (a){\sigma }_A)\right|}$
${\displaystyle f=a\sin (bA)}$	${\displaystyle {\sigma }_f^2\approx {\left[ab\cos (bA){\sigma }_A\right]}^2}$	${\displaystyle {\sigma }_f\approx \left|ab\cos (bA){\sigma }_A\right|}$
${\displaystyle f=a\cos \left(bA\right)}$	${\displaystyle {\sigma }_f^2\approx {\left[ab\sin (bA){\sigma }_A\right]}^2}$	${\displaystyle {\sigma }_f\approx \left|ab\sin (bA){\sigma }_A\right|}$
${\displaystyle f=a\tan \left(bA\right)}$	${\displaystyle {\sigma }_f^2\approx {\left[ab{\sec }^2(bA){\sigma }_A\right]}^2}$	${\displaystyle {\sigma }_f\approx \left|ab{\sec }^2(bA){\sigma }_A\right|}$
${\displaystyle f=A^B}$	${\displaystyle {\sigma }_f^2\approx f^2\left[{\left(\frac{B}{A}{\sigma }_A\right)}^2+{\left(\ln (A){\sigma }_B\right)}^2+2\frac{B\ln (A)}{A}{\sigma }_{AB}\right]}$	${\displaystyle {\sigma }_f\approx \left|f\right|\sqrt{{\left(\frac{B}{A}{\sigma }_A\right)}^2+{\left(\ln (A){\sigma }_B\right)}^2+2\frac{B\ln (A)}{A}{\sigma }_{AB}}}$
${\displaystyle f=\sqrt{a{A}^2\pm b{B}^2}}$	${\displaystyle {\sigma }_f^2\approx {\left(\frac{A}{f}\right)}^2{a}^2{\sigma }_A^2+{\left(\frac{B}{f}\right)}^2{b}^2{\sigma }_B^2\pm 2ab\frac{AB}{f^2}{\sigma }_{AB}}$	${\displaystyle {\sigma }_f\approx \sqrt{{\left(\frac{A}{f}\right)}^2{a}^2{\sigma }_A^2+{\left(\frac{B}{f}\right)}^2{b}^2{\sigma }_B^2\pm 2ab\frac{AB}{f^2}{\sigma }_{AB}}}$

For uncorrelated variables (${\displaystyle {\rho }_{AB}=0}$, ${\displaystyle {\sigma }_{AB}=0}$) expressions for more complicated functions can be derived by combining simpler functions. For example, repeated multiplication, assuming no correlation, gives

${\displaystyle f=ABC;{\left(\frac{{\sigma }_f}{f}\right)}^2\approx {\left(\frac{{\sigma }_A}{A}\right)}^2+{\left(\frac{{\sigma }_B}{B}\right)}^2+{\left(\frac{{\sigma }_C}{C}\right)}^2.}$

For the case ${\displaystyle f=AB}$ we also have Goodman's expression for the exact variance: for the uncorrelated case it is

${\displaystyle V(XY)=E(X{)}^{2}V(Y)+E(Y{)}^{2}V(X)+E((X-E(X){)}^2(Y-E(Y){)}^2)}$

and therefore we have:

${\displaystyle {\sigma }_f^2=A^2{\sigma }_B^2+B^2{\sigma }_A^2+{\sigma }_A^2{\sigma }_B^2}$

Effect of correlation on differences

If A and B are uncorrelated, their difference A-B will have more variance than either of them. An increasing positive correlation (${\displaystyle {\rho }_{AB}\to 1}$) will decrease the variance of the difference, converging to zero variance for perfectly correlated variables with the same variance. On the other hand, a negative correlation (${\displaystyle {\rho }_{AB}\to -1}$) will further increase the variance of the difference, compared to the uncorrelated case.

For example, the self-subtraction f=A-A has zero variance ${\displaystyle {\sigma }_f^2=0}$ only if the variate is perfectly autocorrelated (${\displaystyle {\rho }_A=1}$). If A is uncorrelated, ${\displaystyle {\rho }_A=0}$, then the output variance is twice the input variance, ${\displaystyle {\sigma }_f^2=2{\sigma }_A^2}$. And if A is perfectly anticorrelated, ${\displaystyle {\rho }_A=-1}$, then the input variance is quadrupled in the output, ${\displaystyle {\sigma }_f^2=4{\sigma }_A^2}$ (notice ${\displaystyle 1-{\rho }_A=2}$ for f = aA - aA in the table above).

Example calculations

Inverse tangent function

We can calculate the uncertainty propagation for the inverse tangent function as an example of using partial derivatives to propagate error.

Define

${\displaystyle f(x)=\arctan (x),}$

where ${\displaystyle {\mathrm{\Delta }}_x}$ is the absolute uncertainty on our measurement of x. The derivative of f(x) with respect to x is

${\displaystyle \frac{df}{dx}=\frac{1}{1+x^2}.}$

Therefore, our propagated uncertainty is

${\displaystyle {\mathrm{\Delta }}_f\approx \frac{{\mathrm{\Delta }}_x}{1+x^2},}$

where ${\displaystyle {\mathrm{\Delta }}_f}$ is the absolute propagated uncertainty.

Resistance measurement

A practical application is an experiment in which one measures current, I, and voltage, V, on a resistor in order to determine the resistance, R, using Ohm's law, R = V / I.

Given the measured variables with uncertainties, I ± σ_I and V ± σ_V, and neglecting their possible correlation, the uncertainty in the computed quantity, σ_R, is:

${\displaystyle {\sigma }_R\approx \sqrt{{\sigma }_V^2{\left(\frac{1}{I}\right)}^2+{\sigma }_I^2{\left(\frac{-V}{I^2}\right)}^2}=R\sqrt{{\left(\frac{{\sigma }_V}{V}\right)}^2+{\left(\frac{{\sigma }_I}{I}\right)}^2}.}$

 

Volatility (finance)

From Wikipedia, the free encyclopedia

 

 

The VIX

In finance, volatility (usually denoted by σ) is the degree of variation of a trading price series over time, usually measured by the standard deviation of logarithmic returns.

Historic volatility measures a time series of past market prices. Implied volatility looks forward in time, being derived from the market price of a market-traded derivative (in particular, an option).

Volatility terminology

Volatility as described here refers to the actual volatility, more specifically:

• actual current volatility of a financial instrument for a specified period (for example 30 days or 90 days), based on historical prices over the specified period with the last observation the most recent price.
• actual historical volatility which refers to the volatility of a financial instrument over a specified period but with the last observation on a date in the past
  • near synonymous is realized volatility, the square root of the realized variance, in turn calculated using the sum of squared returns divided by the number of observations.
• actual future volatility which refers to the volatility of a financial instrument over a specified period starting at the current time and ending at a future date (normally the expiry date of an option)

Now turning to implied volatility, we have:

• historical implied volatility which refers to the implied volatility observed from historical prices of the financial instrument (normally options)
• current implied volatility which refers to the implied volatility observed from current prices of the financial instrument
• future implied volatility which refers to the implied volatility observed from future prices of the financial instrument

For a financial instrument whose price follows a Gaussian random walk, or Wiener process, the width of the distribution increases as time increases. This is because there is an increasing probability that the instrument's price will be farther away from the initial price as time increases. However, rather than increase linearly, the volatility increases with the square-root of time as time increases, because some fluctuations are expected to cancel each other out, so the most likely deviation after twice the time will not be twice the distance from zero.

Since observed price changes do not follow Gaussian distributions, others such as the Lévy distribution are often used. These can capture attributes such as "fat tails". Volatility is a statistical measure of dispersion around the average of any random variable such as market parameters etc.

Mathematical definition

For any fund that evolves randomly with time, volatility is defined as the standard deviation of a sequence of random variables, each of which is the return of the fund over some corresponding sequence of (equally sized) times.

Thus, "annualized" volatility σ_annually is the standard deviation of an instrument's yearly logarithmic returns.

The generalized volatility σ_T for time horizon T in years is expressed as:

${\displaystyle {\sigma }_{\text{T}}={\sigma }_{\text{annually}}\sqrt{T}.}$

Therefore, if the daily logarithmic returns of a stock have a standard deviation of σ_daily and the time period of returns is P in trading days, the annualized volatility is

${\displaystyle {\sigma }_{\text{annually}}={\sigma }_{\text{daily}}\sqrt{P}.}$

so

${\displaystyle {\sigma }_{\text{T}}={\sigma }_{\text{daily}}\sqrt{PT}.}$

A common assumption is that P = 252 trading days in any given year. Then, if σ_daily = 0.01, the annualized volatility is

${\displaystyle {\sigma }_{\text{annually}}=0.01\sqrt{252}=\mathrm{0.1587.}}$

The monthly volatility (i.e. ${\displaystyle T=\frac{1}{12}}$ of a year) is

${\displaystyle {\sigma }_{\text{monthly}}=0.01\sqrt{\frac{252}{12}}=\mathrm{0.0458.}}$

The formulas used above to convert returns or volatility measures from one time period to another assume a particular underlying model or process. These formulas are accurate extrapolations of a random walk, or Wiener process, whose steps have finite variance. However, more generally, for natural stochastic processes, the precise relationship between volatility measures for different time periods is more complicated. Some use the Lévy stability exponent α to extrapolate natural processes:

${\displaystyle {\sigma }_T=T^{1/\alpha }\sigma .}$

If α = 2 the Wiener process scaling relation is obtained, but some people believe α < 2 for financial activities such as stocks, indexes and so on. This was discovered by Benoît Mandelbrot, who looked at cotton prices and found that they followed a Lévy alpha-stable distribution with α = 1.7. (See New Scientist, 19 April 1997.)

Volatility origin

Much research has been devoted to modeling and forecasting the volatility of financial returns, and yet few theoretical models explain how volatility comes to exist in the first place.

Roll (1984) shows that volatility is affected by market microstructure. Glosten and Milgrom (1985) shows that at least one source of volatility can be explained by the liquidity provision process. When market makers infer the possibility of adverse selection, they adjust their trading ranges, which in turn increases the band of price oscillation.

In September 2019, JPMorgan Chase determined the effect of US President Donald Trump's tweets, and called it the Volfefe index combining volatility and the covfefe meme.

Volatility for investors

Investors care about volatility for at least eight reasons:

1. The wider the swings in an investment's price, the harder emotionally it is to not worry;
2. Price volatility of a trading instrument can define position sizing in a portfolio;
3. When certain cash flows from selling a security are needed at a specific future date, higher volatility means a greater chance of a shortfall;
4. Higher volatility of returns while saving for retirement results in a wider distribution of possible final portfolio values;
5. Higher volatility of return when retired gives withdrawals a larger permanent impact on the portfolio's value;
6. Price volatility presents opportunities to buy assets cheaply and sell when overpriced;
7. Portfolio volatility has a negative impact on the compound annual growth rate (CAGR) of that portfolio
8. Volatility affects pricing of options, being a parameter of the Black–Scholes model.

Volatility versus direction

Volatility does not measure the direction of price changes, merely their dispersion. This is because when calculating standard deviation (or variance), all differences are squared, so that negative and positive differences are combined into one quantity. Two instruments with different volatilities may have the same expected return, but the instrument with higher volatility will have larger swings in values over a given period of time.

For example, a lower volatility stock may have an expected (average) return of 7%, with annual volatility of 5%. This would indicate returns from approximately negative 3% to positive 17% most of the time (19 times out of 20, or 95% via a two standard deviation rule). A higher volatility stock, with the same expected return of 7% but with annual volatility of 20%, would indicate returns from approximately negative 33% to positive 47% most of the time (19 times out of 20, or 95%). These estimates assume a normal distribution; in reality stocks are found to be leptokurtotic.

Volatility over time

Although the Black-Scholes equation assumes predictable constant volatility, this is not observed in real markets, and amongst the models are Emanuel Derman and Iraj Kani's and Bruno Dupire's local volatility, Poisson process where volatility jumps to new levels with a predictable frequency, and the increasingly popular Heston model of stochastic volatility.

It is common knowledge that types of assets experience periods of high and low volatility. That is, during some periods, prices go up and down quickly, while during other times they barely move at all. In foreign exchange market, price changes are seasonally heteroskedastic with periods of one day and one week.

Periods when prices fall quickly (a crash) are often followed by prices going down even more, or going up by an unusual amount. Also, a time when prices rise quickly (a possible bubble) may often be followed by prices going up even more, or going down by an unusual amount.

Most typically, extreme movements do not appear 'out of nowhere'; they are presaged by larger movements than usual. This is termed autoregressive conditional heteroskedasticity. Whether such large movements have the same direction, or the opposite, is more difficult to say. And an increase in volatility does not always presage a further increase—the volatility may simply go back down again.

Not only the volatility depends on the period when it is measured but also on the selected time resolution. The effect is observed due to the fact that the information flow between short-term and long-term traders is asymmetric. As a result, volatility measured with high resolution contains information that is not covered by low resolution volatility and vice versa.

The risk parity weighted volatility of the three assets Gold, Treasury bonds and Nasdaq acting as proxy for the Marketportfolio seems to have a low point at 4% after turning upwards for the 8th time since 1974 at this reading in the summer of 2014.

Alternative measures of volatility

Some authors point out that realized volatility and implied volatility are backward and forward looking measures, and do not reflect current volatility. To address that issue an alternative, ensemble measures of volatility were suggested. One of the measures is defined as the standard deviation of ensemble returns instead of time series of returns. Another considers the regular sequence of directional-changes as the proxy for the instantaneous volatility.

Implied volatility parametrisation

There exist several known parametrisations of the implied volatility surface, Schonbucher, SVI and gSVI.

Crude volatility estimation

Using a simplification of the above formula it is possible to estimate annualized volatility based solely on approximate observations. Suppose you notice that a market price index, which has a current value near 10,000, has moved about 100 points a day, on average, for many days. This would constitute a 1% daily movement, up or down.

To annualize this, you can use the "rule of 16", that is, multiply by 16 to get 16% as the annual volatility. The rationale for this is that 16 is the square root of 256, which is approximately the number of trading days in a year (252). This also uses the fact that the standard deviation of the sum of n independent variables (with equal standard deviations) is √n times the standard deviation of the individual variables.

The average magnitude of the observations is merely an approximation of the standard deviation of the market index. Assuming that the market index daily changes are normally distributed with mean zero and standard deviation σ, the expected value of the magnitude of the observations is √(2/π)σ = 0.798σ. The net effect is that this crude approach underestimates the true volatility by about 20%.

Estimate of compound annual growth rate (CAGR)

Consider the Taylor series:

${\displaystyle \log (1+y)=y-\frac{1}{2}{y}^2+\frac{1}{3}{y}^3-\frac{1}{4}{y}^4+\cdots }$

Taking only the first two terms one has:

${\displaystyle \mathrm{C}\mathrm{A}\mathrm{G}\mathrm{R}\approx \mathrm{A}\mathrm{R}-\frac{1}{2}{\sigma }^2}$

Volatility thus mathematically represents a drag on the CAGR (formalized as the "volatility tax"). Realistically, most financial assets have negative skewness and leptokurtosis, so this formula tends to be over-optimistic. Some people use the formula:

${\displaystyle \mathrm{C}\mathrm{A}\mathrm{G}\mathrm{R}\approx \mathrm{A}\mathrm{R}-\frac{1}{2}k{\sigma }^2}$

for a rough estimate, where k is an empirical factor (typically five to ten).

Criticisms of volatility forecasting models

Performance of VIX (left) compared to past volatility (right) as 30-day volatility predictors, for the period of Jan 1990-Sep 2009. Volatility is measured as the standard deviation of S&P500 one-day returns over a month's period. The blue lines indicate linear regressions, resulting in the correlation coefficients r shown. Note that VIX has virtually the same predictive power as past volatility, insofar as the shown correlation coefficients are nearly identical.

Despite the sophisticated composition of most volatility forecasting models, critics claim that their predictive power is similar to that of plain-vanilla measures, such as simple past volatility especially out-of-sample, where different data are used to estimate the models and to test them. Other works have agreed, but claim critics failed to correctly implement the more complicated models. Some practitioners and portfolio managers seem to completely ignore or dismiss volatility forecasting models. For example, Nassim Taleb famously titled one of his Journal of Portfolio Management papers "We Don't Quite Know What We are Talking About When We Talk About Volatility". In a similar note, Emanuel Derman expressed his disillusion with the enormous supply of empirical models unsupported by theory. He argues that, while "theories are attempts to uncover the hidden principles underpinning the world around us, as Albert Einstein did with his theory of relativity", we should remember that "models are metaphors – analogies that describe one thing relative to another".