Variance function

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Variance_function

In statistics, the variance function is a smooth function which depicts the variance of a random quantity as a function of its mean. The variance function is a measure of heteroscedasticity and plays a large role in many settings of statistical modelling. It is a main ingredient in the generalized linear model framework and a tool used in non-parametric regression, semiparametric regression and functional data analysis. In parametric modeling, variance functions take on a parametric form and explicitly describe the relationship between the variance and the mean of a random quantity. In a non-parametric setting, the variance function is assumed to be a smooth function.

Intuition

In a regression model setting, the goal is to establish whether or not a relationship exists between a response variable and a set of predictor variables. Further, if a relationship does exist, the goal is then to be able to describe this relationship as best as possible. A main assumption in linear regression is constant variance or (homoscedasticity), meaning that different response variables have the same variance in their errors, at every predictor level. This assumption works well when the response variable and the predictor variable are jointly Normal, see Normal distribution. As we will see later, the variance function in the Normal setting is constant; however, we must find a way to quantify heteroscedasticity (non-constant variance) in the absence of joint Normality.

When it is likely that the response follows a distribution that is a member of the exponential family, a generalized linear model may be more appropriate to use, and moreover, when we wish not to force a parametric model onto our data, a non-parametric regression approach can be useful. The importance of being able to model the variance as a function of the mean lies in improved inference (in a parametric setting), and estimation of the regression function in general, for any setting.

Variance functions play a very important role in parameter estimation and inference. In general, maximum likelihood estimation requires that a likelihood function be defined. This requirement then implies that one must first specify the distribution of the response variables observed. However, to define a quasi-likelihood, one need only specify a relationship between the mean and the variance of the observations to then be able to use the quasi-likelihood function for estimation. Quasi-likelihood estimation is particularly useful when there is overdispersion. Overdispersion occurs when there is more variability in the data than there should otherwise be expected according to the assumed distribution of the data.

In summary, to ensure efficient inference of the regression parameters and the regression function, the heteroscedasticity must be accounted for. Variance functions quantify the relationship between the variance and the mean of the observed data and hence play a significant role in regression estimation and inference.

Types

The variance function and its applications come up in many areas of statistical analysis. A very important use of this function is in the framework of generalized linear models and non-parametric regression.

Generalized linear model

When a member of the exponential family has been specified, the variance function can easily be derived. The general form of the variance function is presented under the exponential family context, as well as specific forms for Normal, Bernoulli, Poisson, and Gamma. In addition, we describe the applications and use of variance functions in maximum likelihood estimation and quasi-likelihood estimation.

Derivation

The generalized linear model (GLM), is a generalization of ordinary regression analysis that extends to any member of the exponential family. It is particularly useful when the response variable is categorical, binary or subject to a constraint (e.g. only positive responses make sense). A quick summary of the components of a GLM are summarized on this page, but for more details and information see the page on generalized linear models.

A GLM consists of three main ingredients:

1. Random Component: a distribution of y from the exponential family, ${\displaystyle E[y\mid X]=\mu }$

2. Linear predictor: ${\displaystyle \eta =XB=\sum _{j=1}^p{X}_{ij}^T{B}_j}$

3. Link function: ${\displaystyle \eta =g(\mu ),\mu =g^{-1}(\eta )}$

First it is important to derive a couple key properties of the exponential family.

Any random variable ${\displaystyle \mathit{\text{y}}}$ in the exponential family has a probability density function of the form,

${\displaystyle f(y,\theta ,\varphi )=\exp \left(\frac{y\theta -b(\theta )}{\varphi }-c(y,\varphi )\right)}$

with loglikelihood,

${\displaystyle \ell (\theta ,y,\varphi )=\log (f(y,\theta ,\varphi ))=\frac{y\theta -b(\theta )}{\varphi }-c(y,\varphi )}$

Here, ${\displaystyle \theta }$ is the canonical parameter and the parameter of interest, and ${\displaystyle \varphi }$ is a nuisance parameter which plays a role in the variance. We use the Bartlett's Identities to derive a general expression for the variance function. The first and second Bartlett results ensures that under suitable conditions (see Leibniz integral rule), for a density function dependent on ${\displaystyle \theta ,f_{\theta }()}$,

${\displaystyle {\mathrm{E}}_{\theta }\left[\frac{\mathrm{\partial }}{\mathrm{\partial }\theta }\log (f_{\theta }(y))\right]=0}$

${\displaystyle {\mathrm{Var}}_{\theta }\left[\frac{\mathrm{\partial }}{\mathrm{\partial }\theta }\log (f_{\theta }(y))\right]+{\mathrm{E}}_{\theta }\left[\frac{{\mathrm{\partial }}^2}{\mathrm{\partial }{\theta }^2}\log (f_{\theta }(y))\right]=0}$

These identities lead to simple calculations of the expected value and variance of any random variable ${\displaystyle \mathit{\text{y}}}$ in the exponential family ${\displaystyle E_{\theta }[y],Va{r}_{\theta }[y]}$.

Expected value of Y: Taking the first derivative with respect to ${\displaystyle \theta }$ of the log of the density in the exponential family form described above, we have

${\displaystyle \frac{\mathrm{\partial }}{\mathrm{\partial }\theta }\log (f(y,\theta ,\varphi ))=\frac{\mathrm{\partial }}{\mathrm{\partial }\theta }\left[\frac{y\theta -b(\theta )}{\varphi }-c(y,\varphi )\right]=\frac{y-b^{\prime }(\theta )}{\varphi }}$

Then taking the expected value and setting it equal to zero leads to,

${\displaystyle {\mathrm{E}}_{\theta }\left[\frac{y-b^{\prime }(\theta )}{\varphi }\right]=\frac{{\mathrm{E}}_{\theta }[y]-b^{\prime }(\theta )}{\varphi }=0}$

${\displaystyle {\mathrm{E}}_{\theta }[y]=b^{\prime }(\theta )}$

Variance of Y: To compute the variance we use the second Bartlett identity,

${\displaystyle {\mathrm{Var}}_{\theta }\left[\frac{\mathrm{\partial }}{\mathrm{\partial }\theta }\left(\frac{y\theta -b(\theta )}{\varphi }-c(y,\varphi )\right)\right]+{\mathrm{E}}_{\theta }\left[\frac{{\mathrm{\partial }}^2}{\mathrm{\partial }{\theta }^2}\left(\frac{y\theta -b(\theta )}{\varphi }-c(y,\varphi )\right)\right]=0}$

${\displaystyle {\mathrm{Var}}_{\theta }\left[\frac{y-b^{\prime }(\theta )}{\varphi }\right]+{\mathrm{E}}_{\theta }\left[\frac{-b^{″}(\theta )}{\varphi }\right]=0}$

${\displaystyle {\mathrm{Var}}_{\theta }\left[y\right]=b^{″}(\theta )\varphi }$

We have now a relationship between ${\displaystyle \mu }$ and ${\displaystyle \theta }$, namely

${\displaystyle \mu =b^{\prime }(\theta )}$ and ${\displaystyle \theta =b^{\prime -1}(\mu )}$, which allows for a relationship between ${\displaystyle \mu }$ and the variance,

${\displaystyle V(\theta )=b^{″}(\theta )=\text{the part of the variance that depends on }\theta }$

${\displaystyle \mathrm{V}(\mu )=b^{″}(b^{\prime -1}(\mu )).}$

Note that because ${\displaystyle {\mathrm{Var}}_{\theta }\left[y\right]>0,b^{″}(\theta )>0}$, then ${\displaystyle b^{\prime }:\theta \to \mu }$ is invertible. We derive the variance function for a few common distributions.

Example – normal

The Normal distribution is a special case where the variance function is a constant. Let ${\displaystyle y\sim N(\mu ,{\sigma }^2)}$ then we put the density function of y in the form of the exponential family described above:

${\displaystyle f(y)=\exp \left(\frac{y\mu -\frac{{\mu }^2}{2}}{{\sigma }^2}-\frac{y^2}{2{\sigma }^2}-\frac{1}{2}\ln 2\pi {\sigma }^2\right)}$

where

${\displaystyle \theta =\mu ,}$

${\displaystyle b(\theta )=\frac{{\mu }^2}{2},}$

${\displaystyle \varphi ={\sigma }^2,}$

${\displaystyle c(y,\varphi )=-\frac{y^2}{2{\sigma }^2}-\frac{1}{2}\ln 2\pi {\sigma }^2}$

To calculate the variance function ${\displaystyle V(\mu )}$, we first express ${\displaystyle \theta }$ as a function of ${\displaystyle \mu }$. Then we transform ${\displaystyle V(\theta )}$ into a function of ${\displaystyle \mu }$

${\displaystyle \theta =\mu }$

${\displaystyle b^{\prime }(\theta )=\theta =\mathrm{E}[y]=\mu }$

${\displaystyle V(\theta )=b^{″}(\theta )=1}$

Therefore, the variance function is constant.

Example – Bernoulli

Let ${\displaystyle y\sim \text{Bernoulli}(p)}$, then we express the density of the Bernoulli distribution in exponential family form,

${\displaystyle f(y)=\exp \left(y\ln \frac{p}{1-p}+\ln (1-p)\right)}$

${\displaystyle \theta =\ln \frac{p}{1-p}=}$ logit(p), which gives us ${\displaystyle p=\frac{e^{\theta }}{1+e^{\theta }}=}$ expit${\displaystyle (\theta )}$

${\displaystyle b(\theta )=\ln (1+e^{\theta })}$ and

${\displaystyle b^{\prime }(\theta )=\frac{e^{\theta }}{1+e^{\theta }}=}$ expit${\displaystyle (\theta )=p=\mu }$

${\displaystyle b^{″}(\theta )=\frac{e^{\theta }}{1+e^{\theta }}-{\left(\frac{e^{\theta }}{1+e^{\theta }}\right)}^2}$

This give us

${\displaystyle V(\mu )=\mu (1-\mu )}$

Example – Poisson

Let ${\displaystyle y\sim \text{Poisson}(\lambda )}$, then we express the density of the Poisson distribution in exponential family form,

${\displaystyle f(y)=\exp (y\ln \lambda -\ln \lambda )}$

${\displaystyle \theta =\ln \lambda =}$ which gives us ${\displaystyle \lambda =e^{\theta }}$

${\displaystyle b(\theta )=e^{\theta }}$ and

${\displaystyle b^{\prime }(\theta )=e^{\theta }=\lambda =\mu }$

${\displaystyle b^{″}(\theta )=e^{\theta }=\mu }$

This give us

${\displaystyle V(\mu )=\mu }$

Here we see the central property of Poisson data, that the variance is equal to the mean.

Example – Gamma

The Gamma distribution and density function can be expressed under different parametrizations. We will use the form of the gamma with parameters ${\displaystyle (\mu ,\nu )}$

${\displaystyle f_{\mu ,\nu }(y)=\frac{1}{\mathrm{\Gamma }(\nu )y}{\left(\frac{\nu y}{\mu }\right)}^{\nu }{e}^{-\frac{\nu y}{\mu }}}$

Then in exponential family form we have

${\displaystyle f_{\mu ,\nu }(y)=\exp \left(\frac{-\frac{1}{\mu }y+\ln (\frac{1}{\mu })}{\frac{1}{\nu }}+\ln \left(\frac{{\nu }^{\nu }{y}^{\nu -1}}{\mathrm{\Gamma }(\nu )}\right)\right)}$

${\displaystyle \theta =\frac{-1}{\mu }\to \mu =\frac{-1}{\theta }}$

${\displaystyle \varphi =\frac{1}{\nu }}$

${\displaystyle b(\theta )=-\ln (-\theta )}$

${\displaystyle b^{\prime }(\theta )=\frac{-1}{\theta }=\frac{-1}{\frac{-1}{\mu }}=\mu }$

${\displaystyle b^{″}(\theta )=\frac{1}{{\theta }^2}={\mu }^2}$

And we have ${\displaystyle V(\mu )={\mu }^2}$

Application – weighted least squares

A very important application of the variance function is its use in parameter estimation and inference when the response variable is of the required exponential family form as well as in some cases when it is not (which we will discuss in quasi-likelihood). Weighted least squares (WLS) is a special case of generalized least squares. Each term in the WLS criterion includes a weight that determines that the influence each observation has on the final parameter estimates. As in regular least squares, the goal is to estimate the unknown parameters in the regression function by finding values for parameter estimates that minimize the sum of the squared deviations between the observed responses and the functional portion of the model.

While WLS assumes independence of observations it does not assume equal variance and is therefore a solution for parameter estimation in the presence of heteroscedasticity. The Gauss–Markov theorem and Aitken demonstrate that the best linear unbiased estimator (BLUE), the unbiased estimator with minimum variance, has each weight equal to the reciprocal of the variance of the measurement.

In the GLM framework, our goal is to estimate parameters ${\displaystyle \beta }$, where ${\displaystyle Z=g(E[y\mid X])=X\beta }$. Therefore, we would like to minimize ${\displaystyle (Z-XB{)}^{T}W(Z-XB)}$ and if we define the weight matrix W as

${\displaystyle \underset{n\times n}{\underbrace{W}}=\left[\begin{array}{ccccc}\frac{1}{\varphi V({\mu }_1)g^{\prime }({\mu }_1{)}^2}& 0& \cdots & 0& 0\\ 0& \frac{1}{\varphi V({\mu }_2)g^{\prime }({\mu }_2{)}^2}& 0& \cdots & 0\\ ⋮& ⋮& ⋮& ⋮& 0\\ ⋮& ⋮& ⋮& ⋮& 0\\ 0& \cdots & \cdots & 0& \frac{1}{\varphi V({\mu }_n)g^{\prime }({\mu }_n{)}^2}\end{array}\right],}$

where ${\displaystyle \varphi ,V(\mu ),g(\mu )}$ are defined in the previous section, it allows for iteratively reweighted least squares (IRLS) estimation of the parameters. See the section on iteratively reweighted least squares for more derivation and information.

Also, important to note is that when the weight matrix is of the form described here, minimizing the expression ${\displaystyle (Z-XB{)}^{T}W(Z-XB)}$ also minimizes the Pearson distance. See Distance correlation for more.

The matrix W falls right out of the estimating equations for estimation of ${\displaystyle \beta }$. Maximum likelihood estimation for each parameter ${\displaystyle {\beta }_r,1\le r\le p}$, requires

${\displaystyle \sum _{i=1}^n\frac{\mathrm{\partial }{l}_i}{\mathrm{\partial }{\beta }_r}=0}$, where ${\displaystyle \mathrm{l}(\theta ,y,\varphi )=\log (\mathrm{f}(y,\theta ,\varphi ))=\frac{y\theta -b(\theta )}{\varphi }-c(y,\varphi )}$ is the log-likelihood.

Looking at a single observation we have,

${\displaystyle \frac{\mathrm{\partial }l}{\mathrm{\partial }{\beta }_r}=\frac{\mathrm{\partial }l}{\mathrm{\partial }\theta }\frac{\mathrm{\partial }\theta }{\mathrm{\partial }\mu }\frac{\mathrm{\partial }\mu }{\mathrm{\partial }\eta }\frac{\mathrm{\partial }\eta }{\mathrm{\partial }{\beta }_r}}$

${\displaystyle \frac{\mathrm{\partial }\eta }{\mathrm{\partial }{\beta }_r}=x_r}$

${\displaystyle \frac{\mathrm{\partial }l}{\mathrm{\partial }\theta }=\frac{y-b^{\prime }(\theta )}{\varphi }=\frac{y-\mu }{\varphi }}$

${\displaystyle \frac{\mathrm{\partial }\theta }{\mathrm{\partial }\mu }=\frac{\mathrm{\partial }{b}^{\prime -1}(\mu )}{\mu }=\frac{1}{b^{″}(b^{\prime }(\mu ))}=\frac{1}{V(\mu )}}$

This gives us

${\displaystyle \frac{\mathrm{\partial }l}{\mathrm{\partial }{\beta }_r}=\frac{y-\mu }{\varphi V(\mu )}\frac{\mathrm{\partial }\mu }{\mathrm{\partial }\eta }{x}_r}$, and noting that

${\displaystyle \frac{\mathrm{\partial }\eta }{\mathrm{\partial }\mu }=g^{\prime }(\mu )}$ we have that

${\displaystyle \frac{\mathrm{\partial }l}{\mathrm{\partial }{\beta }_r}=(y-\mu )W\frac{\mathrm{\partial }\eta }{\mathrm{\partial }\mu }{x}_r}$

The Hessian matrix is determined in a similar manner and can be shown to be,

${\displaystyle H=X^T(y-\mu )\left[\frac{\mathrm{\partial }}{{\beta }_s}W\frac{\mathrm{\partial }}{{\beta }_r}\right]-X^{T}WX}$

Noticing that the Fisher Information (FI),

${\displaystyle \text{FI}=-E[H]=X^{T}WX}$, allows for asymptotic approximation of ${\displaystyle \hat{\beta }}$

${\displaystyle \hat{\beta }\sim N_p(\beta ,(X^{T}WX{)}^{-1})}$, and hence inference can be performed.

Application – quasi-likelihood

Because most features of GLMs only depend on the first two moments of the distribution, rather than the entire distribution, the quasi-likelihood can be developed by just specifying a link function and a variance function. That is, we need to specify

• the link function, ${\displaystyle E[y]=\mu =g^{-1}(\eta )}$
• the variance function, ${\displaystyle V(\mu )}$, where the ${\displaystyle {\mathrm{Var}}_{\theta }(y)={\sigma }^{2}V(\mu )}$

With a specified variance function and link function we can develop, as alternatives to the log-likelihood function, the score function, and the Fisher information, a quasi-likelihood, a quasi-score, and the quasi-information. This allows for full inference of ${\displaystyle \beta }$.

Quasi-likelihood (QL)

Though called a quasi-likelihood, this is in fact a quasi-log-likelihood. The QL for one observation is

${\displaystyle Q_i({\mu }_i,y_i)={\int }_{y_i}^{{\mu }_i}\frac{y_i-t}{{\sigma }^{2}V(t)}dt}$

And therefore the QL for all n observations is

${\displaystyle Q(\mu ,y)=\sum _{i=1}^n{Q}_i({\mu }_i,y_i)=\sum _{i=1}^n{\int }_{y_i}^{{\mu }_i}\frac{y-t}{{\sigma }^{2}V(t)}dt}$

From the QL we have the quasi-score

Quasi-score (QS)

Recall the score function, U, for data with log-likelihood ${\displaystyle \mathrm{l}(\mu \mid y)}$ is

${\displaystyle U=\frac{\mathrm{\partial }l}{d\mu }.}$

We obtain the quasi-score in an identical manner,

${\displaystyle U=\frac{y-\mu }{{\sigma }^{2}V(\mu )}}$

Noting that, for one observation the score is

${\displaystyle \frac{\mathrm{\partial }Q}{\mathrm{\partial }\mu }=\frac{y-\mu }{{\sigma }^{2}V(\mu )}}$

The first two Bartlett equations are satisfied for the quasi-score, namely

${\displaystyle E[U]=0}$

and

${\displaystyle \mathrm{Cov}(U)+E\left[\frac{\mathrm{\partial }U}{\mathrm{\partial }\mu }\right]=0.}$

In addition, the quasi-score is linear in y.

Ultimately the goal is to find information about the parameters of interest ${\displaystyle \beta }$. Both the QS and the QL are actually functions of ${\displaystyle \beta }$. Recall, ${\displaystyle \mu =g^{-1}(\eta )}$, and ${\displaystyle \eta =X\beta }$, therefore,

${\displaystyle \mu =g^{-1}(X\beta ).}$

Quasi-information (QI)

The quasi-information, is similar to the Fisher information,

${\displaystyle i_b=-\mathrm{E}\left[\frac{\mathrm{\partial }U}{\mathrm{\partial }\beta }\right]}$

QL, QS, QI as functions of ${\displaystyle \beta }$

The QL, QS and QI all provide the building blocks for inference about the parameters of interest and therefore it is important to express the QL, QS and QI all as functions of ${\displaystyle \beta }$.

Recalling again that ${\displaystyle \mu =g^{-1}(X\beta )}$, we derive the expressions for QL, QS and QI parametrized under ${\displaystyle \beta }$.

Quasi-likelihood in ${\displaystyle \beta }$,

${\displaystyle Q(\beta ,y)={\int }_y^{\mu (\beta )}\frac{y-t}{{\sigma }^{2}V(t)}dt}$

The QS as a function of ${\displaystyle \beta }$ is therefore

${\displaystyle U_j({\beta }_j)=\frac{\mathrm{\partial }}{\mathrm{\partial }{\beta }_j}Q(\beta ,y)=\sum _{i=1}^n\frac{\mathrm{\partial }{\mu }_i}{\mathrm{\partial }{\beta }_j}\frac{y_i-{\mu }_i({\beta }_j)}{{\sigma }^{2}V({\mu }_i)}}$

${\displaystyle U(\beta )=\left[\begin{array}{c}{U}_1(\beta )\\ U_2(\beta )\\ ⋮\\ ⋮\\ U_p(\beta )\end{array}\right]=D^T{V}^{-1}\frac{(y-\mu )}{{\sigma }^2}}$

Where,

${\displaystyle \underset{n\times p}{\underbrace{D}}=\left[\begin{array}{cccc}\frac{\mathrm{\partial }{\mu }_1}{\mathrm{\partial }{\beta }_1}& \cdots & \cdots & \frac{\mathrm{\partial }{\mu }_1}{\mathrm{\partial }{\beta }_p}\\ \frac{\mathrm{\partial }{\mu }_2}{\mathrm{\partial }{\beta }_1}& \cdots & \cdots & \frac{\mathrm{\partial }{\mu }_2}{\mathrm{\partial }{\beta }_p}\\ ⋮\\ ⋮\\ \frac{\mathrm{\partial }{\mu }_m}{\mathrm{\partial }{\beta }_1}& \cdots & \cdots & \frac{\mathrm{\partial }{\mu }_m}{\mathrm{\partial }{\beta }_p}\end{array}\right]\underset{n\times n}{\underbrace{V}}=\mathrm{diag}(V({\mu }_1),V({\mu }_2),\dots ,\dots ,V({\mu }_n))}$

The quasi-information matrix in ${\displaystyle \beta }$ is,

${\displaystyle i_b=-\frac{\mathrm{\partial }U}{\mathrm{\partial }\beta }=\mathrm{Cov}(U(\beta ))=\frac{D^T{V}^{-1}D}{{\sigma }^2}}$

Obtaining the score function and the information of ${\displaystyle \beta }$ allows for parameter estimation and inference in a similar manner as described in Application – weighted least squares.

Non-parametric regression analysis

A scattor plot of years in the major league against salary (x$1000). The line is the trend in the mean. The plot demonstrates that the variance is not constant.

The smoothed conditional variance against the smoothed conditional mean. The quadratic shape is indicative of the Gamma Distribution. The variance function of a Gamma is V(${\displaystyle \mu }$) = ${\displaystyle {\mu }^2}$

Non-parametric estimation of the variance function and its importance, has been discussed widely in the literature In non-parametric regression analysis, the goal is to express the expected value of your response variable(y) as a function of your predictors (X). That is we are looking to estimate a mean function, ${\displaystyle g(x)=\mathrm{E}[y\mid X=x]}$ without assuming a parametric form. There are many forms of non-parametric smoothing methods to help estimate the function ${\displaystyle g(x)}$. An interesting approach is to also look at a non-parametric variance function, ${\displaystyle g_v(x)=\mathrm{Var}(Y\mid X=x)}$. A non-parametric variance function allows one to look at the mean function as it relates to the variance function and notice patterns in the data.

${\displaystyle g_v(x)=\mathrm{Var}(Y\mid X=x)=\mathrm{E}[y^2\mid X=x]-{\left[\mathrm{E}[y\mid X=x]\right]}^2}$

An example is detailed in the pictures to the right. The goal of the project was to determine (among other things) whether or not the predictor, number of years in the major leagues (baseball,) had an effect on the response, salary, a player made. An initial scatter plot of the data indicates that there is heteroscedasticity in the data as the variance is not constant at each level of the predictor. Because we can visually detect the non-constant variance, it useful now to plot ${\displaystyle g_v(x)=\mathrm{Var}(Y\mid X=x)=\mathrm{E}[y^2\mid X=x]-{\left[\mathrm{E}[y\mid X=x]\right]}^2}$, and look to see if the shape is indicative of any known distribution. One can estimate ${\displaystyle \mathrm{E}[y^2\mid X=x]}$ and ${\displaystyle {\left[\mathrm{E}[y\mid X=x]\right]}^2}$ using a general smoothing method. The plot of the non-parametric smoothed variance function can give the researcher an idea of the relationship between the variance and the mean. The picture to the right indicates a quadratic relationship between the mean and the variance. As we saw above, the Gamma variance function is quadratic in the mean.

Signs and symptoms

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Signs_and_symptoms

Signs (including enlarged liver and spleen) and symptoms (including headache and vomiting) of acute HIV infection.

Signs and symptoms are the observed or detectable signs, and experienced symptoms of an illness, injury, or condition.

Signs are objective and externally observable; symptoms are a person's reported subjective experiences. A sign for example may be a higher or lower temperature than normal, raised or lowered blood pressure or an abnormality showing on a medical scan. A symptom is something out of the ordinary that is experienced by an individual such as feeling feverish, a headache or other pains in the body.

Signs and symptoms

Signs

A medical sign is an objective observable indication of a disease, injury, or medical condition that may be detected during a physical examination. These signs may be visible, such as a rash or bruise, or otherwise detectable such as by using a stethoscope or taking blood pressure. Medical signs, along with symptoms, help in forming a diagnosis. Some examples of signs are nail clubbing of either the fingernails or toenails, an abnormal gait, and a limbal ring a darkened ring around the iris of the eye.

Indications

Main article: Indication (medicine)

A sign is different from an "indication" – the activity of a condition 'pointing to' (thus "indicating") a remedy, not the reverse (viz., it is not a remedy 'pointing to' a condition) – which is a specific reason for using a particular treatment.

Symptoms

A symptom is something felt or experienced, such as pain or dizziness. Signs and symptoms are not mutually exclusive, for example a subjective feeling of fever can be noted as sign by using a thermometer that registers a high reading. The CDC lists various diseases by their signs and symptoms such as for measles which includes a high fever, conjunctivitis, and cough, followed a few days later by the measles rash.

Cardinal signs and symptoms

Cardinal signs and symptoms are specific even to the point of being pathognomonic. A cardinal sign or cardinal symptom can also refer to the major sign or symptom of a disease. Abnormal reflexes can indicate problems with the nervous system. Signs and symptoms are also applied to physiological states outside the context of disease, as for example when referring to the signs and symptoms of pregnancy, or the symptoms of dehydration. Sometimes a disease may be present without showing any signs or symptoms when it is known as being asymptomatic. The disorder may be discovered through tests including scans. An infection may be asymptomatic which may still be transmissible.

Syndrome

Main article: Syndrome

Signs and symptoms are often non-specific, but some combinations can be suggestive of certain diagnoses, helping to narrow down what may be wrong. A particular set of characteristic signs and symptoms that may be associated with a disorder is known as a syndrome. In cases where the underlying cause is known the syndrome is named as for example Down syndrome and Noonan syndrome. Other syndromes such as acute coronary syndrome may have a number of possible causes.

Terms

When a disease is evidenced by symptoms it is known as symptomatic. There are many conditions including subclinical infections that display no symptoms, and these are termed asymptomatic. Signs and symptoms may be mild or severe, brief or longer-lasting when they may become reduced (remission), or then recur (relapse or recrudescence) known as a flare-up. A flare-up may show more severe symptoms.

The term chief complaint, also "presenting problem", is used to describe the initial concern of an individual when seeking medical help, and once this is clearly noted a history of the present illness may be taken. The symptom that ultimately leads to a diagnosis is called a cardinal symptom. Some symptoms can be misleading as a result of referred pain, where for example a pain in the right shoulder may be due to an inflamed gallbladder and not to presumed muscle strain.

Prodrome

Many diseases have an early prodromal stage where a few signs and symptoms may suggest the presence of a disorder before further specific symptoms may emerge. Measles for example has a prodromal presentation that includes a hacking cough, fever, and Koplik's spots in the mouth. Over half of migraine episodes have a prodromal phase. Schizophrenia has a notable prodromal stage, as has dementia.

Nonspecific symptoms

Some symptoms are specific, that is, they are associated with a single, specific medical condition.

Nonspecific symptoms, sometimes also called equivocal symptoms, are not specific to a particular condition. They include unexplained weight loss, headache, pain, fatigue, loss of appetite, night sweats, and malaise. A group of three particular nonspecific symptoms – fever, night sweats, and weight loss – over a period of six months are termed B symptoms associated with lymphoma and indicate a poor prognosis.

Other sub-types of symptoms include:

• constitutional or general symptoms, which affect general well-being or the whole body, such as a fever;
• concomitant symptoms, which are symptoms that occur at the same time as the primary symptom;
• prodromal symptoms, which are the first symptoms of an bigger set of problems;
• delayed symptoms, which happen some time after the trigger; and
• objective symptoms, which are symptoms whose existence can be observed and confirmed by a healthcare provider.

Vital signs

Vital signs are the four signs that can give an immediate measurement of the body's overall functioning and health status. They are temperature, heart rate, breathing rate, and blood pressure. The ranges of these measurements vary with age, weight, gender and with general health.

A digital application has been developed for use in clinical settings that measures three of the vital signs (not temperature) using just a smartphone, and has been approved by NHS England. The application is registered as Lifelight First, and Lifelight Home is under development (2020) for monitoring-use by people at home using just the camera on their smartphone or tablet. This will additionally measure oxygen saturation and atrial fibrillation. Other devices are then not needed.

Syndromes

Further information: List of syndromes and List of medical triads, tetrads, and pentads

Many conditions are indicated by a group of known signs, or signs and symptoms. These can be a group of three known as a triad: a group of four known as a tetrad, and a group of five known as a petrad. An example of a triad is Meltzer's triad presenting purpura a rash, arthralgia painful joints, and myalgia painful and weak muscles. Meltzer's triad indicates the condition cryoglobulinemia. Huntington's disease is a neurodegenerative disease that is characterized by a triad of motor, cognitive, and psychiatric signs and symptoms. A large number of these groups that can be characteristic of a particular disease are known as a syndrome. Noonan syndrome for example, has a diagnostic set of unique facial and musculoskeletal features. Some syndromes such as nephrotic syndrome may have a number of underlying causes that are all related to diseases that affect the kidneys.

Sometimes a child or young adult may have symptoms suggestive of a genetic disorder that cannot be identified even after genetic testing. In such cases the term SWAN (syndrome without a name) may be used. Often a diagnosis may be made at some future point when other more specific symptoms emerge but many cases may remain undiagnosed. The inability to diagnose may be due to a unique combination of symptoms or an overlap of conditions, or to the symptoms being atypical of a known disorder, or to the disorder being extremely rare.

Positive and negative

Sensory symptoms can also be described as positive symptoms, or as negative symptoms depending on whether the symptom is abnormally present such as tingling or itchiness, or abnormally absent such as loss of smell. The following terms are used for negative symptoms – hypoesthesia is a partial loss of sensitivity to moderate stimuli, such as pressure, touch, warmth, cold. Anesthesia is the complete loss of sensitivity to stronger stimuli, such as pinprick. Hypoalgesia (analgesia) is loss of sensation to painful stimuli.

Symptoms are also grouped in to negative and positive for some mental disorders such as schizophrenia.

Positive symptoms are those that are present in the disorder and are not normally experienced by most individuals and reflects an excess or distortion of normal functions. Examples are hallucinations, delusions, and bizarre behavior.

Negative symptoms are functions that are normally found but that are diminished or absent; for example, the negative symptoms of schizophrenia include apathy and anhedonia.

Dynamic and static

Dynamic symptoms are capable of change depending on circumstance, whereas static symptoms are fixed or unchanging regardless of circumstance. For example, the symptoms of exercise intolerance are dynamic as they are brought on by exercise, but alleviate during rest. Fixed muscle weakness is a static symptom as the muscle will be weak regardless of exercise or rest.

A majority of patients with metabolic myopathies have dynamic rather than static findings, typically experiencing exercise intolerance, muscle pain, and cramps with exercise rather than fixed weakness. Those with the metabolic myopathy of McArdle's disease (GSD-V) and some individuals with phosphoglucomutase deficiency (CDG1T/GSD-XIV), initially experience exercise intolerance during mild-moderate aerobic exercise, but the symptoms alleviate after 6–10 minutes in what is known as "second wind".

Neuropsychiatric

Neuropsychiatric symptoms are present in many degenerative disorders including dementia, and Parkinson's disease. Symptoms commonly include apathy, anxiety, and depression. Neurological and psychiatric symptoms are also present in some genetic disorders such as Wilson's disease. Symptoms of executive dysfunction are often found in many disorders including schizophrenia, and ADHD.

Radiologic

Radiologic signs are abnormal medical findings on imaging scanning. These include the Mickey Mouse sign and the Golden S sign. When using imaging to find the cause of a complaint, another unrelated finding may be found known as an incidental finding.

Cardinal

Cardinal signs and symptoms are those that may be diagnostic, and pathognomonic – of a certainty of diagnosis. Inflammation for example has a recognised group of cardinal signs and symptoms, as does exacerbations of chronic bronchitis, and Parkinson's disease.

In contrast to a pathognomonic cardinal sign, the absence of a sign or symptom can often rule out a condition. This is known by the Latin term sine qua non. For example, the absence of known genetic mutations specific for a hereditary disease would rule out that disease. Another example is where the vaginal pH is less than 4.5, a diagnosis of bacterial vaginosis would be excluded.

Reflexes

A reflex is an automatic response in the body to a stimulus. Its absence, reduced (hypoactive), or exaggerated (hyperactive) response can be a sign of damage to the central nervous system or peripheral nervous system. In the patellar reflex (knee-jerk) for example, its reduction or absence is known as Westphal's sign and may indicate damage to lower motor neurons. When the response is exaggerated damage to the upper motor neurons may be indicated.

Facies

A number of medical conditions are associated with a distinctive facial expression or appearance known as a facies An example is elfin facies which has facial features like those of the elf, and this may be associated with Williams syndrome, or Donohue syndrome. The most well-known facies is probably the Hippocratic facies that is seen on a person as they near death.

Anamnestic signs

Anamnestic signs (from anamnēstikós, ἀναμνηστικός, "able to recall to mind") are signs that indicate a past condition, for example paralysis in an arm may indicate a past stroke.

Asymptomatic

Some diseases including cancers, and infections may be present but show no signs or symptoms and these are known as asymptomatic. A gallstone may be asymptomatic and only discovered as an incidental finding. Easily spreadable viral infections such as COVID-19 may be asymptomatic but may still be transmissible.

History

Symptomatology

A symptom (from Greek σύμπτωμα, "accident, misfortune, that which befalls", from συμπίπτω, "I befall", from συν- "together, with" and πίπτω, "I fall") is a departure from normal function or feeling. Symptomatology (also called semiology) is a branch of medicine dealing with the signs and symptoms of a disease. This study also includes the indications of a disease. It was first described as semiotics by Henry Stubbe in 1670 a term now used for the study of sign communication.

Prior to the nineteenth century there was little difference in the powers of observation between physician and patient. Most medical practice was conducted as a co-operative interaction between the physician and patient; this was gradually replaced by a "monolithic consensus of opinion imposed from within the community of medical investigators". Whilst each noticed much the same things, the physician had a more informed interpretation of those things: "the physicians knew what the findings meant and the layman did not".

Development of medical testing

Further information: Medical test

Painting of René Laennec in 1816 using an early method of auscultation on a man with tuberculosis.

A number of advances introduced mostly in the 19th century, allowed for more objective assessment by the physician in search of a diagnosis, and less need of input from the patient. During the 20th century the introduction of a wide range of imaging techniques and other testing methods such as genetic testing, clinical chemistry tests, molecular diagnostics and pathogenomics have made a huge impact on diagnostic capability.

• In 1761 the percussion technique for diagnosing respiratory conditions was discovered by Leopold Auenbrugger. This method of tapping body cavities to note any abnormal sounds had already been in practice for a long time in cardiology. Percussion of the thorax became more widely known after 1808 with the translation of Auenbrugger's work from Latin into French by Jean-Nicolas Corvisart.
• In 1819 the introduction of the stethoscope by René Laennec began to replace the centuries-old technique of immediate auscultation – listening to the heart by placing the ear directly on the chest, with mediate auscultation using the stethoscope to listen to the sounds of the heart and respiratory tract. Laennec's publication was translated into English, 1824, by John Forbes.
• The 1846 introduction by surgeon John Hutchinson (1811–1861) of the spirometer, an apparatus for assessing the mechanical properties of the lungs via measurements of forced exhalation and forced inhalation. (The recorded lung volumes and air flow rates are used to distinguish between restrictive disease (in which the lung volumes are decreased: e.g., cystic fibrosis) and obstructive diseases (in which the lung volume is normal but the air flow rate is impeded; e.g., emphysema).)
• The 1851 invention by Hermann von Helmholtz (1821–1894) of the ophthalmoscope, which allowed physicians to examine the inside of the human eye.
• The (c. 1870) immediate widespread clinical use of Sir Thomas Clifford Allbutt's (1836–1925) six-inch (rather than twelve-inch) pocket clinical thermometer, which he had devised in 1867.
• The 1882 introduction of bacterial cultures by Robert Koch, initially for tuberculosis, being the first laboratory test to confirm bacterial infections.
• The 1895 clinical use of X-rays which began almost immediately after they had been discovered that year by Wilhelm Conrad Röntgen (1845–1923).
• The 1896 introduction of the sphygmomanometer, designed by Scipione Riva-Rocci (1863–1937), to measure blood pressure.

Diagnosis

The recognition of signs, and noting of symptoms may lead to a diagnosis. Otherwise a physical examination may be carried out, and a medical history taken. Further diagnostic medical tests such as blood tests, scans, and biopsies, may be needed. An X-ray for example would soon be diagnostic of a suspected bone fracture. A noted significance detected during an examination or from a medical test may be known as a medical finding.

Examples

See also: List of eponymous medical signs

Further information: List of medical symptoms § Medical signs and symptoms

• Ascites
• Nail clubbing (deformed nails)
• Cough
• Death rattle (last moments of life)
• Hemoptysis (blood-stained sputum)
• Jaundice
• Organomegaly an enlarged organ such as the liver (hepatomegaly)
• Palmar erythema (reddening of hands)
• Hypersalivation excessive (saliva)
• Unintentional weight loss

Sampling (statistics)

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Sampling_(statistics)

A visual representation of the sampling process

In statistics, quality assurance, and survey methodology, sampling is the selection of a subset or a statistical sample (termed sample for short) of individuals from within a statistical population to estimate characteristics of the whole population. Statisticians attempt to collect samples that are representative of the population. Sampling has lower costs and faster data collection compared to recording data from the entire population, and thus, it can provide insights in cases where it is infeasible to measure an entire population.

Each observation measures one or more properties (such as weight, location, colour or mass) of independent objects or individuals. In survey sampling, weights can be applied to the data to adjust for the sample design, particularly in stratified sampling. Results from probability theory and statistical theory are employed to guide the practice. In business and medical research, sampling is widely used for gathering information about a population. Acceptance sampling is used to determine if a production lot of material meets the governing specifications.

Population definition

Successful statistical practice is based on focused problem definition. In sampling, this includes defining the "population" from which our sample is drawn. A population can be defined as including all people or items with the characteristics one wishes to understand. Because there is very rarely enough time or money to gather information from everyone or everything in a population, the goal becomes finding a representative sample (or subset) of that population.

Sometimes what defines a population is obvious. For example, a manufacturer needs to decide whether a batch of material from production is of high enough quality to be released to the customer or should be scrapped or reworked due to poor quality. In this case, the batch is the population.

Although the population of interest often consists of physical objects, sometimes it is necessary to sample over time, space, or some combination of these dimensions. For instance, an investigation of supermarket staffing could examine checkout line length at various times, or a study on endangered penguins might aim to understand their usage of various hunting grounds over time. For the time dimension, the focus may be on periods or discrete occasions.

In other cases, the examined 'population' may be even less tangible. For example, Joseph Jagger studied the behaviour of roulette wheels at a casino in Monte Carlo, and used this to identify a biased wheel. In this case, the 'population' Jagger wanted to investigate was the overall behaviour of the wheel (i.e. the probability distribution of its results over infinitely many trials), while his 'sample' was formed from observed results from that wheel. Similar considerations arise when taking repeated measurements of some physical characteristic such as the electrical conductivity of copper.

This situation often arises when seeking knowledge about the cause system of which the observed population is an outcome. In such cases, sampling theory may treat the observed population as a sample from a larger 'superpopulation'. For example, a researcher might study the success rate of a new 'quit smoking' program on a test group of 100 patients, in order to predict the effects of the program if it were made available nationwide. Here the superpopulation is "everybody in the country, given access to this treatment" – a group that does not yet exist since the program is not yet available to all.

The population from which the sample is drawn may not be the same as the population from which information is desired. Often there is a large but not complete overlap between these two groups due to frame issues etc. (see below). Sometimes they may be entirely separate – for instance, one might study rats in order to get a better understanding of human health, or one might study records from people born in 2008 in order to make predictions about people born in 2009.

Time spent in making the sampled population and population of concern precise is often well spent because it raises many issues, ambiguities, and questions that would otherwise have been overlooked at this stage.

Sampling frame

Main article: Sampling frame

In the most straightforward case, such as the sampling of a batch of material from production (acceptance sampling by lots), it would be most desirable to identify and measure every single item in the population and to include any one of them in our sample. However, in the more general case this is not usually possible or practical. There is no way to identify all rats in the set of all rats. Where voting is not compulsory, there is no way to identify which people will vote at a forthcoming election (in advance of the election). These imprecise populations are not amenable to sampling in any of the ways below and to which we could apply statistical theory.

As a remedy, we seek a sampling frame which has the property that we can identify every single element and include any in our sample. The most straightforward type of frame is a list of elements of the population (preferably the entire population) with appropriate contact information. For example, in an opinion poll, possible sampling frames include an electoral register and a telephone directory.

A probability sample is a sample in which every unit in the population has a chance (greater than zero) of being selected in the sample, and this probability can be accurately determined. The combination of these traits makes it possible to produce unbiased estimates of population totals, by weighting sampled units according to their probability of selection.

Example: We want to estimate the total income of adults living in a given street. We visit each household in that street, identify all adults living there, and randomly select one adult from each household. (For example, we can allocate each person a random number, generated from a uniform distribution between 0 and 1, and select the person with the highest number in each household). We then interview the selected person and find their income.

People living on their own are certain to be selected, so we simply add their income to our estimate of the total. But a person living in a household of two adults has only a one-in-two chance of selection. To reflect this, when we come to such a household, we would count the selected person's income twice towards the total. (The person who is selected from that household can be loosely viewed as also representing the person who isn't selected.)

In the above example, not everybody has the same probability of selection; what makes it a probability sample is the fact that each person's probability is known. When every element in the population does have the same probability of selection, this is known as an 'equal probability of selection' (EPS) design. Such designs are also referred to as 'self-weighting' because all sampled units are given the same weight.

Probability sampling includes: Simple Random Sampling, Systematic Sampling, Stratified Sampling, Probability Proportional to Size Sampling, and Cluster or Multistage Sampling. These various ways of probability sampling have two things in common:

1. Every element has a known nonzero probability of being sampled and
2. involves random selection at some point.

Nonprobability sampling

Main article: Nonprobability sampling

Nonprobability sampling is any sampling method where some elements of the population have no chance of selection (these are sometimes referred to as 'out of coverage'/'undercovered'), or where the probability of selection cannot be accurately determined. It involves the selection of elements based on assumptions regarding the population of interest, which forms the criteria for selection. Hence, because the selection of elements is nonrandom, nonprobability sampling does not allow the estimation of sampling errors. These conditions give rise to exclusion bias, placing limits on how much information a sample can provide about the population. Information about the relationship between sample and population is limited, making it difficult to extrapolate from the sample to the population.

Example: We visit every household in a given street, and interview the first person to answer the door. In any household with more than one occupant, this is a nonprobability sample, because some people are more likely to answer the door (e.g. an unemployed person who spends most of their time at home is more likely to answer than an employed housemate who might be at work when the interviewer calls) and it's not practical to calculate these probabilities.

Nonprobability sampling methods include convenience sampling, quota sampling, and purposive sampling. In addition, nonresponse effects may turn any probability design into a nonprobability design if the characteristics of nonresponse are not well understood, since nonresponse effectively modifies each element's probability of being sampled.

Sampling methods

Within any of the types of frames identified above, a variety of sampling methods can be employed individually or in combination. Factors commonly influencing the choice between these designs include:

• Nature and quality of the frame
• Availability of auxiliary information about units on the frame
• Accuracy requirements, and the need to measure accuracy
• Whether detailed analysis of the sample is expected
• Cost/operational concerns

Simple random sampling

Main article: Simple random sampling

A visual representation of selecting a simple random sample

In a simple random sample (SRS) of a given size, all subsets of a sampling frame have an equal probability of being selected. Each element of the frame thus has an equal probability of selection: the frame is not subdivided or partitioned. Furthermore, any given pair of elements has the same chance of selection as any other such pair (and similarly for triples, and so on). This minimizes bias and simplifies analysis of results. In particular, the variance between individual results within the sample is a good indicator of variance in the overall population, which makes it relatively easy to estimate the accuracy of results.

Simple random sampling can be vulnerable to sampling error because the randomness of the selection may result in a sample that does not reflect the makeup of the population. For instance, a simple random sample of ten people from a given country will on average produce five men and five women, but any given trial is likely to over represent one sex and underrepresent the other. Systematic and stratified techniques attempt to overcome this problem by "using information about the population" to choose a more "representative" sample.

Also, simple random sampling can be cumbersome and tedious when sampling from a large target population. In some cases, investigators are interested in research questions specific to subgroups of the population. For example, researchers might be interested in examining whether cognitive ability as a predictor of job performance is equally applicable across racial groups. Simple random sampling cannot accommodate the needs of researchers in this situation, because it does not provide subsamples of the population, and other sampling strategies, such as stratified sampling, can be used instead.

Systematic sampling

Main article: Systematic sampling

A visual representation of selecting a random sample using the systematic sampling technique

Systematic sampling (also known as interval sampling) relies on arranging the study population according to some ordering scheme and then selecting elements at regular intervals through that ordered list. Systematic sampling involves a random start and then proceeds with the selection of every kth element from then onwards. In this case, k=(population size/sample size). It is important that the starting point is not automatically the first in the list, but is instead randomly chosen from within the first to the kth element in the list. A simple example would be to select every 10th name from the telephone directory (an 'every 10th' sample, also referred to as 'sampling with a skip of 10').

As long as the starting point is randomized, systematic sampling is a type of probability sampling. It is easy to implement and the stratification induced can make it efficient, if the variable by which the list is ordered is correlated with the variable of interest. 'Every 10th' sampling is especially useful for efficient sampling from databases.

For example, suppose we wish to sample people from a long street that starts in a poor area (house No. 1) and ends in an expensive district (house No. 1000). A simple random selection of addresses from this street could easily end up with too many from the high end and too few from the low end (or vice versa), leading to an unrepresentative sample. Selecting (e.g.) every 10th street number along the street ensures that the sample is spread evenly along the length of the street, representing all of these districts. (Note that if we always start at house #1 and end at #991, the sample is slightly biased towards the low end; by randomly selecting the start between #1 and #10, this bias is eliminated.)

However, systematic sampling is especially vulnerable to periodicities in the list. If periodicity is present and the period is a multiple or factor of the interval used, the sample is especially likely to be unrepresentative of the overall population, making the scheme less accurate than simple random sampling.

For example, consider a street where the odd-numbered houses are all on the north (expensive) side of the road, and the even-numbered houses are all on the south (cheap) side. Under the sampling scheme given above, it is impossible to get a representative sample; either the houses sampled will all be from the odd-numbered, expensive side, or they will all be from the even-numbered, cheap side, unless the researcher has previous knowledge of this bias and avoids it by a using a skip which ensures jumping between the two sides (any odd-numbered skip).

Another drawback of systematic sampling is that even in scenarios where it is more accurate than SRS, its theoretical properties make it difficult to quantify that accuracy. (In the two examples of systematic sampling that are given above, much of the potential sampling error is due to variation between neighbouring houses – but because this method never selects two neighbouring houses, the sample will not give us any information on that variation.)

As described above, systematic sampling is an EPS method, because all elements have the same probability of selection (in the example given, one in ten). It is not 'simple random sampling' because different subsets of the same size have different selection probabilities – e.g. the set {4,14,24,...,994} has a one-in-ten probability of selection, but the set {4,13,24,34,...} has zero probability of selection.

Systematic sampling can also be adapted to a non-EPS approach; for an example, see discussion of PPS samples below.

Stratified sampling

Main article: Stratified sampling

A visual representation of selecting a random sample using the stratified sampling technique

When the population embraces a number of distinct categories, the frame can be organized by these categories into separate "strata." Each stratum is then sampled as an independent sub-population, out of which individual elements can be randomly selected. The ratio of the size of this random selection (or sample) to the size of the population is called a sampling fraction. There are several potential benefits to stratified sampling.

First, dividing the population into distinct, independent strata can enable researchers to draw inferences about specific subgroups that may be lost in a more generalized random sample.

Second, utilizing a stratified sampling method can lead to more efficient statistical estimates (provided that strata are selected based upon relevance to the criterion in question, instead of availability of the samples). Even if a stratified sampling approach does not lead to increased statistical efficiency, such a tactic will not result in less efficiency than would simple random sampling, provided that each stratum is proportional to the group's size in the population.

Third, it is sometimes the case that data are more readily available for individual, pre-existing strata within a population than for the overall population; in such cases, using a stratified sampling approach may be more convenient than aggregating data across groups (though this may potentially be at odds with the previously noted importance of utilizing criterion-relevant strata).

Finally, since each stratum is treated as an independent population, different sampling approaches can be applied to different strata, potentially enabling researchers to use the approach best suited (or most cost-effective) for each identified subgroup within the population.

There are, however, some potential drawbacks to using stratified sampling. First, identifying strata and implementing such an approach can increase the cost and complexity of sample selection, as well as leading to increased complexity of population estimates. Second, when examining multiple criteria, stratifying variables may be related to some, but not to others, further complicating the design, and potentially reducing the utility of the strata. Finally, in some cases (such as designs with a large number of strata, or those with a specified minimum sample size per group), stratified sampling can potentially require a larger sample than would other methods (although in most cases, the required sample size would be no larger than would be required for simple random sampling).

A stratified sampling approach is most effective when three conditions are met

1. Variability within strata are minimized
2. Variability between strata are maximized
3. The variables upon which the population is stratified are strongly correlated with the desired dependent variable.

Advantages over other sampling methods

1. Focuses on important subpopulations and ignores irrelevant ones.
2. Allows use of different sampling techniques for different subpopulations.
3. Improves the accuracy/efficiency of estimation.
4. Permits greater balancing of statistical power of tests of differences between strata by sampling equal numbers from strata varying widely in size.

Disadvantages

1. Requires selection of relevant stratification variables which can be difficult.
2. Is not useful when there are no homogeneous subgroups.
3. Can be expensive to implement.

Poststratification

Stratification is sometimes introduced after the sampling phase in a process called "poststratification". This approach is typically implemented due to a lack of prior knowledge of an appropriate stratifying variable or when the experimenter lacks the necessary information to create a stratifying variable during the sampling phase. Although the method is susceptible to the pitfalls of post hoc approaches, it can provide several benefits in the right situation. Implementation usually follows a simple random sample. In addition to allowing for stratification on an ancillary variable, poststratification can be used to implement weighting, which can improve the precision of a sample's estimates.

Oversampling

Choice-based sampling is one of the stratified sampling strategies. In choice-based sampling, the data are stratified on the target and a sample is taken from each stratum so that the rare target class will be more represented in the sample. The model is then built on this biased sample. The effects of the input variables on the target are often estimated with more precision with the choice-based sample even when a smaller overall sample size is taken, compared to a random sample. The results usually must be adjusted to correct for the oversampling.

Probability-proportional-to-size sampling

Main article: Probability-proportional-to-size sampling

In some cases the sample designer has access to an "auxiliary variable" or "size measure", believed to be correlated to the variable of interest, for each element in the population. These data can be used to improve accuracy in sample design. One option is to use the auxiliary variable as a basis for stratification, as discussed above.

Another option is probability proportional to size ('PPS') sampling, in which the selection probability for each element is set to be proportional to its size measure, up to a maximum of 1. In a simple PPS design, these selection probabilities can then be used as the basis for Poisson sampling. However, this has the drawback of variable sample size, and different portions of the population may still be over- or under-represented due to chance variation in selections.

Systematic sampling theory can be used to create a probability proportionate to size sample. This is done by treating each count within the size variable as a single sampling unit. Samples are then identified by selecting at even intervals among these counts within the size variable. This method is sometimes called PPS-sequential or monetary unit sampling in the case of audits or forensic sampling.

Example: Suppose we have six schools with populations of 150, 180, 200, 220, 260, and 490 students respectively (total 1500 students), and we want to use student population as the basis for a PPS sample of size three. To do this, we could allocate the first school numbers 1 to 150, the second school 151 to 330 (= 150 + 180), the third school 331 to 530, and so on to the last school (1011 to 1500). We then generate a random start between 1 and 500 (equal to 1500/3) and count through the school populations by multiples of 500. If our random start was 137, we would select the schools which have been allocated numbers 137, 637, and 1137, i.e. the first, fourth, and sixth schools.

The PPS approach can improve accuracy for a given sample size by concentrating sample on large elements that have the greatest impact on population estimates. PPS sampling is commonly used for surveys of businesses, where element size varies greatly and auxiliary information is often available – for instance, a survey attempting to measure the number of guest-nights spent in hotels might use each hotel's number of rooms as an auxiliary variable. In some cases, an older measurement of the variable of interest can be used as an auxiliary variable when attempting to produce more current estimates.

Cluster sampling

A visual representation of selecting a random sample using the cluster sampling technique

Main article: Cluster sampling

Sometimes it is more cost-effective to select respondents in groups ('clusters'). Sampling is often clustered by geography, or by time periods. (Nearly all samples are in some sense 'clustered' in time – although this is rarely taken into account in the analysis.) For instance, if surveying households within a city, we might choose to select 100 city blocks and then interview every household within the selected blocks.

Clustering can reduce travel and administrative costs. In the example above, an interviewer can make a single trip to visit several households in one block, rather than having to drive to a different block for each household.

It also means that one does not need a sampling frame listing all elements in the target population. Instead, clusters can be chosen from a cluster-level frame, with an element-level frame created only for the selected clusters. In the example above, the sample only requires a block-level city map for initial selections, and then a household-level map of the 100 selected blocks, rather than a household-level map of the whole city.

Cluster sampling (also known as clustered sampling) generally increases the variability of sample estimates above that of simple random sampling, depending on how the clusters differ between one another as compared to the within-cluster variation. For this reason, cluster sampling requires a larger sample than SRS to achieve the same level of accuracy – but cost savings from clustering might still make this a cheaper option.

Cluster sampling is commonly implemented as multistage sampling. This is a complex form of cluster sampling in which two or more levels of units are embedded one in the other. The first stage consists of constructing the clusters that will be used to sample from. In the second stage, a sample of primary units is randomly selected from each cluster (rather than using all units contained in all selected clusters). In following stages, in each of those selected clusters, additional samples of units are selected, and so on. All ultimate units (individuals, for instance) selected at the last step of this procedure are then surveyed. This technique, thus, is essentially the process of taking random subsamples of preceding random samples.

Multistage sampling can substantially reduce sampling costs, where the complete population list would need to be constructed (before other sampling methods could be applied). By eliminating the work involved in describing clusters that are not selected, multistage sampling can reduce the large costs associated with traditional cluster sampling. However, each sample may not be a full representative of the whole population.

Quota sampling

Main article: Quota sampling

In quota sampling, the population is first segmented into mutually exclusive sub-groups, just as in stratified sampling. Then judgement is used to select the subjects or units from each segment based on a specified proportion. For example, an interviewer may be told to sample 200 females and 300 males between the age of 45 and 60.

It is this second step which makes the technique one of non-probability sampling. In quota sampling the selection of the sample is non-random. For example, interviewers might be tempted to interview those who look most helpful. The problem is that these samples may be biased because not everyone gets a chance of selection. This random element is its greatest weakness and quota versus probability has been a matter of controversy for several years.

Minimax sampling

In imbalanced datasets, where the sampling ratio does not follow the population statistics, one can resample the dataset in a conservative manner called minimax sampling. The minimax sampling has its origin in Anderson minimax ratio whose value is proved to be 0.5: in a binary classification, the class-sample sizes should be chosen equally. This ratio can be proved to be minimax ratio only under the assumption of LDA classifier with Gaussian distributions. The notion of minimax sampling is recently developed for a general class of classification rules, called class-wise smart classifiers. In this case, the sampling ratio of classes is selected so that the worst case classifier error over all the possible population statistics for class prior probabilities, would be the best.

Accidental sampling

Accidental sampling (sometimes known as grab, convenience or opportunity sampling) is a type of nonprobability sampling which involves the sample being drawn from that part of the population which is close to hand. That is, a population is selected because it is readily available and convenient. It may be through meeting the person or including a person in the sample when one meets them or chosen by finding them through technological means such as the internet or through phone. The researcher using such a sample cannot scientifically make generalizations about the total population from this sample because it would not be representative enough. For example, if the interviewer were to conduct such a survey at a shopping center early in the morning on a given day, the people that they could interview would be limited to those given there at that given time, which would not represent the views of other members of society in such an area, if the survey were to be conducted at different times of day and several times per week. This type of sampling is most useful for pilot testing. Several important considerations for researchers using convenience samples include:

1. Are there controls within the research design or experiment which can serve to lessen the impact of a non-random convenience sample, thereby ensuring the results will be more representative of the population?
2. Is there good reason to believe that a particular convenience sample would or should respond or behave differently than a random sample from the same population?
3. Is the question being asked by the research one that can adequately be answered using a convenience sample?

In social science research, snowball sampling is a similar technique, where existing study subjects are used to recruit more subjects into the sample. Some variants of snowball sampling, such as respondent driven sampling, allow calculation of selection probabilities and are probability sampling methods under certain conditions.

Voluntary Sampling

Further information: Self-selection bias

The voluntary sampling method is a type of non-probability sampling. Volunteers choose to complete a survey.

Volunteers may be invited through advertisements in social media. The target population for advertisements can be selected by characteristics like location, age, sex, income, occupation, education, or interests using tools provided by the social medium. The advertisement may include a message about the research and link to a survey. After following the link and completing the survey, the volunteer submits the data to be included in the sample population. This method can reach a global population but is limited by the campaign budget. Volunteers outside the invited population may also be included in the sample.

It is difficult to make generalizations from this sample because it may not represent the total population. Often, volunteers have a strong interest in the main topic of the survey.

Line-intercept sampling

Line-intercept sampling is a method of sampling elements in a region whereby an element is sampled if a chosen line segment, called a "transect", intersects the element.

Panel sampling

Panel sampling is the method of first selecting a group of participants through a random sampling method and then asking that group for (potentially the same) information several times over a period of time. Therefore, each participant is interviewed at two or more time points; each period of data collection is called a "wave". The method was developed by sociologist Paul Lazarsfeld in 1938 as a means of studying political campaigns. This longitudinal sampling-method allows estimates of changes in the population, for example with regard to chronic illness to job stress to weekly food expenditures. Panel sampling can also be used to inform researchers about within-person health changes due to age or to help explain changes in continuous dependent variables such as spousal interaction. There have been several proposed methods of analyzing panel data, including MANOVA, growth curves, and structural equation modeling with lagged effects.

Snowball sampling

Snowball sampling involves finding a small group of initial respondents and using them to recruit more respondents. It is particularly useful in cases where the population is hidden or difficult to enumerate.

Theoretical sampling

Theoretical sampling occurs when samples are selected on the basis of the results of the data collected so far with a goal of developing a deeper understanding of the area or develop theories. Extreme or very specific cases might be selected in order to maximize the likelihood a phenomenon will actually be observable.

Replacement of selected units

Sampling schemes may be without replacement ('WOR' – no element can be selected more than once in the same sample) or with replacement ('WR' – an element may appear multiple times in the one sample). For example, if we catch fish, measure them, and immediately return them to the water before continuing with the sample, this is a WR design, because we might end up catching and measuring the same fish more than once. However, if we do not return the fish to the water or tag and release each fish after catching it, this becomes a WOR design.

Sample size determination

Main article: Sample size determination

Formulas, tables, and power function charts are well known approaches to determine sample size.

Steps for using sample size tables:

1. Postulate the effect size of interest, α, and β.
2. Check sample size table
   1. Select the table corresponding to the selected α
   2. Locate the row corresponding to the desired power
   3. Locate the column corresponding to the estimated effect size
   4. The intersection of the column and row is the minimum sample size required.

Sampling and data collection

Good data collection involves:

• Following the defined sampling process
• Keeping the data in time order
• Noting comments and other contextual events
• Recording non-responses

Applications of sampling

Sampling enables the selection of right data points from within the larger data set to estimate the characteristics of the whole population. For example, there are about 600 million tweets produced every day. It is not necessary to look at all of them to determine the topics that are discussed during the day, nor is it necessary to look at all the tweets to determine the sentiment on each of the topics. A theoretical formulation for sampling Twitter data has been developed.

In manufacturing different types of sensory data such as acoustics, vibration, pressure, current, voltage, and controller data are available at short time intervals. To predict down-time it may not be necessary to look at all the data but a sample may be sufficient.

Errors in sample surveys

Main article: Sampling error

Survey results are typically subject to some error. Total errors can be classified into sampling errors and non-sampling errors. The term "error" here includes systematic biases as well as random errors.

Sampling errors and biases

Sampling errors and biases are induced by the sample design. They include:

1. Selection bias: When the true selection probabilities differ from those assumed in calculating the results.
2. Random sampling error: Random variation in the results due to the elements in the sample being selected at random.

Non-sampling error

Main article: Non-sampling error

Non-sampling errors are other errors which can impact final survey estimates, caused by problems in data collection, processing, or sample design. Such errors may include:

1. Over-coverage: inclusion of data from outside of the population
2. Under-coverage: sampling frame does not include elements in the population.
3. Measurement error: e.g. when respondents misunderstand a question, or find it difficult to answer
4. Processing error: mistakes in data coding
5. Non-response or Participation bias: failure to obtain complete data from all selected individuals

After sampling, a review should be held of the exact process followed in sampling, rather than that intended, in order to study any effects that any divergences might have on subsequent analysis.

A particular problem involves non-response. Two major types of non-response exist:

• unit nonresponse (lack of completion of any part of the survey)
• item non-response (submission or participation in survey but failing to complete one or more components/questions of the survey)

In survey sampling, many of the individuals identified as part of the sample may be unwilling to participate, not have the time to participate (opportunity cost), or survey administrators may not have been able to contact them. In this case, there is a risk of differences between respondents and nonrespondents, leading to biased estimates of population parameters. This is often addressed by improving survey design, offering incentives, and conducting follow-up studies which make a repeated attempt to contact the unresponsive and to characterize their similarities and differences with the rest of the frame. The effects can also be mitigated by weighting the data (when population benchmarks are available) or by imputing data based on answers to other questions. Nonresponse is particularly a problem in internet sampling. Reasons for this problem may include improperly designed surveys, over-surveying (or survey fatigue), and the fact that potential participants may have multiple e-mail addresses, which they do not use anymore or do not check regularly.

Survey weights

In many situations the sample fraction may be varied by stratum and data will have to be weighted to correctly represent the population. Thus for example, a simple random sample of individuals in the United Kingdom might not include some in remote Scottish islands who would be inordinately expensive to sample. A cheaper method would be to use a stratified sample with urban and rural strata. The rural sample could be under-represented in the sample, but weighted up appropriately in the analysis to compensate.

More generally, data should usually be weighted if the sample design does not give each individual an equal chance of being selected. For instance, when households have equal selection probabilities but one person is interviewed from within each household, this gives people from large households a smaller chance of being interviewed. This can be accounted for using survey weights. Similarly, households with more than one telephone line have a greater chance of being selected in a random digit dialing sample, and weights can adjust for this.

Weights can also serve other purposes, such as helping to correct for non-response.

Methods of producing random samples

• Random number table
• Mathematical algorithms for pseudo-random number generators
• Physical randomization devices such as coins, playing cards or sophisticated devices such as ERNIE

History

Random sampling by using lots is an old idea, mentioned several times in the Bible. In 1786 Pierre Simon Laplace estimated the population of France by using a sample, along with ratio estimator. He also computed probabilistic estimates of the error. These were not expressed as modern confidence intervals but as the sample size that would be needed to achieve a particular upper bound on the sampling error with probability 1000/1001. His estimates used Bayes' theorem with a uniform prior probability and assumed that his sample was random. Alexander Ivanovich Chuprov introduced sample surveys to Imperial Russia in the 1870s.

In the US the 1936 Literary Digest prediction of a Republican win in the presidential election went badly awry, due to severe bias. More than two million people responded to the study with their names obtained through magazine subscription lists and telephone directories. It was not appreciated that these lists were heavily biased towards Republicans and the resulting sample, though very large, was deeply flawed.

Elections in Singapore have adopted this practice since the 2015 election, also known as the sample counts, whereas according to the Elections Department (ELD), their country's election commission, sample counts help reduce speculation and misinformation, while helping election officials to check against the election result for that electoral division. The reported sample counts yield a fairly accurate indicative result with a 95% confidence interval at a margin of error within 4-5%; ELD reminded the public that sample counts are separate from official results, and only the returning officer will declare the official results once vote counting is complete.