Information field theory

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Information_field_theory 

Information field theory (IFT) is a Bayesian statistical field theory relating to signal reconstruction, cosmography, and other related areas. IFT summarizes the information available on a physical field using Bayesian probabilities. It uses computational techniques developed for quantum field theory and statistical field theory to handle the infinite number of degrees of freedom of a field and to derive algorithms for the calculation of field expectation values. For example, the posterior expectation value of a field generated by a known Gaussian process and measured by a linear device with known Gaussian noise statistics is given by a generalized Wiener filter applied to the measured data. IFT extends such known filter formula to situations with nonlinear physics, nonlinear devices, non-Gaussian field or noise statistics, dependence of the noise statistics on the field values, and partly unknown parameters of measurement. For this it uses Feynman diagrams, renormalisation flow equations, and other methods from mathematical physics.

Motivation

Fields play an important role in science, technology, and economy. They describe the spatial variations of a quantity, like the air temperature, as a function of position. Knowing the configuration of a field can be of large value. Measurements of fields, however, can never provide the precise field configuration with certainty. Physical fields have an infinite number of degrees of freedom, but the data generated by any measurement device is always finite, providing only a finite number of constraints on the field. Thus, an unambiguous deduction of such a field from measurement data alone is impossible and only probabilistic inference remains as a means to make statements about the field. Fortunately, physical fields exhibit correlations and often follow known physical laws. Such information is best fused into the field inference in order to overcome the mismatch of field degrees of freedom to measurement points. To handle this, an information theory for fields is needed, and that is what information field theory is.

Concepts

Bayesian inference

${\displaystyle s(x)}$ is a field value at a location ${\displaystyle x\in \mathrm{\Omega }}$ in a space ${\displaystyle \mathrm{\Omega }}$. The prior knowledge about the unknown signal field ${\displaystyle s}$ is encoded in the probability distribution ${\displaystyle \mathcal{P}(s)}$. The data ${\displaystyle d}$ provides additional information on ${\displaystyle s}$ via the likelihood ${\displaystyle \mathcal{P}(d|s)}$ that gets incorporated into the posterior probability

$${\displaystyle \mathcal{P}(s|d)=\frac{\mathcal{P}(d|s)\mathcal{P}(s)}{\mathcal{P}(d)}}$$

according to Bayes theorem.

Information Hamiltonian

In IFT Bayes theorem is usually rewritten in the language of a statistical field theory,

$${\displaystyle \mathcal{P}(s|d)=\frac{\mathcal{P}(d,s)}{\mathcal{P}(d)}\equiv \frac{e^{-\mathcal{H}(d,s)}}{\mathcal{Z}(d)},}$$

with the information Hamiltonian defined as

$${\displaystyle \mathcal{H}(d,s)\equiv -\ln \mathcal{P}(d,s)=-\ln \mathcal{P}(d|s)-\ln \mathcal{P}(s)\equiv \mathcal{H}(d|s)+\mathcal{H}(s),}$$

the negative logarithm of the joint probability of data and signal and with the partition function being

$${\displaystyle \mathcal{Z}(d)\equiv \mathcal{P}(d)=\int \mathcal{D}s\mathcal{P}(d,s).}$$

This reformulation of Bayes theorem permits the usage of methods of mathematical physics developed for the treatment of statistical field theories and quantum field theories.

Fields

As fields have an infinite number of degrees of freedom, the definition of probabilities over spaces of field configurations has subtleties. Identifying physical fields as elements of function spaces provides the problem that no Lebesgue measure is defined over the latter and therefore probability densities can not be defined there. However, physical fields have much more regularity than most elements of function spaces, as they are continuous and smooth at most of their locations. Therefore less general, but sufficiently flexible constructions can be used to handle the infinite number of degrees of freedom of a field.

A pragmatic approach is to regard the field to be discretized in terms of pixels. Each pixel carries a single field value that is assumed to be constant within the pixel volume. All statements about the continuous field have then to be cast into its pixel representation. This way, one deals with finite dimensional field spaces, over which probability densities are well definable.

In order for this description to be a proper field theory, it is further required that the pixel resolution ${\displaystyle \mathrm{\Delta }x}$ can always be refined, while expectation values of the discretized field ${\displaystyle s_{\mathrm{\Delta }x}}$ converge to finite values:

$${\displaystyle ⟨f(s){⟩}_{(s|d)}\equiv \underset{\mathrm{\Delta }x\to 0}{lim}\int d{s}_{\mathrm{\Delta }x}f(s_{\mathrm{\Delta }x})\mathcal{P}(s_{\mathrm{\Delta }x}).}$$

Path integrals

If this limit exists, one can talk about the field configuration space integral or path integral

$${\displaystyle ⟨f(s){⟩}_{(s|d)}\equiv \int \mathcal{D}sf(s)\mathcal{P}(s).}$$

irrespective of the resolution it might be evaluated numerically.

Gaussian prior

The simplest prior for a field is that of a zero mean Gaussian probability distribution

$${\displaystyle \mathcal{P}(s)=\mathcal{G}(s,S)\equiv \frac{1}{|2\pi S|}{e}^{-\frac{1}{2}{s}^{†}{S}^{-1}s}.}$$

The determinant in the denominator might be ill-defined in the continuum limit ${\displaystyle \mathrm{\Delta }x\to 0}$, however, all what is necessary for IFT to be consistent is that this determinant can be estimated for any finite resolution field representation with ${\displaystyle \mathrm{\Delta }x>0}$ and that this permits the calculation of convergent expectation values.

A Gaussian probability distribution requires the specification of the field two point correlation function ${\displaystyle S\equiv ⟨s{s}^{†}{⟩}_{(s)}}$ with coefficients

$${\displaystyle S_{xy}\equiv ⟨s(x)\overline{s(y)}{⟩}_{(s)}}$$

and a scalar product for continuous fields

$${\displaystyle a^{†}b\equiv {\int }_{\mathrm{\Omega }}dx\overline{a(x)}b(x),}$$

with respect to which the inverse signal field covariance ${\displaystyle S^{-1}}$ is constructed, i.e. ${\displaystyle (S^{-1}S{)}_{xy}\equiv {\int }_{\mathrm{\Omega }}dz(S^{-1}{)}_{xz}{S}_{zy}={\mathbb{1}}_{xy}\equiv \delta (x-y).}$

The corresponding prior information Hamiltonian reads

$${\displaystyle \mathcal{H}(s)=-\ln \mathcal{G}(s,S)=\frac{1}{2}{s}^{†}{S}^{-1}s+\frac{1}{2}\ln |2\pi S|.}$$

Measurement equation

The measurement data ${\displaystyle d}$ was generated with the likelihood ${\displaystyle \mathcal{P}(d|s)}$. In case the instrument was linear, a measurement equation of the form

$${\displaystyle d=Rs+n}$$

can be given, in which ${\displaystyle R}$ is the instrument response, which describes how the data on average reacts to the signal, and ${\displaystyle n}$ is the noise, simply the difference between data ${\displaystyle d}$ and linear signal response ${\displaystyle Rs}$. It is essential to note that the response translates the infinite dimensional signal vector into the finite dimensional data space. In components this reads ${\displaystyle d_i={\int }_{\mathrm{\Omega }}dx{R}_{ix}{s}_x+n_i,}$

where a vector component notation was also introduced for signal and data vectors.

If the noise follows a signal independent zero mean Gaussian statistics with covariance ${\displaystyle N}$, ${\displaystyle \mathcal{P}(n|s)=\mathcal{G}(n,N),}$ then the likelihood is Gaussian as well,

$${\displaystyle \mathcal{P}(d|s)=\mathcal{G}(d-Rs,N),}$$

and the likelihood information Hamiltonian is

$${\displaystyle \mathcal{H}(d|s)=-\ln \mathcal{G}(d-Rs,N)=\frac{1}{2}(d-Rs{)}^{†}{N}^{-1}(d-Rs)+\frac{1}{2}\ln |2\pi N|.}$$

A linear measurement of a Gaussian signal, subject to Gaussian and signal-independent noise leads to a free IFT.

Free theory

Free Hamiltonian

The joint information Hamiltonian of the Gaussian scenario described above is

$${\displaystyle \begin{array}{rl}\mathcal{H}(d,s)& =\mathcal{H}(d|s)+\mathcal{H}(s)\\ & \widehat{=}\frac{1}{2}(d-Rs{)}^{†}{N}^{-1}(d-Rs)+\frac{1}{2}{s}^{†}{S}^{-1}s\\ & \widehat{=}\frac{1}{2}\left[s^{†}\underset{D^{-1}}{\underbrace{(S^{-1}+R^{†}{N}^{-1}R)}}s-s^{†}\underset{j}{\underbrace{R^{†}{N}^{-1}d}}-\underset{j^{†}}{\underbrace{d^{†}{N}^{-1}R}}s\right]\\ & \equiv \frac{1}{2}\left[s^{†}{D}^{-1}s-s^{†}j-j^{†}s\right]\\ & =\frac{1}{2}\left[s^{†}{D}^{-1}s-s^{†}{D}^{-1}\underset{m}{\underbrace{Dj}}-\underset{m^{†}}{\underbrace{j^{†}D}}{D}^{-1}s\right]\\ & \widehat{=}\frac{1}{2}(s-m{)}^{†}{D}^{-1}(s-m),\end{array}}$$

where ${\displaystyle \widehat{=}}$ denotes equality up to irrelevant constants, which, in this case, means expressions that are independent of ${\displaystyle s}$. From this is it clear, that the posterior must be a Gaussian with mean ${\displaystyle m}$ and variance ${\displaystyle D}$,

$${\displaystyle \mathcal{P}(s|d)\propto e^{-\mathcal{H}(d,s)}\propto e^{-\frac{1}{2}(s-m{)}^{†}{D}^{-1}(s-m)}\propto \mathcal{G}(s-m,D)}$$

where equality between the right and left hand sides holds as both distributions are normalized, ${\displaystyle \int \mathcal{D}s\mathcal{P}(s|d)=1=\int \mathcal{D}s\mathcal{G}(s-m,D)}$.

Generalized Wiener filter

The posterior mean

$${\displaystyle m=Dj=(S^{-1}+R^{†}{N}^{-1}R{)}^{-1}{R}^{†}{N}^{-1}d}$$

is also known as the generalized Wiener filter solution and the uncertainty covariance

$${\displaystyle D=(S^{-1}+R^{†}{N}^{-1}R{)}^{-1}}$$

as the Wiener variance.

In IFT, ${\displaystyle j=R^{†}{N}^{-1}d}$ is called the information source, as it acts as a source term to excite the field (knowledge), and ${\displaystyle D}$ the information propagator, as it propagates information from one location to another in

$${\displaystyle m_x={\int }_{\mathrm{\Omega }}dy{D}_{xy}{j}_y.}$$

Interacting theory

Interacting Hamiltonian

If any of the assumptions that lead to the free theory is violated, IFT becomes an interacting theory, with terms that are of higher than quadratic order in the signal field. This happens when the signal or the noise are not following Gaussian statistics, when the response is non-linear, when the noise depends on the signal, or when response or covariances are uncertain.

In this case, the information Hamiltonian might be expandable in a Taylor-Fréchet series,

$${\displaystyle \mathcal{H}(d,s)=\underset{={\mathcal{H}}_{\text{free}}(d,s)}{\underbrace{\frac{1}{2}{s}^{†}{D}^{-1}s-j^{†}s+{\mathcal{H}}_0}}+\underset{={\mathcal{H}}_{\text{int}}(d,s)}{\underbrace{\sum _{n=3}^{\mathrm{\infty }}\frac{1}{n!}{\mathrm{\Lambda }}_{x_1...x_n}^{(n)}{s}_{x_1}...s_{x_n}}},}$$

where ${\displaystyle {\mathcal{H}}_{\text{free}}(d,s)}$ is the free Hamiltonian, which alone would lead to a Gaussian posterior, and ${\displaystyle {\mathcal{H}}_{\text{int}}(d,s)}$ is the interacting Hamiltonian, which encodes non-Gaussian corrections. The first and second order Taylor coefficients are often identified with the (negative) information source ${\displaystyle -j}$ and information propagator ${\displaystyle D}$, respectively. The higher coefficients ${\displaystyle {\mathrm{\Lambda }}_{x_1...x_n}^{(n)}}$are associated with non-linear self-interactions.

Classical field

The classical field ${\displaystyle s_{\text{cl}}}$ minimizes the information Hamiltonian,

$${\displaystyle {\frac{\mathrm{\partial }\mathcal{H}(d,s)}{\mathrm{\partial }s}|}_{s=s_{\text{cl}}}=0,}$$

and therefore maximizes the posterior:

$${\displaystyle {\frac{\mathrm{\partial }\mathcal{P}(s|d)}{\mathrm{\partial }s}|}_{s=s_{\text{cl}}}={\frac{\mathrm{\partial }}{\mathrm{\partial }s}\frac{e^{-\mathcal{H}(d,s)}}{\mathcal{Z}(d)}|}_{s=s_{\text{cl}}}=-\mathcal{P}(d,s)\underset{=0}{\underbrace{{\frac{\mathrm{\partial }\mathcal{H}(d,s)}{\mathrm{\partial }s}|}_{s=s_{\text{cl}}}}}=0}$$

The classical field ${\displaystyle s_{\text{cl}}}$ is therefore the maximum a posteriori estimator of the field inference problem.

Critical filter

The Wiener filter problem requires the two point correlation ${\displaystyle S\equiv ⟨s{s}^{†}{⟩}_{(s)}}$ of a field to be known. If it is unknown, it has to be inferred along with the field itself. This requires the specification of a hyperprior ${\displaystyle \mathcal{P}(S)}$. Often, statistical homogeneity (translation invariance) can be assumed, implying that ${\displaystyle S}$ is diagonal in Fourier space (for ${\displaystyle \mathrm{\Omega }={\mathbb{R}}^u}$ being a ${\displaystyle u}$ dimensional Cartesian space). In this case, only the Fourier space power spectrum ${\displaystyle P_s(\overrightarrow{k})}$ needs to be inferred. Given a further assumption of statistical isotropy, this spectrum depends only on the length ${\displaystyle k=|\overrightarrow{k}|}$ of the Fourier vector ${\displaystyle \overrightarrow{k}}$ and only a one dimensional spectrum ${\displaystyle P_s(k)}$ has to be determined. The prior field covariance reads then in Fourier space coordinates ${\displaystyle S_{\overrightarrow{k}\overrightarrow{q}}=(2\pi {)}^u\delta (\overrightarrow{k}-\overrightarrow{q})P_s(k)}$.

If the prior on ${\displaystyle P_s(k)}$ is flat, the joint probability of data and spectrum is

$${\displaystyle \begin{array}{rl}\mathcal{P}(d,P_s)& =\int \mathcal{D}s\mathcal{P}(d,s,P_s)\\ & =\int \mathcal{D}s\mathcal{P}(d|s,P_s)\mathcal{P}(s|P_s)\mathcal{P}(P_s)\\ & \propto \int \mathcal{D}s\mathcal{G}(d-Rs,N)\mathcal{G}(s,S)\\ & \propto \frac{1}{|S{|}^{\frac{1}{2}}}\int \mathcal{D}s\exp \left[-\frac{1}{2}\left(s^{†}{D}^{-1}s-j^{†}s-s^{†}j\right)\right]\\ & \propto \frac{|D{|}^{\frac{1}{2}}}{|S{|}^{\frac{1}{2}}}\exp \left[\frac{1}{2}{j}^{†}Dj\right],\end{array}}$$

where the notation of the information propagator ${\displaystyle D=(S^{-1}+R^{†}{N}^{-1}R{)}^{-1}}$ and source ${\displaystyle j=R^{†}{N}^{-1}d}$ of the Wiener filter problem was used again. The corresponding information Hamiltonian is

$${\displaystyle \mathcal{H}(d,P_s)\widehat{=}\frac{1}{2}\left[\ln |S{D}^{-1}|-j^{†}Dj\right]=\frac{1}{2}\mathrm{T}\mathrm{r}\left[\ln \left(S{D}^{-1}\right)-j{j}^{†}D\right],}$$

where ${\displaystyle \widehat{=}}$ denotes equality up to irrelevant constants (here: constant with respect to ${\displaystyle P_s}$). Minimizing this with respect to ${\displaystyle P_s}$, in order to get its maximum a posteriori power spectrum estimator, yields

$${\displaystyle \begin{array}{rl}\frac{\mathrm{\partial }\mathcal{H}(d,P_s)}{\mathrm{\partial }{P}_s(k)}& =\frac{1}{2}\mathrm{T}\mathrm{r}\left[D{S}^{-1}\frac{\mathrm{\partial }\left(S{D}^{-1}\right)}{\mathrm{\partial }{P}_s(k)}-j{j}^{†}\frac{\mathrm{\partial }D}{\mathrm{\partial }{P}_s(k)}\right]\\ & =\frac{1}{2}\mathrm{T}\mathrm{r}\left[D{S}^{-1}\frac{\mathrm{\partial }\left(1+S{R}^{†}{N}^{-1}R\right)}{\mathrm{\partial }{P}_s(k)}+j{j}^{†}D\frac{\mathrm{\partial }{D}^{-1}}{\mathrm{\partial }{P}_s(k)}D\right]\\ & =\frac{1}{2}\mathrm{T}\mathrm{r}\left[D{S}^{-1}\frac{\mathrm{\partial }S}{\mathrm{\partial }{P}_s(k)}{R}^{†}{N}^{-1}R+m{m}^{†}\frac{\mathrm{\partial }{S}^{-1}}{\mathrm{\partial }{P}_s(k)}\right]\\ & =\frac{1}{2}\mathrm{T}\mathrm{r}\left[\left(R^{†}{N}^{-1}RD{S}^{-1}-S^{-1}m{m}^{†}{S}^{-1}\right)\frac{\mathrm{\partial }S}{\mathrm{\partial }{P}_s(k)}\right]\\ & =\frac{1}{2}\int {\left(\frac{dq}{2\pi }\right)}^u\int {\left(\frac{d{q}^{\prime }}{2\pi }\right)}^u{\left(\left(D^{-1}-S^{-1}\right)D{S}^{-1}-S^{-1}m{m}^{†}{S}^{-1}\right)}_{\overrightarrow{q}{\overrightarrow{q}}^{\prime }}\frac{\mathrm{\partial }(2\pi {)}^u\delta (\overrightarrow{q}-{\overrightarrow{q}}^{\prime })P_s(q)}{\mathrm{\partial }{P}_s(k)}\\ & =\frac{1}{2}\int {\left(\frac{dq}{2\pi }\right)}^u{\left(S^{-1}-S^{-1}D{S}^{-1}-S^{-1}m{m}^{†}{S}^{-1}\right)}_{\overrightarrow{q}\overrightarrow{q}}\delta (k-q)\\ & =\frac{1}{2}\mathrm{T}\mathrm{r}\left\{S^{-1}\left[S-\left(D+m{m}^{†}\right)\right]S^{-1}{\mathbb{P}}_k\right\}\\ & =\frac{\mathrm{T}\mathrm{r}\left[{\mathbb{P}}_k\right]}{2P_s(k)}-\frac{\mathrm{T}\mathrm{r}\left[\left(D+m{m}^{†}\right){\mathbb{P}}_k\right]}{2{\left[P_s(k)\right]}^2}=0,\end{array}}$$

where the Wiener filter mean ${\displaystyle m=Dj}$ and the spectral band projector ${\displaystyle ({\mathbb{P}}_k{)}_{\overrightarrow{q}{\overrightarrow{q}}^{\prime }}\equiv (2\pi {)}^u\delta (\overrightarrow{q}-{\overrightarrow{q}}^{\prime })\delta (|\overrightarrow{q}|-k)}$were introduced. The latter commutes with ${\displaystyle S^{-1}}$, since ${\displaystyle (S^{-1}{)}_{\overrightarrow{k}\overrightarrow{q}}=(2\pi {)}^u\delta (\overrightarrow{k}-\overrightarrow{q})[P_s(k){]}^{-1}}$ is diagonal in Fourier space. The maximum a posteriori estimator for the power spectrum is therefore

$${\displaystyle P_s(k)=\frac{\mathrm{T}\mathrm{r}\left[\left(m{m}^{†}+D\right){\mathbb{P}}_k\right]}{\mathrm{T}\mathrm{r}\left[{\mathbb{P}}_k\right]}.}$$

It has to be calculated iteratively, as ${\displaystyle m=Dj}$ and ${\displaystyle D=(S^{-1}+R^{†}{N}^{-1}R{)}^{-1}}$ depend both on ${\displaystyle P_s}$ themselves. In an empirical Bayes approach, the estimated ${\displaystyle P_s}$ would be taken as given. As a consequence, the posterior mean estimate for the signal field is the corresponding ${\displaystyle m}$ and its uncertainty the corresponding ${\displaystyle D}$ in the empirical Bayes approximation.

The resulting non-linear filter is called the critical filter. The generalization of the power spectrum estimation formula as

$${\displaystyle P_s(k)=\frac{\mathrm{T}\mathrm{r}\left[\left(m{m}^{†}+\delta D\right){\mathbb{P}}_k\right]}{\mathrm{T}\mathrm{r}\left[{\mathbb{P}}_k\right]}}$$

exhibits a perception thresholds for ${\displaystyle \delta <1}$, meaning that the data variance in a Fourier band has to exceed the expected noise level by a certain threshold before the signal reconstruction ${\displaystyle m}$ becomes non-zero for this band. Whenever the data variance exceeds this threshold slightly, the signal reconstruction jumps to a finite excitation level, similar to a first order phase transition in thermodynamic systems. For filter with ${\displaystyle \delta =1}$ perception of the signal starts continuously as soon the data variance exceeds the noise level. The disappearance of the discontinuous perception at ${\displaystyle \delta =1}$ is similar to a thermodynamic system going through a critical point. Hence the name critical filter.

The critical filter, extensions thereof to non-linear measurements, and the inclusion of non-flat spectrum priors, permitted the application of IFT to real world signal inference problems, for which the signal covariance is usually unknown a priori.

IFT application examples

Radio interferometric image of radio galaxies in the galaxy cluster Abell 2219. The images were constructed by data back-projection (top), the CLEAN algorithm (middle), and the RESOLVE algorithm (bottom). Negative and therefore not physical fluxes are displayed in white.

The generalized Wiener filter, that emerges in free IFT, is in broad usage in signal processing. Algorithms explicitly based on IFT were derived for a number of applications. Many of them are implemented using the Numerical Information Field Theory (NIFTy) library.

• D³PO is a code for Denoising, Deconvolving, and Decomposing Photon Observations. It reconstructs images from individual photon count events taking into account the Poisson statistics of the counts and an instrument response function. It splits the sky emission into an image of diffuse emission and one of point sources, exploiting the different correlation structure and statistics of the two components for their separation. D³PO has been applied to data of the Fermi and the RXTE satellites.
• RESOLVE is a Bayesian algorithm for aperture synthesis imaging in radio astronomy. RESOLVE is similar to D³PO, but it assumes a Gaussian likelihood and a Fourier space response function. It has been applied to data of the Very Large Array.
• PySESA is a Python framework for Spatially Explicit Spectral Analysis for spatially explicit spectral analysis of point clouds and geospatial data.

Advanced theory

Many techniques from quantum field theory can be used to tackle IFT problems, like Feynman diagrams, effective actions, and the field operator formalism.

Feynman diagrams

First three Feynman diagrams contributing to the posterior mean estimate of a field. A line expresses an information propagator, a dot at the end of a line to an information source, and a vertex to an interaction term. The first diagram encodes the Wiener filter, the second a non-linear correction, and the third an uncertainty correction to the Wiener filter.

In case the interaction coefficients ${\displaystyle {\mathrm{\Lambda }}^{(n)}}$ in a Taylor-Fréchet expansion of the information Hamiltonian

$${\displaystyle \mathcal{H}(d,s)=\underset{={\mathcal{H}}_{\text{free}}(d,s)}{\underbrace{\frac{1}{2}{s}^{†}{D}^{-1}s-j^{†}s+{\mathcal{H}}_0}}+\underset{={\mathcal{H}}_{\text{int}}(d,s)}{\underbrace{\sum _{n=3}^{\mathrm{\infty }}\frac{1}{n!}{\mathrm{\Lambda }}_{x_1...x_n}^{(n)}{s}_{x_1}...s_{x_n}}},}$$

are small, the log partition function, or Helmholtz free energy,

$${\displaystyle \ln \mathcal{Z}(d)=\ln \int \mathcal{D}s{e}^{-\mathcal{H}(d,s)}=\sum _{c\in C}c}$$

can be expanded asymptotically in terms of these coefficients. The free Hamiltonian specifies the mean ${\displaystyle m=Dj}$ and variance ${\displaystyle D}$ of the Gaussian distribution ${\displaystyle \mathcal{G}(s-m,D)}$ over which the expansion is integrated. This leads to a sum over the set ${\displaystyle C}$ of all connected Feynman diagrams. From the Helmholtz free energy, any connected moment of the field can be calculated via

$${\displaystyle ⟨s_{x_1}\dots s_{x_n}{⟩}_{(s|d)}^{\text{c}}=\frac{{\mathrm{\partial }}^n\ln \mathcal{Z}}{\mathrm{\partial }{j}_{x_1}\dots \mathrm{\partial }{j}_{x_n}}.}$$

Situations where small expansion parameters exist that are needed for such a diagrammatic expansion to converge are given by nearly Gaussian signal fields, where the non-Gaussianity of the field statistics leads to small interaction coefficients ${\displaystyle {\mathrm{\Lambda }}^{(n)}}$. For example, the statistics of the Cosmic Microwave Background is nearly Gaussian, with small amounts of non-Gaussianities believed to be seeded during the inflationary epoch in the Early Universe.

Effective action

In order to have a stable numerics for IFT problems, a field functional that if minimized provides the posterior mean field is needed. Such is given by the effective action or Gibbs free energy of a field. The Gibbs free energy ${\displaystyle G}$ can be constructed from the Helmholtz free energy via a Legendre transformation. In IFT, it is given by the difference of the internal information energy

$${\displaystyle U=⟨\mathcal{H}(d,s){⟩}_{{\mathcal{P}}^{\prime }(s|d^{\prime })}}$$

and the Shannon entropy

$${\displaystyle \mathcal{S}=-\int \mathcal{D}s{\mathcal{P}}^{\prime }(s|d^{\prime })\ln {\mathcal{P}}^{\prime }(s|d^{\prime })}$$

for temperature ${\displaystyle T=1}$, where a Gaussian posterior approximation ${\displaystyle {\mathcal{P}}^{\prime }(s|d^{\prime })=\mathcal{G}(s-m,D)}$ is used with the approximate data ${\displaystyle d^{\prime }=(m,D)}$ containing the mean and the dispersion of the field.

The Gibbs free energy is then

$${\displaystyle \begin{array}{rl}G(m,D)& =U(m,D)-T\mathcal{S}(m,D)\\ & =⟨\mathcal{H}(d,s)+\ln {\mathcal{P}}^{\prime }(s|d^{\prime }){⟩}_{{\mathcal{P}}^{\prime }(s|d^{\prime })}\\ & =\int \mathcal{D}s{\mathcal{P}}^{\prime }(s|d^{\prime })\ln \frac{{\mathcal{P}}^{\prime }(s|d^{\prime })}{\mathcal{P}(d,s)}\\ & =\int \mathcal{D}s{\mathcal{P}}^{\prime }(s|d^{\prime })\ln \frac{{\mathcal{P}}^{\prime }(s|d^{\prime })}{\mathcal{P}(s|d)\mathcal{P}(d)}\\ & =\int \mathcal{D}s{\mathcal{P}}^{\prime }(s|d^{\prime })\ln \frac{{\mathcal{P}}^{\prime }(s|d^{\prime })}{\mathcal{P}(s|d)}-\ln \mathcal{P}(d)\\ & =\text{KL}({\mathcal{P}}^{\prime }(s|d^{\prime })||\mathcal{P}(s|d))-\ln \mathcal{Z}(d),\end{array}}$$

the Kullback-Leibler divergence ${\displaystyle \text{KL}({\mathcal{P}}^{\prime },\mathcal{P})}$ between approximative and exact posterior plus the Helmholtz free energy. As the latter does not depend on the approximate data ${\displaystyle d^{\prime }=(m,D)}$, minimizing the Gibbs free energy is equivalent to minimizing the Kullback-Leibler divergence between approximate and exact posterior. Thus, the effective action approach of IFT is equivalent to the variational Bayesian methods, which also minimize the Kullback-Leibler divergence between approximate and exact posteriors.

Minimizing the Gibbs free energy provides approximatively the posterior mean field

$${\displaystyle ⟨s{⟩}_{(s|d)}=\int \mathcal{D}ss\mathcal{P}(s|d),}$$

whereas minimizing the information Hamiltonian provides the maximum a posteriori field. As the latter is known to over-fit noise, the former is usually a better field estimator.

Operator formalism

The calculation of the Gibbs free energy requires the calculation of Gaussian integrals over an information Hamiltonian, since the internal information energy is

$${\displaystyle U(m,D)=⟨\mathcal{H}(d,s){⟩}_{{\mathcal{P}}^{\prime }(s|d^{\prime })}=\int \mathcal{D}s\mathcal{H}(d,s)\mathcal{G}(s-m,D).}$$

Such integrals can be calculated via a field operator formalism, in which

$${\displaystyle O_m=m+D\frac{\mathrm{d}}{\mathrm{d}m}}$$

is the field operator. This generates the field expression ${\displaystyle s}$ within the integral if applied to the Gaussian distribution function,

$${\displaystyle \begin{array}{rl}{O}_m\mathcal{G}(s-m,D)& =(m+D\frac{\mathrm{d}}{\mathrm{d}m})\frac{1}{|2\pi D{|}^{\frac{1}{2}}}\exp \left[-\frac{1}{2}(s-m{)}^{†}{D}^{-1}(s-m)\right]\\ & =(m+D{D}^{-1}(s-m))\frac{1}{|2\pi D{|}^{\frac{1}{2}}}\exp \left[-\frac{1}{2}(s-m{)}^{†}{D}^{-1}(s-m)\right]\\ & =s\mathcal{G}(s-m,D),\end{array}}$$

and any higher power of the field if applied several times,

$${\displaystyle \begin{array}{rl}(O_m{)}^n\mathcal{G}(s-m,D)& =s^n\mathcal{G}(s-m,D).\end{array}}$$

If the information Hamiltonian is analytical, all its terms can be generated via the field operator

$${\displaystyle \mathcal{H}(d,O_m)\mathcal{G}(s-m,D)=\mathcal{H}(d,s)\mathcal{G}(s-m,D).}$$

As the field operator does not depend on the field ${\displaystyle s}$ itself, it can be pulled out of the path integral of the internal information energy construction,

$${\displaystyle U(m,D)=\int \mathcal{D}s\mathcal{H}(d,O_m)\mathcal{G}(s-m,D)=\mathcal{H}(d,O_m)\int \mathcal{D}s\mathcal{G}(s-m,D)=\mathcal{H}(d,O_m)1_m,}$$

where ${\displaystyle 1_m=1}$ should be regarded as a functional that always returns the value ${\displaystyle 1}$ irrespective the value of its input ${\displaystyle m}$. The resulting expression can be calculated by commuting the mean field annihilator ${\displaystyle D\frac{\mathrm{d}}{\mathrm{d}m}}$ to the right of the expression, where they vanish since ${\displaystyle \frac{\mathrm{d}}{\mathrm{d}m}{1}_m=0}$. The mean field annihilator ${\displaystyle D\frac{\mathrm{d}}{\mathrm{d}m}}$ commutes with the mean field as

$${\displaystyle \left[D\frac{\mathrm{d}}{\mathrm{d}m},m\right]=D\frac{\mathrm{d}}{\mathrm{d}m}m-mD\frac{\mathrm{d}}{\mathrm{d}m}=D+mD\frac{\mathrm{d}}{\mathrm{d}m}-mD\frac{\mathrm{d}}{\mathrm{d}m}=D.}$$

By the usage of the field operator formalism the Gibbs free energy can be calculated, which permits the (approximate) inference of the posterior mean field via a numerical robust functional minimization.

History

The book of Norbert Wiener might be regarded as one of the first works on field inference. The usage of path integrals for field inference was proposed by a number of authors, e.g. Edmund Bertschinger or William Bialek and A. Zee. The connection of field theory and Bayesian reasoning was made explicit by Jörg Lemm. The term information field theory was coined by Torsten Enßlin. See the latter reference for more information on the history of IFT.

Glutamic acid

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Glutamic_acid

Glutamic acid

Skeletal formula of L-glutamic acid
Ball-and-stick model Space-filling model
Names
IUPAC name Glutamic acid
Systematic IUPAC name 2-Aminopentanedioic acid
Other names 2-Aminoglutaric acid

Glutamic acid (symbol Glu or E; the anionic form is known as glutamate) is an α-amino acid that is used by almost all living beings in the biosynthesis of proteins. It is a non-essential nutrient for humans, meaning that the human body can synthesize enough for its use. It is also the most abundant excitatory neurotransmitter in the vertebrate nervous system. It serves as the precursor for the synthesis of the inhibitory gamma-aminobutyric acid (GABA) in GABAergic neurons.

Its molecular formula is C
₅H
₉NO
₄. Glutamic acid exists in three optically isomeric forms; the dextrorotatory L-form is usually obtained by hydrolysis of gluten or from the waste waters of beet-sugar manufacture or by fermentation. Its molecular structure could be idealized as HOOC−CH(NH
₂)−(CH
₂)₂−COOH, with two carboxyl groups −COOH and one amino group −NH
₂. However, in the solid state and mildly acidic water solutions, the molecule assumes an electrically neutral zwitterion structure ⁻OOC−CH(NH⁺
₃)−(CH
₂)₂−COOH. It is encoded by the codons GAA or GAG.

The acid can lose one proton from its second carboxyl group to form the conjugate base, the singly-negative anion glutamate ⁻OOC−CH(NH⁺
₃)−(CH
₂)₂−COO⁻. This form of the compound is prevalent in neutral solutions. The glutamate neurotransmitter plays the principal role in neural activation. This anion creates the savory umami flavor of foods and is found in glutamate flavorings such as MSG. In Europe it is classified as food additive E620. In highly alkaline solutions the doubly negative anion ⁻OOC−CH(NH
₂)−(CH
₂)₂−COO⁻ prevails. The radical corresponding to glutamate is called glutamyl.

Chemistry

Ionization

The glutamate monoanion.

When glutamic acid is dissolved in water, the amino group (−NH
₂) may gain a proton (H⁺
), and/or the carboxyl groups may lose protons, depending on the acidity of the medium.

In sufficiently acidic environments, the amino group gains a proton and the molecule becomes a cation with a single positive charge, HOOC−CH(NH⁺
₃)−(CH
₂)₂−COOH.

At pH values between about 2.5 and 4.1, the carboxylic acid closer to the amine generally loses a proton, and the acid becomes the neutral zwitterion ⁻OOC−CH(NH⁺
₃)−(CH
₂)₂−COOH. This is also the form of the compound in the crystalline solid state. The change in protonation state is gradual; the two forms are in equal concentrations at pH 2.10.

At even higher pH, the other carboxylic acid group loses its proton and the acid exists almost entirely as the glutamate anion ⁻OOC−CH(NH⁺
₃)−(CH
₂)₂−COO⁻, with a single negative charge overall. The change in protonation state occurs at pH 4.07. This form with both carboxylates lacking protons is dominant in the physiological pH range (7.35–7.45).

At even higher pH, the amino group loses the extra proton, and the prevalent species is the doubly-negative anion ⁻OOC−CH(NH
₂)−(CH
₂)₂−COO⁻. The change in protonation state occurs at pH 9.47.

Optical isomerism

Glutamic acid is chiral; two mirror-image enantiomers exist: d(−), and l(+). The l form is more widely occurring in nature, but the d form occurs in some special contexts, such as the bacterial capsule and cell walls of the bacteria (which produce it from the l form with the enzyme glutamate racemase) and the liver of mammals.

History

Main article: Glutamic acid (flavor)

Although they occur naturally in many foods, the flavor contributions made by glutamic acid and other amino acids were only scientifically identified early in the 20th century. The substance was discovered and identified in the year 1866 by the German chemist Karl Heinrich Ritthausen, who treated wheat gluten (for which it was named) with sulfuric acid. In 1908, Japanese researcher Kikunae Ikeda of the Tokyo Imperial University identified brown crystals left behind after the evaporation of a large amount of kombu broth as glutamic acid. These crystals, when tasted, reproduced the ineffable but undeniable flavor he detected in many foods, most especially in seaweed. Professor Ikeda termed this flavor umami. He then patented a method of mass-producing a crystalline salt of glutamic acid, monosodium glutamate.

Synthesis

Biosynthesis

Reactants		Products	Enzymes
Glutamine + H2O	→	Glu + NH3	GLS, GLS2
NAcGlu + H2O	→	Glu + acetate	N-Acetyl-glutamate synthase
α-Ketoglutarate + NADPH + NH4+	→	Glu + NADP+ + H2O	GLUD1, GLUD2
α-Ketoglutarate + α-amino acid	→	Glu + α-keto acid	Transaminase
1-Pyrroline-5-carboxylate + NAD+ + H2O	→	Glu + NADH	ALDH4A1
N-Formimino-L-glutamate + FH4	→	Glu + 5-formimino-FH4	FTCD
NAAG	→	Glu + NAA	GCPII

Industrial synthesis

Glutamic acid is produced on the largest scale of any amino acid, with an estimated annual production of about 1.5 million tons in 2006. Chemical synthesis was supplanted by the aerobic fermentation of sugars and ammonia in the 1950s, with the organism Corynebacterium glutamicum (also known as Brevibacterium flavum) being the most widely used for production. Isolation and purification can be achieved by concentration and crystallization; it is also widely available as its hydrochloride salt.

Function and uses

Metabolism

Glutamate is a key compound in cellular metabolism. In humans, dietary proteins are broken down by digestion into amino acids, which serve as metabolic fuel for other functional roles in the body. A key process in amino acid degradation is transamination, in which the amino group of an amino acid is transferred to an α-ketoacid, typically catalysed by a transaminase. The reaction can be generalised as such:

R₁-amino acid + R₂-α-ketoacid ⇌ R₁-α-ketoacid + R₂-amino acid

A very common α-keto acid is α-ketoglutarate, an intermediate in the citric acid cycle. Transamination of α-ketoglutarate gives glutamate. The resulting α-ketoacid product is often a useful one as well, which can contribute as fuel or as a substrate for further metabolism processes. Examples are as follows:

Alanine + α-ketoglutarate ⇌ pyruvate + glutamate

Aspartate + α-ketoglutarate ⇌ oxaloacetate + glutamate

Both pyruvate and oxaloacetate are key components of cellular metabolism, contributing as substrates or intermediates in fundamental processes such as glycolysis, gluconeogenesis, and the citric acid cycle.

Glutamate also plays an important role in the body's disposal of excess or waste nitrogen. Glutamate undergoes deamination, an oxidative reaction catalysed by glutamate dehydrogenase, as follows:

glutamate + H₂O + NADP⁺ → α-ketoglutarate + NADPH + NH₃ + H⁺

Ammonia (as ammonium) is then excreted predominantly as urea, synthesised in the liver. Transamination can thus be linked to deamination, effectively allowing nitrogen from the amine groups of amino acids to be removed, via glutamate as an intermediate, and finally excreted from the body in the form of urea.

Glutamate is also a neurotransmitter (see below), which makes it one of the most abundant molecules in the brain. Malignant brain tumors known as glioma or glioblastoma exploit this phenomenon by using glutamate as an energy source, especially when these tumors become more dependent on glutamate due to mutations in the gene IDH1.

See also: Glutamate–glutamine cycle

Neurotransmitter

Main article: Glutamate (neurotransmitter)

Glutamate is the most abundant excitatory neurotransmitter in the vertebrate nervous system. At chemical synapses, glutamate is stored in vesicles. Nerve impulses trigger the release of glutamate from the presynaptic cell. Glutamate acts on ionotropic and metabotropic (G-protein coupled) receptors. In the opposing postsynaptic cell, glutamate receptors, such as the NMDA receptor or the AMPA receptor, bind glutamate and are activated. Because of its role in synaptic plasticity, glutamate is involved in cognitive functions such as learning and memory in the brain. The form of plasticity known as long-term potentiation takes place at glutamatergic synapses in the hippocampus, neocortex, and other parts of the brain. Glutamate works not only as a point-to-point transmitter, but also through spill-over synaptic crosstalk between synapses in which summation of glutamate released from a neighboring synapse creates extrasynaptic signaling/volume transmission. In addition, glutamate plays important roles in the regulation of growth cones and synaptogenesis during brain development as originally described by Mark Mattson.

Brain nonsynaptic glutamatergic signaling circuits

Extracellular glutamate in Drosophila brains has been found to regulate postsynaptic glutamate receptor clustering, via a process involving receptor desensitization. A gene expressed in glial cells actively transports glutamate into the extracellular space, while, in the nucleus accumbens-stimulating group II metabotropic glutamate receptors, this gene was found to reduce extracellular glutamate levels. This raises the possibility that this extracellular glutamate plays an "endocrine-like" role as part of a larger homeostatic system.

GABA precursor

Glutamate also serves as the precursor for the synthesis of the inhibitory gamma-aminobutyric acid (GABA) in GABA-ergic neurons. This reaction is catalyzed by glutamate decarboxylase (GAD), which is most abundant in the cerebellum and pancreas.

Stiff person syndrome is a neurologic disorder caused by anti-GAD antibodies, leading to a decrease in GABA synthesis and, therefore, impaired motor function such as muscle stiffness and spasm. Since the pancreas has abundant GAD, a direct immunological destruction occurs in the pancreas and the patients will have diabetes mellitus.

Flavor enhancer

Main article: Glutamic acid (flavor)

Glutamic acid, being a constituent of protein, is present in foods that contain protein, but it can only be tasted when it is present in an unbound form. Significant amounts of free glutamic acid are present in a wide variety of foods, including cheeses and soy sauce, and glutamic acid is responsible for umami, one of the five basic tastes of the human sense of taste. Glutamic acid often is used as a food additive and flavor enhancer in the form of its sodium salt, known as monosodium glutamate (MSG).

Nutrient

All meats, poultry, fish, eggs, dairy products, and kombu are excellent sources of glutamic acid. Some protein-rich plant foods also serve as sources. 30% to 35% of gluten (much of the protein in wheat) is glutamic acid. Ninety-five percent of the dietary glutamate is metabolized by intestinal cells in a first pass.

Plant growth

Auxigro is a plant growth preparation that contains 30% glutamic acid.

NMR spectroscopy

In recent years,^[when?] there has been much research into the use of residual dipolar coupling (RDC) in nuclear magnetic resonance spectroscopy (NMR). A glutamic acid derivative, poly-γ-benzyl-L-glutamate (PBLG), is often used as an alignment medium to control the scale of the dipolar interactions observed.

Role of glutamate in aging

See also: Aging brain § Glutamate

Pharmacology

The drug phencyclidine (more commonly known as PCP or 'Angel Dust') antagonizes glutamic acid non-competitively at the NMDA receptor. For the same reasons, dextromethorphan and ketamine also have strong dissociative and hallucinogenic effects. Acute infusion of the drug eglumetad (also known as eglumegad or LY354740), an agonist of the metabotropic glutamate receptors 2 and 3) resulted in a marked diminution of yohimbine-induced stress response in bonnet macaques (Macaca radiata); chronic oral administration of eglumetad in those animals led to markedly reduced baseline cortisol levels (approximately 50 percent) in comparison to untreated control subjects. Eglumetad has also been demonstrated to act on the metabotropic glutamate receptor 3 (GRM3) of human adrenocortical cells, downregulating aldosterone synthase, CYP11B1, and the production of adrenal steroids (i.e. aldosterone and cortisol). Glutamate does not easily pass the blood brain barrier, but, instead, is transported by a high-affinity transport system. It can also be converted into glutamine.

Glutamate toxicity can be reduced by antioxidants, and the psychoactive principle of cannabis, tetrahydrocannabinol (THC), and the non psychoactive principle cannabidiol (CBD), and other cannabinoids, is found to block glutamate neurotoxicity with a similar potency, and thereby potent antioxidants.

Runtime system

From Wikipedia, the free encyclopedia
https://en.wikipedia.org/wiki/Runtime_system

In computer programming, a runtime system or runtime environment is a sub-system that exists both in the computer where a program is created, as well as in the computers where the program is intended to be run. The name comes from the compile time and runtime division from compiled languages, which similarly distinguishes the computer processes involved in the creation of a program (compilation) and its execution in the target machine (the run time).

Most programming languages have some form of runtime system that provides an environment in which programs run. This environment may address a number of issues including the management of application memory, how the program accesses variables, mechanisms for passing parameters between procedures, interfacing with the operating system, and otherwise. The compiler makes assumptions depending on the specific runtime system to generate correct code. Typically the runtime system will have some responsibility for setting up and managing the stack and heap, and may include features such as garbage collection, threads or other dynamic features built into the language.

Overview

Every programming language specifies an execution model, and many implement at least part of that model in a runtime system. One possible definition of runtime system behavior, among others, is "any behavior not directly attributable to the program itself". This definition includes putting parameters onto the stack before function calls, parallel execution of related behaviors, and disk I/O.

By this definition, essentially every language has a runtime system, including compiled languages, interpreted languages, and embedded domain-specific languages. Even API-invoked standalone execution models, such as Pthreads (POSIX threads), have a runtime system that implements the execution model's behavior.

Most scholarly papers on runtime systems focus on the implementation details of parallel runtime systems. A notable example of a parallel runtime system is Cilk, a popular parallel programming model. The proto-runtime toolkit was created to simplify the creation of parallel runtime systems.

In addition to execution model behavior, a runtime system may also perform support services such as type checking, debugging, or code generation and optimization.

Comparison between concepts similar to runtime system

Type	Description	Examples
Runtime environment	Software platform that provides an environment for executing code	Node.js, .NET Framework
Engine	Component of a runtime environment that executes code by compiling or interpreting it	JavaScript engine in web browsers, Java Virtual Machine
Interpreter	Type of engine that reads and executes code line by line, without compiling the entire program beforehand	CPython interpreter, Ruby MRI, JavaScript (in some cases)
JIT interpreter	Type of interpreter that dynamically compiles code into machine instructions at runtime, optimizing the code for faster execution	V8, PyPy interpreter

Relation to runtime environments

The runtime system is also the gateway through which a running program interacts with the runtime environment. The runtime environment includes not only accessible state values, but also active entities with which the program can interact during execution. For example, environment variables are features of many operating systems, and are part of the runtime environment; a running program can access them via the runtime system. Likewise, hardware devices such as disks or DVD drives are active entities that a program can interact with via a runtime system.

One unique application of a runtime environment is its use within an operating system that only allows it to run. In other words, from boot until power-down, the entire OS is dedicated to only the application(s) running within that runtime environment. Any other code that tries to run, or any failures in the application(s), will break the runtime environment. Breaking the runtime environment in turn breaks the OS, stopping all processing and requiring a reboot. If the boot is from read-only memory, an extremely secure, simple, single-mission system is created.

Examples of such directly bundled runtime systems include:

• Between 1983 and 1984, Digital Research offered several of their business and educations applications for the IBM PC on bootable floppy diskettes bundled with SpeedStart CP/M-86, a reduced version of CP/M-86 as runtime environment.
• Some stand-alone versions of Ventura Publisher (1986–1993), Artline (1988–1991), Timeworks Publisher (1988–1991) and ViewMAX (1990–1992) contained special runtime versions of Digital Research's GEM as their runtime environment.
• In the late 1990s, JP Software's command line processor 4DOS was optionally available in a special runtime version to be linked with BATCOMP pre-compiled and encrypted batch jobs in order to create unmodifyable executables from batch scripts and run them on systems without 4DOS installed.

Examples

The runtime system of the C language is a particular set of instructions inserted by the compiler into the executable image. Among other things, these instructions manage the process stack, create space for local variables, and copy function call parameters onto the top of the stack.

There are often no clear criteria for determining which language behaviors are part of the runtime system itself and which can be determined by any particular source program. For example, in C, the setup of the stack is part of the runtime system. It is not determined by the semantics of an individual program because the behavior is globally invariant: it holds over all executions. This systematic behavior implements the execution model of the language, as opposed to implementing semantics of the particular program (in which text is directly translated into code that computes results).

This separation between the semantics of a particular program and the runtime environment is reflected by the different ways of compiling a program: compiling source code to an object file that contains all the functions versus compiling an entire program to an executable binary. The object file will only contain assembly code relevant to the included functions, while the executable binary will contain additional code that implements the runtime environment. The object file, on one hand, may be missing information from the runtime environment that will be resolved by linking. On the other hand, the code in the object file still depends on assumptions in the runtime system; for example, a function may read parameters from a particular register or stack location, depending on the calling convention used by the runtime environment.

Another example is the case of using an application programming interface (API) to interact with a runtime system. The calls to that API look the same as calls to a regular software library, however at some point during the call the execution model changes. The runtime system implements an execution model different from that of the language the library is written in terms of. A person reading the code of a normal library would be able to understand the library's behavior by just knowing the language the library was written in. However, a person reading the code of the API that invokes a runtime system would not be able to understand the behavior of the API call just by knowing the language the call was written in. At some point, via some mechanism, the execution model stops being that of the language the call is written in and switches over to being the execution model implemented by the runtime system. For example, the trap instruction is one method of switching execution models. This difference is what distinguishes an API-invoked execution model, such as Pthreads, from a usual software library. Both Pthreads calls and software library calls are invoked via an API, but Pthreads behavior cannot be understood in terms of the language of the call. Rather, Pthreads calls bring into play an outside execution model, which is implemented by the Pthreads runtime system (this runtime system is often the OS kernel).

As an extreme example, the physical CPU itself can be viewed as an implementation of the runtime system of a specific assembly language. In this view, the execution model is implemented by the physical CPU and memory systems. As an analogy, runtime systems for higher-level languages are themselves implemented using some other languages. This creates a hierarchy of runtime systems, with the CPU itself—or actually its logic at the microcode layer or below—acting as the lowest-level runtime system.

Advanced features

Some compiled or interpreted languages provide an interface that allows application code to interact directly with the runtime system. An example is the Thread class in the Java language. The class allows code (that is animated by one thread) to do things such as start and stop other threads. Normally, core aspects of a language's behavior such as task scheduling and resource management are not accessible in this fashion.

Higher-level behaviors implemented by a runtime system may include tasks such as drawing text on the screen or making an Internet connection. It is often the case that operating systems provide these kinds of behaviors as well, and when available, the runtime system is implemented as an abstraction layer that translates the invocation of the runtime system into an invocation of the operating system. This hides the complexity or variations in the services offered by different operating systems. This also implies that the OS kernel can itself be viewed as a runtime system, and that the set of OS calls that invoke OS behaviors may be viewed as interactions with a runtime system.

In the limit, the runtime system may provide services such as a P-code machine or virtual machine, that hide even the processor's instruction set. This is the approach followed by many interpreted languages such as AWK, and some languages like Java, which are meant to be compiled into some machine-independent intermediate representation code (such as bytecode). This arrangement simplifies the task of language implementation and its adaptation to different machines, and improves efficiency of sophisticated language features such as reflection. It also allows the same program to be executed on any machine without an explicit recompiling step, a feature that has become very important since the proliferation of the World Wide Web. To speed up execution, some runtime systems feature just-in-time compilation to machine code.

A modern aspect of runtime systems is parallel execution behaviors, such as the behaviors exhibited by mutex constructs in Pthreads and parallel section constructs in OpenMP. A runtime system with such parallel execution behaviors may be modularized according to the proto-runtime approach.

History

Notable early examples of runtime systems are the interpreters for BASIC and Lisp. These environments also included a garbage collector. Forth is an early example of a language designed to be compiled into intermediate representation code; its runtime system was a virtual machine that interpreted that code. Another popular, if theoretical, example is Donald Knuth's MIX computer.

In C and later languages that supported dynamic memory allocation, the runtime system also included a library that managed the program's memory pool.

In the object-oriented programming languages, the runtime system was often also responsible for dynamic type checking and resolving method references.

Exception handling

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Exception_handling

In computing and computer programming, exception handling is the process of responding to the occurrence of exceptions – anomalous or exceptional conditions requiring special processing – during the execution of a program. In general, an exception breaks the normal flow of execution and executes a pre-registered exception handler; the details of how this is done depend on whether it is a hardware or software exception and how the software exception is implemented.

Exceptions are defined by different layers of a computer system, and the typical layers are CPU-defined interrupts, operating system (OS)-defined signals, programming language-defined exceptions. Each layer requires different ways of exception handling although they may be interrelated, e.g. a CPU interrupt could be turned into an OS signal. Some exceptions, especially hardware ones, may be handled so gracefully that execution can resume where it was interrupted.

Definition

Further information: Interrupt § Terminology

The definition of an exception is based on the observation that each procedure has a precondition, a set of circumstances for which it will terminate "normally". An exception handling mechanism allows the procedure to raise an exception if this precondition is violated, for example if the procedure has been called on an abnormal set of arguments. The exception handling mechanism then handles the exception.

The precondition, and the definition of exception, is subjective. The set of "normal" circumstances is defined entirely by the programmer, e.g. the programmer may deem division by zero to be undefined, hence an exception, or devise some behavior such as returning zero or a special "ZERO DIVIDE" value (circumventing the need for exceptions). Common exceptions include an invalid argument (e.g. value is outside of the domain of a function), an unavailable resource (like a missing file, a network drive error, or out-of-memory errors), or that the routine has detected a normal condition that requires special handling, e.g., attention, end of file. Social pressure is a major influence on the scope of exceptions and use of exception-handling mechanisms, i.e. "examples of use, typically found in core libraries, and code examples in technical books, magazine articles, and online discussion forums, and in an organization’s code standards".

Exception handling solves the semipredicate problem, in that the mechanism distinguishes normal return values from erroneous ones. In languages without built-in exception handling such as C, routines would need to signal the error in some other way, such as the common return code and errno pattern. Taking a broad view, errors can be considered to be a proper subset of exceptions, and explicit error mechanisms such as errno can be considered (verbose) forms of exception handling. The term "exception" is preferred to "error" because it does not imply that anything is wrong - a condition viewed as an error by one procedure or programmer may not be viewed that way by another.

The term "exception" may be misleading because its connotation of "anomaly" indicates that raising an exception is abnormal or unusual, when in fact raising the exception may be a normal and usual situation in the program. For example, suppose a lookup function for an associative array throws an exception if the key has no value associated. Depending on context, this "key absent" exception may occur much more often than a successful lookup.

History

Main articles: Interrupt § History, IEEE 754 § History, and Exception handling (programming) § History

The first hardware exception handling was found in the UNIVAC I from 1951. Arithmetic overflow executed two instructions at address 0 which could transfer control or fix up the result. Software exception handling developed in the 1960s and 1970s. Exception handling was subsequently widely adopted by many programming languages from the 1980s onward.

Hardware exceptions

Main article: Interrupt

There is no clear consensus as to the exact meaning of an exception with respect to hardware. From the implementation point of view, it is handled identically to an interrupt: the processor halts execution of the current program, looks up the interrupt handler in the interrupt vector table for that exception or interrupt condition, saves state, and switches control.

IEEE 754 floating-point exceptions

Main article: IEEE 754 § Exception handling

Exception handling in the IEEE 754 floating-point standard refers in general to exceptional conditions and defines an exception as "an event that occurs when an operation on some particular operands has no outcome suitable for every reasonable application. That operation might signal one or more exceptions by invoking the default or, if explicitly requested, a language-defined alternate handling."

By default, an IEEE 754 exception is resumable and is handled by substituting a predefined value for different exceptions, e.g. infinity for a divide by zero exception, and providing status flags for later checking of whether the exception occurred (see C99 programming language for a typical example of handling of IEEE 754 exceptions). An exception-handling style enabled by the use of status flags involves: first computing an expression using a fast, direct implementation; checking whether it failed by testing status flags; and then, if necessary, calling a slower, more numerically robust, implementation.

The IEEE 754 standard uses the term "trapping" to refer to the calling of a user-supplied exception-handling routine on exceptional conditions, and is an optional feature of the standard. The standard recommends several usage scenarios for this, including the implementation of non-default pre-substitution of a value followed by resumption, to concisely handle removable singularities.

The default IEEE 754 exception handling behaviour of resumption following pre-substitution of a default value avoids the risks inherent in changing flow of program control on numerical exceptions. For example, the 1996 Cluster spacecraft launch ended in a catastrophic explosion due in part to the Ada exception handling policy of aborting computation on arithmetic error. William Kahan claims the default IEEE 754 exception handling behavior would have prevented this.

In programming languages

In computer programming, several language mechanisms exist for exception handling. The term exception is typically used to denote a data structure storing information about an exceptional condition. One mechanism to transfer control, or raise an exception, is known as a throw; the exception is said to be thrown. Execution is transferred to a catch.

In user interfaces

Front-end web development frameworks, such as React and Vue, have introduced error handling mechanisms where errors propagate up the user interface (UI) component hierarchy, in a way that is analogous to how errors propagate up the call stack in executing code. Here the error boundary mechanism serves as an analogue to the typical try-catch mechanism. Thus a component can ensure that errors from its child components are caught and handled, and not propagated up to parent components.

For example, in Vue, a component would catch errors by implementing errorCaptured

Vue.component('parent', {
    template: '<div><slot></slot></div>',
    errorCaptured: (err, vm, info) => alert('An error occurred');
})
Vue.component('child', {
    template: '<div>{{ cause_error() }}</div>'
})

When used like this in markup:

<parent>
    <child></child>
</parent>

The error produced by the child component is caught and handled by the parent component.