Search This Blog

Monday, March 2, 2026

Numerical methods for ordinary differential equations

Illustration of numerical integration for the differential equation
  Blue: Euler method
  Green: Midpoint method
  Red: Exact solution: .
The step size is .
The same illustration for The midpoint method converges faster than the Euler method, as .

Numerical methods for ordinary differential equations are methods used to find numerical approximations to the solutions of ordinary differential equations (ODEs). Their use is also known as "numerical integration", although this term can also refer to the computation of integrals.

Many differential equations cannot be solved exactly. For practical purposes, however – such as in engineering – a numeric approximation to the solution is often sufficient. The algorithms studied here can be used to compute such an approximation. An alternative method is to use techniques from calculus to obtain a series expansion of the solution.

Ordinary differential equations occur in many scientific disciplines, including physics, chemistry, biology, and economics. In addition, some methods in numerical partial differential equations convert the partial differential equation into an ordinary differential equation, which must then be solved.

The problem

A first-order differential equation is an Initial value problem (IVP) of the form,

where is a function , and the initial condition is a given vector. First-order means that only the first derivative of y appears in the equation, and higher derivatives are absent.

Without loss of generality to higher-order systems, we restrict ourselves to first-order differential equations, because a higher-order ODE can be converted into a larger system of first-order equations by introducing extra variables. For example, the second-order equation y′′ = −y can be rewritten as two first-order equations: y′ = z and z′ = −y.

In this section, we describe numerical methods for IVPs, and remark that boundary value problems (BVPs) require a different set of tools. In a BVP, one defines values, or components of the solution y at more than one point. Because of this, different methods need to be used to solve BVPs. For example, the shooting method (and its variants) or global methods like finite differencesGalerkin methods, or collocation methods are appropriate for that class of problems.

The Picard–Lindelöf theorem states that there is a unique solution, provided f is Lipschitz-continuous.

Methods

Numerical methods for solving first-order IVPs often fall into one of two large categories: linear multistep methods, or Runge–Kutta methods. A further division can be realized by dividing methods into those that are explicit and those that are implicit. For example, implicit linear multistep methods include Adams-Moulton methods, and backward differentiation methods (BDF), whereas implicit Runge–Kutta methods include diagonally implicit Runge–Kutta (DIRK), singly diagonally implicit Runge–Kutta (SDIRK), and Gauss–Radau (based on Gaussian quadrature) numerical methods. Explicit examples from the linear multistep family include the Adams–Bashforth methods, and any Runge–Kutta method with a lower diagonal Butcher tableau is explicit. A loose rule of thumb dictates that stiff differential equations require the use of implicit schemes, whereas non-stiff problems can be solved more efficiently with explicit schemes.

The so-called general linear methods (GLMs) are a generalization of the above two large classes of methods.

Euler method

From any point on a curve, you can find an approximation of a nearby point on the curve by moving a short distance along a line tangent to the curve.

Starting with the differential equation (1), we replace the derivative y′ by the finite difference approximation

which when re-arranged yields the following formula and using (1) gives:

This formula is usually applied in the following way. We choose a step size h, and we construct the sequence We denote by a numerical estimate of the exact solution . Motivated by (3), we compute these estimates by the following recursive scheme

This is the Euler method (or forward Euler method, in contrast with the backward Euler method, to be described below). The method is named after Leonhard Euler who described it in 1768.

The Euler method is an example of an explicit method. This means that the new value yn+1 is defined in terms of things that are already known, like yn.

Backward Euler method

If, instead of (2), we use the approximation

we get the backward Euler method:

The backward Euler method is an implicit method, meaning that we have to solve an equation to find yn+1. One often uses fixed-point iteration or (some modification of) the Newton–Raphson method to achieve this.

It costs more time to solve this equation than explicit methods; this cost must be taken into consideration when one selects the method to use. The advantage of implicit methods such as (6) is that they are usually more stable for solving a stiff equation, meaning that a larger step size h can be used.

First-order exponential integrator method

Exponential integrators describe a large class of integrators that have recently seen a lot of development. They date back to at least the 1960s.

In place of (1), we assume the differential equation is either of the form

or it has been locally linearized about a background state to produce a linear term and a nonlinear term .

Exponential integrators are constructed by multiplying (7) by , and exactly integrating the result over a time interval where : This integral equation is exact, but it does not define the integral.

The first-order exponential integrator can be realized by holding constant over the full interval:

Generalizations

The Euler method is often not accurate enough. In more precise terms, it only has order one (the concept of order is explained below). This caused mathematicians to look for higher-order methods.

One possibility is to use not only the previously computed value yn to determine yn+1, but to make the solution depend on more past values. This yields a so-called multistep method. Perhaps the simplest is the leapfrog method which is second order and (roughly speaking) relies on two time values.

Almost all practical multistep methods fall within the family of linear multistep methods, which have the form

Another possibility is to use more points in the interval . This leads to the family of Runge–Kutta methods, named after Carl Runge and Martin Kutta. One of their fourth-order methods is especially popular.

Advanced features

A good implementation of one of these methods for solving an ODE entails more than the time-stepping formula.

It is often inefficient to use the same step size all the time, so variable step-size methods have been developed. Usually, the step size is chosen such that the (local) error per step is below some tolerance level. This means that the methods must also compute an error indicator, an estimate of the local error.

An extension of this idea is to choose dynamically between different methods of different orders (this is called a variable order method). Methods based on Richardson extrapolation, such as the Bulirsch–Stoer algorithm, are often used to construct various methods of different orders.

Other desirable features include:

  • dense output: cheap numerical approximations for the whole integration interval, and not only at the points t0, t1, t2, ...
  • event location: finding the times where, say, a particular function vanishes. This typically requires the use of a root-finding algorithm.
  • support for parallel computing.
  • when used for integrating with respect to time, time reversibility

Alternative methods

Many methods do not fall within the framework discussed here. Some classes of alternative methods are:

  • multiderivative methods, which use not only the function f but also its derivatives. This class includes Hermite–Obreschkoff methods and Fehlberg methods, as well as methods like the Parker–Sochacki method or Bychkov–Scherbakov method, which compute the coefficients of the Taylor series of the solution y recursively.
  • methods for second order ODEs. We said that all higher-order ODEs can be transformed to first-order ODEs of the form (1). While this is certainly true, it may not be the best way to proceed. In particular, Nyström methods work directly with second-order equations.
  • geometric integration methods are especially designed for special classes of ODEs (for example, symplectic integrators for the solution of Hamiltonian equations). They take care that the numerical solution respects the underlying structure or geometry of these classes.
  • Quantized state systems methods are a family of ODE integration methods based on the idea of state quantization. They are efficient when simulating sparse systems with frequent discontinuities.

Parallel-in-time methods

Some IVPs require integration at such high temporal resolution and/or over such long time intervals that classical serial time-stepping methods become computationally infeasible to run in real-time (e.g. IVPs in numerical weather prediction, plasma modelling, and molecular dynamics). Parallel-in-time (PinT) methods have been developed in response to these issues in order to reduce simulation runtimes through the use of parallel computing.

Early PinT methods (the earliest being proposed in the 1960s) were initially overlooked by researchers due to the fact that the parallel computing architectures that they required were not yet widely available. With more computing power available, interest was renewed in the early 2000s with the development of Parareal, a flexible, easy-to-use PinT algorithm that is suitable for solving a wide variety of IVPs. The advent of exascale computing has meant that PinT algorithms are attracting increasing research attention and are being developed in such a way that they can harness the world's most powerful supercomputers. The most popular methods as of 2023 include Parareal, PFASST, ParaDiag, and MGRIT.

Analysis

Numerical analysis is not only the design of numerical methods, but also their analysis. Three central concepts in this analysis are:

  • convergence: whether the method approximates the solution,
  • order: how well it approximates the solution, and
  • stability: whether errors are damped out.

Convergence

A numerical method is said to be convergent if the numerical solution approaches the exact solution as the step size h goes to 0. More precisely, we require that for every ODE (1) with a Lipschitz function f and every t* > 0,

All the methods mentioned above are convergent.

Consistency and order

Suppose the numerical method is

The local (truncation) error of the method is the error committed by one step of the method. That is, it is the difference between the result given by the method, assuming that no error was made in earlier steps, and the exact solution:

The method is said to be consistent if The method has order if Hence a method is consistent if it has an order greater than 0. The (forward) Euler method (4) and the backward Euler method (6) introduced above both have order 1, so they are consistent. Most methods being used in practice attain higher order. Consistency is a necessary condition for convergence, but not sufficient; for a method to be convergent, it must be both consistent and zero-stable.

A related concept is the global (truncation) error, the error sustained in all the steps one needs to reach a fixed time . Explicitly, the global error at time is where . The global error of a -th order one-step method is ; in particular, such a method is convergent. This statement is not necessarily true for multi-step methods.

Stability and stiffness

For some differential equations, application of standard methods—such as the Euler method, explicit Runge–Kutta methods, or multistep methods (for example, Adams–Bashforth methods)—exhibit instability in the solutions, though other methods may produce stable solutions. This "difficult behaviour" in the equation (which may not necessarily be complex itself) is described as stiffness, and is often caused by the presence of different time scales in the underlying problem. For example, a collision in a mechanical system like in an impact oscillator typically occurs at much smaller time scale than the time for the motion of objects; this discrepancy makes for very "sharp turns" in the curves of the state parameters.

Stiff problems are ubiquitous in chemical kinetics, control theory, solid mechanics, weather forecasting, biology, plasma physics, and electronics. One way to overcome stiffness is to extend the notion of differential equation to that of differential inclusion, which allows for and models non-smoothness.

History

Below is a timeline of some important developments in this field.

Numerical solutions to second-order one-dimensional boundary value problems

Boundary value problems (BVPs) are usually solved numerically by solving an approximately equivalent matrix problem obtained by discretizing the original BVP. The most commonly used method for numerically solving BVPs in one dimension is called the Finite Difference Method. This method takes advantage of linear combinations of point values to construct finite difference coefficients that describe derivatives of the function. For example, the second-order central difference approximation to the first derivative is given by:

and the second-order central difference for the second derivative is given by:

In both of these formulae, is the distance between neighbouring x values on the discretized domain. One then constructs a linear system that can then be solved by standard matrix methods. For example, suppose the equation to be solved is:

The next step would be to discretize the problem and use linear derivative approximations such as

and solve the resulting system of linear equations. This would lead to equations such as:

On first viewing, this system of equations appears to have difficulty associated with the fact that the equation involves no terms that are not multiplied by variables, but in fact this is false. At i = 1 and n − 1 there is a term involving the boundary values and and since these two values are known, one can simply substitute them into this equation and as a result have a non-homogeneous system of linear equations that has non-trivial solutions.

Gut–brain axis

From Wikipedia, the free encyclopedia
Gut–brain axis overview

The gut–brain axis is the two-way biochemical signaling that takes place between the gastrointestinal tract (GI tract) and the central nervous system (CNS). The term "microbiota–gut–brain axis" highlights the putative role of gut microbiota interacting with brain functions, according to preliminary research. Broadly defined, the gut–brain axis includes the central nervous system, neuroendocrine system, neuroimmune systems, the hypothalamic–pituitary–adrenal axis (HPA axis), sympathetic and parasympathetic arms of the autonomic nervous system, the enteric nervous system, vagus nerve, and the gut microbiota.

Chemicals released by the gut microbiome can influence brain development, starting from birth. A review from 2015 states that the gut microbiome influences the CNS by "regulating brain chemistry and influencing neuro-endocrine systems associated with stress response, anxiety and memory function".

Various factors influence the human gut microbiota composition, accounting for the vast variability found amongst humans. Factors such as diet, medication exposure such as antibiotics, genetics, and environment all affect the framework of the human gut microbiota. These differences across populations may affect how the gut-brain axis is interpreted and studied. The diversity in the gut microbiomes across individuals has led to inconsistencies in this field of research.

The bidirectional communication may involve immune, endocrine, humoral and neural connections between the gastrointestinal tract and the central nervous system. A 2019 review of laboratory research suggests that the gut microbiome may influence brain function by releasing chemical signals, possibly including cytokines, neurotransmitters, neuropeptides, chemokines, endocrine messengers and microbial metabolites, such as "short-chain fatty acids, branched chain amino acids, and peptidoglycans". These chemical signals are then transported to the brain via the blood, neuropod cells, nerves, endocrine cells, where they may impact different metabolic processes.

The first of the brain–gut interactions shown, was the cephalic phase of digestion, in the release of gastric and pancreatic secretions in response to sensory signals, such as the smell and sight of food. This was first demonstrated by Pavlov through Nobel prize winning research in 1904.

As of October 2016, most of the work done on the role of gut microbiota in the gut–brain axis had been conducted in animals, or on characterizing the various neuroactive compounds that gut microbiota can produce.

Enteric nervous system

Gut-brain communication

The enteric nervous system is one of the main divisions of the nervous system and consists of a mesh-like system of neurons that governs the function of the gastrointestinal system; it has been described as a "second brain" for several reasons. The enteric nervous system can operate autonomously. It normally communicates with the central nervous system (CNS) through the parasympathetic (e.g., via the vagus nerve) and sympathetic (e.g., via the prevertebral ganglia) nervous systems. However, vertebrate studies show that when the vagus nerve is severed, the enteric nervous system continues to function.

In vertebrates, the enteric nervous system includes efferent neurons, afferent neurons, and interneurons, all of which make the enteric nervous system capable of carrying reflexes in the absence of CNS input. The sensory neurons report on mechanical and chemical conditions. Through intestinal muscles, the motor neurons control peristalsis and churning of intestinal contents. Other neurons control the secretion of enzymes. The enteric nervous system also makes use of more than 30 neurotransmitters, most of which are identical to the ones found in CNS, such as acetylcholine, dopamine, and serotonin. More than 90% of the body's serotonin lies in the gut, as well as about 50% of the body's dopamine; the dual function of these neurotransmitters is an active part of gut–brain research.

The first of the gut–brain interactions was shown to be between the sight and smell of food and the release of gastric secretions, known as the cephalic phase, or cephalic response of digestion.

Tryptophan metabolism by human gut microbiota
The image above contains clickable links
This diagram shows the biosynthesis of bioactive compounds (indole and certain other derivatives) from tryptophan by bacteria in the gut. Indole is produced from tryptophan by bacteria that express tryptophanaseClostridium sporogenes metabolizes tryptophan into indole and subsequently 3-indolepropionic acid (IPA), a highly potent neuroprotective antioxidant that scavenges hydroxyl radicals. IPA binds to the pregnane X receptor (PXR) in intestinal cells, thereby facilitating mucosal homeostasis and barrier function. Following absorption from the intestine and distribution to the brain, IPA confers a neuroprotective effect against cerebral ischemia and Alzheimer's diseaseLactobacillaceae (Lactobacillus s.l.) species metabolize tryptophan into indole-3-aldehyde (I3A) which acts on the aryl hydrocarbon receptor (AhR) in intestinal immune cells, in turn increasing interleukin-22 (IL-22) production. Indole itself triggers the secretion of glucagon-like peptide-1 (GLP-1) in intestinal L cells and acts as a ligand for AhR. Indole can also be metabolized by the liver into indoxyl sulfate, a compound that is toxic in high concentrations and associated with vascular disease and renal dysfunction. AST-120 (activated charcoal), an intestinal sorbent that is taken by mouth, adsorbs indole, in turn decreasing the concentration of indoxyl sulfate in blood plasma.

Gut


microbiota

The gut microbiota is the complex community of microorganisms that live in the digestive tracts of humans and other animals. The gut metagenome is the aggregate of all the genomes of gut microbiota. The gut is one niche that human microbiota inhabit.

In humans, the gut microbiota has the largest quantity of bacteria and the greatest number of species, compared to other areas of the body. In humans, the gut flora is established at one to two years after birth; by that time, the intestinal epithelium and the intestinal mucosal barrier that it secretes have co-developed in a way that is tolerant to, and even supportive of, the gut flora and that also provides a barrier to pathogenic organisms.

The relationship between gut microbiota and humans is not merely commensal (a non-harmful coexistence), but rather a mutualistic relationship. Human gut microorganisms benefit the host by collecting the energy from the fermentation of undigested carbohydrates and the subsequent absorption of short-chain fatty acids (SCFAs), acetate, butyrate, and propionate. Intestinal bacteria also play a role in synthesizing vitamin B and vitamin K as well as metabolizing bile acids, sterols, and xenobiotics. The systemic importance of the SCFAs and other compounds they produce are like hormones and the gut flora itself appears to function like an endocrine organ; dysregulation of the gut flora has been correlated with a host of inflammatory and autoimmune conditions.

The composition of human gut microbiota changes over time, when the diet changes, and as overall health changes. In general, the average human has over 1000 species of bacteria in their gut microbiome, with Bacteroidetes and Firmicutes being the dominant phyla. Diets higher in processed foods and unnatural chemicals can negatively alter the ratios of these species, while diets high in whole foods can positively alter the ratios. Additional health factors that may skew the composition of the gut microbiota are antibiotics and probiotics. Antibiotics have severe impacts on gut microbiota, ridding of both good and bad bacteria. Without proper rehabilitation, it can be easy for harmful bacteria to become dominant. Probiotics may help to mitigate this by supplying healthy bacteria into the gut and replenishing the richness and diversity of the gut microbiota. There are many strains of probiotics that can be administered depending on the needs of a specific individual.

Gut–brain integration

The gut–brain axis, a bidirectional neurohumoral communication system, is important for maintaining homeostasis and is regulated through the central and enteric nervous systems and the neural, endocrine, immune, and metabolic pathways, and especially including the hypothalamic–pituitary–adrenal axis (HPA axis). That term has been expanded to include the role of the gut microbiota as part of the "microbiome-gut-brain axis", a linkage of functions including the gut microbiota.

Interest in the field was sparked by a 2004 study (Nobuyuki Sudo and Yoichi Chida) showing that germ-free mice (genetically homogeneous laboratory mice, birthed and raised in an antiseptic environment) showed an exaggerated HPA axis response to stress, compared to non-GF laboratory mice.

The gut microbiota can produce a range of neuroactive molecules, such as acetylcholine, catecholamines, γ-aminobutyric acid, histamine, melatonin, and serotonin, which are essential for regulating peristalsis and sensation in the gut. Changes in the composition of the gut microbiota due to diet, drugs, or disease correlate with changes in levels of circulating cytokines, some of which can affect brain function. The gut microbiota also release molecules that can directly activate the vagus nerve, which transmits information about the state of the intestines to the brain.

Multiple pathways are implicated in the gut-brain axis. These pathways include neural signaling through the vagus nerve, endocrine signaling through stress hormones, and immune signaling mediated by cytokines. Gut barrier function and immune activity is influenced by metabolites such as short-chain fatty acids, which in turn affect the signaling received by the central nervous system. The pathways described are thought to work together simultaneously rather than function independently.

Likewise, chronic or acutely stressful situations activate the hypothalamic–pituitary–adrenal axis, causing changes in the gut microbiota and intestinal epithelium, and possibly having systemic effects. Additionally, the cholinergic anti-inflammatory pathway, signaling through the vagus nerve, affects the gut epithelium and microbiota. Hunger and satiety are integrated in the brain, and the presence or absence of food in the gut and types of food present also affect the composition and activity of gut microbiota.

Most of the work that has been done on the role of gut microbiota in the gut–brain axis has been conducted in animals, including the highly artificial germ-free mice. As of 2016, studies with humans measuring changes to gut microbiota in response to stress, or measuring effects of various probiotics, have generally been small and cannot be generalized; whether changes to gut microbiota are a result of disease, a cause of disease, or both in any number of possible feedback loops in the gut–brain axis, remains unclear.

Recent research has explored the gut-brain axis and its role in neurological or psychiatric conditions. Several studies have found associations between gut microbiota and psychiatric disorders such as depression, anxiety, and bipolar disorder; however, most of these come from animal studies and small human cohorts. Causal mechanisms are still being researched and at this point, the current literature considers these associations correlational and not causal.

The concept is of special interest in autoimmune diseases such as multiple sclerosis. This process is thought to be regulated via the gut microbiota, which ferment indigestible dietary fibre and resistant starch; the fermentation process produces short chain fatty acids (SCFAs) such as propionate, butyrate, and acetate. The history of ideas about a relationship between the gut and the mind dates from the nineteenth century.

Accelerationism

From Wikipedia, the free encyclopedia https://en.wikipedia.org/wiki/Accelerat...