Entropy (statistical thermodynamics)

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Entropy_(statistical_thermodynamics)

The concept entropy was first developed by German physicist Rudolf Clausius in the mid-nineteenth century as a thermodynamic property that predicts that certain spontaneous processes are irreversible or impossible. In statistical mechanics, entropy is formulated as a statistical property using probability theory. The statistical entropy perspective was introduced in 1870 by Austrian physicist Ludwig Boltzmann, who established a new field of physics that provided the descriptive linkage between the macroscopic observation of nature and the microscopic view based on the rigorous treatment of large ensembles of microscopic states that constitute thermodynamic systems.

Boltzmann's principle

Main article: Boltzmann's entropy formula

Ludwig Boltzmann defined entropy as a measure of the number of possible microscopic states (microstates) of a system in thermodynamic equilibrium, consistent with its macroscopic thermodynamic properties, which constitute the macrostate of the system. A useful illustration is the example of a sample of gas contained in a container. The easily measurable parameters volume, pressure, and temperature of the gas describe its macroscopic condition (state). At a microscopic level, the gas consists of a vast number of freely moving atoms or molecules, which randomly collide with one another and with the walls of the container. The collisions with the walls produce the macroscopic pressure of the gas, which illustrates the connection between microscopic and macroscopic phenomena.

A microstate of the system is a description of the positions and momenta of all its particles. The large number of particles of the gas provides a near infinite number of possible microstates for the sample, but collectively they exhibit a well-defined average of configuration, which is exhibited as the macrostate of the system, to which each individual microstate contribution is negligibly small. The ensemble of microstates comprises a statistical distribution of probability for each microstate, and the group of most probable configurations accounts for the macroscopic state. Therefore, the system can be described as a whole by only a few macroscopic parameters, called the thermodynamic variables: the total energy E, volume V, pressure P, temperature T, and so forth. However, this description is relatively simple only when the system is in a state of equilibrium.

Equilibrium may be illustrated with a simple example of a drop of food coloring falling into a glass of water. The dye diffuses in a complicated manner, which is difficult to precisely predict. However, after sufficient time has passed, the system reaches a uniform color, a state much easier to describe and explain.

Boltzmann formulated a simple relationship between entropy and the number of possible microstates of a system, which is denoted by the symbol Ω. The entropy S is proportional to the natural logarithm of this number: $${\displaystyle S=k_{\text{B}}\ln \mathrm{\Omega }}$$ The proportionality constant k_B is one of the fundamental constants of physics and is named the Boltzmann constant in honor of its discoverer.

Boltzmann's entropy describes the system when all the accessible microstates are equally likely. It is the configuration corresponding to the maximum of entropy at equilibrium. The randomness or disorder is maximal, and so is the lack of distinction (or information) of each microstate.

Entropy is a thermodynamic property just like pressure, volume, or temperature. Therefore, it connects the microscopic and the macroscopic world view.

Boltzmann's principle is regarded as the foundation of statistical mechanics.

Gibbs entropy formula

The macroscopic state of a system is characterized by a distribution on the microstates. The entropy of this distribution is given by the Gibbs entropy formula, named after J. Willard Gibbs. For a classical system (i.e., a collection of classical particles) with a discrete set of microstates, if E_i is the energy of microstate i, and p_i is the probability that it occurs during the system's fluctuations, then the entropy of the system is $${\displaystyle S=-k_{\text{B}}\sum _i{p}_i\ln (p_i)}$$

Entropy changes for systems in a canonical state

A system with a well-defined temperature, i.e., one in thermal equilibrium with a thermal reservoir, has a probability of being in a microstate i given by Boltzmann's distribution.

Changes in the entropy caused by changes in the external constraints are then given by: $${\displaystyle \begin{array}{rl}dS& =-k_{\text{B}}\sum _{i}d{p}_i\ln p_i\\ & =-k_{\text{B}}\sum _{i}d{p}_i(-E_i/k_{\text{B}}T-\ln Z)\\ & =\sum _i{E}_{i}d{p}_i/T\\ & =\sum _i[d(E_i{p}_i)-(d{E}_i)p_i]/T\end{array}}$$ where we have twice used the conservation of probability, ∑_i dp_i = 0.

Now, ∑_i d(E_i p_i) is the expectation value of the change in the total energy of the system.

If the changes are sufficiently slow, so that the system remains in the same microscopic state, but the state slowly (and reversibly) changes, then ∑_i (dE_i) p_i is the expectation value of the work done on the system through this reversible process, dW_rev.

But from the first law of thermodynamics, dE = δQ + δW. Therefore, $${\displaystyle dS=\frac{\delta ⟨Q_{\text{rev}}⟩}{T}}$$

In the thermodynamic limit, the fluctuation of the macroscopic quantities from their average values becomes negligible; so this reproduces the definition of entropy from classical thermodynamics, given above.

The quantity ${\displaystyle k_{\text{B}}}$ is the Boltzmann constant, a multiplier of the summation expression. The summation is dimensionless, since the value ${\displaystyle p_i}$ is a probability and therefore dimensionless, and ln is the natural logarithm. Hence the SI unit on both sides of the equation is that of heat capacity: $${\displaystyle [S]=[k_{\text{B}}]=\frac{\mathrm{J}}{\mathrm{K}}}$$

This definition remains meaningful even when the system is far away from equilibrium. Other definitions assume that the system is in thermal equilibrium, either as an isolated system, or as a system in exchange with its surroundings. The set of microstates (with probability distribution) over which the sum is found is called a statistical ensemble. Each type of statistical ensemble (micro-canonical, canonical, grand-canonical, etc.) describes a different configuration of the system's exchanges with the outside, varying from a completely isolated system to a system that can exchange one or more quantities with a reservoir, like energy, volume or molecules. In every ensemble, the equilibrium configuration of the system is dictated by the maximization of the entropy of the union of the system and its reservoir, according to the second law of thermodynamics (see the statistical mechanics article).

Neglecting correlations (or, more generally, statistical dependencies) between the states of individual particles will lead to an incorrect probability distribution on the microstates and hence to an overestimate of the entropy. Such correlations occur in any system with nontrivially interacting particles, that is, in all systems more complex than an ideal gas.

This S is almost universally called simply the entropy. It can also be called the statistical entropy or the thermodynamic entropy without changing the meaning. Note the above expression of the statistical entropy is a discretized version of Shannon entropy. The von Neumann entropy formula is an extension of the Gibbs entropy formula to the quantum-mechanical case.

It has been shown that the Gibbs entropy is equal to the classical "heat engine" entropy characterized by ${\displaystyle dS=\delta Q/T}$, and the generalized Boltzmann distribution is a sufficient and necessary condition for this equivalence. Furthermore, the Gibbs entropy is the only entropy measure that is equivalent to the classical "heat engine" entropy under the following postulates:

1. The probability density function is proportional to some function of the ensemble parameters and random variables.
2. Thermodynamic state functions are described by ensemble averages of random variables.
3. In the limit of infinite temperature, all the microstates have the same probability.

Ensembles

The various ensembles used in statistical thermodynamics are linked to the entropy by the following relations:^{[clarification needed]} $${\displaystyle S=k_{\text{B}}\ln {\mathrm{\Omega }}_{\text{mic}}=k_{\text{B}}(\ln Z_{\text{can}}+\beta \overline{E})=k_{\text{B}}(\ln {\mathcal{Z}}_{\text{gr}}+\beta (\overline{E}-\mu \overline{N}))}$$

• ${\displaystyle {\mathrm{\Omega }}_{\text{mic}}}$ is the microcanonical partition function
• ${\displaystyle Z_{\text{can}}}$ is the canonical partition function
• ${\displaystyle {\mathcal{Z}}_{\text{gr}}}$ is the grand canonical partition function

Order through chaos and the second law of thermodynamics

We can think of Ω as a measure of our lack of knowledge about a system. To illustrate this idea, consider a set of 100 coins, each of which is either heads up or tails up. In this example, let us suppose that the macrostates are specified by the total number of heads and tails, while the microstates are specified by the facings of each individual coin (i.e., the exact order in which heads and tails occur). For the macrostates of 100 heads or 100 tails, there is exactly one possible configuration, so our knowledge of the system is complete. At the opposite extreme, the macrostate which gives us the least knowledge about the system consists of 50 heads and 50 tails in any order, for which there are 100891344545564193334812497256 (100 choose 50) ≈ 10²⁹ possible microstates.

Even when a system is entirely isolated from external influences, its microstate is constantly changing. For instance, the particles in a gas are constantly moving, and thus occupy a different position at each moment of time; their momenta are also constantly changing as they collide with each other or with the container walls. Suppose we prepare the system in an artificially highly ordered equilibrium state. For instance, imagine dividing a container with a partition and placing a gas on one side of the partition, with a vacuum on the other side. If we remove the partition and watch the subsequent behavior of the gas, we will find that its microstate evolves according to some chaotic and unpredictable pattern, and that on average these microstates will correspond to a more disordered macrostate than before. It is possible, but extremely unlikely, for the gas molecules to bounce off one another in such a way that they remain in one half of the container. It is overwhelmingly probable for the gas to spread out to fill the container evenly, which is the new equilibrium macrostate of the system.

This is an example illustrating the second law of thermodynamics:

the total entropy of any isolated thermodynamic system tends to increase over time, approaching a maximum value.

Since its discovery, this idea has been the focus of a great deal of thought, some of it confused. A chief point of confusion is the fact that the Second Law applies only to isolated systems. For example, the Earth is not an isolated system because it is constantly receiving energy in the form of sunlight. In contrast, the universe may be considered an isolated system, so that its total entropy is constantly increasing. (Needs clarification. See: Second law of thermodynamics#cite note-Grandy 151-21)

Counting of microstates

In classical statistical mechanics, the number of microstates is actually uncountably infinite, since the properties of classical systems are continuous. For example, a microstate of a classical ideal gas is specified by the positions and momenta of all the atoms, which range continuously over the real numbers. If we want to define Ω, we have to come up with a method of grouping the microstates together to obtain a countable set. This procedure is known as coarse graining. In the case of the ideal gas, we count two states of an atom as the "same" state if their positions and momenta are within δx and δp of each other. Since the values of δx and δp can be chosen arbitrarily, the entropy is not uniquely defined. It is defined only up to an additive constant. (As we will see, the thermodynamic definition of entropy is also defined only up to a constant.)

To avoid coarse graining one can take the entropy as defined by the H-theorem. $${\displaystyle S=-k_{\text{B}}{H}_{\text{B}}:=-k_{\text{B}}\int f(q_i,p_i)\ln f(q_i,p_i)d{q}_{1}d{p}_1\cdots d{q}_{N}d{p}_N}$$

However, this ambiguity can be resolved with quantum mechanics. The quantum state of a system can be expressed as a superposition of "basis" states, which can be chosen to be energy eigenstates (i.e. eigenstates of the quantum Hamiltonian). Usually, the quantum states are discrete, even though there may be an infinite number of them. For a system with some specified energy E, one takes Ω to be the number of energy eigenstates within a macroscopically small energy range between E and E + δE. In the thermodynamical limit, the specific entropy becomes independent on the choice of δE.

An important result, known as Nernst's theorem or the third law of thermodynamics, states that the entropy of a system at zero absolute temperature is a well-defined constant. This is because a system at zero temperature exists in its lowest-energy state, or ground state, so that its entropy is determined by the degeneracy of the ground state. Many systems, such as crystal lattices, have a unique ground state, and (since ln(1) = 0) this means that they have zero entropy at absolute zero. Other systems have more than one state with the same, lowest energy, and have a non-vanishing "zero-point entropy". For instance, ordinary ice has a zero-point entropy of 3.41 J/(mol⋅K), because its underlying crystal structure possesses multiple configurations with the same energy (a phenomenon known as geometrical frustration).

The third law of thermodynamics states that the entropy of a perfect crystal at absolute zero (0 K) is zero. This means that nearly all molecular motion should cease. The oscillator equation for predicting quantized vibrational levels shows that even when the vibrational quantum number is 0, the molecule still has vibrational energy: $${\displaystyle E_{\nu }=h{\nu }_0\left(n+\frac{1}{2}\right)}$$ where ${\displaystyle h}$ is the Planck constant, ${\displaystyle {\nu }_0}$ is the characteristic frequency of the vibration, and ${\displaystyle n}$ is the vibrational quantum number. Even when ${\displaystyle n=0}$ (the zero-point energy), ${\displaystyle E_n}$ does not equal 0, in adherence to the Heisenberg uncertainty principle.

Quantum indeterminacy

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Quantum_indeterminacy 

Quantum indeterminacy is the apparent necessary incompleteness in the description of a physical system, that has become one of the characteristics of the standard description of quantum physics. Prior to quantum physics, it was thought that

1. a physical system had a determinate state that uniquely determined all the values of its measurable properties, and
2. conversely, the values of its measurable properties uniquely determined the state.

Quantum indeterminacy can be quantitatively characterized by a probability distribution on the set of outcomes of measurements of an observable. The distribution is uniquely determined by the system state, and moreover quantum mechanics provides a recipe for calculating this probability distribution.

Indeterminacy in measurement was not an innovation of quantum mechanics, since it had been established early on by experimentalists that errors in measurement may lead to indeterminate outcomes. By the later half of the 18th century, measurement errors were well understood, and it was known that they could either be reduced by better equipment or accounted for by statistical error models. In quantum mechanics, however, indeterminacy is of a much more fundamental nature, having nothing to do with errors or disturbance.

Measurement

An adequate account of quantum indeterminacy requires a theory of measurement. Many theories have been proposed since the beginning of quantum mechanics and quantum measurement continues to be an active research area in both theoretical and experimental physics. Possibly the first systematic attempt at a mathematical theory was developed by John von Neumann. The kinds of measurements he investigated are now called projective measurements. That theory was based in turn on the theory of projection-valued measures for self-adjoint operators that had been recently developed (by von Neumann and independently by Marshall Stone) and the Hilbert space formulation of quantum mechanics (attributed by von Neumann to Paul Dirac).

In this formulation, the state of a physical system corresponds to a vector of length 1 in a Hilbert space H over the complex numbers. An observable is represented by a self-adjoint (i.e. Hermitian) operator A on H. If H is finite dimensional, by the spectral theorem, A has an orthonormal basis of eigenvectors. If the system is in state ψ, then immediately after measurement the system will occupy a state that is an eigenvector e of A and the observed value λ will be the corresponding eigenvalue of the equation Ae = λe. It is immediate from this that measurement in general will be non-deterministic. Quantum mechanics, moreover, gives a recipe for computing a probability distribution Pr on the possible outcomes given the initial system state is ψ. The probability is $${\displaystyle \mathrm{Pr}(\lambda )=⟨\mathrm{E}(\lambda )\psi \mid \psi ⟩}$$ where E(λ) is the projection onto the space of eigenvectors of A with eigenvalue λ.

Example

Bloch sphere showing eigenvectors for Pauli Spin matrices. The Bloch sphere is a two-dimensional surface the points of which correspond to the state space of a spin 1/2 particle. At the state ψ the values of σ₁ are +1 whereas the values of σ₂ and σ₃ take the values +1, −1 with probability 1/2.

In this example, we consider a single spin 1/2 particle (such as an electron) in which we only consider the spin degree of freedom. The corresponding Hilbert space is the two-dimensional complex Hilbert space C², with each quantum state corresponding to a unit vector in C² (unique up to phase). In this case, the state space can be geometrically represented as the surface of a sphere, as shown in the figure on the right.

The Pauli spin matrices $${\displaystyle {\sigma }_1=\left(\begin{array}{cc}0& 1\\ 1& 0\end{array}\right),{\sigma }_2=\left(\begin{array}{cc}0& -i\\ i& 0\end{array}\right),{\sigma }_3=\left(\begin{array}{cc}1& 0\\ 0& -1\end{array}\right)}$$ are self-adjoint and correspond to spin-measurements along the 3 coordinate axes.

The Pauli matrices all have the eigenvalues +1, −1.

• For σ₁, these eigenvalues correspond to the eigenvectors $${\displaystyle \frac{1}{\sqrt{2}}(1,1),\frac{1}{\sqrt{2}}(1,-1)}$$
• For σ₃, they correspond to the eigenvectors $${\displaystyle (1,0),(0,1)}$$

Thus in the state $${\displaystyle \psi =\frac{1}{\sqrt{2}}(1,1),}$$ σ₁ has the determinate value +1, while measurement of σ₃ can produce either +1, −1 each with probability 1/2. In fact, there is no state in which measurement of both σ₁ and σ₃ have determinate values.

There are various questions that can be asked about the above indeterminacy assertion.

1. Can the apparent indeterminacy be construed as in fact deterministic, but dependent upon quantities not modeled in the current theory, which would therefore be incomplete? More precisely, are there hidden variables that could account for the statistical indeterminacy in a completely classical way?
2. Can the indeterminacy be understood as a disturbance of the system being measured?

Von Neumann formulated the question 1) and provided an argument why the answer had to be no, if one accepted the formalism he was proposing. However, according to Bell, von Neumann's formal proof did not justify his informal conclusion. A definitive but partial negative answer to 1) has been established by experiment: because Bell's inequalities are violated, any such hidden variable(s) cannot be local (see Bell test experiments).

The answer to 2) depends on how disturbance is understood, particularly since measurement entails disturbance (however note that this is the observer effect, which is distinct from the uncertainty principle). Still, in the most natural interpretation the answer is also no. To see this, consider two sequences of measurements: (A) that measures exclusively σ₁ and (B) that measures only σ₃ of a spin system in the state ψ. The measurement outcomes of (A) are all +1, while the statistical distribution of the measurements (B) is still divided between +1, −1 with equal probability.

Other examples of indeterminacy

Quantum indeterminacy can also be illustrated in terms of a particle with a definitely measured momentum for which there must be a fundamental limit to how precisely its location can be specified. This quantum uncertainty principle can be expressed in terms of other variables, for example, a particle with a definitely measured energy has a fundamental limit to how precisely one can specify how long it will have that energy. The magnitude involved in quantum uncertainty is on the order of the Planck constant (6.62607015×10⁻³⁴ J⋅Hz⁻¹).

Indeterminacy and incompleteness

Quantum indeterminacy is the assertion that the state of a system does not determine a unique collection of values for all its measurable properties. Indeed, according to the Kochen–Specker theorem, in the quantum mechanical formalism it is impossible that, for a given quantum state, each one of these measurable properties (observables) has a determinate (sharp) value. The values of an observable will be obtained non-deterministically in accordance with a probability distribution that is uniquely determined by the system state. Note that the state is destroyed by measurement, so when we refer to a collection of values, each measured value in this collection must be obtained using a freshly prepared state.

This indeterminacy might be regarded as a kind of essential incompleteness in our description of a physical system. Notice however, that the indeterminacy as stated above only applies to values of measurements not to the quantum state. For example, in the spin 1/2 example discussed above, the system can be prepared in the state ψ by using measurement of σ₁ as a filter that retains only those particles such that σ₁ yields +1. By the von Neumann (so-called) postulates, immediately after the measurement the system is assuredly in the state ψ.

However, Albert Einstein believed that quantum state cannot be a complete description of a physical system and, it is commonly thought, never came to terms with quantum mechanics. In fact, Einstein, Boris Podolsky and Nathan Rosen showed that if quantum mechanics is correct, then the classical view of how the real world works (at least after special relativity) is no longer tenable. This view included the following two ideas:

1. A measurable property of a physical system whose value can be predicted with certainty is actually an element of (local) reality (this was the terminology used by EPR).
2. Effects of local actions have a finite propagation speed.

This failure of the classical view was one of the conclusions of the EPR thought experiment in which two remotely located observers, now commonly referred to as Alice and Bob, perform independent measurements of spin on a pair of electrons, prepared at a source in a special state called a spin singlet state. It was a conclusion of EPR, using the formal apparatus of quantum theory, that once Alice measured spin in the x direction, Bob's measurement in the x direction was determined with certainty, whereas immediately before Alice's measurement Bob's outcome was only statistically determined. From this it follows that either value of spin in the x direction is not an element of reality or that the effect of Alice's measurement has infinite speed of propagation.

Indeterminacy for mixed states

We have described indeterminacy for a quantum system that is in a pure state. Mixed states are a more general kind of state obtained by a statistical mixture of pure states. For mixed states the "quantum recipe" for determining the probability distribution of a measurement is determined as follows:

Let A be an observable of a quantum mechanical system. A is given by a densely defined self-adjoint operator on H. The spectral measure of A is a projection-valued measure defined by the condition

${\displaystyle {\mathrm{E}}_A(U)={\int }_U\lambda d\mathrm{E}(\lambda ),}$

for every Borel subset U of R. Given a mixed state S, we introduce the distribution of A under S as follows:

${\displaystyle {\mathrm{D}}_A(U)=\mathrm{Tr}({\mathrm{E}}_A(U)S).}$

This is a probability measure defined on the Borel subsets of R that is the probability distribution obtained by measuring A in S.

Logical independence and quantum randomness

Quantum indeterminacy is often understood as information (or lack of it) whose existence we infer, occurring in individual quantum systems, prior to measurement. Quantum randomness is the statistical manifestation of that indeterminacy, witnessable in results of experiments repeated many times. However, the relationship between quantum indeterminacy and randomness is subtle and can be considered differently.

In classical physics, experiments of chance, such as coin-tossing and dice-throwing, are deterministic, in the sense that, perfect knowledge of the initial conditions would render outcomes perfectly predictable. The ‘randomness’ stems from ignorance of physical information in the initial toss or throw. In diametrical contrast, in the case of quantum physics, the theorems of Kochen and Specker, the inequalities of John Bell, and experimental evidence of Alain Aspect, all indicate that quantum randomness does not stem from any such physical information.

In 2008, Tomasz Paterek et al. provided an explanation in mathematical information. They proved that quantum randomness is, exclusively, the output of measurement experiments whose input settings introduce logical independence into quantum systems.

Logical independence is a well-known phenomenon in Mathematical Logic. It refers to the null logical connectivity that exists between mathematical propositions (in the same language) that neither prove nor disprove one another.

In the work of Paterek et al., the researchers demonstrate a link connecting quantum randomness and logical independence in a formal system of Boolean propositions. In experiments measuring photon polarisation, Paterek et al. demonstrate statistics correlating predictable outcomes with logically dependent mathematical propositions, and random outcomes with propositions that are logically independent.