Bayesian epistemology

From Wikipedia, the free encyclopedia

https://en.wikipedia.org/wiki/Bayesian_epistemology 

 

Part of a series on
Bayesian statistics

Bayesian epistemology is a formal approach to various topics in epistemology that has its roots in Thomas Bayes' work in the field of probability theory. One advantage of its formal method in contrast to traditional epistemology is that its concepts and theorems can be defined with a high degree of precision. It is based on the idea that beliefs can be interpreted as subjective probabilities. As such, they are subject to the laws of probability theory, which act as the norms of rationality. These norms can be divided into static constraints, governing the rationality of beliefs at any moment, and dynamic constraints, governing how rational agents should change their beliefs upon receiving new evidence. The most characteristic Bayesian expression of these principles is found in the form of Dutch books, which illustrate irrationality in agents through a series of bets that lead to a loss for the agent no matter which of the probabilistic events occurs. Bayesians have applied these fundamental principles to various epistemological topics but Bayesianism does not cover all topics of traditional epistemology. The problem of confirmation in the philosophy of science, for example, can be approached through the Bayesian principle of conditionalization by holding that a piece of evidence confirms a theory if it raises the likelihood that this theory is true. Various proposals have been made to define the concept of coherence in terms of probability, usually in the sense that two propositions cohere if the probability of their conjunction is higher than if they were neutrally related to each other. The Bayesian approach has also been fruitful in the field of social epistemology, for example, concerning the problem of testimony or the problem of group belief. Bayesianism still faces various theoretical objections that have not been fully solved.

Relation to traditional epistemology

Traditional epistemology and Bayesian epistemology are both forms of epistemology, but they differ in various respects, for example, concerning their methodology, their interpretation of belief, the role justification or confirmation plays in them and some of their research interests. Traditional epistemology focuses on topics such as the analysis of the nature of knowledge, usually in terms of justified true beliefs, the sources of knowledge, like perception or testimony, the structure of a body of knowledge, for example in the form of foundationalism or coherentism, and the problem of philosophical skepticism or the question of whether knowledge is possible at all. These inquiries are usually based on epistemic intuitions and regard beliefs as either present or absent. Bayesian epistemology, on the other hand, works by formalizing concepts and problems, which are often vague in the traditional approach. It thereby focuses more on mathematical intuitions and promises a higher degree of precision. It sees belief as a continuous phenomenon that comes in various degrees, so-called credences. Some Bayesians have even suggested that the regular notion of belief should be abandoned. But there are also proposals to connect the two, for example, the Lockean thesis, which defines belief as credence above a certain threshold. Justification plays a central role in traditional epistemology while Bayesians have focused on the related notions of confirmation and disconfirmation through evidence. The notion of evidence is important for both approaches but only the traditional approach has been interested in studying the sources of evidence, like perception and memory. Bayesianism, on the other hand, has focused on the role of evidence for rationality: how someone's credence should be adjusted upon receiving new evidence. There is an analogy between the Bayesian norms of rationality in terms of probabilistic laws and the traditional norms of rationality in terms of deductive consistency. Certain traditional problems, like the topic of skepticism about our knowledge of the external world, are difficult to express in Bayesian terms.

Fundamentals

Bayesian epistemology is based only on a few fundamental principles, which can be used to define various other notions and can be applied to many topics in epistemology. At their core, these principles constitute constraints on how we should assign credences to propositions. They determine what an ideally rational agent would believe. The basic principles can be divided into synchronic or static principles, which govern how credences are to be assigned at any moment, and diachronic or dynamic principles, which determine how the agent should change her beliefs upon receiving new evidence. The axioms of probability and the principal principle belong to the static principles while the principle of conditionalization governs the dynamic aspects as a form of probabilistic inference. The most characteristic Bayesian expression of these principles is found in the form of Dutch books, which illustrate irrationality in agents through a series of bets that lead to a loss for the agent no matter which of the probabilistic events occurs. This test for determining irrationality has been referred to as the "pragmatic self-defeat test".

Beliefs, probability and bets

One important difference to traditional epistemology is that Bayesian epistemology focuses not on the notion of simple belief but on the notion of degrees of belief, so-called credences. This approach tries to capture the idea of certainty: we believe in all kinds of claims but we are more certain about some, like that the earth is round, than about others, like that Plato was the author of the First Alcibiades. These degrees come in values between 0 and 1. 0 corresponds to full disbelief, 1 corresponds to full belief and 0.5 corresponds to suspension of belief. According to the Bayesian interpretation of probability, credences stand for subjective probabilities. Following Frank P. Ramsey, they are interpreted in terms of the willingness to bet money on a claim. So having a credence of 0.8 (i.e. 80 %) that your favorite soccer team will win the next game would mean being willing to bet up to four dollars for the chance to make one dollar profit. This account draws a tight connection between Bayesian epistemology and decision theory. It might seem that betting-behavior is only one special area and as such not suited for defining such a general notion as credences. But, as Ramsey argues, we bet all the time when understood in the widest sense. For example, in going to the train station, we bet on the train being there on time, otherwise we would have stayed at home. It follows from the interpretation of credence in terms of willingness to make bets that it would be irrational to ascribe a credence of 0 or 1 to any proposition, except for contradictions and tautologies. The reason for this is that ascribing these extreme values would mean that one would be willing to bet anything, including one's life, even if the payoff was minimal. Another negative side-effect of such extreme credences is that they are permanently fixed and cannot be updated anymore upon acquiring new evidence.

This central tenet of Bayesianism, that credences are interpreted as subjective probabilities and are therefore governed by the norms of probability, has been referred to as probabilism. These norms express the nature of the credences of ideally rational agents. They do not put demands on what credence we should have on any single given belief, for example, whether it will rain tomorrow. Instead, they constraint the system of beliefs as a whole. For example, if your credence that it will rain tomorrow is 0.8 then your credence in the opposite proposition, i.e. that it will not rain tomorrow, should be 0.2, not 0.1 or 0.5. According to Stephan Hartmann and Jan Sprenger, the axioms of probability can be expressed through the following two laws: (1) ${\displaystyle P(A)=1}$ for any tautology ${\displaystyle A}$; (2) For incompatible (mutually exclusive) propositions ${\displaystyle A}$ and ${\displaystyle B}$, ${\displaystyle P(A\lor B)=P(A)+P(B)}$.

Another important Bayesian principle of degrees of beliefs is the principal principle due to David Lewis. It states that our knowledge of objective probabilities should correspond to our subjective probabilities in the form of credences. So if you know that the objective chance of a coin landing heads is 50% then your credence that the coin will land heads should be 0.5.

The axioms of probability together with the principal principle determines the static or synchronic aspect of rationality: what an agent's beliefs should be like when only considering one moment. But rationality also involves a dynamic or diachronic aspect, which comes to play for changing one's credences upon being confronted with new evidence. This aspect is determined by the principle of conditionalization.

Principle of conditionalization

The principle of conditionalization governs how the agent's credence in a hypothesis should change upon receiving new evidence for or against this hypothesis. As such, it expresses the dynamic aspect of how ideal rational agents would behave. It is based on the notion of conditional probability, which is the measure of the probability that one event occurs given that another event has already occurred. The unconditional probability that ${\displaystyle A}$ will occur is usually expressed as ${\displaystyle P(A)}$ while the conditional probability that ${\displaystyle A}$ will occur given that B has already occurred is written as ${\displaystyle P(A\mid B)}$. For example, the probability of flipping a coin two times and the coin landing heads two times is only 25%. But the conditional probability of this occurring given that the coin has landed heads on the first flip is then 50%. The principle of conditionalization applies this idea to credences:  we should change our credence that the coin will land heads two times upon receiving evidence that it has already landed heads on the first flip. The probability assigned to the hypothesis before the event is called prior probability. The probability afterward is called posterior probability. According to the simple principle of conditionalization, this can be expressed in the following way: ${\displaystyle P_{\text{posterior}}(H)=P_{\text{prior}}(H\mid E)={\frac {P_{\text{prior}}(H\land E)}{P_{\text{prior}}(E)}}}$. So the posterior probability that the hypothesis is true is equal to the conditional prior probability that the hypothesis is true relative to the evidence, which is equal to the prior probability that both the hypothesis and the evidence are true, divided by the prior probability that the evidence is true. The original expression of this principle, referred to as Bayes' theorem, can be directly deduced from this formulation.

The simple principle of conditionalization makes the assumption that our credence in the acquired evidence, i.e. its posterior probability, is 1, which is unrealistic. For example, scientists sometimes need to discard previously accepted evidence upon making new discoveries, which would be impossible if the corresponding credence was 1. An alternative form of conditionalization, proposed by Richard Jeffrey, adjusts the formula to take the probability of the evidence into account: ${\displaystyle P_{\text{posterior}}(H)=P_{\text{prior}}(H\mid E)\cdot P_{\text{posterior}}(E)+P_{\text{prior}}(H\mid \lnot E)\cdot P_{\text{posterior}}(\lnot E)}$.

Dutch books

A Dutch book is a series of bets that necessarily results in a loss. An agent is vulnerable to a Dutch book if her credences violate the laws of probability. This can be either in synchronic cases, in which the conflict happens between beliefs held at the same time, or in diachronic cases, in which the agent does not respond properly to new evidence. In the most simple synchronic case, only two credences are involved: the credence in a proposition and in its negation. The laws of probability hold that these two credences together should amount to 1 since either the proposition or its negation are true. Agents who violate this law are vulnerable to a synchronic Dutch book. For example, given the proposition that it will rain tomorrow, suppose that an agent's degree of belief that it is true is 0.51 and the degree that it is false is also 0.51. In this case, the agent would be willing to accept two bets at $0.51 for the chance to win $1: one that it will rain and another that it will not rain. The two bets together cost $1.02, resulting in a loss of $0.02, no matter whether it will rain or not. The principle behind diachronic Dutch books is the same, but they are more complicated since they involve making bets before and after receiving new evidence and have to take into account that there is a loss in each case no matter how the evidence turns out to be.

There are different interpretations about what it means that an agent is vulnerable to a Dutch book. On the traditional interpretation, such a vulnerability reveals that the agent is irrational since she would willingly engage in behavior that is not in her best self-interest. One problem with this interpretation is that it assumes logical omniscience as a requirement for rationality, which is problematic especially in complicated diachronic cases. An alternative interpretation uses Dutch books as "a kind of heuristic for determining when one's degrees of belief have the potential to be pragmatically self-defeating". This interpretation is compatible with holding a more realistic view of rationality in the face of human limitations.

Dutch books are closely related to the axioms of probability. The Dutch book theorem holds that only credence assignments that do not follow the axioms of probability are vulnerable to Dutch books. The converse Dutch book theorem states that no credence assignment following these axioms is vulnerable to a Dutch book.

Applications

Confirmation theory

In the philosophy of science, confirmation refers to the relation between a piece of evidence and a hypothesis confirmed by it. Confirmation theory is the study of confirmation and disconfirmation: how scientific hypotheses are supported or refuted by evidence. Bayesian confirmation theory provides a model of confirmation based on the principle of conditionalization. A piece of evidence confirms a theory if the conditional probability of that theory relative to the evidence is higher than the unconditional probability of the theory by itself. Expressed formally: ${\displaystyle P(H\mid E)>P(H)}$. If the evidence lowers the probability of the hypothesis then it disconfirms it. Scientists are usually not just interested in whether a piece of evidence supports a theory but also in how much support it provides. There are different ways how this degree can be determined. The simplest version just measures the difference between the conditional probability of the hypothesis relative to the evidence and the unconditional probability of the hypothesis, i.e. the degree of support is ${\displaystyle P(H\mid E)-P(H)}$. The problem with measuring this degree is that it depends on how certain the theory already is prior to receiving the evidence. So if a scientist is already very certain that a theory is true then one further piece of evidence will not affect her credence much, even if the evidence would be very strong. There are other constraints for how an evidence measure should behave, for example, surprising evidence, i.e. evidence that had a low probability on its own, should provide more support. Scientists are often faced with the problem of having to decide between two competing theories. In such cases, the interest is not so much in absolute confirmation, or how much a new piece of evidence would support this or that theory, but in relative confirmation, i.e. in which theory is supported more by the new evidence.

A well-known problem in confirmation theory is Carl Gustav Hempel's raven paradox.Hempel starts by pointing out that seeing a black raven counts as evidence for the hypothesis that all ravens are black while seeing a green apple is usually not taken to be evidence for or against this hypothesis. The paradox consists in the consideration that the hypothesis "all ravens are black" is logically equivalent to the hypothesis "if something is not black, then it is not a raven". So since seeing a green apple counts as evidence for the second hypothesis, it should also count as evidence for the first one. Bayesianism allows that seeing a green apple supports the raven-hypothesis while explaining our initial intuition otherwise. This result is reached if we assume that seeing a green apple provides minimal but still positive support for the raven-hypothesis while spotting a black raven provides significantly more support.

Coherence

Coherence plays a central role in various epistemological theories, for example, in the coherence theory of truth or in the coherence theory of justification. It is often assumed that sets of beliefs are more likely to be true if they are coherent than otherwise. For example, we would be more likely to trust a detective who can connect all the pieces of evidence into a coherent story. But there is no general agreement as to how coherence is to be defined. Bayesianism has been applied to this field by suggesting precise definitions of coherence in terms of probability, which can then be employed to tackle other problems surrounding coherence. One such definition was proposed by Tomoji Shogenji, who suggests that the coherence between two beliefs is equal to the probability of their conjunction divided by the probabilities of each by itself, i.e. ${\displaystyle Coherence(A,B)={\frac {P(A\land B)}{(P(A)\cdot P(B))}}}$. Intuitively, this measures how likely it is that the two beliefs are true at the same time, compared to how likely this would be if they were neutrally related to each other. The coherence is high if the two beliefs are relevant to each other. Coherence defined this way is relative to a credence assignment. This means that two propositions may have high coherence for one agent and a low coherence for another agent due to the difference in prior probabilities of the agents' credences.

Social epistemology

Social epistemology studies the relevance of social factors for knowledge. In the field of science, for example, this is relevant since individual scientists often have to place their trust in the discoveries of other scientists in order to progress. The Bayesian approach can be applied to various topics in social epistemology. For example, probabilistic reasoning can be used in the field of testimony to evaluate how reliable a given report is. In this way, it can be formally shown that witness reports that are probabilistically independent of each other provide more support than otherwise. Another topic in social epistemology concerns the question of how to aggregate the beliefs of the individuals within a group to arrive at the belief of the group as a whole. Bayesianism approaches this problem by aggregating the probability assignments of the different individuals.

Objections

Problem of priors

In order to draw probabilistic inferences based on new evidence, it is necessary to already have a prior probability assigned to the proposition in question. But this is not always the case: there are many propositions that the agent never considered and therefore lacks a credence. This problem is usually solved by assigning a probability to the proposition in question in order to learn from the new evidence through conditionalization. The problem of priors concerns the question of how this initial assignment should be done. Subjective Bayesians hold that there are no or few constraints besides probabilistic coherence that determine how we assign the initial probabilities. The argument for this freedom in choosing the initial credence is that the credences will change as we acquire more evidence and will converge on the same value after enough steps no matter where we start. Objective Bayesians, on the other hand, assert that there are various constraints that determine the initial assignment. One important constraint is the principle of indifference. It states that the credences should be distributed equally among all the possible outcomes. For example, the agent wants to predict the color of balls drawn from an urn containing only red and black balls without any information about the ratio of red to black balls. Applied to this situation, the principle of indifference states that the agent should initially assume that the probability to draw a red ball is 50%. This is due to symmetric considerations: it is the only assignment in which the prior probabilities are invariant to a change in label. While this approach works for some cases it produces paradoxes in others. Another objection is that one should not assign prior probabilities based on initial ignorance.

Problem of logical omniscience

The norms of rationality according to the standard definitions of Bayesian epistemology assume logical omniscience: the agent has to make sure to exactly follow all the laws of probability for all her credences in order to count as rational. Whoever fails to do so is vulnerable to Dutch books and is therefore irrational. This is an unrealistic standard for human beings, as critics have pointed out.

Problem of old evidence

The problem of old evidence concerns cases in which the agent does not know at the time of acquiring a piece of evidence that it confirms a hypothesis but only learns about this supporting-relation later. Normally, the agent would increase her belief in the hypothesis after discovering this relation. But this is not allowed in Bayesian confirmation theory since conditionalization can only happen upon a change of the probability of the evidential statement, which is not the case. For example, the observation of certain anomalies in the orbit of Mercury is evidence for the theory of general relativity. But this data had been obtained before the theory was formulated, thereby counting as old evidence.