Evolutionary game theory (EGT) is the application of game theory to evolving populations in biology. It defines a framework of contests, strategies, and analytics into which Darwinian competition can be modelled. It originated in 1973 with John Maynard Smith and George R. Price's
formalisation of contests, analysed as strategies, and the mathematical
criteria that can be used to predict the results of competing
strategies.
Evolutionary game theory differs from classical game theory in focusing more on the dynamics of strategy change. This is influenced by the frequency of the competing strategies in the population.
Evolutionary game theory has helped to explain the basis of altruistic behaviours in Darwinian evolution. It has in turn become of interest to economists, sociologists, anthropologists, and philosophers.
Evolutionary game theory differs from classical game theory in focusing more on the dynamics of strategy change. This is influenced by the frequency of the competing strategies in the population.
Evolutionary game theory has helped to explain the basis of altruistic behaviours in Darwinian evolution. It has in turn become of interest to economists, sociologists, anthropologists, and philosophers.
History
Classical game theory
Classical non-cooperative game theory was conceived by John von Neumann
to determine optimal strategies in competitions between adversaries. A
contest involves players, all of whom have a choice of moves. Games can
be a single round or repetitive. The approach a player takes in making
his moves constitutes his strategy. Rules govern the outcome for the
moves taken by the players, and outcomes produce payoffs for the
players; rules and resulting payoffs can be expressed as decision trees or in a payoff matrix.
Classical theory requires the players to make rational choices. Each
player must consider the strategic analysis that his opponents are
making to make his own choice of moves.
The problem of ritualized behaviour
The mathematical biologist John Maynard Smith modelled evolutionary games.
Evolutionary game theory started with the problem of how to explain
ritualized animal behaviour in a conflict situation; "why are animals so
'gentlemanly or ladylike' in contests for resources?" The leading ethologists Niko Tinbergen and Konrad Lorenz proposed that such behaviour exists for the benefit of the species. John Maynard Smith considered that incompatible with Darwinian thought,
where selection occurs at an individual level, so self-interest is
rewarded while seeking the common good is not. Maynard Smith, a
mathematical biologist, turned to game theory as suggested by George
Price, though Richard Lewontin's attempts to use the theory had failed.
Adapting game theory to evolutionary games
Maynard
Smith realised that an evolutionary version of game theory does not
require players to act rationally —– only that they have a strategy. The
results of a game shows how good that strategy was, just as evolution
tests alternative strategies for the ability to survive and reproduce.
In biology, strategies are genetically inherited traits that control an
individual's action, analogous with computer programs. The success of a
strategy is determined by how good the strategy is in the presence of
competing strategies (including itself), and of the frequency with which
those strategies are used. Maynard Smith described his work in his book Evolution and the Theory of Games.
Participants aim to produce as many replicas of themselves as
they can, and the payoff is in units of fitness (relative worth in being
able to reproduce). It is always a multi-player game with many
competitors. Rules include replicator dynamics, in other words how the
fitter players will spawn more replicas of themselves into the
population and how the less fit will be culled, in a replicator equation.
The replicator dynamics models heredity but not mutation, and assumes
asexual reproduction for the sake of simplicity. Games are run
repetitively with no terminating conditions. Results include the
dynamics of changes in the population, the success of strategies, and
any equilibrium states reached. Unlike in classical game theory, players
do not choose their strategy and cannot change it: they are born with a
strategy and their offspring inherit that same strategy.
Evolutionary games
Models
Evolutionary game theory analyses Darwinian mechanisms with a system model with three main components – Population, Game, and Replicator Dynamics. The system process has four phases:
1) The model (as evolution itself) deals with a Population (Pn). The population will exhibit Variation among Competing individuals. In the model this competition is represented by the Game.
2) The Game tests the strategies of the individuals under the “rules of the game”. These rules produce different payoffs – in units of Fitness (the production rate of offspring). The contesting individuals meet in pairwise contests with others, normally in a highly mixed distribution of the population. The mix of strategies in the population affects the payoff results by altering the odds that any individual may meet up in contests with various strategies. The individuals leave the game pairwise contest with a resulting fitness determined by the contest outcome, represented in a Payoff Matrix.
3) Based on this resulting fitness each member of the population then undergoes replication or culling determined by the exact mathematics of the Replicator Dynamics Process. This overall process then produces a New Generation P(n+1). Each surviving individual now has a new fitness level determined by the game result.
4) The new generation then takes the place of the previous one and the cycle repeats. The population mix may converge to an Evolutionarily Stable State that cannot be invaded by any mutant strategy.
1) The model (as evolution itself) deals with a Population (Pn). The population will exhibit Variation among Competing individuals. In the model this competition is represented by the Game.
2) The Game tests the strategies of the individuals under the “rules of the game”. These rules produce different payoffs – in units of Fitness (the production rate of offspring). The contesting individuals meet in pairwise contests with others, normally in a highly mixed distribution of the population. The mix of strategies in the population affects the payoff results by altering the odds that any individual may meet up in contests with various strategies. The individuals leave the game pairwise contest with a resulting fitness determined by the contest outcome, represented in a Payoff Matrix.
3) Based on this resulting fitness each member of the population then undergoes replication or culling determined by the exact mathematics of the Replicator Dynamics Process. This overall process then produces a New Generation P(n+1). Each surviving individual now has a new fitness level determined by the game result.
4) The new generation then takes the place of the previous one and the cycle repeats. The population mix may converge to an Evolutionarily Stable State that cannot be invaded by any mutant strategy.
Evolutionary game theory encompasses Darwinian evolution, including
competition (the game), natural selection (replicator dynamics), and
heredity. Evolutionary game theory has contributed to the understanding
of group selection, sexual selection, altruism, parental care, co-evolution, and ecological
dynamics. Many counter-intuitive situations in these areas have been
put on a firm mathematical footing by the use of these models.
The common way to study the evolutionary dynamics in games is through replicator equations.
These show the growth rate of the proportion of organisms using a
certain strategy and that rate is equal to the difference between the
average payoff of that strategy and the average payoff of the population
as a whole. Continuous replicator equations assume infinite populations, continuous time, complete mixing and that strategies breed true. The attractors (stable fixed points) of the equations are equivalent with evolutionarily stable states.
A strategy which can survive all "mutant" strategies is considered
evolutionarily stable. In the context of animal behavior, this usually
means such strategies are programmed and heavily influenced by genetics, thus making any player or organism's strategy determined by these biological factors.
Evolutionary games are mathematical objects with different rules,
payoffs, and mathematical behaviours. Each "game" represents different
problems that organisms have to deal with, and the strategies they might
adopt to survive and reproduce. Evolutionary games are often given
colourful names and cover stories which describe the general situation
of a particular game. Representative games include hawk-dove, war of attrition, stag hunt, producer-scrounger, tragedy of the commons, and prisoner's dilemma.
Strategies for these games include Hawk, Dove, Bourgeois, Prober,
Defector, Assessor, and Retaliator. The various strategies compete under
the particular game's rules, and the mathematics are used to determine
the results and behaviours.
Hawk Dove
Solution of the Hawk Dove
game for V=2, C=10 and fitness starting base B=4. The fitness of a Hawk
for different population mixes is plotted as a black line, that of Dove
in red. An ESS (a stationary point) will exist when Hawk and Dove
fitness are equal: Hawks are 20% of population and Doves are 80% of the
population.
The first game that Maynard Smith analysed is the classic Hawk Dove
game. It was conceived to analyse Lorenz and Tinbergen's problem, a
contest over a shareable resource. The contestants can be either Hawk
or Dove. These are two subtypes or morphs of one species with different
strategies. The Hawk first displays aggression, then escalates into a
fight until it either wins or is injured (loses). The Dove first
displays aggression, but if faced with major escalation runs for safety.
If not faced with such escalation, the Dove attempts to share the
resource.
| meets Hawk | meets Dove | |
| if Hawk | V/2 − C/2 | V |
| if Dove | 0 | V/2 |
Given that the resource is given the value V, the damage from losing a fight is given cost C:
- If a Hawk meets a Dove he gets the full resource V to himself
- If a Hawk meets a Hawk – half the time he wins, half the time he loses...so his average outcome is then V/2 minus C/2
- If a Dove meets a Hawk he will back off and get nothing – 0
- If a Dove meets a Dove both share the resource and get V/2
The actual payoff however depends on the probability of meeting a
Hawk or Dove, which in turn is a representation of the percentage of
Hawks and Doves in the population when a particular contest takes place.
That in turn is determined by the results of all of the previous
contests. If the cost of losing C is greater than the value of winning V
(the normal situation in the natural world) the mathematics ends in an
ESS, a mix of the two strategies where the population of Hawks is V/C.
The population regresses to this equilibrium point if any new Hawks or
Doves make a temporary perturbation in the population.
The solution of the Hawk Dove Game explains why most animal contests
involve only ritual fighting behaviours in contests rather than outright
battles. The result does not at all depend on good of the species behaviours as suggested by Lorenz, but solely on the implication of actions of so-called selfish genes.
War of attrition
In the Hawk Dove game the resource is
shareable, which gives payoffs
to both Doves meeting in a pairwise contest. Where the resource is not
shareable, but an alternative resource might be available by backing off
and trying elsewhere, pure Hawk or Dove strategies are less effective.
If an unshareable resource is combined with a high cost of losing a
contest (injury or possible death) both Hawk and Dove payoffs are
further diminished. A safer strategy of lower cost display, bluffing
and waiting to win, is then viable – a Bluffer strategy. The game then
becomes one of accumulating costs, either the costs of displaying or the
costs of prolonged unresolved engagement. It is effectively an auction;
the winner is the contestant who will swallow the greater cost while
the loser gets the same cost as the winner but no resource. The
resulting evolutionary game theory mathematics leads to an optimal
strategy of timed bluffing.
War of attrition
for different values of resource. Note the time it takes for an
accumulation of 50% of the contestants to quit vs. the Value(V) of
resource contested for.
This is because in the war of attrition
any strategy that is
unwavering and predictable is unstable, because it will ultimately be
displaced by a mutant strategy which relies on the fact that it can best
the existing predictable strategy by investing an extra small delta of
waiting resource to ensure that it wins. Therefore, only a random
unpredictable strategy can maintain itself in a population of Bluffers.
The contestants in effect choose an acceptable cost to be incurred
related to the value of the resource being sought, effectively making a
random bid as part of a mixed strategy (a strategy where a contestant
has several, or even many, possible actions in his strategy). This
implements a distribution of bids for a resource of specific value V,
where the bid for any specific contest is chosen at random from that
distribution. The distribution (an ESS) can be computed using the
Bishop-Cannings theorem, which holds true for any mixed strategy ESS.
The distribution function in these contests was determined by Parker and
Thompson to be:
The result is that the cumulative population of quitters for any particular cost m in this "mixed strategy" solution is:
as shown in the adjacent graph. The intuitive sense that greater
values of resource sought leads to greater waiting times is borne out.
This is observed in nature, as in male dung flies contesting for mating
sites, where the timing of disengagement in contests is as predicted by
evolutionary theory mathematics.
Asymmetries that allow new strategies
Dung Fly (Scatophaga stercoraria) – a War of Attrition player
-APiazza.JPG)
The mantis shrimp guarding its home with the Bourgeois Strategy
Animal
Strategy Examples: by examining the behaviours, then determining both
the Costs and the Value of resources attained in a contest the strategy
of an organism can be verified
In the War of Attrition there must be nothing that signals the size
of a bid to an opponent, otherwise the opponent can use the cue in an
effective counter-strategy. There is however a mutant strategy which
can better a Bluffer in the War of Attrition
Game if a suitable asymmetry exists, the Bourgeois strategy. Bourgeois
uses an asymmetry of some sort to break the deadlock. In nature one such
asymmetry is possession of a resource. The strategy is to play a Hawk
if in possession of the resource, but to display then retreat if not in
possession. This requires greater cognitive capability than Hawk, but
Bourgeois is common in many animal contests, such as in contests among mantis shrimps and among speckled wood butterflies.
Social behaviour
Alternatives for game theoretic social interaction
Games like Hawk Dove and War of Attrition represent pure competition
between individuals and have no attendant social elements. Where social
influences apply, competitors have four possible alternatives for
strategic interaction. This is shown on the adjacent figure, where a
plus sign represents a benefit and a minus sign represents a cost.
- In a Cooperative or Mutualistic relationship both "donor" and "recipient" are almost indistinguishable as both gain a benefit in the game by co-operating, i.e. the pair are in a game-wise situation where both can gain by executing a certain strategy, or alternatively both must act in concert because of some encompassing constraints that effectively puts them "in the same boat".
- In an Altruistic relationship the donor, at a cost to himself provides a benefit to the recipient. In the general case the recipient will have a kin relationship to the donor and the donation is one-way. Behaviours where benefits are donated alternatively (in both directions) at a cost, are often called altruistic, but on analysis such "altruism" can be seen to arise from optimised "selfish" strategies
- Spite is essentially a "reversed" form of altruism where an ally is aided by damaging the ally's competitor(s). The general case is that the ally is kin related and the benefit is an easier competitive environment for the ally. Note: George Price, one of the early mathematical modellers of both altruism and spite, found this equivalence particularly disturbing at an emotional level.
- Selfishness is the base criteria of all strategic choice from
a game theory perspective – strategies not aimed at self-survival and
self-replication are not long for any game. Critically however, this
situation is impacted by the fact that competition is taking place on
multiple levels – i.e. at a genetic, an individual and a group level.
Contests of selfish genes
Female Belding's ground squirrels
risk their lives giving loud alarm calls, protecting closely related
female colony members; males are less closely related and do not call.
At first glance it may appear that the contestants of evolutionary
games are the individuals present in each generation who directly
participate in the game. But individuals live only through one game
cycle, and instead it is the strategies that really contest with one
another over the duration of these many-generation games. So it is
ultimately genes that play out a full contest – selfish genes of
strategy. The contesting genes are present in an individual and to a
degree in all of the individual's kin. This can sometimes profoundly
affect which strategies survive, especially with issues of cooperation
and defection. William Hamilton, known for his theory of kin selection,
explored many of these cases using game theoretic models. Kin related
treatment of game contests helps to explain many aspects of the
behaviour of social insects, the altruistic behaviour in parent/offspring interactions, mutual protection behaviours, and co-operative care of offspring. For such games Hamilton defined an extended form of fitness – inclusive fitness, which includes an individual's offspring as well as any offspring equivalents found in kin.
| The Mathematics of Kin Selection |
|---|
The concept of Kin Selection is that:
The inclusive fitness of an individual wi is the sum of its specific fitness of itself ai plus the specific fitness of each and every relative weighted by the degree of relatedness which equates to the summation of all rj*bj....... where rj is relatedness of a specific relative and bj is that specific relative's fitness – producing:
|
Hamilton went beyond kin relatedness to work with Robert Axelrod, analysing games of co-operation under conditions not involving kin where reciprocal altruism comes into play.
Eusociality and kin selection
Meat ant
workers (always female) are related To mother or father=0.5 To
sister+=0.75 To own daughter or son=0.5 To brother=0.25.......
Therefore, it is more advantageous to help produce a sister than to have
a child oneself.
Eusocial
insect workers forfeit reproductive rights to their queen. It has been
suggested that Kin Selection, based on the genetic makeup of these
workers, may predispose them to altruistic behaviour. Most eusocial insect societies have haplodiploid sexual determination, which means that workers are unusually closely related.
This explanation of insect eusociality has however been
challenged by a few highly noted evolutionary game theorists (Nowak and
Wilson)
who have published a controversial alternative game theoretic
explanation based on a sequential development and group selection
effects proposed for these insect species.
Prisoner's dilemma
A difficulty of the theory of evolution, recognised by Darwin himself, was the problem of altruism.
If the basis for selection is at individual level, altruism makes no
sense at all. But universal selection at the group level (for the good
of the species, not the individual) fails to pass the test of the
mathematics of game theory and is certainly not the general case in
nature.
Yet in many social animals, altruistic behaviour exists. The solution
to this paradox can be found in the application of evolutionary game
theory to the prisoner's dilemma
game – a game which tests the payoffs of cooperating or in defecting
from cooperation. It is certainly the most studied game in all of game
theory.
The analysis of prisoner's dilemma is as a repetitive game. This
affords competitors the possibility of retaliating for defection in
previous rounds of the game. Many strategies have been tested; the best
competitive strategies are general cooperation with a reserved
retaliatory response if necessary. The most famous and one of the most successful of these is tit-for-tat with a simple algorithm.
procedure tit-for-tat
EventBit:=Trust;
do while Contest=ON;
if Eventbit=Trust then
Cooperate
else
Defect;
if Opponent_Move=Cooperate then
EventBit:=Trust
else
Eventbit:=NOT(Trust);
end;
The pay-off for any single round of the game is defined by the
pay-off matrix for a single round game (shown in bar chart 1 below). In
multi-round games the different choices – Co-operate or Defect – can be
made in any particular round, resulting in a certain round payoff. It
is, however, the possible accumulated pay-offs over the multiple rounds
that count in shaping the overall pay-offs for differing multi-round
strategies such as Tit-for-Tat.
Payoffs in two varieties of prisoner's dilemma game.
Prisoner's dilemma: Co-operate or Defect?
Prisoner's dilemma: Co-operate or Defect?
Payoff (Temptation in Defecting vs. Co-operation) > Payoff (Mutual Co-operation) > Payoff(Joint Defection) > Payoff(Sucker co-operates but opponent defects)
Example 1: The straightforward single round prisoner's dilemma game.
The classic prisoner's dilemma game payoffs gives a player a maximum
payoff if he defect and his partner co-operates (this choice is known as
temptation). If however the player co-operates and his partner defects,
he gets the worst possible result (the suckers payoff). In these payoff
conditions the best choice (a Nash equilibrium) is to defect.
Example 2: Prisoner's dilemma played repeatedly. The strategy
employed is Tit-for-Tat which alters behaviors based on the action taken
by a partner in the previous round – i.e. reward co-operation and
punish defection. The effect of this strategy in accumulated payoff
over many rounds is to produce a higher payoff for both players
co-operation and a lower payoff for defection. This removes the
Temptation to defect. The suckers payoff also becomes less, although
"invasion" by a pure defection strategy is not entirely eliminated.
Routes to altruism
Altruism
takes place when one individual, at a cost C to itself, exercises a
strategy that provides a benefit B to another individual. The cost may
consist of a loss of capability or resource which helps in the battle
for survival and reproduction, or an added risk to its own survival.
Altruism strategies can arise through:
| Type | Applies to: | Situation | Mathematical effect |
|---|---|---|---|
| Kin Selection – (inclusive fitness of related contestants) | Kin – genetically related individuals | Evolutionary Game participants are genes of strategy. The best payoff for an individual is not necessarily the best payoff for the gene. In any generation the player gene is NOT only in one individual, it is in a Kin-Group. The highest fitness payoff for the Kin Group is selected by natural selection. Therefore, strategies that include self-sacrifice on the part of individuals are often game winners – the evolutionarily stable strategy. Animals must live in kin-group during part of the game for the opportunity for this altruistic sacrifice ever to take place. | Games must take into account Inclusive Fitness. Fitness function is
the combined fitness of a group of related contestants – each weighted
by the degree of relatedness – relative to the total genetic population.
The mathematical analysis of this gene centric view of the game leads
to Hamilton's rule, that the relatedness of the altruistic donor must
exceed the cost-benefit ratio of the altruistic act itself:
|
| Direct reciprocity | Contestants that trade favours in paired relationships | A game theoretic embodiment of "I'll scratch your back if you scratch mine". A pair of individuals exchange favours in a multi-round game. The individuals are recognisable to one another as partnered. The term "direct" applies because the return favour is specifically given back to the pair partner only. | The characteristics of the multi-round game produce a danger of defection and the potentially lesser payoffs of cooperation in each round, but any such defection can lead to punishment in a following round – establishing the game as repeated prisoner's dilemma. Therefore, the family of tit-for-tat strategies come to the fore. |
| Indirect Reciprocity | Related or non related contestants trade favours but without partnering. A return favour is "implied" but with no specific identified source who is to give it. | This behaviour is akin to "I'll scratch your back, you scratch
someone else's back, another someone else will scratch mine (probably)".
The return favour is not derived from any particular established
partner. The potential for indirect reciprocity exists for a specific
organism if it lives in a cluster of individuals who can interact over
an extended period of time.
It has been argued that human behaviours in establishing moral system
as well as the expending of significant energies in human society for
tracking individual reputation is a direct effect of societies reliance
on strategies of indirect reciprocation.
|
The game is highly susceptible to defection, as direct retaliation
is impossible. Therefore, indirect reciprocity will not work without
keeping a social score, a measure of past co-operative behaviour. The
mathematics leads to a modified version of Hamilton's Rule where:
|
The evolutionarily stable strategy
The
Payoff Matrix for the Hawk Dove Game with the addition of the Assessor
Strategy. This "studies its opponent", behaving as a Hawk when matched
with an opponent it judges "weaker", like a Dove when the opponent seems
bigger and stronger. Assessor is an ESS, since it can invade both Hawk
and Dove populations, and can withstand invasion by either Hawk or Dove
mutants.
The evolutionarily stable strategy (ESS) is akin to Nash equilibrium
in classical game theory, but with mathematically extended criteria.
Nash Equilibrium is a game equilibrium where it is not rational for any
player to deviate from his present strategy, provided that the others
adhere to their strategies. An ESS is a state of game dynamics where, in
a very large population of competitors, another mutant strategy cannot
successfully enter the population to disturb the existing dynamic (which
itself depends on the population mix). Therefore, a successful strategy
(with an ESS) must be both effective against competitors when it is
rare – to enter the previous competing population, and successful when
later in high proportion in the population – to defend itself. This in
turn means that the strategy must be successful when it contends with
others exactly like itself.
An ESS is not:
- An optimal strategy: that would maximize Fitness, and many ESS states are far below the maximum fitness achievable in a fitness landscape. (see Hawk Dove graph above as an example of this)
- A singular solution: often several ESS conditions can exist in a competitive situation. A particular contest might stabilize into any one of these possibilities, but later a major perturbation in conditions can move the solution into one of the alternative ESS states.
- Always present: it is possible for there to be no ESS. An evolutionary game with no ESS is Rock-Scissors-Paper, as found in species such as the side-blotched lizard (Uta stansburiana).
- An unbeatable strategy: the ESS is only an uninvadeable strategy.
Female
funnel web spiders (Agelenopsis aperta) contest with one another for
the possession of their desert spider webs using the Assessor strategy.
The ESS state can be solved for by exploring either the dynamics of
population change to determine an ESS, or by solving equations for the
stable stationary point conditions which define an ESS.
For example, in the Hawk Dove Game we can look for whether there is a
static population mix condition where the fitness of Doves will be
exactly the same as fitness of Hawks (therefore both having equivalent
growth rates – a static point).
Let the chance of meeting a Hawk=p so therefore the chance of meeting a dove is (1-p)
Let WHawk equal the Payoff for Hawk.....
WHawk=Payoff in the chance of meeting a Dove + Payoff in the chance of meeting a Hawk
Taking the PAYOFF MATRIX results and plugging them into the above equation:
WHawk= V·(1-p)+(V/2-C/2)·p
Similarly for a Dove:
WDove= V/2·(1-p)+0·(p)
so....
WDove= V/2·(1-p)
Equating the two fitnesses, Hawk and Dove
V·(1-p)+(V/2-C/2)·p= V/2·(1-p)
... and solving for p
p= V/C
so for this "static point" where the Population Percent is an ESS solves to be ESS(percent Hawk)=V/C
Similarly, using inequalities, it can be shown that an additional
Hawk or Dove mutant entering this ESS state eventually results in less
fitness for their kind – both a true Nash and an ESS equilibrium. This
example shows that when the risks of contest injury or death (the Cost
C) is significantly greater than the potential reward (the benefit value
V), the stable population will be mixed between aggressors and doves,
and the proportion of doves will exceed that of the aggressors. This
explains behaviours observed in nature.
Unstable games, cyclic patterns
Rock paper scissors
Rock Paper Scissors
Mutant Invasion for Rock Paper Scissors payoff matrix – an endless cycle
An evolutionary game that turns out to be a children's game is rock paper scissors.
The game is simple – rock beats scissors (blunts it), scissors beats
paper (cuts it), and paper beats rock (wraps it up). Anyone who has
ever played this simple game knows that it is not sensible to have any
favoured play – the opponent will soon notice this and switch to the
winning counter-play. The best strategy (a Nash equilibrium) is to play
a mixed random game with any of the three plays taken a third of the
time. This, in evolutionary game theory terms, is a mixed strategy.
But many lifeforms are incapable of mixed behavior – they only exhibit
one strategy (known as a pure strategy). If the game is played only with
the pure Rock, Paper and Scissors strategies the evolutionary game is
dynamically unstable:
Rock mutants can enter an all scissor population, but then –
Paper mutants can take over an all Rock population, but then –
Scissor mutants can take over an all Paper population – and on and
on....
This is easily seen on the game payoff matrix, where if the paths of
mutant invasion are noted, it can be seen that the mutant "invasion
paths" form into a loop. This in triggers a cyclic invasion pattern.
A computer simulation of the Rock Scissors Paper game. The associated RPS Game Payoff Matrix
is shown. Starting with an arbitrary population the percentage of the
three morphs builds up into a continuously cycling pattern.
Rock paper scissors incorporated into an evolutionary game has been used for modelling natural processes in the study of ecology.
Using experimental economics
methods, scientists have used RPS game to test human social
evolutionary dynamical behaviors in laboratory. The social cyclic
behaviors, predicted by evolutionary game theory, have been observed in
various laboratory experiments.
The side-blotched lizard
The side-blotched lizard (Uta stansburiana) is polymorphic with three morphs that each pursues a different mating strategy
The side-blotched lizard effectively uses a rock-paper-scissors mating strategy.
- The orange throat is very aggressive and operates over a large territory – attempting to mate with numerous females within this larger area
- The unaggressive yellow throat mimics the markings and behavior of female lizards, and "sneakily" slips into the orange throat's territory to mate with the females there (thereby taking over the population), and
- The blue throat mates with and carefully guards one female – making it impossible for the sneakers to succeed and therefore overtakes their place in a population…
However the blue throats cannot overcome the more aggressive orange
throats. The overall situation corresponds to the Rock, Scissors, Paper
game, creating a six-year population cycle. When he read that these
lizards were essentially engaged in a game with rock-paper-scissors
structure, John Maynard Smith is said to have exclaimed "They have read
my book!"
Signalling, sexual selection and the handicap principle
The peacock's tail may be an instance of the handicap principle in action.
Aside from the difficulty of explaining how altruism exists in many
evolved organisms, Darwin was also bothered by a second conundrum – why
do a significant number of species have phenotypical attributes that are
patently disadvantageous to them with respect to their survival – and
should by the process of natural section be selected against – e.g. the
massive inconvenient feather structure found in a peacock's tail?
Regarding this issue Darwin wrote to a colleague "The sight of a feather
in a peacock's tail, whenever I gaze at it, makes me sick."
It is the mathematics of evolutionary game theory, which has not only
explained the existence of altruism but also explains the totally
counterintuitive existence of the peacock's tail and other such
biological encumbrances.
On analysis, problems of biological life are not at all unlike
the problems that define economics – eating (akin to resource
acquisition and management), survival (competitive strategy) and
reproduction (investment, risk and return). Game theory was originally
conceived as a mathematical analysis of economic processes and indeed
this is why it has proven so useful in explaining so many biological
behaviours. One important further refinement of the evolutionary game
theory model that has economic overtones rests on the analysis of COSTS.
A simple model of cost assumes that all competitors suffer the same
penalty imposed by the Game costs, but this is not the case. More
successful players will be endowed with or will have accumulated a
higher "wealth reserve" or "affordability" than less successful players.
This wealth effect in evolutionary game theory is represented
mathematically by "resource holding potential
(RHP)" and shows that the effective cost to a competitor with higher
RHP are not as great as for a competitor with a lower RHP. As a higher
RHP individual is more desirable mate in producing potentially
successful offspring, it is only logical that with sexual selection RHP
should have evolved to be signalled in some way by the competing rivals,
and for this to work this signalling must be done honestly. Amotz Zahavi has developed this thinking in what is known as the handicap principle,
where superior competitors signal their superiority by a costly
display. As higher RHP individuals can properly afford such a costly
display this signalling is inherently honest, and can be taken as such
by the signal receiver. Nowhere in nature is this better illustrated
than in the magnificent and costly plumage of the peacock. The mathematical proof of the handicap principle was developed by Alan Grafen using evolutionary game-theoretic modelling.
Coevolution
Two types of dynamics have been discussed so far in this article:
- Evolutionary games which lead to a stable situation or point of stasis for contending strategies which result in an evolutionarily stable strategy
- Evolutionary games which exhibit a cyclic behaviour (as with RPS game) where the proportions of contending strategies continuously cycle over time within the overall population
Competitive Coevolution
- The rough-skinned newt (Tarricha granulosa) is highly toxic, due to an evolutionary arms race with a predator, the common garter snake (Thamnophis sirtalis), which in turn is highly tolerant of the poison. The two are locked in a Red Queen arms race.
Mutualistic Coevolution
- Darwin's orchid (Angraecum sesquipedale) and the moth Morgan's sphinx (Xanthopan morgani) have a mutual relationship where the moth gains pollen and the flower is pollinated.
A third, coevolutionary,
dynamic combines intra-specific and inter-specific competition.
Examples include predator-prey competition and host-parasite
co-evolution, as well as mutualism. Evolutionary game models have been
created for pairwise and multi-species coevolutionary systems. The
general dynamic differs between competitive systems and mutualistic
systems.
In competitive (non-mutualistic) inter-species coevolutionary
system the species are involved in an arms race – where adaptations that
are better at competing against the other species tend to be preserved.
Both game payoffs and replicator dynamics reflect this. This leads to a
Red Queen dynamic where the protagonists must "run as fast as they can to just stay in one place".
A number of evolutionary game theory models have been produced to
encompass coevolutionary situations. A key factor applicable in these
coevolutionary systems is the continuous adaptation of strategy in such
arms races. Coevolutionary modelling therefore often includes genetic algorithms
to reflect mutational effects, while computers simulate the dynamics of
the overall coevolutionary game. The resulting dynamics are studied
as various parameters are modified. Because several variables are
simultaneously at play, solutions become the province of multi-variable
optimisation. The mathematical criteria of determining stable points are
Pareto efficiency and Pareto dominance, a measure of solution optimality peaks in multivariable systems.
Carl Bergstrom and Michael Lachmann apply evolutionary game theory to the division of benefits in mutualistic
interactions between organisms. Darwinian assumptions about fitness are
modeled using replicator dynamics to show that the organism evolving at
a slower rate in a mutualistic relationship gains a disproportionately
high share of the benefits or payoffs.
Extending the model
A mathematical model
analysing the behaviour of a system needs initially to be as simple as
possible to aid in developing a base understanding the fundamentals, or
“first order effects”, pertaining to what is being studied. With this
understanding in place it is then appropriate to see if other, more
subtle, parameters (second order effects) further impact the primary
behaviours or shape additional behaviours in the system. Following
Maynard Smith's seminal work in evolutionary game theory, the subject
has had a number of very significant extensions which have shed more
light on understanding evolutionary dynamics, particularly in the area
of altruistic behaviors. Some of these key extensions to evolutionary
game theory are:
A Spatial Game
In a spatial evolutionary game contestants meet in contests at fixed grid positions and only interact with immediate neighbors. Shown here are the dynamics of a Hawk Dove contest, showing Hawk and Dove contestants as well as the changes of strategy taking place in the various cells
In a spatial evolutionary game contestants meet in contests at fixed grid positions and only interact with immediate neighbors. Shown here are the dynamics of a Hawk Dove contest, showing Hawk and Dove contestants as well as the changes of strategy taking place in the various cells
Spatial Games
Geographic factors in evolution include gene flow and horizontal gene transfer.
Spatial game models represent geometry by putting contestants in a
lattice of cells: contests take place only with immediate neighbours.
Winning strategies take over these immediate neighbourhoods and then
interact with adjacent neighbourhoods. This model is useful in showing
how pockets of co-operators can invade and introduce altruism in the
Prisoners Dilemma game,
where Tit for Tat (TFT) is a Nash Equilibrium but NOT also an ESS.
Spatial structure is sometimes abstracted into a general network of
interactions. This is the foundation of evolutionary graph theory.
Effects of having information
In evolutionary game theory as in conventional Game Theory
the effect of Signalling (the acquisition of information) is of
critical importance, as in Indirect Reciprocity in Prisoners Dilemma
(where contests between the SAME paired individuals are NOT repetitive).
This models the reality of most normal social interactions which are
non-kin related. Unless a probability measure of reputation is available
in Prisoners Dilemma only direct reciprocity can be achieved. With this information indirect reciprocity is also supported.
Alternatively, agents might have access to an arbitrary signal
initially uncorrelated to strategy but becomes correlated due to
evolutionary dynamics. This is the green-beard effect
or evolution of ethnocentrism in humans. Depending on the game, it can
allow the evolution of either cooperation or irrational hostility.
From molecular to multicellular level, a signaling game
model with information asymmetry between sender and receiver might be
appropriate, such as in mate attraction or evolution of translation
machinery from RNA strings.
Finite populations
Many
evolutionary games have been modelled in finite populations to see the
effect this may have, for example in the success of mixed strategies.
