by Chris Meredith
(Click here for an overview of this article)
Long before humans started playing games, natural selection discovered the fundamentals of game theory and shaped animal societies according to its rules. Within species, individuals adopt alternative competing strategies with frequencies that reflect the success of each strategy. Evolutionary Stable Strategies occur when alternative competing strategies are at equilibrium. Competition within species has generated many Evolutionary Stable Strategies with colourful titles like: Bourgeois, Scrounger, Sneaky, Satellite, Transvestite, and Sex-change. However, co-operation within and between species has generated only one Evolutionary Stable Strategy., TIT FOR TAT.
The importance of TIT FOR TAT to the evolution of co-operative behaviour was discovered in a very unusual way, through a worldwide computer competition to find the winning strategy for the well known paradox 'The Prisoner's Dilemma'. In 1981 TIT FOR TAT won that competition, and ever since then it has grown in stature to where it now dominates our thinking about the evolution of co-operative behaviour in animal and human societies.
Animals behave in ways that improve their chances of survival and reproduction. Common sense tells us that this is how they should behave, and it is easy to accept that animal behaviour is a product of natural selection. Nevertheless, animal behaviour is complex and quite difficult to analyse unless it is broken down into its component strategies. A species' behaviour is best described as a collection of strategies, each one of which promotes survival and reproduction in its own way. Some strategies are employed by every member of a species, some are not.
An example of the former is the pre-hibernation behaviour of British red squirrels. At the beginning of winter all these squirrels provide for hibernation in exactly the same way because their provisioning behaviour has been naturally selected by a fixed environmental condition - the onset of winter. A strategy like this, which all the members of a species adopt, is called a single 'optimum' strategy.
Within every species many strategies cannot be classified as single 'optimum' strategies because they are not universally adopted. Instead, alternative competing strategies exist, and individuals can display one of several strategies at any given time. For example, in most animal species the males adopt more than one strategy for attracting females and more than one strategy for acquiring resources. A new type of explanation is required to explain the evolution of these alternative competing strategies. It turns out that we all know this explanation because we all play games. The first rule of playing games is, 'watch the other players.' Your best game strategy often depends upon the strategies adopted by your opponents. It should come as no surprise that natural selection discovered this rule long before humans started playing games. Here is the explanation for the evolution of alternative competing strategies within animal societies. In their quest for survival and reproduction, animals are pitted against the other members of their species in exactly the same way as game players confront each other. This point was first realised by Maynard Smith (1972) and his application of game theory to the evolution of animal strategies launched an entirely new way of analysing animal behaviour.
Hawks and doves
Evolutionary game theory is derived from the theory of games first formalised by Von Neumann and Morgenstern (1953). Their aim was to describe economic behaviour but, paradoxically, it turns out that their theory is far more successful in biology than in economics. The main reason for this is the fact that the variables in biology are less volatile than those in economics. Natural selection and population dynamics are far more predictable than human rationality and self-interest.
As a simple demonstration of evolutionary game theory consider a game where there are just two sorts of strategies. 'Hawks' always fight to injure or kill their opponents, though in the process they risk injury to themselves. 'Doves' simply display and never engage in serious rights. These two strategies are chosen to represent the two possible extremes that we may see in nature. How would evolution proceed in this particular game? Consider what would happen if all the individuals were Doves. Every contest would be between a Dove and a Dove so a truant Hawk would do very well and its genes would spread. It is clear that a Dove population is not stable because it can be invaded by a mutant adopting the Hawk strategy.
Now consider a population comprising Hawks. Every contest would be between a Hawk and a Hawk so the costs would be very high. A mutant Dove in such a population would do better than the Hawks so Dove genes would spread. In this model of Hawks and Doves natural selection would favour a mixed population of both Doves and Hawks, and the stable equilibrium would be when the average payoffs for a Hawk are equal to those for a Dove. When alternative competing strategies are at equilibrium it is described as an Evolutionary Stable State.
Evolutionary Stable States arise when the strategy options open to an individual are determined for life by factors like size, genetics or colour, etc. For example, some males might put their resources into highly competitive ability and have bright colours and weapons. This lifestyle would be 'fast and furious', the male mating at a fast rate but living for only a short time because of the high costs invoked. Other males might put their resources into survival and although they would reproduce at a slower rate this disadvantage may be offset by a longer lifespan. A particularly good example of this occurs in some figwasps. Some males of these species are wingless and put their resources into fighting; they have large heads and mandibles that can chop another figwasp in half. These males have no wings and remain inside the fig where they hatch and fight to mate with newly hatched females that develop from larvae in that fig. Other males of the same species are winged and put their resources into dispersal; they have tiny heads and mandibles, are not aggressive and fly off to mate with females that have emerged from the fruit.
The difference between fighting and dispersing males is a genetic difference, each maintained at an equilibrium frequency determined by its average relative pay-off. Sometimes dispersing will do better than fighting and vice versa. For example, if figs contain only a few emerging females, dispersal will pay. If, on the other hand, many individuals are born in the same fruit, a male would have access to many females and it would pay to stay home and fight.
A similar example occurs in the bee Centris pallida (Alcock et al, 1977). In this species small body size is fixed throughout life as a consequence of poor feeding conditions when young. Large males search for females by patrolling over the ground, searching for buried virgin females about to emerge. When they discover an emerging female they dig her up and copulate. It takes several minutes to dig up a female, during which time the other males are attracted to the site by the activity. There are often violent fights and only large males are able to defend their discoveries successfully. It is not surprising, therefore, that only large males adopt the strategy of patrolling and digging. Small males search for mates by hovering above the emergence areas and pursuing airborne females who have escaped the diggers. Observations have shown that large males have the greatest mating success so it is probable that the smaller males are forced to adopt hovering throughout their lives to make the best of a bad job.
In the Centris pallida Evolutionary Stable State we can see the beginning of choice, even if it is a forced choice. The small males would have been diggers but their small size forced them into hovering for a living. Intermediate males do make a choice, digging sometimes and hovering others. They assess their chances and behave as a digger or hoverer with frequencies that reflect their success with each strategy. Perhaps a more accurate way of describing the Evolutionary Stable State of the bee Centris pallida is to say that the males have adopted a strategy that is - 'If small, hover; if large, patrol and dig'. This is a better way of describing the males' mating behaviour because it introduces choice based on an individual's assessment of a situation. When alternative competing strategies occur, most of the time assessment and choice are involved. By giving Centris pallida one strategy with choice rather than two discrete strategies, the real situation is more accurately modelled. A strategy like this which is stable and has choice built into it is called an Evolutionary Stable Strategy. It differs from evolutionary stable states and single optimum strategies because it has choice, and natural selection has determined the frequency with which each behaviour (choice) occurs within the strategy.
The concept of Evolutionary Stable Strategies (ESS) relies upon the notion that animals can assess the relative pay-offs for the different roles within a strategy. For example, in our earlier Hawk-Dove game imagine that each individual could assess whether to play Hawk or Dove. The frequency of Hawk behaviour would remain at 66.6% and that of Dove at 33.3%, but individuals would exert a choice, playing each role when they thought appropriate. This is a Hawk-Dove Evolutionary Stable Strategy (ESS) - as opposed to the Hawk-Dove Evolutionary Stable State described earlier - because role choice based on assessment is now involved.
A strategy very similar to a Hawk-Dove ESS is widespread in nature. It is called 'Bourgeois' and operates on the rule - 'play Hawk if you are an owner and Dove if you are an intruder'. Such a Bourgeois strategy is an ESS and cannot be invaded by Hawk or Dove strategies because Bourgeois individuals avoid more damaging encounters than the pure Hawks and win more encounters than pure Doves.
One example from Nature involves male Hamadryas baboon contests over females (Kummer et al, 1974). In the wild, a male Hamadryas baboon forms a long-lasting relationship with several females. Kummer showed that if male A was permitted to form a bond with a strange female, then a second male B, who had watched the interaction, will not subsequently challenge A for ownership. If, on a later occasion, male B forms a bond with a female he will not be challenged by A. Escalated fights do occur between two males if each perceives himself as the owner of the same female. It seems clear that for Male Hamadryas baboons, ownership, and not any perceived difference in size or strength, is decisive in settling contests.
A similar Bourgeois strategy exists amongst male lions where clear ownership of an oestrus female by a consorting male will prevent other males from challenging him. It makes good sense for lions to avoid fighting amongst themselves.
There are many examples of ESSs where individuals select their strategy after assessing their opponents. Male dung flies, Scafophaga stercoraria have to decide whether to stay or go as a cowpat grows stale. They hang around cowpats and attempt to mate with females as they arrive but progressively fewer females arrive as the cowpat grows stale. If other males stay it pays to go and vice versa. Parker (1978) found that male stay times are distributed so as to give the same reproductive success to the males adopting the different strategies. Staying-Going was an ESS for male dung flies.
Quite a large number of ESSs fit a model called the Producer-Scrounger model. A producer invests time and energy in guarding or creating some resource, which scroungers parasitise. The ESS point occurs when producer and scrounger fitness are equal.
Many animal behaviours in nature fit the Producer-Scrounger model. Successful male red deer control harems of about five hinds. Excluded males hang around and wait for an opportunity to break up harems. If the antlers of two harem owners become locked together in a rutting contest, then a motley collection of geriatric, and Juvenile 'sneaky rutters' move in (Clifton-Brock et al, 1979).
Excluded young male elephant seals are a bit more subtle than excluded red deer. At four years of age they are too small to compete for harem ownership, being only about the size of a female, but they use this to their advantage and sneak into harems as 'pretend' females. When his lordship is otherwise engaged in some titanic struggle, they quickly throw away their disguise, expose their growing tusks and pounce upon a not entirely unsuspecting female. The females tolerate these juveniles but protest loudly during copulation. They seem to be using such incursions to keep their bull seal on the tips of his flippers (Le Boeuf, 1974).
So far we have only considered animal competition and have ignored situations where sharing a resource might benefit 'contestants'. For example, territorial battles are over a divisible resource and it is quite obvious that sometimes two 'contestants' could benefit in fitness by sharing a space rather than by engaging in an escalated contest. This does happen and male lions share territories and females, as do some baboons, chimpanzees, and many other primates. Compromise is a reality, particularly if the costs of escalated fighting are high. This is not meant to imply that animals who share resources or engage in bargaining are more intelligent than those who never bargain. The only suggestion is that natural selection has favoured Bargaining ESS for some species and not for others. In its simplest form a Bargaining ESS consists of two roles - to agree to a bargain or not to agree to a bargain. As always, the ratio of 'bargain' to 'no bargain' is a measure of the success of each role in a Bargaining ESS. Of course the model could be complicated by dividing bargainers into honest-bargainers and bluff-bargainers, but there is no data from nature that I know of to support this view. Human bargaining certainly does have a lot of bluff about it, but it seems that bargaining between animals has to be done honestly. Bluff-bargaining may pose too great a threat to the central notion of bargaining which is to avoid the costs of conflict.
A striking example of honest trading or bargaining has been described (Hazlett, 1980). In hermit crabs there is good evidence that empty shells are a limiting resource.
An individual may find itself in a shell which is either too large or too small. A large crab in a small shell and a small crab in a big shell can both benefit by exchange, and such exchanges do in fact take place. One crab will initiate an exchange by tapping or shaking the shell of another in a manner that is characteristic of the species. The non-initiating crab may stay inside its shell, or it may come out of its shell after first tapping the initiator on its shell. If it comes out of its shell an exchange of shells takes place. When an exchange of shells would leave a non-initiating crab in a shell further from its preferred size, then no exchange takes place. Neither the size nor the sex of the initiating crab influences the likelihood of an exchange, so it seems that an exchange requires mutual benefit, and cannot be enforced by the initiator.
Exchanges take place between members of different species, provided the initiator has in its repertoire a signal appropriate to the other species.
Since different species of crab prefer different types of shell, interspecific exchanges afford opportunities of mutual benefit additional to those arising in intraspecific interactions.
The presence of honesty in this bargaining system is easy to explain because there is no special disadvantage associated with failure to agree. It pays each crab to acquire accurate information about whether an exchange would benefit it.
From what we know, animal bargaining is honest and based on complete information. In contrast, human bargaining, if not entirely dishonest, is at best based on incomplete information. Consider the following imaginary example of wage bargaining. The management would prefer to give no rise at all, but would pay 10% rather than face a strike. The union would like as big a rise as possible, but would be willing to settle for 5% rather than strike. Clearly a settlement would be welcomed by both sides at some point between 5% to 10%. The union, however, does not know that the management will go to 10% and the management does not know that the union would settle for 5%. Furthermore, it would not pay the union to announce right away that it would settle for 5% because, if it did, that is all it would get. This then is a game of incomplete information; each side knows something that the other does not.
What eventuates in these situations are 'delicate negotiations' where the players employ a varied set of signals designed to explore the other side's position with a view to achieving a compromise. It is, in effect, a cloak and dagger game of give and take played very seriously by both sides. It is a compromise not true co-operation, however bargaining does employ a strategy that led to the evolution of co-operation.
You scratch my back ...
Evolutionary biologists have had considerable trouble explaining the evolution of co-operative behaviour. The problem is that co-operation can always be exploited by selfish individuals who cheat. It seems that natural selection should always favour the cheats over the co-operators. Co-operation involves doing and receiving favours and this means that the opportunity to cheat and not return a favour is a very real possibility. Trivers (1971) tackled this problem and developed the theory of reciprocal altruism based on the idea that co-operation could evolve in species clever enough to discriminate between co-operators and cheats. The concept is summarised in the saying 'you scratch my back and I'll scratch yours'. Trivers' theory of reciprocal altruism is particularly successful in explaining human behaviour because reciprocal altruism is a major part of all human activities.
As a first means of eliciting reciprocity we use displays of generosity, gratitude, sympathy and sincerity. These 'guarantors' of reciprocity typically operate at the family, friend, and local community levels. If they fail to generate appropriate reciprocity we employ moralistic aggression in the form of sermons and lectures designed to bully all the cheats back into line. Moralistic aggression is the number one weapon of religions around the world. The strength and weakness of religions lies in their promise of 'reciprocation after death'. The sky is offered but how can we tell if it is true" Religions have found that moralistic aggression of the hell-fire-and-damnation variety is needed to calm such doubts and keep the flow of altruism coming their way.
As a final act of determination to keep us all reciprocating, we punish the non-reciprocators with fines, gaol, torture and death. So it really is true that crimes against property are more serious than crimes against people. They threaten our reciprocal altruism adaptation far more than the odd killing or rape ever will. Societies founded on reciprocal altruism fear cheats more than anything else, even although we only seem to be able to catch the stupid ones.
Trivers' theory of reciprocal altruism was an important advance in our understanding of the evolution of co-operation but it was a 'special theory' rather than a 'general theory'. The discovery of how co-operative behaviour could evolve in species far less intelligent than humans, came in a surprising way - from a detailed study of the well known paradox 'The Prisoner's Dilemma'.
The prisoner's dilemma
The prisoner's dilemma refers to an imaginary situation in which two individuals are imprisoned and are accused of having co-operated to perform some crime. The two prisoners are held separately, and attempts are made to induce each one to implicate the other. If neither one does, both are set free. This is the co-operative strategy available to both prisoners. In order to tempt one or both to defect, each is told that a confession implicating the other will lead to his or her release and, as an added incentive, to a small reward. If both confess, each one is imprisoned. But if one individual implicated the other and not vice versa, then the implicated partner receives a harsher sentence than if each had implicated the other.
The prisoner's dilemma is that if they both think rationally then each one will decide that the best course of action is to implicate the other even although they would both be better off trusting each other. Consider how one prisoner thinks. If his partner fails to implicate him then he should implicate his partner and get the best possible pay-off. If his partner has implicated him he should still 'cheat' - since he suffers less than if he trusts his partner. However, the situation is more complicated than this analysis suggests. It is fairly obvious that the players' strategic decisions will also depend upon their likelihood of future encounters. If they know that they are destined never to meet again, defection is the only rational choice. Both individuals will cheat and both will end up relatively badly-off. But if the prisoner's dilemma is repeated a number of times, then it may be advantageous to co-operate on the early moves and cheat only towards the end of the game. When people know the total number of games of prisoner's dilemma, they do indeed cheat more often in the final games.
Robert Axelrod was interested in finding a winning strategy for repeated prisoner's dilemmas games. He conducted a computer tournament where people were invited to submit strategies for playing 200 games of prisoner's dilemma (Axlerod and Hamilton, 1981). Fourteen game theorists in disciplines such as economics and mathematics submitted entries. These 14, and a totally random strategy, were paired with each other in a round robin tournament. Some of these strategies were highly intricate. But the result of the tournament was that the simplest of all strategies submitted attained the highest average score. This strategy, called TIT FOR TAT by its submitter Anatol Rapoport, had only two rules. On the first move co-operate. ON each succeeding move do what your opponent did the previous move. Thus, TIT FOR TAT was a strategy of co-operation based on reciprocity. By conceptualising reciprocal altruism as a series of prisoner's dilemmas we can see that TIT FOR TAT might be the Evolutionary Stable Strategy for our reciprocal altruism adaptation. It might even help to explain the evolution of co-operation in a more general way than Trivers' theory of reciprocal altruism.
TIT FOR TAT
The results of Axelrod's tournament were published and people were invited to submit programs for a second tournament. This was identical in form to the first, except that matches were not of exactly 200 games, but were of a random length with median 200; this avoided the complication of programs that might have special cheating rules for the last game. This time there were 62 entries from six countries. Most of the contestants were computer hobbyists but also present were professors of evolutionary biology, physics and computer science as well as the disciplines represented earlier. Rapoport again submitted TIT FOR TAT and again it won with a leg in the air. Ultimately it displaced all other strategies and became the equivalent of an ESS for prisoner's dilemma.
From an analysis of the 3-million choices made in the second competition, four features of TIT FOR TAT emerged:
1. Never be the first to defect
2. Retaliate only after your partner has defected
3. Be prepared to forgive after carrying out just one act of retaliation
4. Adopt this strategy only if the probability of meeting the same player again exceeds 2/3.
These results provide a model for the evolution of co-operative behaviour. At first sight it might seem that the model is relevant only to higher animals which can distinguish between their various opponents. If so, TIT FOR TAT would simply be Trivers' theory of reciprocal altruism restated. But TIT FOR TAT is more than this and can be applied to animals that cannot recognise each other - as long as each individual starts co-operative encounters with very minor, low cost moves and gradually escalates as reciprocation occurs.
Axelrod and Hamilton emphasise that a formal theory for the evolution of co-operation needs to answer three questions.
1. How can a co-operative strategy get an initial foothold in an environment which is predominantly non-co-operative?
2. What type of strategy can thrive in a varied environment composed of other individuals using a wide diversity of more or less sophisticated strategies?
3. Under what conditions can such a strategy, once fully established, resist invasion by mutant strategies (such as cheating)?
The studies of TIT FOR TAT answer these questions about initial viability, robustness and stability. Provided that the probability of future interaction between two individuals is sufficiently great, co-operation based on reciprocity can indeed get started in an asocial world, can flourish in a variegated environment and can defend itself once fully established.
According to Axelrod, TIT FOR TAT is a successful ESS because it is 'nice', 'provokable' and 'forgiving'. A nice strategy is one which is never first to defect. In a match between two nice strategies, both do well. A provokable strategy responds by defecting at once in response to defection. A forgiving strategy is one which readily returns to co-operation if its opponent does so; unforgiving strategies are likely to produce isolation and end co-operative encounters.
Since the appearance of TIT FOR TAT as a model for the evolution of co-operation, there have been many strategies derived from it: TIT FOR TWO TATS, SUSPICIOUS TIT FOR TAT and ALWAYS DEFECT to name just three. Under varying conditions all achieve some success but none demonstrate the robustness of TIT FOR TAT. However the real proof of this theory is in nature where TIT FOR TAT is beginning to be identified.
Vervet monkeys and olive baboons are more likely to go to the assistance of an animal that has helped them in the past (Packer, 1977). One might expect vervets and baboons to play TIT FOR TAT as they live in complex societies with many chances to interact. Of course they are also intelligent enough to distinguish between reciprocation and cheats, so, if Axelrod and Hamilton are correct, we would expect TIT FOR TAT to evolve as their co-operative strategy. This is what appears to have happened.
Recent evidence, however, suggests that much less sociable and much less intelligent animals may also play TIT FOR TAT. Tree swallows, Tachycineta bicolour, live in groups, but not all the birds in a group are parents. The non-breeders hang around and may kill young birds then usurp a nest. Parents, however, generally do not chase off non-breeders, which suggests that some mutual restraint is present. Non-breeders, by staying near the colony, learn the characteristics of a good nest site. Parents benefit by having extra birds around to challenge predators. So the elements of the Prisoner's Dilemma are there; both types of bird gain if they show restraint - the non-breeders gain information and, perhaps, the nesting site while the breeders produce extra young.
Michael Lombardo (1985) changed things around experimentally. He made it seem as if the non-breeders had defected by putting two stuffed nestlings in the nest in place of the live young. The parents attacked the non-breeders as if they were the culprits but stopped their attacks and resumed their co-operative relationship as soon as the live nestlings were returned to the nest. Many other helper-parent relationships are now being re-analysed for TIT FOR TAT.
Possibly the most beautiful empirical test of the TIT FOR TAT model comes from Manfred Milinski's laboratory experiments with stickleback fish (Miliniski, 1987). His experiment was based upon the observation that, during the early stages of an attack by a stalking pike, some minnows or three spined sticklebacks leave their shoal to approach within 4-6 body lengths of the predator, for what has been called a 'predator inspection visit'. In the wild, sticklebacks often approach a stalking predator, probably to identify it accurately and gauge its readiness to attack. If the little fish do so together they can get closer to the predator and, should it attack, they might be better protected by being in a group and confusing the predator. Two fish engaged in such an inspection behaviour can be regarded as co-operating if they either stay close together or take turns in leading the advance towards the predator. If one fish consistently lags behind, it may be regarded as a defector (gaining the advantages of inspection with less accompanying risk). There is, therefore, a series of choices to be made. Each time one fish swims closer, the companion can co-operate and go along with it, or defect. If it defects, it runs less risk of being eaten itself, and it may gain more information than the 'sucker' as it watches its fate.
Milinski gave sticklebacks, Gasterosteus acofeatus, the chance to alter their behaviour according to that of a companion. He put a stickleback in a tank from which it could see a large predatory cicid - a fish that resembles the perch, a common predator of sticklebacks. Also in the tank was a mirror angled either to be a co-operating mirror or a defecting mirror.
When the co-operating mirror was in place a stickleback had the illusion of a co-operating companion, but with a defecting mirror the companion lagged behind and eventually disappeared.
In this experiment, those fish with a co-operating mirror went closer to the ciclid and stayed there longer than the fish with a defecting mirror. Milinski observes that the sticklebacks acted as if they perceived that a companion was either following them or staying increasingly behind. Other aspects of TIT FOR TAT seem to be fulfilled too. The fish often forgave its cowardly companion image, approaching the ciclid again and again. This is because at first the companion moves forward too, irrespective of which mirror is in place. It eventually defects if the defecting mirror is in place, but since its first move was co-operative, it is forgiven for its previous defections - exactly what the theory of TIT FOR TAT predicted would happen.
It is beginning to appear that the strategy of TIT FOR TAT is very bit as robust in real life as it is in computer competitions. Laboratory tests of TIT FOR TAT have become a growth industry as the theory gains in stature. We can expect new revelations about its worth as a theory to explain the evolution of co-operative behaviour. But whatever the outcome of this debate, one fact remains unchallenged. TIT FOR TAT is a major regulator of human behaviour. It may be a Culturally Stable Strategy (CSS) - one that humans just learned as a way of regulating our co-operative behaviour - or it may indeed be a very necessary, naturally selected co-operative Evolutionary Stable Strategy.
© 1998 Australian Broadcasting Corporation