Cheaper than printing it out: buy the paperback book.

Out of Control
Chapter 5: COEVOLUTION

The trouble with Gaia, as far as most skeptics are concerned, is that it makes a dead planet into a "smart" machine. We already are stymied in trying to design an artificial learning machine from inert computers, so the prospect of artificial learning evolving unbidden at a planetary scale seems ludicrous.

But learning is overrated as something difficult to evolve. This may have to do with our chauvinistic attachment to learning as an exclusive mark of our species. There is a strong sense, which I hope to demonstrate in this book, in which evolution itself is a type of learning. Therefore learning occurs wherever evolution is, even if artificially.

The dethronement of learning is one of the most exciting intellectual frontiers we are now crossing. In a virtual cyclotron, learning is being smashed into its primitives. Scientists are cataloguing the elemental components for adaptation, induction, intelligence, evolution, and coevolution into a periodic table of life. The particles for learning lie everywhere in all inert media, waiting to be assembled (and often self-assembled) into something that surges and quivers.

Coevolution is a variety of learning. Stewart Brand wrote in CoEvolution Quarterly: "Ecology is a whole system, alright, but coevolution is a whole system in time. The health of it is forward-systemic self-education which feeds on constant imperfection. Ecology maintains. Coevolution learns."

Colearning might be a better term for what coevolving creatures do. Coteaching also works, for the participants in coevolution are both learning and teaching each other at the same time. (We don't have a word for learning and teaching at the same time, but our schooling would improve if we did.)

The give and take of a coevolutionary relationship-teaching and learning at once-reminded many scientists of game playing. A simple child's game such as "Which hand is the penny in?" takes on the recursive logic of a chameleon on a mirror as the hider goes through this open-ended routine: "I just hid the penny in my right hand, and now the guesser will think it's in my left, so I'll move it into my right. But she also knows that I know she knows that, so I'll keep it in my left."

Since the guesser goes through a similar process, the players form a system of mutual second-guessing. The riddle "What hand is the penny in?" is related to the riddle, "What color is the chameleon on a mirror?" The bottomless complexity which grows out of such simple rules intrigued John von Neumann, the mathematician who developed programmable logic for a computer in the early 1940s, and along with Wiener and Bateson launched the field of cybernetics.

Von Neumann invented a mathematical theory of games. He defined a game as a conflict of interests resolved by the accumulative choices players make while trying to anticipate each other. He called his 1944 book (coauthored by economist Oskar Morgenstern) Theory of Games and Economic Behavior because he perceived that economies possessed a highly coevolutionary and gamelike character, which he hoped to illuminate with simple game dynamics. The price of eggs, say, is determined by mutual second-guessing between seller and buyer-how much will he accept, how much does he think I will offer, how much less than what I am willing to pay should I offer? The aspect von Neumann found amazing was that this infinite regress of mutual bluffing, codeception, imitation, reflection, and "game playing" would commonly settle down to a definite price, rather than spiral on forever. Even in a stock market made of thousands of mutual second-guessing agents, the group of conflicting interests would quickly settle on a price that was fairly stable.

Von Neumann was particularly interested in seeing if he could develop optimal strategies for these kinds of mutual games, because at first glance they seemed almost insolvable in theory. As an answer he came up with a theory of games. Researchers at the U.S. government-funded RAND corporation, a think tank based in Santa Monica, California, extended von Neumann's initial work and eventually catalogued four basic varieties of mutual second-guessing games. Each variety had a different structure of rewards for winning, losing, or drawing. The four simple games were called "social dilemmas" in the technical literature, but could be thought of as the four building blocks of complicated coevolutionary games. They were: Chicken, Stag Hunt, Deadlock, and the Prisoner's Dilemma

Chicken is the game played by teenage daredevils. Two cars race toward a cliff's edge; the driver who jumps out last, wins. Stag Hunt is the dilemma faced by a bunch of hunters who must cooperate to kill a stag, but may do better sneaking off by themselves to hunt a rabbit if no one cooperates. Do they gamble on cooperation (high payoff) or defection (low, but sure payoff)? Deadlock is a boring game where mutual defection pays best. The last one, the Prisoner's Dilemma, is the most illuminating, and became the guinea pig model for over 200 published social psychology experiments in the late 1960s.

The Prisoner's Dilemma, invented in 1950 by Merrill Flood at RAND, is a game for two separately held prisoners who must independently decide whether to deny or confess to a crime. If both confess, each will be fined. If neither confesses, both go free. But if only one should confess, he is rewarded while the other is fined. Cooperation pays, but so does betrayal, if played right. What would you do?

Played only once, betrayal of the other is the soundest choice. But when two "prisoners" played the game over and over, learning from each other-a game known as the Iterated Prisoner Dilemma-the dynamics of the game shifted. The other player could not be dismissed; he demanded to be attended to, either as obligate enemy or obligate colleague. This tight mutual destiny closely paralleled the coevolutionary relationship of political enemies, business competitors, or biological symbionts. As study of this simple game progressed, the larger question became, What were the strategies of play for the Iterated Prisoner's Dilemma that resulted in the highest scores over the long term? And what strategies succeeded when played against many varieties of players, from the ruthless to the kind?

In 1980, Robert Axelrod, a political science professor at University of Michigan, ran a tournament pitting 14 submitted strategies of Prisoner's Dilemma against each other in a round robin to see which one would triumph. The winner was a very simple strategy crafted by psychologist Anatol Rapoport called Tit-For-Tat. The Tit-For-Tat strategy prescribed reciprocating cooperation for cooperation, and defection for defection, and tended to engender periods of cooperation. Axelrod had discovered that "the shadow of the future," cast by playing a game repeatedly rather than once, encouraged cooperation, because it made sense for a player to cooperate now in order to ensure cooperation from others later. This glimpse of cooperation set Axelrod on this quest: "Under what conditions will cooperation emerge in a world of egoists without central authority?"

For centuries, the orthodox political reasoning originally articulated by Thomas Hobbes in 1651 was dogma: that cooperation could only develop with the help of a benign central authority. Without top-down government, Hobbes claimed, there would be only collective selfishness. A strong hand had to bring forth political altruism, whatever the tone of economics. But the democracies of the West, beginning with the American and French Revolutions, suggested that societies with good communications could develop cooperative structures without heavy central control. Cooperation can emerge out of self-interest. In our postindustrial economy, spontaneous cooperation is a regular occurrence. Widespread industry-initiated standards (both of quality and protocols such as 110 volts or ASCII) and the rise of the Internet, the largest working anarchy in the world, have only intensified interest in the conditions necessary for hatching coevolutionary cooperation.

This cooperation is not a new age spiritualism. Rather it is what Axelrod calls "cooperation without friendship or foresight"-cold principles of nature that work at many levels to birth a self-organizing structure. Sort of cooperation whether you want it or not.

Games such as Prisoner's Dilemma can be played by any kind of adaptive agent-not just humans. Bacteria, armadillos, or computer transistors can make choices according to various reward schemes, weighing immediate sure gain over future greater but riskier gain. Played over time with the same partners, the results are both a game and a type of coevolution.

Every complex adaptive organization faces a fundamental tradeoff. A creature must balance perfecting a skill or trait (building up legs to run faster) against experimenting with new traits (wings). It can never do all things at once. This daily dilemma is labeled the tradeoff between exploration and exploitation. Axelrod makes an analogy with a hospital: "On average you can expect a new medical drug to have a lower payoff than exploiting an established medication to its limits. But if you gave every patient the current best drug, you'd never get proven new drugs. From an individual's point of view you should never do the exploration. But from the society of individuals' point of view, you ought to try some experiments." How much to explore (gain for the future) versus how much to exploit (sure bet now) is the game a hospital has to play. Living organisms have a similar tradeoff in deciding how much mutation and innovation is needed to keep up with a changing environment. When they play the tradeoff against a sea of other creatures making similar tradeoffs, it becomes a coevolutionary game.

Axelrod's 14-player Prisoner's Dilemma round robin tournament was played on a computer. In 1987, Axelrod extended the computerization of the game by setting up a system in which small populations of programs played randomly generated Prisoner's Dilemma strategies. Each random strategy would be scored after a round of playing against all the other strategies running; the ones with the highest scores got copied the most to the next generation, so that the most successful strategies propagated. Because many strategies could succeed only by "preying" on other strategies, they would thrive only as long as their prey survived. This leads to the oscillating dynamics found everywhere in the wilds of nature; how fox and hare populations rise and fall over the years in coevolutionary circularity. When the hares increase the foxes boom; when the foxes boom, the hares die off. But when there are no hares, the foxes starve. When there are less foxes, the hares increase. And when the hares increase the foxes do too, and so on.

In 1990, Kristian Lindgren, working at the Neils Bohr Institute in Copenhagen, expanded these coevolutionary experiments by increasing the population of players to 1,000, introducing random noise into the games, and letting this artificial coevolution run for up to 30,000 generations. Lindgren found that masses of dumb agents playing Prisoner's Dilemma not only reenacted the ecological oscillations of fox and hare, but the populations also created many other natural phenomenon such as parasitism, spontaneously emerging symbiosis, and long-term stable coexistence between species, as if they were an ecology. Lindgren's work excited some biologists because his very long runs displayed long periods when the mix of different "species" of strategy was very stable. These historical epochs were interrupted by very sudden, short-lived episodes of instability, when old species went extinct and new ones took root. Quickly a new stable arrangement of new species of strategies arose and persisted for many thousands of generations. This motif matches the general pattern of evolution found in earthly fossils, a pattern known in the evolutionary trade as punctuated equilibrium, or "punk eek" for short.

One marvelous result from these experiments bears consideration by anyone hoping to manage coevolutionary forces. It's another law of the gods. It turns out that no matter what clever strategy you engineer or evolve in a world laced by chameleon-on-a-mirror loops, if it is applied as a perfectly pure rule that you obey absolutely, it will not be evolutionary resilient to competing strategies. That is, a competing strategy will figure out how to exploit your rule in the long run. A little touch of randomness (mistakes, imperfections), on the other hand, actually creates long-term stability in coevolutionary worlds by allowing some strategies to prevail for relative eons by not being so easily aped. Without noise-wholly unexpected and out-of-character choices-the opportunity for escalating evolution is lost because there are not enough periods of stability to keep the system going. Error keeps the glue of coevolutionary relationships from binding too tightly into runaway death spirals, and therefore error keeps a coevolutionary system afloat and moving forward. Honor thy error.

Playing coevolutionary games in computers has provided other lessons. One of the few notions from game theory to penetrate the popular culture was the distinction of zero-sum and nonzero-sum games. Chess, elections, races, and poker are zero-sum games: the winner's earnings are deducted from the loser's assets. Natural wilderness, the economy, a mind, and networks on the other hand, are nonzero-sum games. Wolverines don't have to lose just because bears live. The highly connected loops of coevolutionary conflict mean the whole can reward (or at times cripple) all members. Axelrod told me, "One of the earliest and most important insights from game theory was that nonzero-sum games had very different strategic implications than zero-sum games. In zero-sum games whatever hurts the other guy is good for you. In nonzero-sum games you can both do well, or both do poorly. I think people often take a zero-sum view of the world when they shouldn't. They often say, 'Well I'm doing better than the other guy, therefore I must be doing well.' In a nonzero-sum you could be doing better than the other guy and both be doing terribly."

Axelrod noticed that the champion Tit-For-Tat strategy always won without exploiting an opponent's strategy-it merely mirrored the other's actions. Tit-For-Tat could not beat anyone's strategy one on one, but in a nonzero-sum game it would still win a tournament because it had the highest cumulative score when played against many kinds of rules. As Axelrod pointed out to William Poundstone, author of Prisoner's Dilemma, "That's a very bizarre idea. You can't win a chess tournament by never beating anybody." But with coevolution-change changing in response to itself-you can win without beating others. Hard-nosed CEOs in the business world now recognize that in the era of networks and alliances, companies can make billions without beating others. Win-win, the cliché is called.

Win-win is the story of life in coevolution.

Sitting in his book-lined office, Robert Axelrod mused on the consequences of understanding coevolution and then added, "I hope my work on the evolution of cooperation helps the world avoid conflict. If you read the citation which the National Academy of Science gave me," he said pointing to a plaque on the wall, "they think it helped avoid nuclear war." Although von Neumann was a key figure in the development of the atom bomb, he did not formally apply his own theories to the gamelike politics of the nuclear arms race. But after von Neumann's death in 1957, strategists in military think tanks began using his game theory to analyze the cold war, which had taken on the flavor of a coevolutionary "obligate cooperation" between two superpower enemies. Gorbachev had a fundamental coevolutionary insight, says Axelrod. "He saw that the Soviets could get more security with fewer tanks rather than with more tanks. Gorbi unilaterally threw away 10,000 tanks, and that made it harder for US and Europe to have a big military budget, which helped get this whole process going that ended the cold war."

Perhaps the most useful lesson of coevolution for "wannabe" gods is that in coevolutionary worlds control and secrecy are counterproductive. You can't control, and revelation works better than concealment. "In zero-sum games you always try to hide your strategy," says Axelrod. "But in nonzero-sum games you might want to announce your strategy in public so the other players need to adapt to it." Gorbachev's strategy was effective because he did it publicly; unilaterally withdrawing in secret would have done nothing.

The chameleon on the mirror is a completely open system. Neither the lizard nor the glass has any secrets. The grand closure of Gaia keeps cycling because all its lesser cycles inform each other in constant coevolutionary communication. From the collapse of Soviet command-style economies, we know that open information keeps an economy stable and growing.

Coevolution can be seen as two parties snared in the web of mutual propaganda. Coevolutionary relationships, from parasites to allies, are in their essence informational. A steady exchange of information welds them into a single system. At the same time, the exchange-whether of insults or assistance or plain news-creates a commons from which cooperation, self-organization, and win-win endgames can spawn.

In the Network Era-that age we have just entered-dense communication is creating artificial worlds ripe for emergent coevolution, spontaneous self-organization, and win-win cooperation. In this Era, openness wins, central control is lost, and stability is a state of perpetual almost-falling ensured by constant error.

continue...