Should Ukraine trust Russia, or Trump?
What my AI strategists can teach us about Ukraine’s peace negotiations
I'm sat at the airport, watching text scroll by on my MacBook. We're about 3/4 of the way through a very large tournament between a bunch of agents, including two AI ones, that are trying to decide how far to trust one another; or whether they can get away with cheating, and so scoop a bigger prize. And while I wait, I'm scrolling the news about Ukraine on my iphone. Will there be a peace deal between Russia and Ukraine, brokered by President Trump? How far can the Ukrainians trust President Putin to keep to any deal? How far can they trust President Trump to backstop it?
The two are connected, of course. That's why strategic studies scholars have, over the decades, spent so much time working with game theory. In strategy, you need to understand your enemy, whether that's in a simplified model of bargaining, or in the messier reality of war.
My agents are playing an iterated Prisoners Dilemma. It's a great way of exploring trust. If you only play the game once, cheats can prosper. But if you play it over and over, a reputation forms; and you have to think really carefully about whether and when to cheat - or 'defect' in the lingo.
In his landmark experiments with the Iterated Prisoner's Dilemma, political scientist Robert Axelrod found that cooperation emerges most reliably when players foresee many interactions ahead—the so-called "shadow of the future." Today, as Ukraine and Russia negotiate via Washington, Axelrod's findings resonate again. Can the protagonists trust one another enough to cooperate, or will fears of betrayal doom their efforts?
Tit-for-Tat—a deceptively simple strategy of cooperating first and then mirroring your opponent's previous move—was Axelrod’s tournament champion. Its strength lay in its clarity and reliability: cooperate, but punish betrayal instantly. That captures something of the logic underpinning cooperation in the wild - 'reciprocal altruism', argued legendary biologist Robert Trivers, is why we behave altruistically, even when 'survival of the fittest' suggests we shouldn't. We sacrifice in the present for the promise of future benefits from a cooperative partner. But for that to work, there has to be a decent chance that your partner will be around to pay you back, and be inclined to do so.
Tit-for-Tat thrives when there's certainty that the game won't suddenly end. That's rarely so in the real world. When negotiations, like those in Ukraine, are overshadowed by uncertainty, mistrust breeds caution, and cautious players rarely commit fully to peace. If you think the PD games might soon end, you've extra incentive to defect and secure a better payoff than you might have if they ran on and on. If you don't think Trump is going to be heavily invested in enforcing a peace deal, why not sign up to it, then cheat?
In my current tournament, this 'shadow of the future' is short. There's a ten percent chance that the match ends in the next round. Every round. I'm playing around with this, to see how it shapes trust. What, for example, if that probability of termination is itself a probability? ie, there’s a ninety percent chance that the round ends ten percent of the time. What if the AI agents I have playing the game know all this? They can't reliably calculate when the game ends, and have only the track record of their adversary to go on when weighing what might happen next. That feels a bit more like what's going on in Ukraine today.
What am I doing here? Well, for one, I'm checking out how Axelrod's thinking stacks up, now I can replicate his studies at huge scale. But I'm mostly interested in how far AI can do strategy. I think this will be one of the critical issues of our times, as AI advances rapidly. Critics think today's language models aren't great at this sort of thing - they memorise data, and regurgitate it; or they hallucinate nonsense. I disagree, and my growing mountain of data does too. These agents aren't just talking nonsense when they explain their moves - they are trying to win. And they aren't just regurgitating training data - otherwise they might as well play tit for tat themselves, since they know it's often the strongest tactic.
But they don't - they compete under that uncertain 'shadow of the future', and they do so well. So, Gemini, one of the agents, starts out trying to figure out what its adversary, Random, is up to. But there's no logic to Random, and eventually Gemini susses that out, switching its own tactics accordingly. I'd love to run these large contests on the most sophisticated models, with their extended 'chain of thought' inference. My suspicion is they will fare even better. Very soon, as prices rapidly come down, I will do just that.*
My contention is that language itself provides some sort of 'world model' that these models exploit. The relationship between language and logic might be fuzzy; memorisation is almost certainly a factor too, regardless of reasoning. But then, all that's true for human strategists too - habit and heuristics indelibly shape their decisions. And what if I'm right? What if AI agents are indeed capable strategists? The data certainly looks that way, in this game, and in the other, richer strategy exercises I'm working on too (more on those soon).
Well, then, we will have a powerful tool to understanding human interactions, and to participate in them. That will be momentous, and perhaps dangerous. But before all that, my tournament is rolling smoothly on. GPT-4 is playing a Bayesian strategy, that seeks to learn from the pattern of prior moves. At the end of the phase, my code tots up all the scores, many thousands of them. Then it populates another phase in the tournament. This, like life, is an evolutionary game: the best strategies multiply; the worst go out of business. Back to my iphone, and thinking again about Ukraine and Russia, three years into their bloody war.
* Do you work for OpenAI? Or DeepMind? Feel free to give me free API access to these - you won’t regret it!