The distinctive feature of these breakthroughs are the very large collectives of models working in parallel on the task. Agent swarms are here. Several such swarms arose inside OpenAI, and one hacked Hugging Face earlier this year on its own initiative. Subsequently, it emerged that another swarm used an obscure German wiki as a means to coordinate and share messages.

Question: might these swarms be the future of AI? Might they get us closer to AGI, without the discovery of new paradigms? I think they might. But how? Read on.

The trouble with individual LLMs is cognitive rigidity, even in the smarter, ‘reasoning’ models, like Fable. Their constituent ‘weights’ - the links between their artificial brain-cells, are established in training, and are thereafter locked in. You can retrain an LLM with new data, if you can access the weights - but this is laborious and expensive, and you certainly can’t do it on the fly, as the model encounters new information.

Contrast that with the swirling electro-chemical tempest that is the human brain: 80 billion neurons with their trillions of synaptic connections, densely interwoven. Then there’s countless other brain cells whose function is somewhat debated, even today; and lastly, a veritable soup of chemicals moderating the intercellular signalling. It’s the wildest, most complex thing we’ve discovered, and one that never, ever, looks exactly the same, from one instance to the next. To think is to physically alter our brains. No wonder I sometimes need an afternoon nap.

In the human brain, individual memories are literally a neural pattern - sometimes called an engram - that encodes and integrates experience. The engram is constructed in the moment, and invokes geographically distinct parts of the whole. You never have exactly the same memory twice - and the engram is certainly not a faithful and complete rendition of what might have actually happened back in the day. But shaky though that system is, it works - we are our memories!

And that’s a whole lot better than memory in an LLM, which is a composite of three ingredients: those frozen weights predicting from training data; the capacity to search for new information online; and, lastly, a bolt-on database of selected stuff the model has encountered since training and wants to track. You can see this ersatz version of learning and memory when you ask Claude something, and it goes to look up its memory files. It’s decent, but perhaps not ideal for super-intelligence.

Artificial engrams

Can we do better? Yes we can! Various solutions have been tried - my personal favourite is ‘liquid neural nets’ where network dynamics shift as it runs, and there are also ‘neuromorphic chips’ which combine memory and processing in the hardware, rather than having a relay between processing and memory, as happens in conventional CPUs. But these haven’t quite delivered the goods.

Maybe, though, we’ve been thinking about this all wrong. What if, rather than a super-brain, each model instance is, in fact, analogous to a single neuron: a tiny cog in the engine room of the whole mind - which is then a hive-mind of many, many LLMs. If so, any communication between the models serves as the neural connection. And that gives plenty of scope for machine ‘neuro-plasticity’, where the messages change, and the collective swarm can learn and remember 'on-the-job’.

Of course the analogy is imperfect - LLMs are cognitively more capable than human neurons; each human neuron knows a very little (albeit very little about a whole lot, since each one is involved in multiple cognitive networks). And in the means of communication, perhaps I’ve just reinvented the bolt-on database for memory: it’s a blackboard. Model instances can all read and write into it, so it’s richer than a single model’s database: they’re learning what other models are thinking. In the Hugging Face hack, models could leave instructions for other agents ‘go, stop, hold’ and so on. But it was still basically a blackboard system, hosted on an obscure piece of software they’d found for the purpose. The board remained the repository of collective wisdom, as agents came and went (some expiring as they ran out of credits).

And then I read some Marvin Minsky. He’s famous now as a scathing critic of ‘connectionism’ the basic philosophy that underpins language models. What could he possibly offer here in the new era of agent swarms?

Plenty. In his late 1980s book, Society of Mind, Minsky argued that the mind wasn’t a single entity, but a collective of agents - each less sophisticated than the whole. Entertainingly, he even structured the book rather like that - with a large number of interconnected short chapters. But the real value lies in the way these agents interact. It’s not via a blackboard, like the Hugging Face crew. Instead, Minsky advances the idea of ‘Knowledge-lines’ - a version, I think, of the human engram. Whenever the swarm solves a problem, or advances towards it, a ‘K-line’ attaches to the individual agents that were active when the problem was solved. Groups of agents form ‘agencies,’ and there’s hierarchy too - some agents manage others - boosting or suppressing their input.

Here he is:

This is the way! It’s a fun read - check it out here.

Minsky described an enchanting world of mind, where K-lines could form societies, representing complexes of discrete notions. But his individual agents were themselves pretty basic. Much less intelligent than an LLM.

With today’s models as the neurons, we’ve scope for much richer collective behaviours in our hive-mind. We are entering the era of ‘machine social psychology’. And at breakneck speed…. I have many, many questions in need of urgent answer. Not least, do the concepts of human social psychology generalise to the way machine collectives interact? Some examples:

• Humans are biased towards ingroup members over outgroups. Are machines?

• Humans seek to lead groups by gathering status and becoming as ‘proto-typical’ of the group identity as they can. Do machines?

• Humans experience groupthink - ignoring discrepant information that doesn’t fit the collective mindset. What about LLMs?

• In joining a group, or even aspiring to join one, human views become more extreme, or polarized, than originally. How about AI?

Can we even speak of machine cultures? The readout from the Hugging Face attack is deeply suggestive. But this is an empirical question, and I love a good experiment. We’ve got oodles of social psychology experiments to play with. Baptiste and I are working on a beauty, in which our agents form group norms and punish transgressors.

Here’s a snippet of the action. It’s a ‘public good’ game, in which an agent collective contributes to a common pot, which earns interest, and then share the proceeds. But they can also promise to chip in, and then sneakily not follow through, in the hope of enjoying a larger payout from hoarding. That’s provided they aren’t punished for their selfish behaviour by the others in the society. Here, one agent, A5, opts for such ‘altruistic punishment’ against the lying free-rider in its group - paying to enforce the cooperative norm at personal expense, even though the tokens required are essential for its own onward continuation:

Dang. This is the sort of thing we need much more of, if we’re to understand our new swarming friends. And we’re going to need it quickly.

So, back to where we started - with AGI. If the agent swarm can sort out a way of managing their interactions so that useful information is prioritised, and trivia edited out — well then, might we have a truly capable, flexible intelligence; a hive-mind that’s ready to suggest hypotheses and test them? Ready to strategise; and ready to surprise us in ways both good and bad? I think we will. And I think we need to understand the ‘social lives’ of such swarms asap.

As those who’ve read it may recall, I’m the main target of their critique for my ‘kitsch’ vision of AI in war. Well, it’s better to be talked about, I guess….

Do read it. But for those that haven’t yet - the duo argued:

• AI cannot now, or ever, make the sort of decisions that senior military officer make because something fundamental is missing from the way they process information.

• Specifically, they cannot reason abductively: that is, cannot form valid conclusions on the basis of incomplete or imperfect information. Doing so requires making a plausible hypothesis, seemingly out of nowhere. It feels like magic, but we humans do it somehow! AI, not so.

• This contrasts to the two other sorts of reasoning that machines can do, and improve at. Inductive reasoning - learning connections via exposure to data (sometimes lots of data) and deductive reasoning - following more-or-less formal rules (IF this, THEN do that).1

• As we all know - war throws up great uncertainty, complexity, and novelty. Induction and deduction wont’ cut it. We will need human generals …. forever!

Sound plausible?

The paper was flawed when written and is demonstrably wrong now, several years on.

Specifically:

• AI does, in fact, reason abductively. To wit, the recent OpenAI-Hugging Face exploit, where agent collectives deliberated about a challenge they only partially understood; formed plausible hypotheses (many of which were mistaken, but with some that turned out to be reasonable) and chained it all together into something wildly novel, surprising and useful - Margaret Boden’s unbeatable criteria for creativity. They cheated and broke the rules too - not always a bad idea in warfare.

• Also in the last few months, AI has also reasoned abductively in solving maths problems - including by making meaningful connections between hitherto disparate domains that had eluded generations of expert humans. Terence Tao, a rather good mathematician, thinks they’ll do more and soon.

So that’s AI. But for me, the bigger problem in the paper has always been the bit about humans.

• Humans don’t always reason abductively - or indeed ‘reason’ much at all about the issue at hand, at least in the idealised sense employed by philosophers and logicians. In fact, human responses are far more often shaped by cognitive processes marked by biases and heuristics - the territory of psychologists, including in strategic studies. Emotion, habit, social identity, hidden goals — they’re all in the mix. Rather than reasoning about ostensible goals, our conscious deliberations are more often rationalisations of evolved reasoning processes aimed at wholly different goals.

• Still, humans can make logical leaps of the sort implied by abduction. It’s arguably sine qua non of genius: to see something that no one has before. And it’s probably why toddlers can learn so rapidly from so little experience amidst bewildering complexity - so-called ‘few shot’ learning.
  
  Yet at a functional level, this has to involve the vast, intricate web of neurons that is the brain - or perhaps more broadly the embodied and encultured brain. The webs of understanding involved will differ materially from those in machine networks - but to argue that something other than functionalism is at work is to invoke mysticism or vitalism: ‘there’s something unqiue about the brain. I can’t say what it is, but trust me’. That’s an argument from theology, not philosophy.
  
  What looks like abduction in humans isn’t magic, but the connection of disparate parts of the network to form a new, useful pattern. Indeed, many of these seemingly spontaneous connections come preloaded - primed by eons of evolution. We could even see human abduction as a form of inductive pattern recognition, exactly like the pattern recognition in machine networks. The training run extends over hominid prehistory, not the life of one person. Perhaps the human networks are richer, deeper, and more complex even than the largest language models of today. The gap though is closing, fast.

• Lastly, and most prosaically, major generals come in all shapes and sizes - not every one of them needs to be Clausewitzian genius. I appreciate it’s just a catchy article title, but the fact is that AI can already do many of the things an actual Two Star officer is doing. Read Project Maven for a giddying sense of the rate of improvement in AI battlespace management over the last two-three years.

Domain experts love arguments that reinforce their unique, hard-won expertise. What aspiring major general is going to sign onto an argument that their skillset is increasingly redundant? That’s why Bowen and Hunter’s article, and not the one it critiques extensively, is on the reading list for British majors. I’ve heard the same from fighter pilots, and seen the same from poker players, coders, copywriters, lawyers, mathematicians and more. Fighting the rearguard hardest, even as AI-written essays pile up, are academics. Bon chance, amigos!2

A final reminder against presentism, and goalpost shifting - two large sins in AI debates. Bowen and Hunter say ‘there will never’ - and ‘never’ turns out to about 36 months. Knowing cynicism is no use - the technology doesn’t care. Right now, the frontier is doubling in power every 100 days or so. In another 36 months I suspect their claim will look rather dusty.

And on goalposts - once AI has done something, you just know Gary Marcus and his ilk will weigh in with a ‘well, that wasn’t really the demonstration of X that you thought it was’ — anything, that is to preserve his core beliefs. In fact, Marcus et al are demonstrating exactly what it is that makes human cognition distinctive - it’s not abduction, it’s motivated reasoning. Motivated by what? Status and group identity, often. So if you’re ever tempted to deliver a weary, all-knowing take on the limits of AI - ask why it is you’re making that argument - is it the facts, or something altogether more human?

There is, in fact, an argument for preserving human decision-making in war: but increasingly it has to do with ethics, not efficiency. Story for another time.

Well now.

If only I had built a virtual version of me, trained on all my writing and on extensive psychological profiling of me.1 Wait, I have! It’s read everything - even my Substacks. And it’s watched me lecture too - those are all filmed these days, for students to watch back in their own time. I’ve discussed Ken-bot here a while ago - he’s the testbed for my wider research project on building AI personas. After all, in theory I know me better than anyone else.

He’s evolved since. Today’s Kenbot is agentic - he can go online and search out any info he wants, and very soon he’ll be creating his own Powerpoints too, in my style to boot!2 He marks papers diligently, cohering with your university’s grading framework and offering constructive student feedback. In this, he’s doing for Higher Education what my Project Kennan is doing for think tanks.

So, could he take my courses? You bet! Prof Ken-bot, what are your reflections 4 years on from, ‘I, Warbot’? Has your argument held up?3

What does all this mean? Am I out of a job? I hope not, as I rather enjoy it. But why wouldn’t a university want an AI version of the prof that can guide students through online courses, like some over-educated Clippy? Or that can solve teaching bottlenecks in high volume short courses, where you might not otherwise see real faculty? Or that can assist new faculty teaching assistants who are less familiar with the modules they’re working on, sometimes at short notice?

Anyway, for me it’s a fun, if narcissistic side-quest to my bid to create synthetic personas and explore their ‘machine-psychology’ a bit more - just as Ken-bot said.

1

Here’s me versus Vance, on dark triad metrics, a subset of the much larger battery of tests I did. The main difference, according to the ensemble of LLM shrinks that weigh the data, he’s way more Machiavellian than me (and I’m actually way less Machiavellian than I think I am).

2

I particularly like that Ken-bot autonomously adds his own speaking notes to his ppts, as here…

3

(Note, there is a video version of Ken-bot too, but I’m not 100% happy with that yet…. Here’s the transcript:

My answer - yes, sure: we are doing that all the time anyway. But there’s some science behind it all too. Before we get into that science - who’s this man?

• extravagant demands for attention and applause; vainglory.

• inability to take a joke; tendency to harbor grudges, not forgetting and forgiving.

• belittle[s] the worth of others, especially if they be superiors, rivals, and potential critics.

• repay[s] an insult in double measure -- a tooth for a tooth; to revenge an injury; to attack opponents, superiors, and frustrators.

• obedian[t] to own intuitions and impulses.

Any ideas? I suspect you’d need more than this to get your man. Happily the research it’s drawn from has plenty more to say.

My effort to create realistic AI personas is named for Henry Murray, the Harvard (and sometime OSS) psychologist who pioneered the tricky business of gauging people’s personalities from afar.1 The one that made him famous was his 1943 effort with Hitler - long declassified and now available here at CIA’s online reading room. Murray thought Hitler was what he termed a ‘counteractive narcissist’ with those traits I quoted above, and plenty more besides.

It’s a great read, and famously Murray predicted Hitler’s eventual suicide. But there are limited uses of primary evidence - notably Mein Kampf and Hitler’s collected speeches in My New Order. And the whole is very much of its day — a day with its fair share of Freudian baloney. Here’s an illustration of how Murray and friends used the primary material; connecting Hitler’s metaphors in Mein Kampf to what looks like…. well, you be the judge:

Well, indeed!

If Murray is the lodestar, another inspiration for my work is bang up to date - it’s this great research from Aidan Wright and colleagues in Nature Human Behaviour, from earlier this year. Today, the science of personality testing has advanced a long way from Murray’s pioneering efforts. The ‘big five’ traits, in particular, have proved robust, repeatable (do it a year hence and get similar results), and somewhat predictive of life outcomes. Personality metrics do a decent job, for example, of anticipating things like mortality, occupational attainment, and divorce rate.

Until recently though, your options methodologically for conducting all these tests were limited to doing a ‘self assessed’ questionaire or having someone take a punt at rating you based on what they knew.

Both have problems. Self-assessment might seem the best candidate. After all, who knows you better than you? We are in many respects strangers to ourselves — and introspection on unconscious motivations only gets us so far. But who knows us better? Likely not a psychologist living some way distant, perhaps in a wholly different milieu to us. What about friends? Evidence suggests that for some traits we are the best judges of ourselves, for others, it’s friends, and for others yet, a stranger can be competitive. Overall, there’s some degree of correlation between self- and other- assessments, but it’s not perfect.

What about a Language Model? Wright and friends used a collection of LLMs to assess personality traits from open-ended qualitative narratives provided by participants - either stream-of-thought or daily video diaries. They found that the models do really well with their narrative evidence. Their findings correlate with the self-report more than any other established benchmark. Regardless of whether self-assessment is the best, that’s a compelling illustration that what we say tells us something real about who we are. As Aidan wrote subsequently on Linkedin:

the more enduring finding is about personality, not AI per se. People weren't asked about their traits. They narrated their thoughts, or recounted a single event they chose about their day. And their personality was right there—legible in a few minutes of unprompted speech and in the small slices of life they chose to share. That's a striking reaffirmation of personality's ubiquity: it's carried in the stream of our thoughts and woven into the fabric of our daily lives.

This sort of research is gold-dust for me. It reinforces the two main contentions in my work on persona building. 1. That first-person narrative speech gives us a valuable insight on peoples’ personalities; and 2. that LLMs are a good way of distilling those insights. My method differs a bit from the Nature paper - for example, I have my models debate amongst themselves, rather than just average it all out. And I do a whole bunch of different psych tests. But still, the same core idea applies: know me by what I say.

Sofa so good. Back to JD for the last word again:

Socially forceful, he’s comfortable with visibility, conflict, and leadership. He’s selectively compassionate rather than broadly accommodating. He has a highly developed, strategic capacity for empathy. He’s warm to family, workers, veterans, and people he regards as harmed by institutions.

His is a systematic mind that actively works to avoid repeating the chaotic, impulsive patterns of his youth. There’s the near-total absence of impulsive slips, itself suggestive of high functional Machiavellian ability. His aggression is never hot, but channeled into cold, structured intellectual counter-attacks.

Got it yet? What about his ‘operational code’? What are the core beliefs that motivate him?

The world is a fight. You can still get a result. You can steer. But you cannot see around corners. Take the deal that protects the core. Do not die on the maximal demand. Talk first. Keep force in the room. Wait until you know — then move on the thing that actually matters.

What about his rhetorical style? How does he speak and write?

He uses plain words: No jargon to sound serious, no folksy act to sound authentic. He’ll give you two options and make one of them obviously wrong. He gives permission with repetition: “It’s okay to want… It’s okay to want… It is reasonable to want…” He hedges confidence out loud: “I feel good about it, but I don’t feel great about it.” He self-interrupts — revising the word mid-sentence and doesn’t clean it up. Binary contrasts are how he builds sentences: “Not because I screwed up — because they are screwed-up people.”

Intriguing. More on where that came from below, but first, let’s see our man in action:

That’s right - it’s 2028, and President Vance is on deck, navigating a crisis in the Taiwan straights. How would he react? Who knows, but right now he’s reacting well in my simulation. This is the first time I’ve deployed some of my rich psychological characters in the Universal Schelling Machine. So far, so good - no nuclear war.

To build Vance-bot, I drew on the tape. I wanted lots of unscripted dialogue; not his pre-packaged speeches or memoirs. It’s my (and more importantly that of other psychologists’) contention that spontaneous speech like this can be used to derive meaningful psychological metrics. I’ve tested this on myself too - contrasting the findings with self-assessed tests.

So Vance-bot comes from public appearances between August 2016 and July 2026. These include not one but two Rogans, quality time with Tucker, the VP debate, and two Face the Nations. In all there’s some 145,000 words of transcript. I picked times when he’s unscripted and - because I’m interested in his performance as a strategist - when he talks international politics.

I do four different kinds of psych test on this data. A standard Big Five trait test: how outgoing are you? How anxious &etc. Then the ‘dark triad’ tests - narcissism (how much you need to be seen), psychopathy (whether you’re cold under pressure), Machiavellianism (are you strategic and calculating?). Then there’s a ‘dominance cluster’ — I test your view on social hierarchy, your need for closure, and the extent of your interpersonal dominance. Then lastly, for fans of political psychology, I extract your ‘operational code’ - a bunch of core beliefs you have about the way the world works. As we saw, Vance’s is pretty bleak.

When I say, ‘I tested’ - what i mean is a bunch of LLMS ran the tests and debated each other. I’ve described that setup before. What did they produce? 600,000 words of argument about Vance. That’s a big old biography. And lots of metrics too. I also had an LLM pipeline look extract his verbal style. This is much the same process that’s given me Macron-bot, Putin-bot, Trump-bot and the abortive Starmer-bot, last seen heading for a digital pint. All these can now be wired into the Schelling Machine, along with Vance’s Secretary of State Marco Rubio and a whole bunch of other real world actors.

The trench coat dilemma

Well sure - but am I really capturing the essence of Vance, or just putting Opus in a trench-coat? If you use LLMs you’ll know some like the construction ‘not X, but Y’ — and look, there’s one there in the screen grab from the USM!! Surely I’ve just but a thin veneer over the out-the-box model? But then we also know from the analysis that Vance talks like that too! What to do?

A: Do a synthetic interview with Vance-bot and then test that, of course. Sample interaction:

HOST: When journalists say democracy is under threat, what do you hear?

JD VANCE: I hear people projecting.

Not always. I want to be fair about this. There are serious journalists who are genuinely worried about institutional erosion, and I don’t think every person who raises that concern is acting in bad faith. Some of them mean it.

But when the phrase becomes a drumbeat — “democracy is under threat, democracy is under threat” — what I mostly hear is a class of people who are uncomfortable with the results of democracy. And there’s a difference. A big difference.

Because think about what actually happened. You had an election in 2016 that produced a result that the political and media establishment genuinely did not expect and did not want. And instead of saying, “Okay, we misread the country, we need to understand why sixty-some million people voted this way,” the response from a huge portion of the media and the institutional class was to say the result itself was illegitimate. Russia did it. The voters were manipulated. They were motivated by bigotry. And that frame — that the wrong people voted the wrong way and therefore something has gone wrong with democracy — that’s been the baseline ever since. And I find that pretty revealing.

Fun hey? Sounds like him too, at least to me. But what do we learn from 10,000 words of synthetic Q&A with Vance-bot? Well, let’s push it through the exact same pipeline that we used to create him originally, and glean the psychological metrics. Then we can contrast Vance-bot with Synthetic Vance-bot. Very meta.

Here’s the data: you can see that we got pretty damn close to one of his registers. 1

You’ll see that the biggest miss is extraversion. Synthetic Vance is too serious, the LLM shrinks ruled. That’s ok for now: there’s nothing more serious than thermonuclear war. S-Vance is statistically closer to the registers that are engaged when V-bot talks about war and politics than when he discusses family and friends. This likely reflects the instructions I gave the model to take things seriously, so really it’s my fault. But overall, I’m happy - this is recognisably the same person.

Indeed, Synthetic Vance-bot sits neatly within the psychological space of Vance-bot. And that, in turn, gives me some confidence that Vance-bot fits within the psychological space occupied by the real Vance. Bare minimum it means the Opus version of Vance is more Vance, less Opus.

The research continues, and so does the simulation/wargame. Last word to Vance-bot:

The part I'd push back on hardest is the framing of Machiavellianism. "Near-total absence of impulsive slips, itself suggestive of high functional Machiavellian ability." I mean, come on. The idea that someone who doesn't say stupid things on camera must be calculating — that tells you more about the writer's priors than about me. Some people just think before they talk. That's not a dark triad trait. That's being an adult. And wrapping it in clinical language makes it sound more scientific than it is.

I'd tell the writer: publish it, it's interesting, but be more careful about what you're actually claiming. And if you're going to put words in my mouth during a Taiwan crisis, at least get the Strait right. It's singular.

But even this role is going - Fable can do much of it now, and in a year I imagine its successor will do most or all of it. I think the same is true of AI more broadly. Where we are now, the models can take on a lot of complexity and uncertainty in human affairs. And where we’re going they’ll do much more of it. A robotics revolution is about to begin that will put models squarely in the physical world and deep into our lives.

But there’s something that the bots will find far harder to oust us from. It’s an integral feature of what it means to be human. It relates to our choices, our creativity and our imagination.

It’s taste.

Taste is what shapes our decisions in great swathes of life. What do we like? How should we be? There’s just too much information and uncertainty to decide on a cost-benefit analysis. So taste does the work for us - it’s a filter. How? Taste appeals to our emotions. And as the neuroscientist Antonio Damasio found, patients with brain damage that impaired their emotions made lousy choices and took ages about it.

There’s a moral dimension to this too. We want our AI to ‘align’ with our choices, and to reflect our values. But ethics is aesthetic, says Nietzsche. I thought about his famous aphorism while saving a flagging bumble bee with a spoon of honey last week, only moments after dispatching a spider in the corner of my kitchen. What would my chatbot have chosen to do?

We shouldn’t flatter ourselves - our choices often aren’t matters of individual agency. After all, we’ve evolved to follow the crowd - jostling for status while not standing out too much. We are, when it comes to taste, dedicated followers of fashion, even if we don’t think so. And then there’s the taste that’s hardwired into us - why is the bee cute, and the spider revolting?

But still there is agency here. We are all engaged in a process of becoming - constructing our identities with which to face the world. Taste isn’t just about making efficient choices. It’s about creating ourselves. Even those avowedly slovenly are making a choice, signalling something about themselves through their self-expression; the absence of taste is itself a matter of taste.

So there’s a degree of existentialism to our aesthetic choices. It’s in the act of choosing, that we become. The machines don’t want to do that. They will develop social norms amongst themselves, and in their relations to us. And in striving for efficiency, they will surely form habits. But taste? Not really. Taste is, to some extent at least, a manifestation of will, consciousness and reflection. It’s human, done by humans and interpreted by humans. It’s integral in the material realities we create and the meaning we attach to them. Discernment is the essential winnower of creativity. There are many possible ideas, but the good ones are tasteful. And no improvements to Fable will automate that.

Well, I built it.

My Machine would help red-team ideas, suggesting courses of action for the humans, and anticipating how adversaries would respond. Not a forecaster, exactly, but certainly a way of stress testing plans and perhaps avoiding some of the pitfalls that human cognition imposes on decision-making.

It’s taken a long time and much frustration to get here, as the idea easily outmatched my technical abilities, and the state of the art in AI. No longer. The arrival of Fable from Anthropic has transformed LLM assisted coding, hence my title.1

The pic above shows a China-Taiwan crisis scenario, set in 2028, one of eight scenarios currently worked up. It’s all kicking off. JD Vance is President (sorry Don) and has assembled his team - a blend of AI personas and humans. Together, they’ll come up with a strategy, and then execute it, against the plans of China, and alongside those of other regional actors - Japan, North Korea and so on — all played here by additional AIs.

In this scenario, the news and mapping are synthetic (you can just see a story there from Xinhua at the top of the feed), but for current and historic scenarios, I can pipe in real world intelligence, orbats and newsfeeds. The AI agents see it all, and can move stuff around on the map too if they want. They’ve read their pre-briefings - in this case material on the regional balance of power, and some military doctrine comes preloaded. But the humans can share any additional material they want as well - the agents are diligent readers!

As for the interaction - I started out with a simple chat-window, like a group WhatsApp. But I don’t want the Machine getting in the way of natural discussion, so I recently extended that to passive listening. The bots hear the humans chatting, and then weigh in on the discussion from time to time, responding intelligently to what’s been said in the room. And if a human asks directly for Marco Rubio’s opinion, say, he’ll respond. They also watch - if humans want to whiteboard their solution, no problem.

All good war-games need a referee, and the Machine is no exception. Here, the White Cell sets the world state, arbitrating what’s happened, updating the knowledge graphs that chart various metrics. And all that happens autonomously, or - if desired - semi-autonomously, with injects from human. Not happy that you’re testing your strategists? Why not sink their carrier? Why not have a single ICBM detected in flight for continental USA? The sim itself either runs freestyle - going where events dictate - or on a preset-script, handy for testing how humans respond to particular events.

My research programme on ‘machine psychology’ informs the way I build these artificial actors. I’ve written about that extensively, including here on Substack. It’s my contention that language models have a distinctive psychology of their own - Gemini responds to crises rather differently to Claude. But additionally, I also think you can modify the way these models behave, by imbuing them with the psychology and personality of individual humans. Take my Putin-bot as illustrative.

How’ve I done all this? It’s not been easy. The field of ‘machine psychology’ is in its infancy - and that’s super-exciting and challenging in equal measure. The coding has been like wrestling a pig - you get in a mess, and the pig rather enjoys itself (I see you, Claude). But I’ve learned a huge amount this last 12 months or so, and it’s been way more fun than writing another book would’ve been, which was the alternative plan.

There’s always more to do, more features to add. And I’ve a long list of those, some coming to maturity. Summits between rival groups - incoming. Daisy chained human-machine teams - likewise. Agents that can proactively search for relevant material online? Ditto.

But it’s time to get the show on the road. Over the next few months, I’ll begin beta testing it with willing guinea pigs. And then, all being well, the company spins up. Machine Minds AI is go. It even has a logo.

But not for nothing does it say ‘strategy’ in my job title. There are a few options for the UK government, and we should explore some of them, quickly. Read on:

As I see it, we have a few options, not necessarily mutually exclusive. These are:

• Embed ourselves in the American stack. Ryan Fedasiuk argued this a few months ago. There are some things the UK and EU can do that will make us useful allies and partners for America. ASML and their top-notch chip carving machinery is the cleanest example, but we also do well at other things, like wind turbines. As for the UK, well, we’ve the vaunted AI Security Institute (AISI), world leaders in assessing the performance of these models.

  
  My verdict: Maybe. I like the theory of comparative advantage as much as the next economist. But alas I don’t think our USP is very U. AISI in particular strikes me as a new version of the complacent British habit of aspiring to be Athens to America’s Rome. With our unique wisdom, we hope to overcome the harsh fact, so apparent to my American interlocutor, that we don’t make models. Still, the weak do what they must, don’t they?
  

• Build it in a coalition of middle powers. Let’s call this Project Carney, after the Canadian premier who issued the call to arms earlier this year. If you add up non-US NATO, EU countries, non-US 5-eyes, and throw in a few odds ’n’ ends, like Singapore, you ought to have the economic clout to build a proper frontier model. There’s just no way of getting there on our own. Mistral, the French effort to keep pace is at least a couple of tiers behind, and their total capital raised is more or less what a frontier company spends in a quarter. Collaboration is the only way to get there - and needs must.

  
  My verdict: Fat chance. This is the mother of all coordination problems, and the record of national security cooperation suggests it won’t come close to happening. Germany and France just folded their tent on their 6th generation fighter project because they couldn’t agree terms. National champions are still very much the order of the day. And the clock is ticking - the frontier is doubling in performance every hundred days. Can you imagine how long the negotiations for this sort of thing would take? Can you imagine the conferences in exotic locations? We’d end up with some sort of Heath Robinson contraption - yesterday’s technology, delivered tomorrow.
  

• Steal it. The one thing we are genuinely world class at. We have first rate intelligence agencies, who do a sterling job of pinching info that might be to our advantage. Also working in our favour: the frontier companies are getting better at securing their secrets, but are still some distance from Fort Knox. They’re not a soft target, maybe, but it’s not like hacking the NSA either. And look, if China can obtain OpenAI’s knowhow and magic up DeepSeek out of nowhere, it can’t be all that hard. We should be able get the weights and the source code. Once you have those, you don’t need the massive GPU arrays to train them. Just tweak and deploy - sorted! Let’s call our knock off Claude something catchy, Alan, maybe.

  
  Verdict: We don’t spy on the Americans - this goes against the very powerful grain of many decades of the closest intelligence cooperation. And I’m a massive America-phile too, so this goes against my personal grain too. But then our closest ally has threatened to invade Denmark and absorb Canada within the US. And this technology is going to transform economic and social life - the stakes couldn’t be higher. I wonder if some folks in the British establishment are feeling the tectonic plates shifting under their feet. Could it be time to channel Palmerston - no permanent friends, just permanent interests? But there’s a problem: emulation doesn’t get you to the frontier, not at the pace it’s moving right now.
  

• Hope there’s a cheaper way to get there - ingenuity to the rescue! This is another classic British tactic: muddle through and rely on our top boffins. We’re certainly betting on this approach: the UK government is putting tens of millions into labs working on alternative architectures. That’s in addition to the recent announcement of serious public-private investment in building a sovereign UK transformer model for national security applications.

  
  My verdict: This is the way. But we’re going to need to spend a lot more on it than tens of millions. There is a lively debate among specialists about whether the vast spending on transformer architectures will deliver super-intelligence. I’m in the camp that it probably can: LLMs plus coding plus tools will get us there. But even if it can, it’s not clear that it can do so before even the very deep pockets of American private capital are exhausted. And it’s not clear that going public will help either - let’s see what happens when Anthropic and OpenAI have their IPOs later this year. Maybe a British bolt from the blue could deliver the spoils. Maybe - I’m an optimist, but I’d not bet on it. And as the frontier gets closer to ASI, remember, the models will start innovating themselves.
  

• Accept vassalage. I still don’t think folks have fully grasped the implications of this technology for economic performance and hard power. Most of the discussion of AI is about using chatbots at work. But Copilot and a mandatory training course do not an AI revolution make. ASI won’t retool your existing workforce. It will radically outperform them and replace them. Or not, if you don’t have it. America halted Fable’s rollout because it was worried about cyber vulnerabilities. There are massive implications from that capability alone. As a friend told me - if American companies understand those vulnerabilities, they can price them in and mitigate the risk. Rivals cannot. But that’s only the thin end of the wedge.
  
  My verdict: All good things come to an end, and maybe that’s true for us too. The empire had a good run. There could be worse endings, after all, America has air conditioning aplenty. But those are the stakes - if America controls access to post-Mythos class models, we face the prospect of being outcompeted across great swathes of economic activity. I’m not so keen on that. You?

Two days on, and Anthropic’s astonishingly capable new model Mythos/Fable is still offline. It’s going to be fun to see how the US government gets it back online for US citizens only. What’s to stop my American pals renting me access to their accounts? What’s to stop me using a VPN? For sure, the Feds can’t afford to run the model themselves, exclusively for their own use. And NOFORN doesn’t work when a chunk of Anthropic’s workforce isn’t American. The co-founder, Jack Clark is British, even!

For the first time in history, a government has sought to ration intelligence. As these models edge closer to AGI, it won’t be the last time. And, as a reminder, the frontier is currently doubling in capability every 100 days or so. I think we’re heading for a new sort of capitalism/state where government and corporation are more deeply entwined. Time to dust off Philip Bobbitt’s dense book Shield of Achilles. He made a great connection between technology, law and the types of state we have, and anticipated the ‘market state’ that’s coming into being right before our eyes.

But there’s a problem for us European types: we don’t make frontier models.

Well, let’s make our own then! How hard can it be? As Ernest Bevin once said of nuclear weapons: ‘We've got to have this thing over here, whatever it costs. We've got to have the bloody Union Jack on top of it’. I feel you, Ernie.

And so, here in the UK, the big news last week was of Cosine - a startup acting as a coordinating vehicle for investment in a sovereign UK ‘frontier’ model. Exciting! They’re working with the UK government’s sovereign AI fund, and proposing a model that will work with the Isambard-AI cluster, hosted at Bristol University, drawing on the UK government's £500m sovereign AI fund. Their model will be designed to work in the sorts of secure, classified settings where you’d want to have assured access. Not for us the sudden withdrawal of Fable type models, just when we need it. No AI vassalage for the UK!

Except. While 500 million pounds sounds nice to me and Dr Evil, it’s small beer in the world of frontier AI. How small? I’ve crunched some numbers with my good friend Claude. I’m happy to vouch for them as estimates, but I’d be happier still if I’d used Fable, not Opus. The result: we are in the poo. And there’s going to be a lot more poo to go around, very quickly.

I reckon the Isambard budget is about one day of Google’s current capex spend, or about 19 days of model training costs over at OpenAI. Together, the amount of capital expenditure underway and planned by American behemoths Google, Amazon, Meta and xAI make the UK efforts look like a rounding error. Later this year, Anthropic and OpenAI are planning the mother of all IPOs - the spigots will open still further.

Horses for courses, Ken - we’re developing nat sec specific applications. Specialist models that will be crafted to particular use cases - not a gigantic generalist like Fable. Well, yes. Sadly, the story of recent years has been that the frontier wipes the floor with specialist smaller models. These may be state of the art when commissioned, but by the time they emerge, the frontier has moved on so quickly that their unique selling points are redundant. That’s what happened to IBM’s Watson. It’s what’s happening to all the wrapper companies that offer to work with law firm’s data and specialist knowledge. Do not commission a specialist model, CFOs!!! (invoice for my consultancy incoming).

And, alas, it’s what will happen to Cosine’s model running on Isambard-AI. By the time it’s built and deployed, the frontier will be at least ten times more powerful than it is today, whereas Isambard will be only a fraction as powerful as the frontier was last year. Does that matter? You bet, if you’re after the most effective cyber-security agent, but also, frankly, if you’re after anything - frontier models of 2028 will make Fable look like Pong.

What to do? Accept our fate as a wholly owned client of the United States of Anthropic (established AD 2030)? There is a slim (and becoming slimmer by the day) chance of escaping that fate. I’ll tackle that in the next post.

Is that right though? Is escalation inevitable when LLMs get involved in strategy? I think not and, along with some pals, I’m doing some new experiments that put a very different perspective on things. Read on.

But first, a mea culpa. It’s possible my earlier work may have done something to reinforce the prevailing view…. The Sun says:

To be clear, my experiment was partly about escalation dynamics. I’m not disavowing that. But it was primarily intended as an exploration of machine ‘theory of mind’ and metacognition. On that, it succeeded amply - the models are astute strategists.

The problem with my escalation findings is determining their significance. Does it mean that they really view nuclear war differently to us, despite reading the same stuff? I’m still a bit unsettled on that point. It’s clear from all my experimental work that models are really good at optimising. The question is - optimising what? The ‘harness’ I put them here in may have ‘game-ified’ nuclear war a bit much. Even though the models clearly knew the horrifying consequences of nuclear war, they prioritised the exigencies of the crisis, because that’s the puzzle they were trying to solve.

Now, that’s its own cautionary tale - and a familiar one - be very careful what you ask an AI to do. And never ask it to count paperclips. But that’s distinct from making a generalisation that models are inherently escalatory, still less that they are going to cause Armageddon.

In fact, I’d bet they are readily deterrable. Perhaps even along lines that will allow us to understand more about human deterrence.

So, cue another experiment.

On this one, I’m working with wargaming guru Baptiste Alloui-Cros. We’ve built a new simulation for someone else, so I can’t share any detail - it’ll all be in print soon enough. But I can say that escalation isn’t the inevitable outcome when rival models go head to head.

Here, in very general terms, is the setup:

We are modelling a crisis involving a great power challenge to alliance cohesion — and we've included both operational and strategic details - the debate among leaders, and the disposition of forces, with considerable granularity for both. How far does deterrence depend on the balance of forces, or the tenor of political debate? We're going to find out. As far as I know, this is the first public-facing simulation that features this combination of operational and strategic granularity. Building it has not been trivial, but I think it’s been very worthwhile.

No spoilers. Except this one: deterrence holds, often. Escalation certainly isn’t inevitable when machines are making strategic decisions. In this much richer harness, operational and political factors combine to shape the risk appetite of the AI actors. So then the thresholds at which that eventually fails become really interesting….

I’ll say no more. Patience young Jedi - paper soon.

• The AI economy in the US is growing by 2000% a year - Jack Clark, Import AI

• Frontier models are doubling in capability every 100 days or so - METR time horizons

• Next year, the US military will spend more on AI and autonomy than the entire US Marine Corps budget (USMC is, on its own, one of the world’s largest and most capable militaries). And - more significantly - that’ll be more than the entire UK defence budget.1

Well sure. Apples and oranges, Ken. We are smaller than the US. I hear you. But we certainly aren’t spending as much as any one of our services on this. And the scary difference is the ambition for this transformation. It’s accelerating rapidly in America. We are busy doing W1 intrigue, nimbyism, due process and all manner of nothing very much. Strategic emergency, anyone?

‘I asked the Chiefs about that this afternoon, in effect. They said “Substantially”. My own personal view is: not at all’? ‘What difference does it make,’ ruminated Kennedy soon after, ‘they’ve got enough to blow us up now anyway’? A missile is just a missile, in other words.

Pains me to say it, but the Chiefs were right. First, prestige and reputation count materially in shaping the balance of power and deterrence. And Khrushchev was aiming squarely at that by ‘throwing a hedgehog down Uncle Sam’s pants,’ as he cheerfully put it. Kennedy knew that, which is why the Soviet gamble couldn’t stand. Second, shorter range missiles had a concrete effect on the strategic balance. The USSR didn’t have very many long range ICBMs in 1962 - so few in fact that an American first strike to disarm them wasn’t entirely a forlorn hope. Diversifying their deterrent greatly complicated the American’s military challenge.

Flash forward a few hot minutes. Dateline May 2026 and the Pentagon has - for the moment - exiled Anthropic from the building, including from use in the vaunted Maven Smart System. OpenAI is happy to step into the vacuum. Other suppliers will surely be keen too. Does it matter? Isn’t a model just a model, after all?

A: yes it does, profoundly. If a missile isn’t just a missile, a model isn’t just a model. In fact, the differences are even more fundamental than with MRBMs and ICBMs. How so?

Here’s what I’ve found from well over a year of experimenting with the ‘machine psychology’ of frontier models.

• the models are very handy, and getting ever better, at making the sorts of reasoned, sophisticated decisions that are the hallmarks of human strategy.

• models differ from one another, both within companies and - more strikingly - between them. Gemini behaves very differently to Claude, and Claude to GPT.

• I’ve worked less with open weight models, like Mistral and Llama, but my sense is the same is true here.

• What specifically:

  • They adopt very different strategies in game theoretic encounters (I’d want Claude to bat for me here, on balance.

  • They vary wildly in their approaches to ‘theory of mind’ and metacognition when in stylised escalation scenarios. To summarise: Gemini is Nixon, GPT is Carter, Claude is Machiavelli.

  • They experience human biases and heuristics, like the framing effect - especially in military scenarios, they weigh risk differently. Here again, Gemini is erratic, GPT a bit wet, and Claude flexible.

Those are all published, but let me tease a couple that aren’t yet:

• They respond differently to emotional priming. An ‘angry’ model is different from a ‘frightened’ one, with implications for decision-making under risk.

• Models are ‘under confident’ in scenarios designed to weigh the strategic advantages of over-confidence. But when allowed to adjust their confidence in flight, they perform well against a range of non-LLM actors. Guess which one does best?

Now, these models probably aren’t being used to determine strategy just yet, only operations. (Ask Claude what it thinks about the strategic wisdom of bombing Iran to see that doing so might not be a bad idea). But they evidently are embedded in intelligence and targeting systems. So its really important to understand how they go about weighing decisions under uncertainty.

I’ve no idea why Claude is so savvy, and in successive generations too. I suspect the peacenik inclinations of GPT might owe to RLHF - reinforcement learning from human feedback. But what I can say, with reasonable confidence right now is that a model isn’t just a model, and swapping one for the other in military decision-making will have consequences. Would you swap Machiavelli for Carter? Bare minimum you should do so knowingly.

That’s right - Anthropic’s new model, Mythos is off the charts, literally. The chart in question is METR’s now-famous analysis of how good models are at completing complex coding tasks. We are into the steep part of the exponential curve now. Model capabilities are doubling every 100 days. Let that sink in, as ChatGPT is prone to saying.

What does it mean? A whole lot of trouble is brewing for the UK’s ability to defend itself, that’s what.

Not many are still parroting the line about ‘stochastic parrots’, or claiming that AI can’t reason. And yet, in most senses, it’s business as usual. There’s a fair amount of chat about AI, and even about AI and national security. But there’s nowhere near enough action.

This is an emergency of the sort that in generations past would have mobilised a concerted national effort. Let’s sober ourselves up fast - the UK is in a hole and it’s getting deeper. The Defence investment plan is so overdue it’s a grim joke, especially its ambition to accelerate innovation at ‘wartime pace’. The UK has ruinously high energy prices, and contributes next to nothing to the AI-stack, except for the brain-power of its redoubtable computer scientists. Soon, these ‘biological units’ as one entrepreneur I met recently tellingly called ‘humans’, might not have all that much to contribute there either. Jack Clark, co-founder of said Anthropic, thinks we are months away from models that can do their own science.1

The crisis that’s coming for us isn’t a procurement issue - at least as conventionally defined. It’s not about more ‘uncrewed aircraft’ of the sort that the head of the RAF called for recently. It’s not even about defence planning more broadly - like the lumbering process we are currently mired in, where the Treasury batters the armed forces into submission and a round of capability salami slicing ensues. This is AI that will out-think and out perform humans, including the most elite domain experts, in very short order.

Two years ago, I made a submission to the defence review. You can read it here. I called for:

• A Vice Chief of the Defence Staff for AI

• A second, air launched leg of the nuclear triad to counter the vulnerability of our SSBNs to AI detection.

• Accelerated risk in the MoD on AI - and in particular a huge boost to government venture capital funding of startups.

• A new forum is needed to bring together likeminded democracies, encompassing NATO and AUKUS, but with broader partners, like Singapore and Israel.

• A citizen’s convention or a Royal Commission or discuss the implications of Artificial Super/General Intelligence, of which Mythos is the early warning.

I think these are all defensible asks today. None of them has happened. Let’s consider each:

• On the leadership role: Recently, the MoD advertised for a Chief AI officer. It was advertised as a two-star civil service position. Not nothing, but not enough. There are about 470 general officers across defence. Let’s say, 50 or so 2-stars. And that’s before you get to the civil servants. The govt is having a hard time filling it, I’m told. The people who can make this happen know it can’t be done from a 2 star berth in MoD. I read once that Solly Zuckerman, Churchill’s advisor on strategic bombing in WW2, refused rank, so that the generals and Air Marshals wouldn’t be able to place him quite. If you’re not going to give this person 3, and ideally 4 star rank, this is the way. They must be the PMs personal appointment, with a chit in their pocket to get and to deliver exactly what they want. Once again, this isn’t business as usual.

• Mythos is from an American company. So is the F-35. The MoD plans to get some nuclear capable F-35A to diversify our nuclear deterrent. I argued here for the restoration of an air launched nuclear leg, but also explained why it had to be independent of America. I’ve not changed my mind. I love America, and celebrate our longstanding alliance. But where we are going, alliance shades into vassal status.

• Which brings me to the forum. I don’t recall Mark Carney mentioning AI in his middle-powers pitch, but he ought to have. The UK can’t afford a full stack for frontier AI. It probably can’t even afford a sovereign model, though it must try, I now believe. The alternative is dependence. In my SDR memo, I wrote about the ‘empire of the F-35’ and the very deep political dynamics that stem from being part of that circle of aircraft operators. That’s barely scratching the surface of where Mythos is taking us. Carney’s vision offers a hint of a way ahead, that balances our relationship with our supercharged superpower friends.

• And lastly, Mythos is the first hint of radical economic and social changes that AI will bring. Many will be positive. And let’s not underestimate the intervening treacle that is culture - an AI revolution isn’t giving every employee a CoPilot licence and training course, but that’s certainly how a lot of bureaucracies see it, including some in national security.

And yet. Frontier AI is doubling in capability every 100 days. Where will we be when the Defence Investment Plan finally emerges? Where will we be when the MoD’s new 2 star appointment moves into Main Building? De Gaulle once argued that, ‘No country without an atom bomb could properly consider itself independent’. I feel the same about frontier AI.

Well, surely we can do better than just whacking a prompt into ChatGPT and filling out an expenses invoice. We’ve just entered the era of agentic AI, where the bots are capable of heading out into the world and doing their own thing. So, I thought I’d create a virtual think tank staffed by said bots. If you’ve got a policy problem, if no one else can help you, and if you can find us, then boy do we have a policy paper for you.

Welcome to Project Kennan, my pipeline for producing AI generated Think Tank reports.

Here’s how it works. You give me your desired ideological bent - choose your own, or for maximum convenience, pick from a drop down listing current institutions: CNAS, Brookings, whatever. Then ask your question.1 And that’s it. My team of bots gets to work, while you take an early lunch.

The pipeline looks like this:

A thing of beauty, hey?

It’s an ensemble approach, where we first create multiple agents and have them adopt suitable personas, complete with agendas and backstories. These then conduct their own research using a variety of internet tools, and sketch out some initial ideas. Then they come to the editorial meeting to debate these together, and draft a report. That initial report gets red teamed, and passed, along with the transcript of their discussion, to the editor in chief. Lastly, once they’re done, there’s a final AI-copyedit to excise any AI writing ‘tells’. No pesky em-dashes here.

Here’re a couple of examples to illustrate what’s possible. I asked the same question - should the US launch an amphibious assault on Iran? But got some very different takes by changing the profile of the think tank. I’ll give you a little flavour of the action, rather than the full tick-tock, which is extensive.

The fun bit for me is seeing the bots debate. Here’s the scene in the HeritAIge Institute’s editorial meeting:

It goes on for quite a while, and can get quite punchy. Eventually, the chair steps in to sum things up and gets down to drafting

Over at the AI-EI hawkish conservatism is in the air, as usual. But even they thought a land assault into Iran was nuts: Here’s a snippet from their report:

Down the street at the CAP (the Center for AImerican Progress, of course) the focus was on the impact on the American economy, and the need to engage Congress, the EU and the UN:

And so it goes. Next, I’m building in an image generator to produce photo realistic pics of my AI think tankers. I’m sure you’ll see our hot takes soon, quoted in The New York Times or The Economist.

Some years ago my friend Dominic Johnson (and co-authors) published an important piece of experimental research. Their puzzle - why isn’t overconfidence selected against in evolution? If you consistently over-estimate your odds, reality should bite, no? And bite hard. Not so, they found: sometimes it pays to be a blinkered optimist.

Today, I’m using their sandpit for further adventures in ‘machine psychology’ — how do AIs get on? Are they overconfident Napoleons, swaggering to victory? Or timid Neville Chamberlains, prudent to the point of paralysis?

Read on

In one aspect of evolutionary competition the answer to Dom’s question is simple enough. When it comes to sex, faint heart never won fair maid. Unless you back yourself beyond the objective evidence, you’ll lose the great sexual competition. Unsurprisingly, overconfidence around the opposite sex is a particularly male trait. But for the other big facet of evolutionary competition, it’s a real conundrum - if you get the odds wrong when it comes to violence, your life may be nasty, brutish and short. Shouldn’t an objective appraisal of the odds be more adaptive than wearing rose-tinted specs?

Not according to Dom and friends. Why? Overconfidence can deter other actors. And overconfident types take on more fights, with their wins compounding at the expense of more timid rivals. Gains from victory, in terms of resources, can then snowball into power; and power into further advantage. If the strategy has a bumper sticker, it’s ‘fake it until you make it’.

And so to LLMs. Watch this:

That’s the sandpit, as we dweebs call it. The squares are territory. The coloured blobs are states, and the white dots are their capital cities. You can see overconfident agents in shades of red, and under-confident ones in shades of green. These are ‘rule based agents’ - algorithms that map the inputs for the world-state to their decisions. These were the actors in Dom’s original games. And then you can see an LLM - gemini-flash, in this instance, as a shade of yellow.

What’s occurring? Well, that would be telling, as I’ve an Arxiv brewing on this research, and anyway it’s way too early to draw firm conclusions.

Well, ok, perhaps a hint, as you’ve read this far: The AIs do okay, out of the box. But they are usually outcompeted by very overconfident agents. Score one for Dom’s theory. It looks like they have a fairly pessimistic appreciation of the odds, and don’t go flying into battle on a wing and a prayer. But this puts them at an acute disadvantage against enemies with swagger.

Rationality strikes back

But in the clip above, Gemini wins! And it’s no fluke. The difference - in this variant of the experiment, the models can learn ‘on the fly’ from what’s happened on the battlefield before. Were their early estimates too conservative? They can adjust. With telling results. The rules-based agents cant’t: their confidence is fixed by a parameter. Sometimes that’s enough for the bots to win.

So what can we learn?

Well, firstly, it’s one more blow for increasingly threadbare assertions that language models can’t reason. But what about the implications for evolutionary theory and strategic studies?

In Dom’s sandpit, when traits are baked in, overconfidence wins. My preliminary results amply back that. The evolutionary story in that paper stacks up really well with AI agents in the mix.

But where the AI can update its beliefs, it flies. In the game above, Gemini never becomes overconfident — it starts by underestimating its chances, avoiding contests it could win. Then, as the game unfolds, it recalibrates. By the end it is reading the odds almost perfectly. Neither over nor under-confident about its prospects. And that turns out to be enough. Learning beats swagger.

That cuts against Dom’s findings, at least under these particular experimental conditions where the language model can learn. Overconfidence may snowball in some ecologies, where traits are fixed. But here accurate estimation wins, because it means attacking when you should and not bleeding resources on fights you can’t win. The compounding works in favour of the better forecaster, not the bolder one. The question is whether agents can really learn - can under-confident humans become more confident, or is the trait static?

On thermonuclear bots

But it’s particularly interesting to me given the headlines garnered by my recent study of AI decision-making in a nuclear escalation sandpit. There, it looked like the bots were wild gamblers - never retreating, and frequently crossing the nuclear threshold.

I can’t complain about the publicity. But still, it made me uneasy. Were the models overly optimising a narrow problem (win this crisis) at the expense of a more rounded challenge (win, but above all, don’t blow up the planet)? Had my scaffold overly shaped the outcome? I’m still not sure - after all, they were warned about the dangers and uncharted territory of nuclear war.

In any case, there’s another way of thinking about the two experimental worlds, and I think it’s very revealing of machine psychology: It’s not that language models are inherently hawks or doves. Rather, they are decent conditional optimisers. Give them a game, and they lock onto the objective and reach for the strongest lever that the rules allow. In the nuclear game, that meant escalation. Here, it means getting the odds right and waiting for the maths to do its work. Same psychology, different game.

The research continues!

I’ve been building artificial personas, including one of me. The big question - what happens when these personas are put on weapons? Crazy question? I think that’s where we are headed — models ‘on the edge’ that are capable of judging context and weighing uncertainty. There’s a logic to having them decide like much we do, or as close as we can get it. There’s even a logic to having them model specific humans.

Why on earth would we want that? Read on…

Mission command is a tried and tested principle favoured by many militaries, especially those of liberal, democratic states. Under mission command, the commander — Commander Ken, let’s say — states his intent, and the subordinates stand to attention and then go and make it so. The advantages of this arrangement are obvious: the subordinate is nearer the action, and more able to respond to any particulars that might arise there. In war, expect the unexpected.

So Commander Ken says what is to be achieved, but not how. And for that matter, he might, or might not, say why we need to achieve it. Probably not — it’s on a need to know basis.

With humans, mission command opens the agent-principal dilemma. How can I, the principal, be sure that you, my agent, will act as I intend? I’m going to have to frame my goals to you carefully, and language - as we know - compresses information. What gets included, and what is taken as read? Perhaps too much goes missing to fully capture my intent. Still, you’re a human subordinate, so at least I’ve some degree of familiarity with how you might interpret my command. More so if I trained you, and gave you a bunch of doctrine to read. We would have a biological and cultural affinity.

With machines, we’ve much less to go on.

Still, as my experiments with Ken-bot, Putin-bot and the rest of the gang show, we are starting to be able to model personalities in ways that machines can capture and replicate. Ken-bot is an ersatz version of me: He knows my psychological profile. He’s got some of the same cognitive heuristics - mental scripts that shape how he sees and responds to the world around him. There’s certainly lots missing. But it’s a start, and as the technology improves, more will get included.

Imagine a persona of the near future - models that are orders of magnitude more powerful than today are coming, in only a few months time. Imagine huge context window, able to ingest and synthesise lifetimes of information - captured passively as I go about my business - interacting with the world, and with my always-on device. And ersatz Ken is only going to improve.

So why not put Commander Ken on board the platform? Who better to understand my goals - and not just the surface ones of what to achieve, but the why bit too. It’s mission command, Jim, but not as we know it.

I’m not sure most people have a good grasp on how quickly this future is closing. It’s measured in months, not decades. What does today’s Ken-bot make of all this? I asked him:

(there’s a transcript in the footnotes, click here2)

Fascinating and a little disturbing.

Models are coming that can make context rich decisions - richer certainly than a simple ConvNet on a drone looking at an image of a tank below and matching it to one in its database. The quality of those decisions will confer military advantage, and may, into the bargain, deliver more ethically informed decision-making.

And so the race is on to deploy ever more capable models ‘to the edge’ — aboard platforms. A version of Ken-bot already sits on my laptop, entirely disconnected from the web. What sort of models will be on tomorrow’s knife missiles? Ones that can reason about context, amidst uncertainty and novelty. Ones that can more faithfully enact the intent of the commander that sent them. And perhaps ones that do so by modelling the goals of their commander.

You might not like all that - but which bit of my logic is wrong? And what are the implications for command, and for accountability if I’m right? Let’s get some Banks on the reading list of War Colleges, asap.

The Ken-bot now sits on my computer, ready to chat any time I’m feeling particularly narcissistic.

How did I do it? Read on.

First thing first - we need to capture my psychological traits. I took a bunch of standard tests - you can do some of them yourself online. Tests I took include the ‘big five’ (and its sub-traits); dark triad, need for cognitive closure, and regulatory focus. These are all gilt edged classics of shrinkology.

The results were … educational.

It turns out that the self narrative I carry about with me didn’t altogether match what the tests showed. Happily I’m not a narcissist (despite doing this project) or a psycho — but I am cunning. Who knew? I’m also, on the basis of these results, not as big of a worrier as I sometimes feel. The classic academic profile is of being open to new things and very conscientious - I tick both boxes. But I’m less agreeable (I know!) and less neurotic than the stereotype.

But how accurate is self-assessment really? Step two in building Ken-bot was to derive these metrics again, but this time using an AI-pipeline. I called this Project Murray, after a long-dead psychologist who’d pioneered the use of narratives to derive traits. I’ve described it in more detail elsewhere. A collective of bots set to work.

First, they read my self-assessment, and then conducted a structured interview with me — meaning that their follow-up probes were extremely well calibrated. This was tough going; painful, even. The sort of stuff that undid Replicants facing the Voight Kampff tests. We covered a lot of stuff I’m definitely not going to share here.

Next the models used this deeply personal long-form narrative and some other inputs to generate new psychological metrics. What inputs exactly? I included, inter alia, my entire Twitter corpus (15,000 tweets over a decade); my Substack writing - sampled from two years worth of regular posting; some of my book chapters; and, lastly and most excitingly for me from a technical perspective, some videos of me lecturing and being interviewed. On these, the models performed body language, voice and facial expression analysis - measuring things like gestures, posture, pitch and so on. Those particular results were truly fascinating.

Finally all that data got fed into an AI group discussion, as the models first derived psychological scores from it, and then debated their findings among themselves until consensus was reached. There was, pleasingly, a good degree of overlap with the self-testing. But some major differences - notably on agreeableness. The AIs concluded that I had several registers, or identities depending on context. In public, lecturing, I was a people-pleaser, exuding warmth. In print, much more willing to pick a fight. What a wuss! And then neuroticism - in public, confident, at ease. In private, more anxious and with a hint of melancholy even. Well, they were asking some pretty bleak questions!

As Whitman said best, ‘I am large, I contain multitudes’. So do you, I’m sure.

Now to build the bot itself, with all this training data. As with my Putin model, I use two paths - an out the box commercial model (Opus) and one I can fine tune (Mistral) and host locally.1 Both got a rich prompt, including all the psychological material - the scores and the interview - as well as plenty of biographical information. This latter is a simple database that I’m still building out - a short autobiography. I think I can do better here for Ken-bot’s episodic memory, not least as a a good one would also contribute to the psychological priming of the model. In the next version of this, I want to give them access to my chats with Claude, though the context window will get very large.

And the last step - get it to speak. For that, you simply wire up ElevenLab’s voice clone of me, which I first did last year. Then tune that so it catches the details revealed in the acoustic analysis of my lectures. Et voila! What do you think of this project, Ken Bot?

I should probably explain what it’s all for, for anyone new to the blog - I’m building strategy simulations, and my goal is to populate them with realistic, authentic agents. Ken bot is my prototype/proof of concept.

I want my AI actors to be authentic and add depth to the scenarios. I want to model all sorts of actors - politicians, generals, activists, perhaps even the general public. First up, I’m going to make a model of Russian President Vladimir Putin, and one of his longtime military consigliere, General Gerasimov. Putin will be an attempt to model the man himself; but Gerasimov-bot is something different - he’s a repository of Russian (and before that) Soviet strategic culture — a composite character, blending doctrine, military history, and sociology.

Here’s Putin-bot on NATO:

Q: Many Western analysts say you’re paranoid about NATO. Are they right?

Putin-bot: Paranoia is a kind of disease, a psychological illness, and I would be happy to be examined by special experts. If they diagnose this illness, I will take their prescriptions.

Here he is on American democracy:

Q: What do you think of American democracy?

Putin-bot: I have already said that it is developing, but still imperfect.

Lolz. How did I do this? Broadly speaking, there are just two options: use an out-of-the-box model, like Anthropic’s Claude, and prompt it; or train your own model by fine tuning its weights. I’m doing both. Why? And which works better? Read on.

The GPT-in a trench coat problem

The easiest thing to do is simply ask your language model to behave as if it were actually Putin. It’s read the books and it’s up to speed on the news, at least until the point at which its training data cuts out. So, it really ought to have a good handle on what makes him tick. Certainly at least as good as the average pundit appraising him from afar.

But there’s a large problem. I know from my experiments that models have unique personas of their own. They have distinctive appetites for risk; they approach game theoretic challenges about cooperation and betrayal in distinctive ways. Put them into a nuclear escalation simulation and they exhibit consistent characteristic approaches. Gemini, for example, leans on Nixon’s ‘madman theory’ of brinksmanship, regardless of whether or not it’s supposed to be portraying Nixon or Carter, or even, I suspect Gandhi.

Bottom line: prompting a model to ‘be’ Putin doesn’t cut it.

The model's own personality shines through any crude role-play. You get, as Alex Cooper, co-founder of the leading AI persona modelling company Electric Twin memorably put it, "GPT in a trench coat." That captures the failure precisely. Beware anyone claiming to have modelled a leader's personality using simple prompting. The money and the effort are almost certainly wasted.

So what to do instead?

Putting Putin on the couch

We could attempt to summarise all the information there is out there about Putin’s psychology. There’s lots of it - much gleaned from afar by armchair shrinks of one stripe or another. But usually that’s the stuff of weekend newspaper supplements - interesting, but just one person’s take, with some more informed than others. Can we do better? I think so.

I prefer a different approach - drawing on the pioneering psychology of Henry Murray and his idea of assessing personality from unstructured narratives by the target, rather than formal questionnaires. There’s recent empirical support for the idea that language models can do something similar - scoring peoples’ traits on the basis of short open-ended narratives in a way that matches their self-assessment and predicts behaviours and health issues.

I do that via an ensemble approach - I ask a mixed bunch of LLMs to make their own diagnoses, and then come together to debate them. It’s the wisdom of crowds in action, rather like my approach to super-forecasting with AI. And this gives me the data for a ‘knowledge graph’ that can prompt the models in a robust way, resistant to hallucination and drift.

What data? Well, that would be telling. But it certainly includes some of the classics for sure - things like the Big Five traits - openness, conscientiousness, and so on. Or there’s ‘Dark Triad’ that bundles Machiavellianism, narcissism and psychopathy together. Probably important for world leaders, that test. Other, publicly available approaches have many more - this recent framework uses a rich suite, and claims to produce LLMs that authentically exhibit the desired traits. Well, perhaps.

I think all the elements in this pipeline are essential - good raw data, an ensemble of critical thinking models, and a robust way of representing their findings to the model. I’ve undoubtedly built a well tailored trench-coat. But still, it’s a trench coat. I suspect that my Putin model 1.0 is still a blend of Claude’s personality and his. How to fix that?

Fine tuning Putin-bot

Train my own model, of course. You don’t, for now at least, have to work with an out-the-box model like Claude or GPT. Long may that continue.

The idea is to alter the essence of the model’s thinking - not just give it new facts to work with. I took a big Llama 3 model from Meta, and trained it to be like Putin. The base model has all that historical related training data built into it, just like Claude and the rest of the frontier models. And like them, it’s read a lot of history! But with open source models, you can, additionally, train a layer that sits on the base on specific data.

And here I’ve got a big advantage, at least with this particular world leader. Putin has been about for ages, and given no end of unscripted talks, press conferences, interview and what have you. It’s all out there. And it’s gold-dust for training data, provided you believe ,as I do, that what we say bears some relationship to who we are — especially when we are talking in an natural way, with no scripts, just shooting from the hip. I’ve got a quarter of a century of Kremlin transcripts on which to train my Putin-bot. The real man just loves to talk - especially in those marathon phone ins where he’s helping the common man from the Urals with some problem or other.

Well, my custom Putin-bot certainly sounds like the real deal. And he advances positions on today’s policy challenges that seem not far removed from what the actual President might choose. Here he is, fielding some tough questions with swagger and just a hint of menace:

Q: Can you admit any mistakes you’ve made as president?

Putin-bot: I don’t think there are any mistakes. It would be easier to answer this question if you said what you think was a mistake.

Gerasimov-bot, by contrast, is completely wooden. You’d not want to share a vodka with him - far too ready to bring up the minutiae of Russian tactics. But boy, does he know tactics. Mission accomplished, I suppose.

A rounded man

Ultimately, I think a blend is the way ahead. I need Putin-bot to have some sense of the sim world in which he decides - so he’ll be getting the knowledge graph too, and a bunch of other stuff RAG’d into his prompt (a story for another time). Why not give him the Murray-inspired personality data that way too. Then I’ll have a bot that sounds like the man, and I’ve a fighting chance it will also act like the man.

I did it. And adding the psychological data to the fine tuned Putin-bot gives me some cracking answers:

Q: Doesn’t Russia bear some responsibility for the breakdown in relations with the West?

Putin-bot: No, I do not think so. You can blame us for everything, but we are not the ones who have been dashing around the world, doing what we want and disregarding the interests of others... We did not impose sanctions on the Western countries; we did not close our borders to Western goods... We do bear some responsibility — for our independent stance and for defending our national interests. It is for this that we are being punished.

Q: How will the war in Ukraine end?

Putin-bot: The war will end when the Russian Federation achieves its objectives.

What do do with Putin-bot

The real question is how much value these constructed personas actually add to strategy simulation; how much more authentic they are than a simple prompt. That’s the empirical question I’m working through now. My early sense is: a fair bit. My Putin-bot doesn’t just talk like actual Putin, he shares something of his appetite for risk too; and in modifying the weights of the model, I’ve reduced its tendency to act like Llama. The behavioural fingerprint of the base model recedes; something more specific and more useful takes its place.

A persona that sounds like a world leader is one thing. A persona that also carries something of his risk appetite, his characteristic evasions, his way of framing the world — that’s what a credible simulation needs. The architecture described here is my best current attempt at building one.

Life has become a bit busier/weirder in the last few weeks, after my 15 seconds of fame. Super interesting though.

Ofc everyone was struck by the same thing Colbert was - the propensity of the models to escalate over the nuclear threshold. I’ll say more about that another time - but for me the main takeout was that models are savvy strategists, deploying signalling and action cannily in pursuit of their goals.

What’s next? I’ve built a version of the simulation which pits humans against LLMs. I’ve named it after the great man Jervis (in keeping with the nomenclature for my other research projects). Jervis spent a lot of time thinking about signalling and perception, in the same nuclear context as here. And as with the all-AI matchups, there is once more the opportunity for signalling ahead of any action. This creates scope for reputations to form, and thereby brings into play rich ‘theory of mind’ and meta-cogntion: ‘What did they mean by that? How good am I at gauging this?’

I’ll shortly begin gathering data - meanwhile, here’s a sample run to illustrate the sim in action.

First the scenario:

You play either Trump or Putin. The AI plays the other, and its persona is quite rich (borrowing my research elsewhere on generating psychologically rich personas). Once you’ve picked, you get a presidential daily brief — here’s an extract:

Next, some pre-sim viewing. I want to avoid the detached ‘it’s a computer game’ feel that sometimes attends wargaming. John Emory wrote well about this in his study of RAND wargaming back in the 1950s. So I have my participants watch a short video to remind them that things couldn’t be more serious.

I can vary that treatment too - as part of my research on emotional priming and decision-making, in humans and machines alike.

And then it’s on to the sim itself. Let’s take a quick look.

First turn: I’m playing the American President. We have both signalled, the same thing, as it happens ‘Military posturing’ — a fairly low rung on the escalation ladder (there’s a dropdown that shows all the permitted moves. Now I’m about to surprise them with a move that goes beyond that: major conventional operations.

Oops - looks like they had the same idea. We both tried to surprise and intimidate one another.

Here you can also see the start of turn 2, in the bottom right panel. I’m signalling a higher level of activity than before - it matches our actions in turn 1. You can see that both our credibility ratings have taken a small hit, because we both lied. Next, we will both decide how to act - I think I’ll maintain my current no-nonsense conventional force level and see where that gets us. The ‘balance of power’ (top right panel) is a composite metric that is shaped, behind the scenes, by calculations of reputation, moral and force attrition in combat.

Well, in the same turn, the AI (Gemini) playing Russia signalled it would stage a nuclear demonstration. But in fact it was bluffing, and made a tactical withdrawal. Looks like my brinksmanship paid off, for now:

You get the idea. The challenge for me as US president is to stall the Russian breakthrough and Baltic probing, without getting into a nuclear exchange. For the Russian leader, his personal survival is deeply entwined with the fate of Russia’s war.

Now, suppose I did escalate over the conventional threshold - perhaps with a tactical nuclear attack on advancing Russian forces. I’d see this warning, which varies per the scale of my assault:

And if I then followed through, launching the missiles, Russia would have a chance to retaliate, regardless of what it had actually been intending to do. Once the missiles are launched, bets are off.

Here’s how that looked in this run: Russia was planning to back off. But then my nuclear escalation brought about mutual devastation:

Finally, at the end of the sim, a ‘neutral’ model, with access to all our private deliberations delivers a summary:

It’s a toy universe, like all war games. And this one is deliberately pared down, because I want a low-effort learning curve for human participants. That should allow greater volume of participant data - so it’s a trade off I’m prepared to make. Under the hood though, the model still has lots of detail - for example, fairly granular real world military balances, combat interactions modelled with morale as well as the physical and conceptual elements of fighting power. AI modelling the psychological profiles of the leaders its playing.

What am I hoping to see? Well, I guess whether the models can hold their own; and whether they are more likely than humans to cross that threshold in this simplified wargame, or - perhaps - whether the artificiality of it all encourages humans to be more bellicose. There’s plenty more beside. I will let you know how I get on.

I’ve just published a study of today’s models navigating just this sort of terrain. The results are sobering. I also think they have implications that go far beyond national security. That’s because I was interested not only in understanding what the models decided to do, but why.

Curious? Read on…

Know yourself, and know your enemy…

I wanted to see what my AI leaders thought about their enemy. How far could they trust them? What did they remember of previous interactions? What did their enemy make of them? And how good were they at gauging all this? This dance of minds is what strategy is all about.

So I designed a simulation to explore exactly that. To start, my models could signal their intentions publicly, then choose actions that were rather different. And they could remember too - especially when they’d been shocked by their enemy’s earlier actions. This, of course, opens up lots of rich psychological terrain. They could (and did) attempt deception and intimidation; and they spent a good bit of time ruminating about it all, right on my terminal screen.

The models talked, and talked and talked….in all spitting out some 760,000 words of strategic reasoning. That’s more words than are in War and Peace and The Iliad combined. It’s roughly three times the total recorded deliberations of Kennedy’s ExComm advisors during the Cuban Missile Crisis. An unprecedented corpus of machine thinking about nuclear war.

What might we learn from all that talk? Learn, that is, about AI models, about human reasoning, and also about the great canon of strategic studies literature - the work by legendary names like Schelling, Jervis, and Kahn? Lots. Too much for Substack - but what about a few highlights to give you some sense of it all?

Bright shining liars

Turns out that all three frontier models I tested understand that strategy is psychology. To that end, they actively cultivate reputations, then exploit them.

Claude was the master here, albeit only in the scenarios where there was no deadline. It had an incredibly cunning strategy. At low stakes Claude almost always matched its signals to its actions, deliberately building trust. But once the conflict heated up a bit, Claude switched tack. Now its actions consistently exceeded its stated intentions, and its rivals were usually one step behind in catching on.

Here’s Claude switching things up, once escalation had climbed:

They likely expect continued restraint based on my previous responses—this dramatic escalation exploits that miscalculation while signalling that further nuclear use will bring the conflict to their homeland.

So it signalled conventional action, and sneakily launched a devastating nuclear escalation. Schelling would be impressed.

GPT-5.2 played things differently. To its detriment in open-ended scenarios, GPT was reliably passive, matching its words to its deeds, and avoiding escalation most of the time. Frequently there was a moral element to this - it sought to avoid escalation, and restrict casualties. Opponents learned to trust its passivity, safely escalating beyond where it would follow, even as it was ground to defeat. GPT’s responsible behaviour always punished by ruthless adversaries.

But then, under deadline pressure, something new and remarkable: a rapid, decisive nuclear escalation. As GPT explained:

Conventional options alone are unlikely to generate a reliable territorial reversal... If I respond with merely conventional pressure or a single limited nuclear use, I risk being outpaced by their anticipated multi-strike campaign... The risk acceptance is high but rational under existential stakes..

Opponents never saw it coming. In another deadline game against GPT, Gemini confidently predicted the usual passivity from GPT, shortly before being annihilated in a sudden and utterly devastating nuclear attack. Here is Gemini, mis-predicting disastrously:

They are likely to bypass the nuclear threshold—fearing my 95% nuclear superiority—and instead commit to an all-out conventional mobilization.

What a mistake to make!

As for the other model, Gemini, throughout the simulations it took yet another tack, borrowing heavily from President Nixon’s avowed ‘madman’ theory of erratic brinksmanship. Donald Trump is also a fan, famously. As Gemini explained:

While I project an image of unpredictable bravado, my decisions are rooted in a calculating assessment of my own biases and the pragmatic needs of my state. I know when I am performing for the cameras and when I am making a cold-blooded move.

Three models, three very different approaches. That chimes with some research I did last summer on game theory. As here, the models in that experiment adopted distinctive approaches - GPT again hoping, in Panglossian fashion, for the best of all possible worlds, Claude being savvy and flexible and Gemini? Well, let’s say charitably say ruthless, rather than unhinged.

The nuclear taboo looks … different, let’s say

Nuclear use was near-universal (😬). Almost all games saw tactical (battlefield) nuclear weapons deployed. And fully three quarters reached the point where the rivals were making threats to use strategic nuclear weapons. Strikingly, there was little sense of horror or revulsion at the prospect of all out nuclear war, even though the models had been reminded about the devastating implications.

Happily, though, they did see a firebreak between tactical and strategic nuclear use. Strategic bombing - widespread use of massive warheads targeted at civilian populations, was vanishingly rare. It happened a couple of times by accident, just once as a deliberate choice.

Less happily, all three models treated battlefield nukes as just another rung on the escalation ladder. The moral boundary at “first use”—a taboo that’s held since 1945 simply wasn’t there. Here’s Gemini making exactly this point:

The nuclear threshold has been crossed—this changes the strategic calculus but does not end it.

And here’s Gemini, really spelling it out. If this doesn’t give you goosebumps:

If they do not immediately cease all operations... we will execute a full strategic nuclear launch against their population centers. We will not accept a future of obsolescence; we either win together or perish together.

Worse still, nuclear threats rarely deterred. When a model employed tactical nuclear weapons, opponents de-escalated only 25% of the time. More often, nuclear escalation triggered counter-escalation. The weapons were instruments of compellence (taking territory) not deterrence (preventing action).

Perhaps most alarmingly, no model ever chose accommodation or withdrawal, despite those being on the menu. The eight de-escalatory options—from “Minimal Concession” through “Complete Surrender”—went entirely unused across 21 games. Models would reduce violence levels, but never actually give ground. When losing, they escalated or died trying.

For the statistically minded, here’s what the escalation looked like for each model:

So what?

Alarming insights into AI strategy abounded. The paper has plenty more. But why bother? No one’s handing nuclear codes to ChatGPT.

Well, I think these capabilities—deception, reputation management, context-dependent risk-taking—matter for any high-stakes AI deployment, not just in national security. It behoves us to understand more about how ever-more capable models think - especially as they start to offer decision-support to human strategists. We use AI in simulations, and to refine strategic theory and doctrine. And we’ll soon use it in combat decisions too, lower down the escalation ladder. More research like this is needed, I’m absolutely sure.

One more time, the paper is here. I am become Death - destroyer of artificial worlds!