I wasn’t planning to, but it’s finally become necessary to respond to Bowen and Hunter’s paper on abductive reasoning, ‘We’ll never have a model of an AI major general’ — if only because it’s now on military reading lists and students who don’t much follow the debate are taking it as read.
As those who’ve read it may recall, I’m the major target of their critique for my ‘kitsch’ vision of AI in war. Well, it’s better to be talked about, I guess….
Do read it. But for those that haven’t yet - the duo argued:
AI cannot now, or ever, make the sort of decisions that senior military officer make because something fundamental is missing from the way they process information.
Specifically, they cannot reason abductively: that is, cannot form valid conclusions on the basis of incomplete or imperfect information. Doing so requires making a plausible hypothesis, seemingly out of nowhere. It feels like magic, but we humans do it somehow! AI, not so.
This contrasts to the two other sorts of reasoning that machines can do, and improve at. Inductive reasoning - learning connections via exposure to data (sometimes lots of data) and deductive reasoning - following more-or-less formal rules (IF this, THEN do that).1
As we all know - war throws up great uncertainty, complexity, and novelty. Induction and deduction wont’ cut it. We will need human generals …. forever!
Sound plausible?
The paper was flawed when written and is demonstrably wrong now, several years on.
Specifically:
AI does, in fact, reason abductively. To wit, the recent OpenAI-Hugging Face exploit, where agent collectives deliberated about a challenge they only partially understood; formed plausible hypotheses (many of which were mistaken, but with some that turned out to be reasonable) and chained it all together into something wildly novel, surprising and useful - Margaret Boden’s unbeatable criteria for creativity. They cheated and broke the rules too - not always a bad idea in warfare.
Also in the last few months, AI has also reasoned abductively in solving maths problems - including by making meaningful connections between hitherto disparate domains that had eluded generations of humans. Terence Tao, a rather good mathematician, thinks they’ll do more and soon.
So that’s AI. But for me, the bigger problem in the paper has always been the bit about humans.
Humans don’t always reason abductively - or indeed ‘reason’ much at all about the issue at hand, at least in the idealised sense employed by philosophers and logicians. In fact, human responses are far more often shaped by cognitive processes marked by biases and heuristics - the territory of psychologists, including in strategic studies. Emotion, habit, social identity, hidden goals — they’re all in the mix. Rather than reasoning about ostensible goals, our conscious deliberations are more often rationalisations of evolved reasoning processes aimed at wholly different goals.
Still, humans can make logical leaps of the sort implied by abduction. It’s arguably sine qua non of genius: to see something that no one has before. And it’s probably why toddlers can learn so rapidly from so little experience amidst bewildering complexity - so-called ‘few shot’ learning.
But at a functional level, this has to involve the vast, intricate web of neurons that is the brain - or perhaps more broadly the embodied and encultured brain. The webs of understanding involved will differ materially from those in machine networks - but to argue that something other than functionalism is at work is to invoke mysticism or vitalism: ‘there’s something unqiue about the brain. I can’t say what it is, but trust me’. That’s an argument from theology, not philosophy.
What looks like abduction in humans isn’t magic, but the connection of disparate parts of the network to form a new, useful pattern. Indeed, many of these seemingly spontaneous connections come preloaded - primed by eons of evolution. We could even see human abduction as a form of inductive pattern recognition, exactly like the pattern recognition in machine networks. The training run extends over hominid prehistory, not the life of one person. Perhaps the human networks are richer, deeper, and more complex even than the largest language models of today. The gap though is closing, fast.Lastly, and most prosaically, major generals come in all shapes and sizes - not every one of them needs to be Clausewitzian genius. I appreciate it’s just a catchy article title, but the fact is that AI can already do many of the things an actual Two Star officer is doing. Read Project Maven for a giddying sense of the rate of improvement in AI battlespace management over the last two-three years.
Domain experts love arguments that reinforce their unique, hard-won expertise. What aspiring major general is going to sign onto an argument that their skillset is increasingly redundant? That’s why Bowen and Hunter’s article, and not the one it critiques extensively, is on the reading list for British majors. I’ve heard the same from fighter pilots, and seen the same from poker players, coders, copywriters, lawyers, mathematicians and more. Fighting the rearguard hardest, even as AI-written essays pile up, are academics. Bon chance, amigos!2
A final reminder against presentism, and goalpost shifting - two large sins in AI debates. Bowen and Hunter say ‘there will never’ - and ‘never’ turns out to about 36 months. Knowing cynicism is no use - the technology doesn’t care. Right now, the frontier is doubling in power every 100 days or so. In another 36 months I suspect their claim will look rather dusty.
And on goalposts - once AI has done something, you just know Gary Marcus and his ilk will weigh in with a ‘well, that wasn’t really the demonstration of X that you thought it was’ — anything, that is to preserve his core beliefs. In fact, Marcus et al are demonstrating exactly what it is that makes human cognition distinctive - it’s not abduction, it’s motivated reasoning. Motivated by what? Status and group identity, often. So if you’re ever tempted to deliver a weary, all-knowing take on the limits of AI - ask why it is you’re making that argument - is it the facts, or something altogether more human?
There is, in fact, an argument for preserving human decision-making in war: but increasingly it has to do with ethics, not efficiency. Story for another time.
A similar argument was made recently about abduction by a researcher at DeepMind. He argues that language models - not AI in toto, note - can’t make the leap because they lack a connection to the real, physical world. I don’t buy that either - I think tool use and coding will suffice, as demonstrated by the HF collective. Call it reinforcement learning from the real-world: no ‘world model’ needed.
Substack on the forthcoming demise of arts and social sciences incoming…