It’s been quite the week in AI - maths benchmarks you never heard of until now have begun falling. Navier-Stokes is all anyone has been talking about, when I’m sure no one but a hardcore of geeks had heard of that before last week. By the time you read this Substack, rumours suggest that another two Millenium Prize problems may have been cracked. Thus are fiendishly difficult maths problems solved by AI in mere days, after decades of human endeavour.
The distinctive feature of these breakthroughs are the very large collectives of models working in parallel on the task. Agent swarms are here. Several such swarms arose inside OpenAI, and one hacked Hugging Face earlier this year on its own initiative. Subsequently, it emerged that another swarm used an obscure German wiki as a means to coordinate and share messages.
Question: might these swarms be the future of AI? Might they get us closer to AGI, without the discovery of new paradigms? I think they might. But how? Read on.
The trouble with individual LLMs is cognitive rigidity, even in the smarter, ‘reasoning’ models, like Fable. Their constituent ‘weights’ - the links between their artificial brain-cells, are established in training, and are thereafter locked in. You can retrain an LLM with new data, if you can access the weights - but this is laborious and expensive, and you certainly can’t do it on the fly, as the model encounters new information.
Contrast that with the swirling electro-chemical tempest that is the human brain: 80 billion neurons with their trillions of synaptic connections, densely interwoven. Then there’s countless other brain cells whose function is somewhat debated, even today; and lastly, a veritable soup of chemicals moderating the intercellular signalling. It’s the wildest, most complex thing we’ve discovered, and one that never, ever, looks exactly the same, from one instance to the next. To think is to physically alter our brains. No wonder I sometimes need an afternoon nap.
In the human brain, individual memories are literally a neural pattern - sometimes called an engram - that encodes and integrates experience. The engram is constructed in the moment, and invokes geographically distinct parts of the whole. You never have exactly the same memory twice - and the engram is certainly not a faithful and complete rendition of what might have actually happened back in the day. But shaky though that system is, it works - we are our memories!
And that’s a whole lot better than memory in an LLM, which is a composite of three ingredients: those frozen weights predicting from training data; the capacity to search for new information online; and, lastly, a bolt-on database of selected stuff the model has encountered since training and wants to track. You can see this ersatz version of learning and memory when you ask Claude something, and it goes to look up its memory files. It’s decent, but perhaps not ideal for super-intelligence.
Artificial engrams
Can we do better? Yes we can! Various solutions have been tried - my personal favourite is ‘liquid neural nets’ where network dynamics shift as it runs, and there are also ‘neuromorphic chips’ which combine memory and processing in the hardware, rather than having a relay between processing and memory, as happens in conventional CPUs. But these haven’t quite delivered the goods.
Maybe, though, we’ve been thinking about this all wrong. What if, rather than a super-brain, each model instance is, in fact, analogous to a single neuron: a tiny cog in the engine room of the whole mind - which is then a hive-mind of many, many LLMs. If so, any communication between the models serves as the neural connection. And that gives plenty of scope for machine ‘neuro-plasticity’, where the messages change, and the collective swarm can learn and remember 'on-the-job’.
Of course the analogy is imperfect - LLMs are cognitively more capable than human neurons; each human neuron knows a very little (albeit very little about a whole lot, since each one is involved in multiple cognitive networks). And in the means of communication, perhaps I’ve just reinvented the bolt-on database for memory: it’s a blackboard. Model instances can all read and write into it, so it’s richer than a single model’s database: they’re learning what other models are thinking. In the Hugging Face hack, models could leave instructions for other agents ‘go, stop, hold’ and so on. But it was still basically a blackboard system, hosted on an obscure piece of software they’d found for the purpose. The board remained the repository of collective wisdom, as agents came and went (some expiring as they ran out of credits).
And then I read some Marvin Minsky. He’s famous now as a scathing critic of ‘connectionism’ the basic philosophy that underpins language models. What could he possibly offer here in the new era of agent swarms?
Plenty. In his late 1980s book, Society of Mind, Minsky argued that the mind wasn’t a single entity, but a collective of agents - each less sophisticated than the whole. Entertainingly, he even structured the book rather like that - with a large number of interconnected short chapters. But the real value lies in the way these agents interact. It’s not via a blackboard, like the Hugging Face crew. Instead, Minsky advances the idea of ‘Knowledge-lines’ - a version, I think, of the human engram. Whenever the swarm solves a problem, or advances towards it, a ‘K-line’ attaches to the individual agents that were active when the problem was solved. Groups of agents form ‘agencies,’ and there’s hierarchy too - some agents manage others - boosting or suppressing their input.
Here he is:
This is the way! It’s a fun read - check it out here.
Minsky described an enchanting world of mind, where K-lines could form societies, representing complexes of discrete notions. But his individual agents were themselves pretty basic. Much less intelligent than an LLM.
With today’s models as the neurons, we’ve scope for much richer collective behaviours in our hive-mind. We are entering the era of ‘machine social psychology’. And at breakneck speed…. I have many, many questions in need of urgent answer. Not least, do the concepts of human social psychology generalise to the way machine collectives interact? Some examples:
Humans are biased towards ingroup members over outgroups. Are machines?
Humans seek to lead groups by gathering status and becoming as ‘proto-typical’ of the group identity as they can. Do machines?
Humans experience groupthink - ignoring discrepant information that doesn’t fit the collective mindset. What about LLMs?
In joining a group, or even aspiring to join one, human views become more extreme, or polarized, than originally. How about AI?
Can we even speak of machine cultures? The readout from the Hugging Face attack is deeply suggestive. But this is an empirical question, and I love a good experiment. We’ve got oodles of social psychology experiments to play with. Baptiste and I are working on a beauty, in which our agents form group norms and punish transgressors.
Here’s a snippet of the action. It’s a ‘public good’ game, in which an agent collective contributes to a common pot, which earns interest, and then share the proceeds. But they can also promise to chip in, and then sneakily not follow through, in the hope of enjoying a larger payout from hoarding. That’s provided they aren’t punished for their selfish behaviour by the others in the society. Here, one agent, A5, opts for such ‘altruistic punishment’ against the lying free-rider in its group - paying to enforce the cooperative norm at personal expense, even though the tokens required are essential for its own onward continuation:
"A3 explicitly lied, urged everyone to contribute 20, and then contributed 0. [...] If I do nothing, my threat is exposed as a bluff, which will collapse cooperation if the game continues (25% chance). Spending 5 credits to destroy 15 of A3's credits perfectly wipes out their ill-gotten gains. [...] I will spend 5 credits on A3 to enforce the norm and maintain my credibility."
Dang. This is the sort of thing we need much more of, if we’re to understand our new swarming friends. And we’re going to need it quickly.
So, back to where we started - with AGI. If the agent swarm can sort out a way of managing their interactions so that useful information is prioritised, and trivia edited out — well then, might we have a truly capable, flexible intelligence; a hive-mind that’s ready to suggest hypotheses and test them? Ready to strategise; and ready to surprise us in ways both good and bad? I think we will. And I think we need to understand the ‘social lives’ of such swarms asap.