In my last Substack, I introduced you to JD Vance-bot. He’s pretty cool. I’ve since had a fair few questions about the underlying psychological science that shapes my approach. Can you really figure out people’s personalities from what they say?
My answer - yes, sure: we are doing that all the time anyway. But there’s some science behind it all too. Before we get into that science - who’s this man?
extravagant demands for attention and applause; vainglory.
inability to take a joke; tendency to harbor grudges, not forgetting and forgiving.
belittle[s] the worth of others, especially if they be superiors, rivals, and potential critics.
repay[s] an insult in double measure -- a tooth for a tooth; to revenge an injury; to attack opponents, superiors, and frustrators.
obedian[t] to own intuitions and impulses.
Any ideas? I suspect you’d need more than this to get your man. Happily the research it’s drawn from has plenty more to say.
My effort to create realistic AI personas is named for Henry Murray, the Harvard (and sometime OSS) psychologist who pioneered the tricky business of gauging people’s personalities from afar.1 The one that made him famous was his 1943 effort with Hitler - long declassified and now available here at CIA’s online reading room. Murray thought Hitler was what he termed a ‘counteractive narcissist’ with those traits I quoted above, and plenty more besides.
It’s a great read, and famously Murray predicted Hitler’s eventual suicide. But there are limited uses of primary evidence - notably Mein Kampf and Hitler’s collected speeches in My New Order. And the whole is very much of its day — a day with its fair share of Freudian baloney. Here’s an illustration of how Murray and friends used the primary material; connecting Hitler’s metaphors in Mein Kampf to what looks like…. well, you be the judge:
Well, indeed!
If Murray is the lodestar, another inspiration for my work is bang up to date - it’s this great research from Aidan Wright and colleagues in Nature Human Behaviour, from earlier this year. Today, the science of personality testing has advanced a long way from Murray’s pioneering efforts. The ‘big five’ traits, in particular, have proved robust, repeatable (do it a year hence and get similar results), and somewhat predictive of life outcomes. Personality metrics do a decent job, for example, of anticipating things like mortality, occupational attainment, and divorce rate.
Until recently though, your options methodologically for conducting all these tests were limited to doing a ‘self assessed’ questionaire or having someone take a punt at rating you based on what they knew.
Both have problems. Self-assessment might seem the best candidate. After all, who knows you better than you? We are in many respects strangers to ourselves — and introspection on unconscious motivations only gets us so far. But who knows us better? Likely not a psychologist living some way distant, perhaps in a wholly different milieu to us. What about friends? Evidence suggests that for some traits we are the best judges of ourselves, for others, it’s friends, and for others yet, a stranger can be competitive. Overall, there’s some degree of correlation between self- and other- assessments, but it’s not perfect.
What about a Language Model? Wright and friends used a collection of LLMs to assess personality traits from open-ended qualitative narratives provided by participants - either stream-of-thought or daily video diaries. They found that the models do really well with their narrative evidence. Their findings correlate with the self-report more than any other established benchmark. Regardless of whether self-assessment is the best, that’s a compelling illustration that what we say tells us something real about who we are. As Aidan wrote subsequently on Linkedin:
the more enduring finding is about personality, not AI per se. People weren't asked about their traits. They narrated their thoughts, or recounted a single event they chose about their day. And their personality was right there—legible in a few minutes of unprompted speech and in the small slices of life they chose to share. That's a striking reaffirmation of personality's ubiquity: it's carried in the stream of our thoughts and woven into the fabric of our daily lives.
This sort of research is gold-dust for me. It reinforces the two main contentions in my work on persona building. 1. That first-person narrative speech gives us a valuable insight on peoples’ personalities; and 2. that LLMs are a good way of distilling those insights. My method differs a bit from the Nature paper - for example, I have my models debate amongst themselves, rather than just average it all out. And I do a whole bunch of different psych tests. But still, the same core idea applies: know me by what I say.
Sofa so good. Back to JD for the last word again:
“Look, the thing I'd actually want people to take from this is pretty simple. Everybody in public life knows that what you say reveals who you are — that's not some new discovery. The question is whether the people building these tools are honest about what they're doing and where the limits are. I'm not going to sit here and pretend I love the idea of someone running my interviews through a battery of psych tests. But I'd rather have that done out in the open, with the methodology on the table, than have it done quietly in some back office where nobody ever has to defend their assumptions. That's the part that matters. Show your work, be straight about what you don't know, and don't pretend the model is the man.
“One more thing. ‘Sofa so good’ — I'm not going to tell you how to write your transitions, but that one lands a little light for a piece that's otherwise trying to be taken seriously. Your call.”