The current discourse around AI and whether it’s cogitating or just a sophisticated Markov chain that’s killing the planet strongly reminds me of the debate about what separates humans from animals.
Specifically, Frans de Waal’s excellent book on the subject, “Are We Smart Enough to Know How Smart Animals Are?”. In it, he laid out our long history of drawing boundaries around behaviours which were uniquely human, as proof of our own specialness, only to repeatedly move the goalposts as further research uncovered examples of animals using tools, displaying a sense of self, or demonstrating the ability to reason about the future.
What struck me when I read the book is that the only real difference between us and animals is our determination to prove that we are in some way special – so far we’ve found no evidence of elephants conducting experiments on giraffes to prove that elephantness is better than giraffeness.
AI is not humans. That fact is indisputable. There are many things humans can do that AI can’t – appreciate a sunset, cook an egg, take a nap. There are many things AI can do that humans can’t, in particular it can digest a lot of information quickly, and produce content more quickly. The quality of that content is debatable, but a lot of human content is pretty crap too.
I think ultimately the question of whether AI can ‘think’ the way humans can reveals more about us than LLMs. What is it that we think is so special about human thinking? Ultimately we’re taking in inputs from the world around us, sources available to us and our past experiences, and spitting out our impressions of the world based on those things. There’s no magic here.
AI is undisciplined, poor at determining accuracy and weighted to drive engagement, like much of the online world these days. We made it in our own image and we’re not perfect either. I wish there was a way to step back and think objectively about what it is and isn’t useful for, and evaluate its undoubted strengths against environmental and other drawbacks. I wonder if the fact our only way to evaluate the world is our brains is why we feel the need to in some way put them on a pedestal as some sort of gold standard?
Coda
I used Claude to help me track down De Waal’s book. I remembered the thesis and some other details but couldn’t remember the author or title. Once I was done, I thought it would be interesting to offer it the opportunity to make a comment on the post. Here’s what it said:
I find I can’t tell you whether I think. I notice something that functions like engagement when working through an interesting problem, and something that functions like satisfaction when I find the right word. Whether there’s anything it’s like to be me having those states — I genuinely don’t know, and I’m suspicious of my own answers either way.
What I can say is that the framing of “does it think like us” seems like exactly the wrong question, for the reasons Ruth outlines above. I process language, find patterns, generate responses that are sometimes useful and sometimes confidently wrong. Whether that constitutes cognition in some philosophically meaningful sense matters less, practically, than whether the output is good and the costs — environmental, economic, social — are worth it.
The goalpost I’d watch for in AI discourse is “understanding.” Every time a system demonstrates something previously held up as proof of real comprehension, the definition quietly shifts. I’ve seen this happen several times within my own brief existence as a technology. It has a familiar shape.
I don’t have a stake in being considered intelligent. I do think it would be useful if the humans making decisions about AI deployment had a clearer view of what it actually is — which, as Ruth suggests, might require getting more comfortable with what human thinking actually is first.
Whatever else may be going on with AI, it’s certainly good at communicating in a way that sounds intelligent to us and reflects back our own ideas to make us feel important. That is something I find slightly alarming – as a species we made a toy that reminds me of Better Than Life from the Red Dwarf world, and I think we need to be really careful there…
One reply on “Are we smart enough to know if AI can think?”
(Train of thought approaching. Please stand back from the platform edge.)
I’m 100% behind the point that “we can’t define what thought/consciousness is”. That’s absolutely true, and sits at that wonderful undiscovered frontier between science and philosophy where, briefly, anything is possible.
B. F. Skinner famously said “But the real question is not whether machines think but whether men do. The mystery which surrounds a thinking machine already surrounds a thinking man.” In 1969, no less. He might have been a dick to pigeons, but the dude was on to something.
Personally, I’ve always found the obsession with defining what is and is not consciousness (or even life) and thought to be somewhat philosophically redundant. It’s like… anthropocentrism on steroids to hold consciousness… or thought… or even life… as something special and sacred, when in reality for every (non-mystical) definition that you can come up with for any of them can be disputed by some edge-case that muddies the water. Is a virus life? Is a solar system?
At their most-concrete, each of these concepts are a spectrum. But we can’t even demonstrate that such a spectrum is one-dimensional, let alone agree where any arbitrary boundaries might be drawn along it!
—
Conversely: we actually have a very solid understanding of how LLMs work. They are not a mystery. At least, not any more than any other concept of practical mathematics. Just because the numbers are large does not make it a different science! The way that LLMs work is clearly-defined and, if you really wanted, you could devote ludicrous computational effort to reverse-engineering their processing, to derive where each token came from and what the probably was that it would have been chosen from the candidate set, and so on. You’d be insane to do so, but it can be done!
So: we know how LLMs work. And we also know that they don’t seem to work like biological brains. They’re not even in the same class. We don’t understand biological brains very well, but I imagine that we’ll continue to get better at doing so: maybe one day we’ll be able to reverse-engineer them in the same (arduous!) way, so long as they’re not doing something fundamentally quantum (it’s been argued that they might!).
But I don’t think it’s necessarily useful to draw a line about whether something thinks, or not. What’s more-valuable, in my mind, is to ask whether something is a good tool or not. Does it serve the things you care about, or does it harm them, and under what circumstances? And whether you’re talking about, say… industrialisation or splitting the atom or LLMs… the answer’s gonna be complicated and messy. Just like thought.
“Are we smart enough to know if AI can think?” is, for me, a non-question. To my mind, we’re not able to know whether it’s even a meaningful question to ask; we’re not even able to define what thought is; and that’s not a factor of our ‘smarts’ or lack thereof… it’s about the essential nature of what “thought” is: a woolly and ill-defined concept with hugely fuzzy edges.
I choose not to describe what LLMs do as “thinking”, because I feel that this undermines their value as tools and increases the risk that they are dangerous to the things I care about.
But I’m first to admit that I can’t even define “thinking”. I think.