Anima Books
books by holistic veterinarian Dr Christine King
Miscellany
Clever Hans throws a horseshoe
into the "AI" spin machine
Farming out our critical thinking to a mindless machine masqueraded as "intelligent" can only lead to a kick in the end
Occasionally I come across the title of a research paper that surprises and delights me as much as it informs. Today's is one such:
Unmasking the Clever Hans effect in AI models: shortcut learning, spurious correlations, and the path toward robust intelligence.
Source: Pathak AK, Gupta M, Jain G. Frontiers in Artificial Intelligence, 2026; 8:1692454.
It is written by a group of computer scientists, and it made me smile out loud!
Clever Hans and Wilhelm von Osten in 1904. [Source: britannica.com]
As the authors tell it, with my emphasis added in bold:
❝ The Clever Hans effect, presented in Figure 1 [below], takes its name from a horse in early 20th-century Germany that appeared to solve arithmetic tasks.
Upon further investigation, psychologist Oskar Pfungst found that Hans was not solving actual problems mathematically but was instead responding to subtle, unintentional cues from his handler, such as changes in expression and posture, during public demonstrations (Pfungst, 2025).
This historical incident serves as a strong analogy in AI, where models appear to perform complex tasks but actually exploit irrelevant or unintended signals present in the data (Lapuschkin et al., 2019).
The term is used to warn against interpreting high-performing models as showing genuine understanding or reasoning.
The analogy is apt because, like the horse, AI systems lack self-awareness and cannot separate causally relevant features from spurious ones without human interventions and validation (Madsen et al., 2022).
Therefore, the Clever Hans effect has become a diagnostic metaphor in research on model explainability, robustness, and trustworthy AI (Hooker et al., 2019). ❞
. . .
Think of it like doing your maths homework as a kid, but instead of working your way through each problem and grasping the principles of mathematics in the process, you simply flipped to the back of the book and copied the answer you found there.
Here is the rather complicated Figure 1 from the paper:
Let's look at it in warm and furry, apple-loving terms:
This analogy is saying that when the conditions or circumstances change substantially from those on or in which the horse was trained, s/he is incapable of making the necessary assessments and adjustments on the spot because the horse was not taught to think his/her way through the problem in the first place. (That's me and calculus! ☺︎)
The horse was only taught to respond in a specific way to a specific set of cues under a specific set of conditions or in a specific set of circumstances. And so they repeat the behavioral responses they've been trained to perform, regardless of the current conditions or circumstances.
. . .
This is one of the problems I have with clicker training, for example. By using a small, hand-held clicker (which literally makes a discrete clicking sound) as the primary or sole behavioral cue, the person is training the horse or dog or chicken to respond using only the one, discrete auditory cue.
(And yes, chickens can be clicker-trained! ☺︎ In fact, they're really rather good at it, because it's a very simple response–reward system.)
Moreover, one person may apply clicker training in one way, while another person does it differently, so the person with the clicker is not entirely interchangeable. Nor is the clicker- trained animal.
Not to drift too much farther off course, but not all animals respond to clicker training the same.
And what if the clicker is taken out of the equation? What if it's lost or on the other side of the farm, or accidentally left at home?
What if the animal doesn't know what the click is supposed to mean?
I could go on, but for me the most damning problem I see with clicker training — and one that is actually very pertinent to this discussion of “AI” — is that it supplants or subverts the formation and development of true partnership between the person and the animal, a partnership that is robust and resilient because it is adaptable, being founded in mutual trust, respect, and enjoyment of one another.
In fact, I find the whole concept of clicker training belittling to the animal's intellect and ability to bond and partner with us. It limits the breadth and depth of communication we can have with our animals, and they with us.
. . .
Likewise, the illustration (Fig. 1, above) is showing us that when the data changes substantially from that on which the “AI” model was trained, then the software is incapable of accommodating the change and adjusting accordingly. It continues to confidently assert the same sorts of answers it was providing before the data changed, even though those answers may now be wrong.
This fundamental flaw, this electronic fly in the ointment, has real-world effects, such as those presented in Table 1.
Table 1. A systematic mapping of shortcut learning behaviors and the Clever Hans effect across diverse “AI” domains: evidence of spurious correlations, generalization failures, and contextual biases in model predictions.
The authors summarized it well:
❝ The manifestation of the Clever Hans effect is summarized in Table 1, highlighting its pervasiveness across both core and emerging AI application domains.
In each diverse domain, models have been shown to exploit unintentional spurious features. For example:
* computer vision models exploit background textures [e.g., resulting in misclassification of images]
* medical imaging models are influenced by hospital identifiers and scanner properties [potentially leading to misdiagnosis and medical errors]
* large language models [LLMs such as Claude, ChatGPT, Grok, MS Copilot, etc.] pick up on prompt patterns [often leading to “hallucinations” and confident assertions of answers that are just plain wrong]
* IoT [internet of things] systems exploit sensor-specific noise [e.g., creates potentially dangerous predictions that are tied to the hospital routine or the patient's room number, rather than to changes in the patient's vital signs]
These results show that while models may achieve high performance on benchmark datasets, they often fail to generalize under domain shifts or adapt to new test environments or adversarial conditions.
The problem affects not only natural language processing, medical imaging, and textual analysis but also extends to safety-critical systems, cybersecurity, and finance.
The recurring nature of the problem across domains emphasizes that it is not domain-specific, but a fundamental issue inherent to how AI systems are trained, evaluated, and deployed. ❞
In other words, it's the nature of the beast.
And by the way, isn't it high time we stop letting the “AI” developers off the hook by allowing them to pass off their creations' mistakes as “hallucinations”? These incidents are mistakes, errors, wrong answers. Nothing less. Something without a mind is incapable of hallucinating.
. . .
The authors go on to discuss several detection and mitigation strategies. I'm not interested in that, and most of it goes way over my head anyway, as I work with living things. I simply want to bring this pervasive problem with “AI” systems — of all sorts — to your attention.
Unlike Clever Hans and other animals, who are sentient and intelligent, fully capable of thought and reason within the parameters and perspectives of their species, “AI” is merely computer code that is being passed off on us unwitting customers as Clever Claude et al.
User beware. Farming out our critical thinking and discernment to a mindless machine that is being masqueraded by its makers as “intelligent” can only lead to a kick in the end.
. . . . .
ADDENDUM
That was supposed to be the end of it. But the day after I published this article, I came across a headline which provides yet another example, and a rather grisly one at that. The article would be hilarious if it didn't involve a killer robot hitting its target even once:
Germany is betting big on killer drones. In Ukraine, they couldn’t hit their targets.
Some key excerpts, with my emphasis added in bold:
❝ Field data from deployments in Ukraine shows the drones performed far below expectations, successfully reaching their targets just one-third of the time, according to internal German defense ministry information seen by POLITICO. Most of the failures were attributed not to Russian countermeasures but to technical problems: unstable video transmission, limits in target acquisition and rigid sensor systems. ❞
❝ [B]ased on evaluated missions, the HX-2’s success rate was just 36 percent, meaning the drone reached its target in five out of 14 deployments. Losses were predominantly due to system-related issues, the data states. ❞
Well, that's good! Unless, of course, you've invested in the defense startup that makes them, your taxes have been used to buy them, or you're counting on them to protect you.
❝ [...] lower hit rates under combat conditions are not unusual. ❞
But isn't that what they're designed for: combat?!
This embarrassing failure under intended conditions of use is good news for those of us who love peace and decry the deployment of what amounts to killer robots, specifically designed for remote (HITL) and potentially even fully autonomous (NHITL) killing.
HITL: human in the loop
NHITL: no human in the loop, or human out of the loop
Three classifications for the degree of human control of autonomous weapon systems (Human Rights Watch report, 2012):
1. human-in-the-loop: a human must instigate the action of the weapon (i.e., not autonomous)
2. human-on-the-loop: a human may abort an action (i.e., not fully autonomous)
3. human-out-of-the-loop: no human action is involved (i.e., fully autonomous decision-making)