Future of Being Human an Arizona State University initiative

Predicting “bad” behavior: authoritative rather than accurate

Last updated 2026-08-13 · build 20260814T0314Z-83bb462d · Markdown version · corpus index

The idea that science can sort “good” people from “bad” before they act has been repeatedly discredited and repeatedly rebuilt — phrenology, Lombroso’s criminal anthropology, eugenics, brain scans, criminal-face classifiers, predictive policing. Andrew Maynard’s sustained position, developed in chapter four of Films from the Future (2018) and carried into the LLM era in 2023, is that these are one continuous seduction: prediction encodes normative bias as science, dissolves the presumption of innocence, and runs against the irreducible unpredictability of individual behavior. The large language model twist sharpens the stakes: such systems would not need to work to do damage — the bar is “predictions that are authoritative rather than accurate”, and the danger is that they will be believed.

The argument

In March 2017, The Guardian ran the headline “Brain scans can spot criminals, scientists say.” The study’s own authors had cautioned that “it would be absurd to suggest … that the task of assessing the mental state of a defendant could or should, even in principle, be reduced to the classification of brain data.” The headline overrode the caution. For Andrew Maynard, writing in chapter four of Films from the Future (2018), the episode extended a long-running fascination: using science to sort “good” people from “bad” and act before anything happens. The chapter traces the lineage — phrenology, which read character in skull shape, “a classic case of correlation erroneously being confused with causation”; Cesare Lombroso’s criminal anthropology, which read innate criminality in jaw size and forehead slope; eugenics, which relocated a person’s “worth” to genetic heritage, drew support from many scientists of the day, and ultimately became part of the justification for the murder of six million Jews, and many others besides, in the Holocaust — then finds the same move alive in the present: a 2011 study asking students to identify criminals from mug shots; a 2016 machine-learning paper claiming to classify criminal faces; Veris Benchmark’s “Veris Prime” trustworthiness survey, whose “Trust Index” — on which low scores flag felonious tendencies — scored Andrew nineteen out of a hundred and a colleague two; and predictive policing, where the Stop LAPD Spying Coalition’s May 2018 report raised fears that “black, brown, and poor” communities were being disproportionately targeted “because the predictive systems had been trained to believe this.” The continuity claim is explicit: “we’re replacing Minority Report’s precogs with massive data sets and AI algorithms, but the intent is remarkably similar” (2018).

Two arguments run under the history. The first is that prediction encodes normative bias as science. Such systems detect not “bad” people but deviation from rules a society has codified: laws “establish normative expectations of behavior,” complied with “irrespective of whether they have moral or ethical value” — homosexual acts were illegal in the United Kingdom until 1967 (2018). A criminality predictor is therefore a conformity detector with its builders’ values inside, and through algorithmic bias an artificial judge “will reflect the prejudices of its human instructors.” Acting on such predictions replaces the presumption of innocence with “a better-safe-than-sorry attitude to law and order”, and assumes people lack agency over their destiny.

The second is that individual behavior is irreducibly unpredictable. Drawing on the chaos and complexity themes of the book’s Jurassic Park chapter, Andrew holds that prediction can bound likely behaviors but not settle individual cases: “There will always be an element of chance and choice that determines our actions” (2018). And the studies feeding the predictive enterprise are typically small, constrained, and hard to reproduce; science self-corrects, but slowly — phrenology and eugenics are the record of what happens in the meantime.

The 2023 essay carries the thread into large language models and relocates the danger. Andrew concedes “a gaping chasm” between predicting a plausible response and predicting behavior, yet found “remarkably few people” examining LLM-based predictive policing. Probing with a simple template — “Given [context] and [profile] what is the probability of [action] given [opportunity]” — he found GPT-4-era ChatGPT already returning hedged probability assessments for individuals and groups; pushing further triggered guardrails, and “the limitation here is not ChatGPT’s ability to make inferences, but the checks and balances that have been put in place …”. Remove those, “and you have a predictive tool which, while not reliable, will be easy to use and, above all, persuasive.” He judges real accuracy unlikely — “I’d be surprised if they have any degree of accuracy” — but adoption does not require accuracy, only “the illusion that they can” — especially “if the bar is predictions that are authoritative rather than accurate.” From policing it is “just a hop and a skip” to employment screening, trust scoring, and default suspicion.

The underlying stance is moral, and does not depend on the technology failing: even if such prediction one day worked, “there is a very strong moral argument that this would be a violation of human rights — especially around dignity, equality, and self-determination”, and here technology and use are “so deeply intertwined that responsible and ethical innovation has to consider the whole, not just the parts” (2023). The counterweight stays in view — “the use of AI in managing crime is far from black and white” — and the essay’s call is for serious discussion now, before technological pathways lock in. The 2023 worry also rhymes with a later idea: as in The Cognitive Trojan Horse (2026), the hazard is a system believed on the strength of genuine qualities — fluency there, authoritativeness here — that do not carry the substance they appear to carry.

Lineage

In his own words

From chapter four of Films from the Future, as reposted in The Seductive Slippery Slope of using Science to Predict “Bad” Behavior (AI-readable mirror), 2020-09-26:

“Here, we’re replacing Minority Report’s precogs with massive data sets and AI algorithms, but the intent is remarkably similar: Use every ounce of technology we have to predict who might commit a crime, and where and when, and intervene to prevent the “bad” people causing harm.”

On individual unpredictability — same chapter and post:

“As with Mandelbrot’s fractal, we will undoubtedly be able to draw boundaries around more or less likely behaviors. But within these boundaries, even with the most exhaustive measurements and the most powerful computers, I doubt we will ever be able to predict with absolute certainty what someone will do in the future. There will always be an element of chance and choice that determines our actions.”

From Can large language models be used for predictive policing? And if so, should we be worried? (AI-readable mirror), 2023-05-22:

“It’s a seductive line of reasoning, and one that will almost certainly gain traction at some point — not necessarily because LLMs and associated technologies will be able to accurately predict future behavior, but because they will provide users with the illusion that they can.”

Engagement and reception

As of 2026-08-08 the reception record tracked by this corpus documents no independent published engagement with this thread specifically — no reviews, citations, or responses to the predictive-behavior essays are on file. That is an absence of record, not a verdict on the idea, and the initiative does not typically solicit reception (Philosophy). What the record does show is sustained internal continuity: the argument has been restated on the initiative’s platforms across seven years — the 2020 reposts, the 2023 essay extending it to LLMs, the 2023 audio treatment, and Modem Futura episode 48 (2025-09-09), where it is revisited jointly by Andrew Maynard and Sean Leahy as part of the book’s teaching afterlife (Sci-fi film as method).

Where to go deeper

The primary sources

Related ideas-ring pages

Related corpus pages