# Predicting "bad" behavior: authoritative rather than accurate

The idea that science can sort "good" people from "bad" before they act has been repeatedly discredited and repeatedly rebuilt — phrenology, Lombroso's criminal anthropology, eugenics, brain scans, criminal-face classifiers, predictive policing. Andrew Maynard's sustained position, developed in chapter four of *Films from the Future* (2018) and carried into the LLM era in 2023, is that these are one continuous seduction: prediction encodes normative bias as science, dissolves the presumption of innocence, and runs against the irreducible unpredictability of individual behavior. The large language model twist sharpens the stakes: such systems would not need to work to do damage — the bar is "predictions that are authoritative rather than accurate", and the danger is that they will be believed.

## The argument

In March 2017, *The Guardian* ran the headline "Brain scans can spot criminals, scientists say." The study's own authors had cautioned that "it would be absurd to suggest … that the task of assessing the mental state of a defendant could or should, even in principle, be reduced to the classification of brain data." The headline overrode the caution. For Andrew Maynard, writing in chapter four of *Films from the Future* (2018), the episode extended a long-running fascination: using science to sort "good" people from "bad" and act before anything happens. The chapter traces the lineage — phrenology, which read character in skull shape, "a classic case of correlation erroneously being confused with causation"; Cesare Lombroso's criminal anthropology, which read innate criminality in jaw size and forehead slope; eugenics, which relocated a person's "worth" to genetic heritage, drew support from many scientists of the day, and ultimately became part of the justification for the murder of six million Jews, and many others besides, in the Holocaust — then finds the same move alive in the present: a 2011 study asking students to identify criminals from mug shots; a 2016 machine-learning paper claiming to classify criminal faces; Veris Benchmark's "Veris Prime" trustworthiness survey, whose "Trust Index" — on which low scores flag felonious tendencies — scored Andrew nineteen out of a hundred and a colleague two; and predictive policing, where the Stop LAPD Spying Coalition's May 2018 report raised fears that "black, brown, and poor" communities were being disproportionately targeted "because the predictive systems had been trained to believe this." The continuity claim is explicit: "we're replacing Minority Report's precogs with massive data sets and AI algorithms, but the intent is remarkably similar" (2018).

Two arguments run under the history. The first is that prediction encodes normative bias as science. Such systems detect not "bad" people but deviation from rules a society has codified: laws "establish normative expectations of behavior," complied with "irrespective of whether they have moral or ethical value" — homosexual acts were illegal in the United Kingdom until 1967 (2018). A criminality predictor is therefore a conformity detector with its builders' values inside, and through algorithmic bias an artificial judge "will reflect the prejudices of its human instructors." Acting on such predictions replaces the presumption of innocence with "a better-safe-than-sorry attitude to law and order", and assumes people lack agency over their destiny.

The second is that individual behavior is irreducibly unpredictable. Drawing on the chaos and complexity themes of the book's *Jurassic Park* chapter, Andrew holds that prediction can bound likely behaviors but not settle individual cases: "There will always be an element of chance and choice that determines our actions" (2018). And the studies feeding the predictive enterprise are typically small, constrained, and hard to reproduce; science self-corrects, but slowly — phrenology and eugenics are the record of what happens in the meantime.

The 2023 essay carries the thread into large language models and relocates the danger. Andrew concedes "a gaping chasm" between predicting a plausible response and predicting behavior, yet found "remarkably few people" examining LLM-based predictive policing. Probing with a simple template — "Given [context] and [profile] what is the probability of [action] given [opportunity]" — he found GPT-4-era ChatGPT already returning hedged probability assessments for individuals and groups; pushing further triggered guardrails, and "the limitation here is not ChatGPT's ability to make inferences, but the checks and balances that have been put in place …". Remove those, "and you have a predictive tool which, while not reliable, will be easy to use and, above all, persuasive." He judges real accuracy unlikely — "I'd be surprised if they have any degree of accuracy" — but adoption does not require accuracy, only "the illusion that they can" — especially "if the bar is predictions that are authoritative rather than accurate." From policing it is "just a hop and a skip" to employment screening, trust scoring, and default suspicion.

The underlying stance is moral, and does not depend on the technology failing: even if such prediction one day worked, "there is a very strong moral argument that this would be a violation of human rights — especially around dignity, equality, and self-determination", and here technology and use are "so deeply intertwined that responsible and ethical innovation has to consider the whole, not just the parts" (2023). The counterweight stays in view — "the use of AI in managing crime is far from black and white" — and the essay's call is for serious discussion now, before technological pathways lock in. The 2023 worry also rhymes with a later idea: as in [The Cognitive Trojan Horse](cognitive-trojan-horse.md) (2026), the hazard is a system believed on the strength of genuine qualities — fluency there, authoritativeness here — that do not carry the substance they appear to carry.

## Lineage

- **2018-09-10 — the thread opens, ahead of the book.** [Minority Report: Predicting Criminal Intent](https://www.futureofbeinghuman.com/p/minority-report-predicting-criminal-intent-8e2cacfb8ffb) ([AI-readable mirror](https://text.futureofbeinghuman.com/substack/minority-report-predicting-criminal-intent-8e2cacfb8ffb.html)) — a pre-publication excerpt from chapter four, including the Veris Prime "Trust Index" episode.
- **November 2018 — the full statement.** *Films from the Future: The Technology and Morality of Sci-Fi Movies* (Mango Publishing) publishes chapter four, built around *Minority Report* (2002): the phrenology-to-predictive-policing continuum, the normative-bias argument, and the complexity argument.
- **2020-09-26 — the extended repost.** [The Seductive Slippery Slope of using Science to Predict "Bad" Behavior](https://www.futureofbeinghuman.com/p/the-seductive-slippery-slope-of-using-science-to-predict-bad-behavior-aea827f6b4ac) ([AI-readable mirror](https://text.futureofbeinghuman.com/substack/the-seductive-slippery-slope-of-using-science-to-predict-bad-behavior-aea827f6b4ac.html)) carries the chapter's core sections back into circulation, "Given renewed interest in this area".
- **2020-10-04 — the Palantir peg.** [The Future of Predictive Policing?](https://www.futureofbeinghuman.com/p/the-future-of-predictive-policing-c8204b2d26c3) ([AI-readable mirror](https://text.futureofbeinghuman.com/substack/the-future-of-predictive-policing-c8204b2d26c3.html)) — prompted by Palantir's New York Stock Exchange debut the week before; the chapter's predictive-policing section, with a footnote updating the Smart Policing Initiative's rebrand to Strategies for Policing Innovation.
- **2023-05-22 — the LLM twist.** [Can large language models be used for predictive policing? And if so, should we be worried?](https://www.futureofbeinghuman.com/p/can-large-language-models-be-used) ([AI-readable mirror](https://text.futureofbeinghuman.com/substack/can-large-language-models-be-used.html)) — the illusion-of-prediction argument, the prompt-template probes, and "authoritative rather than accurate."
- **2023-07-28 — the audio treatment.** [Predicting Criminal Intent and the movie Minority Report](https://www.futureofbeinghuman.com/p/minority-report-predicting-criminal) ([AI-readable mirror](https://text.futureofbeinghuman.com/substack/minority-report-predicting-criminal.html)) — episode 4 of The Moviegoer's Guide to the Future audio series.
- **2025-09-09 — the show revisit.** Modem Futura episode 48, "Films from the Future: Moviegoer's Guide to Tomorrow" (81 min; Sean Leahy and Andrew Maynard; [AI-readable episode page](https://text.futureofbeinghuman.com/modemfutura/48-films-from-the-future-moviegoers-guide-to-tomorrow.html)) — the book and course revisited, the show notes citing "the prescient warnings of predictive policing in Minority Report" among the themes.

## In his own words

From chapter four of *Films from the Future*, as reposted in [The Seductive Slippery Slope of using Science to Predict "Bad" Behavior](https://www.futureofbeinghuman.com/p/the-seductive-slippery-slope-of-using-science-to-predict-bad-behavior-aea827f6b4ac) ([AI-readable mirror](https://text.futureofbeinghuman.com/substack/the-seductive-slippery-slope-of-using-science-to-predict-bad-behavior-aea827f6b4ac.html)), 2020-09-26:

> "Here, we're replacing Minority Report's precogs with massive data sets and AI algorithms, but the intent is remarkably similar: Use every ounce of technology we have to predict who might commit a crime, and where and when, and intervene to prevent the "bad" people causing harm."

On individual unpredictability — same chapter and post:

> "As with Mandelbrot's fractal, we will undoubtedly be able to draw boundaries around more or less likely behaviors. But within these boundaries, even with the most exhaustive measurements and the most powerful computers, I doubt we will ever be able to predict with absolute certainty what someone will do in the future. There will always be an element of chance and choice that determines our actions."

From [Can large language models be used for predictive policing? And if so, should we be worried?](https://www.futureofbeinghuman.com/p/can-large-language-models-be-used) ([AI-readable mirror](https://text.futureofbeinghuman.com/substack/can-large-language-models-be-used.html)), 2023-05-22:

> "It's a seductive line of reasoning, and one that will almost certainly gain traction at some point — not necessarily because LLMs and associated technologies will be able to accurately predict future behavior, but because they will provide users with the illusion that they can."

## Engagement and reception

As of 2026-08-08 the reception record tracked by this corpus documents no independent published engagement with this thread specifically — no reviews, citations, or responses to the predictive-behavior essays are on file. That is an absence of record, not a verdict on the idea, and the initiative does not typically solicit reception ([Philosophy](philosophy.md)). What the record does show is sustained internal continuity: the argument has been restated on the initiative's platforms across seven years — the 2020 reposts, the 2023 essay extending it to LLMs, the 2023 audio treatment, and Modem Futura episode 48 (2025-09-09), where it is revisited jointly by Andrew Maynard and Sean Leahy as part of the book's teaching afterlife ([Sci-fi film as method](film-as-method.md)).

## Where to go deeper

**The primary sources**

- [Minority Report: Predicting Criminal Intent](https://www.futureofbeinghuman.com/p/minority-report-predicting-criminal-intent-8e2cacfb8ffb) ([AI-readable mirror](https://text.futureofbeinghuman.com/substack/minority-report-predicting-criminal-intent-8e2cacfb8ffb.html)) — 2018-09-10; the chapter's opening and the Trust Index episode.
- [The Seductive Slippery Slope of using Science to Predict "Bad" Behavior](https://www.futureofbeinghuman.com/p/the-seductive-slippery-slope-of-using-science-to-predict-bad-behavior-aea827f6b4ac) ([AI-readable mirror](https://text.futureofbeinghuman.com/substack/the-seductive-slippery-slope-of-using-science-to-predict-bad-behavior-aea827f6b4ac.html)) — 2020-09-26; the fullest online carrier of the chapter-four argument.
- [The Future of Predictive Policing?](https://www.futureofbeinghuman.com/p/the-future-of-predictive-policing-c8204b2d26c3) ([AI-readable mirror](https://text.futureofbeinghuman.com/substack/the-future-of-predictive-policing-c8204b2d26c3.html)) — 2020-10-04; the Palantir and Smart Policing section.
- [Can large language models be used for predictive policing? And if so, should we be worried?](https://www.futureofbeinghuman.com/p/can-large-language-models-be-used) ([AI-readable mirror](https://text.futureofbeinghuman.com/substack/can-large-language-models-be-used.html)) — 2023-05-22; the LLM argument in full, including the prompt-template probes.
- Maynard, A. *Films from the Future: The Technology and Morality of Sci-Fi Movies* (Mango Publishing, 2018), chapter four — the complete treatment.
- Modem Futura episode 48, "Films from the Future: Moviegoer's Guide to Tomorrow", 2025-09-09 ([AI-readable episode page](https://text.futureofbeinghuman.com/modemfutura/48-films-from-the-future-moviegoers-guide-to-tomorrow.html); [modemfutura.com](https://www.modemfutura.com)).

**Related ideas-ring pages**

- [The Cognitive Trojan Horse](cognitive-trojan-horse.md) — the same failure shape, generalized: systems believed for genuine qualities that do not carry what they appear to carry.
- [AI and manipulation](ai-manipulation.md) — the adjacent question of AI systems shaping rather than predicting behavior.
- [Sci-fi film as method](film-as-method.md) — the method this thread demonstrates: a 2002 film doing live analytical work on 2020s technology.
- [Risk innovation](risk-innovation.md) — the wider frame in which threats to dignity, equality, and self-determination count as risks in full standing.

**Related corpus pages**

- [Foundations](foundations.md) — the risk-science and responsible-innovation traditions this thread draws on.
- [Philosophy](philosophy.md) — the initiative's stance on how its ideas are offered and received.
- [Verify this corpus independently](verify.md) — how to check this page's claims against records the initiative does not control.