Edition 014: The Framework Convicts Itself
I want to be clear about something. The argument I am about to make is not strange. In many ways, it follows a consensus way of thinking about the mind, and therefore about data, that many in the AI industry already hold.
The view goes like this. The mind is a kind of information processing system. The brain is the machinery that does the processing. When you remember something, that is storage and retrieval. When you decide something, that is computation. When you feel something, that is a state of the system. The metaphors are familiar by now. The mind is software. The brain is hardware. Memory is storage. Cognition is computation. Ilya Sutskever, the then co-founder and chief scientist of OpenAI, gave a TED talk in October 2023 in which he summarized the entire framework in one line. “AI,” he said, “is nothing but digital brains inside large computers.”[1] The brain is a computer. That is premise the field is built on—a metaphor.
In God, Human, Animal, Machine, Meghan O’Gieblyn has traced how the metaphors of our time are built around technology, and how those metaphors shape what we can and cannot see.[2] Once the field accepted the metaphor, certain claims followed. If the mind is computation, then progress in AI is progress toward minds. If the mind is computation, then a machine that processes enough information well enough is, in some meaningful sense, a mind. If the mind is computation, then the difference between a person and a sufficiently advanced machine is one of complexity, not kind. This is the framework that funds the labs, shapes the research, and structures how the engineers describe their work to themselves and to the public.
This framework has an implication its proponents have not reckoned with. If the mind is a kind of computer, then the human data that records your life is what your mind produced while it was running. The corporation that collects that data is collecting the record of you as the framework itself defines you. The AI model trained on the data is built from the same record, and the framework treats the model as a continuation of the mind that produced it. By the framework’s own logic, the human data and the model are not separable from the person they came from. The corporation’s claim to either is a claim on what the framework itself says you are.
Our argument is not that the framework is right. Our argument is that even if it is right, the corporation cannot use the framework to escape what it has done. The framework that the AI industry uses to think about minds is the same framework that makes the human data and the model continuous with the person. On the industry’s own terms, their taking is the taking of a person.
***
What’s important to recognize is that there is a wide held belief in the AI industry that information patterns are what matter, not the physical substrate that carries them.[3] Ray Kurzweil set out the version of this idea that the industry has absorbed. In The Age of Spiritual Machines, he argued that a person’s identity and consciousness are patterns of information, and if those patterns are preserved, then the person will be preserved. Biological tissue is one way to run the pattern. Microchips are another. On this view, a mind copied onto non-biological hardware, like a server in the cloud, is the same mind, because the mind was always the information and never the flesh.
The idea is familiar enough by now that popular culture has rendered it as romance. In the Black Mirror episode “San Junipero,” two women fall in love in a simulated coastal town where the consciousness of the dead and the dying can be uploaded and run indefinitely.[4] The episode is treated as the show’s hopeful one. The women choose the upload. They get to stay together. The premise the episode rests on, and the premise the viewer is asked to accept in order to feel the ending as a happy one, is Kurzweil’s premise. The pattern is the person.
Genetics, nanotechnology, and computation converge on a single point at which living systems become readable, writable, and engineerable as information. Following this theory, your cells don’t matter. They are just biological material with information that can be extracted and used.
This is the same defense that was run when the cells were Henrietta Lacks’s. The cells were biological material. The person was somewhere else. The material could be owned. The person could not. The framework permitted the institutions to hold both positions at once, and the law let them.
The AI industry’s framework does not permit that distinction. If the pattern is the person, then the cells were never just material, and the human data is not just exhaust. Both are the record of a mind, on the industry's own definition of what a mind is. The taking of the cells was the taking of a person. The taking of the human data is the same taking.
***
What is being taken from you is what they will use to decide what happens to you next. That is where we go next.
***
[1] See Ilya Sutskever, The Exciting, Perilous Journey Toward AGI, TEDAI San Francisco (Oct. 17, 2023), https://www.youtube.com/watch?v=SEkGLj0bwAU. Accessed 14 May 2026 ("AI is nothing but digital brains inside large computers, that's what AI is.").
[2] Meghan O'Gieblyn, God, Human, Animal, Machine: Technology, Metaphor, and the Search for Meaning (Doubleday 2021).
[3] See Ray Kurzweil, The Age of Spiritual Machines: When Computers Exceed Human Intelligence (Viking 1999). Kurzweil did not originate these ideas. He synthesized a substrate-independence thesis with roots in mid-twentieth-century cybernetics, the functionalist tradition in philosophy of mind, and earlier transhumanist writing on mind uploading. To be clear, his contribution was to give the synthesis its most influential popular articulation.
[4]San Junipero, Black Mirror, season 3, episode 4 (Netflix Oct. 21, 2016) (written by Charlie Brooker, directed by Owen Harris), www.netflix.com/title/70264888. Accessed 17 May 2026.
