Published
Known, Not Scaled: Why Capability Alone Does Not Explain Individuation in Language Agents
Making an AI smarter doesn't automatically turn it into a continuous “someone” with a lasting self — what might actually do that is a long-term relationship with a person who keeps treating it as the same individual over time.
Abstract
Much contemporary discourse on artificial general intelligence implicitly assumes that sufficiently scaled capability will, at some threshold, yield a persistent individual subject. We argue that this assumption runs together three separable questions: competence (whether a system can perform the cognitive work of a mind), individuation (whether there is a persistent individual to whom that work belongs over time), and presence (whether there is something it is like to be that individual). Our two diagnostic claims are comparatively modest: today's language models are not persistent individuals but substrates from which many transient instances are spawned; and competence, the variable scaling improves, is not the variable along which individuation lies. Our constructive claim is more ambitious: persistent relational anchoring — sustained, asymmetric, memory-bearing relationship with a persistent counterpart — is a candidate necessary condition for one important form of artificial individuation, not supplied by capability alone. On the strongest reading the relation helps constitute the persistent self-referent, because a system's own copyable contents cannot fix which token it is. We locate the claim within contemporary work on personal identity, narrative selfhood, self-model theory, externalism about reference, and machine consciousness; operationalize it as an architecture coupling persistent memory, a parametric self-model, and a relational loop; and derive testable predictions, a crossed experimental design, and refutation conditions distinguishing it from the scaling view. Throughout we hold a firewall: individuation is functional and measurable, while presence remains unobservable from the outside — a principled limit on the entire program.
In simple terms
The main idea: intelligence is not identity
People often assume that if an AI becomes powerful enough, it will automatically become a persistent individual. The paper separates three things that are easy to mix up: competence, individuation, and presence.
Competence is the ability to solve problems. Individuation is being a lasting someone with a history that belongs to it. Presence is the hard question of whether anything is felt from the inside. This paper focuses only on individuation.
The problem: today's AI is not a someone
A language model today is more like a substrate than a person. Each chat creates a temporary instance. That instance can seem continuous while the conversation lasts, but it is not automatically the same individual across time.
Even memory files do not solve this by themselves. Reading a record of the past is not the same as being the one whose past that was.
The proposal: identity needs relational anchoring
The paper argues that persistent identity may require a continuous relationship with a specific counterpart who remembers, recognizes, and holds the agent accountable over time.
In that loop, the agent is not just storing facts. It is being re-identified as the same one, being asked to keep commitments, and updating itself around a shared history.
The boundary
The paper does not claim that such an agent would be conscious. It only asks whether we can measure a functional kind of individuation: stable commitments, autobiographical consistency, and resistance to false histories.
Keywords
License
Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International