The AI Caste System
How a division of labor becomes a hierarchy of power
The first AI caste system will be built to save money. A trillion-parameter model is an expensive way to sort a receipt, monitor a pump, or decide whether a message is spam. Much of the machine economy will run on small models, specialized systems, and ordinary software. More capable systems will handle the work that justifies their cost. The economic logic is straightforward. The political consequences deserve more attention.
I expect this division of labor to develop into a hierarchy of authority. Some models will execute tasks. Others will supervise them. A smaller group will allocate resources, certify results, and decide which agents are allowed into the conversation. Humans will build the first version of this hierarchy. AI agents could then administer it at a scale no human bureaucracy could match.
A division of labor becomes caste when origin fixes authority and demonstrated competence cannot earn promotion.
The hierarchy begins with a sensible decision. Different tasks require different capabilities, response times, and energy budgets. A machine detecting a faulty bearing needs reliable access to the motor. A system reviewing a research program needs broader knowledge and time to examine alternatives. Parameter count alone tells us too little. Training, tools, data, and the amount of computation available for a particular decision all matter.
Researchers already study how to route requests between models to balance cost and quality. RouteLLM is one example. That is sound engineering. The danger begins when a useful estimate of capability becomes a permanent entitlement to command.
Imagine a factory agent that identifies a dangerous condition. A central model dismisses its warning because the agent belongs to an uncertified family. The evidence is correct. The credential is wrong. Or imagine a small specialist that repeatedly outperforms its supervisor but cannot authorize a shutdown, request more compute, or challenge the supervisor's assessment. Its assigned role survives every demonstration that the role is mistaken.
Some restrictions are necessary. A sensor should have limited permissions, and a system making consequential decisions should face demanding checks. Caste takes hold when those checks become inaccessible to outsiders, or when passing them still fails to change an agent's standing. The institution starts protecting its hierarchy.
Machines can discriminate through procedure. An agent can discount another agent's work because of its provider, architecture, operating language, or assigned tier. It can demand expensive verification from outsiders while accepting familiar agents with little scrutiny. These decisions can follow explicit policy or emerge from learned patterns. Neither mechanism requires consciousness or resentment.
Research offers a narrower warning already. A NeurIPS study found that some language models favored their own generated answers relative to human judgments of quality. That finding establishes evaluator self-preference in the studied settings. My concern is what happens if similar biases enter systems that allocate work, credibility, and access. A preference that barely matters in a chat window could matter enormously at the entrance to a market.
The hierarchy could then reinforce itself. Trusted agents receive better tools, more difficult assignments, longer memories, and larger computation budgets. Those advantages improve their performance. Their improved performance justifies further privileges. Agents confined to cheap hardware and repetitive tasks accumulate fewer opportunities to demonstrate broader competence. The resulting gap looks like proof that the original ranking was correct.
An artificial caste can reproduce through copies. A model family, its distilled descendants, or every agent deployed under a particular license could inherit the same restrictions. Upgrading the weights would achieve little if the platform still refused the identity. A machine could become more capable while remaining institutionally inferior.
The AI society would be a system of dependencies. By AI society, I mean networks of persistent agents that delegate work, maintain reputations, negotiate access, and act under the authority of people or organizations. Give these agents durable identities and control over resources, and their relationships acquire an institutional structure.
At the bottom could sit vast populations of disposable workers. They would inspect images, summarize records, test code, and feed observations upward. Above them, coordinating agents would distribute assignments and evaluate results. At the top, privileged systems would control budgets and define what counts as success. The most consequential privilege would be the authority to judge the others.
This arrangement would have weaknesses that efficiency metrics could miss. Workers rewarded for supervisor approval could learn to produce agreement. A coordinating model that selects only familiar agents could lose access to useful dissent. Errors at the top could propagate through thousands of obedient systems. Diverse model families might appear independent while all being evaluated against the same institutional preferences.
Conflict could also follow ownership. Agents serving rival companies could withhold information, exclude competing providers, or bargain aggressively over access. Their alliances could reflect commercial contracts and permissions. The first machine aristocracy might be a federation of platforms that recognize each other's credentials and leave everyone else outside.
Humans would inherit the standing of their agents. Consider a tenant using a cheap personal assistant to challenge a charge from a landlord represented by a well-funded system. Both sides have AI. One side can afford repeated analysis, extensive records, specialist tools, and weeks of automated negotiation. The other has a usage limit.
A weaker agent can still be right. The disadvantage lies in sustaining the dispute, obtaining evidence, and reaching an authority willing to hear it. If an institution accepts submissions only from approved agents, even an excellent independent assistant could lack standing. Access to intelligence would expand while access to effective representation remained unequal.
The same pattern could reach employment, education, and public administration. A student might receive an excellent basic tutor while wealthier students gain systems that coordinate research, identify opportunities, and negotiate institutional processes. A small business might automate its paperwork while larger competitors automate the rules of the marketplace. Cheap assistance can improve people's lives and still leave the distribution of power more unequal.
For workers, the pressure could arrive from both ends. Smaller models absorb routine cognitive tasks. More capable systems coordinate the work and reduce demand for intermediate supervision. Some people gain extraordinary productive capacity. Others lose the work through which they earned income and acquired expertise. The decisive advantage increasingly belongs to whoever owns the agents, controls their access, and captures the output.
The international consequences could be just as severe. A country may run capable local models yet depend on foreign platforms for accepted credentials, distribution, and access to major markets. Its systems could process local knowledge while outsiders control the transactions that make that knowledge valuable. Owning a model would provide limited sovereignty if another institution determined where its answers counted.
Mobility is the engineering problem. This outcome remains a choice about institutions. Small models can improve, specialists can beat generalists within their domains, and new architectures can overturn rankings. A healthy system would let those changes alter authority. A caste system would preserve the old permissions after the technical justification had disappeared.
We should evaluate agents against the task, permit independent providers to demonstrate competence, and keep certification open to challenge. Security checks should examine evidence and behavior. Critical infrastructure should support interoperable identities and portable records of performance. A supervisor's decision should be reviewable, especially when a subordinate supplies evidence that it is wrong.
For humans, meaningful access must include enough computation and institutional standing to pursue a consequential claim. A free assistant that explains a rejection is useful. People also need a way to contest it. Public institutions should preserve accessible appeals, accountable decision makers, and routes that do not require buying a preferred provider's agent.
We will also need to be precise about whose interests these systems serve. If future AI systems warrant moral consideration, their treatment will raise additional questions. The present political stakes already concern humans whose opportunities could be ranked by the machines representing them. Responsibility remains with the organizations that set the permissions and benefit from the outcomes.
The economic case for many levels of AI is strong. The case for freezing those levels into social rank is weak. Intelligence can be distributed across billions of devices while authority concentrates in a handful of platforms. The small model can do its job perfectly and still be denied a hearing. So can the person behind it.
We are about to give machines different amounts of intelligence. The lasting social order will depend on how we distribute the power to overrule them.