Edition 024: The Bargain
Let’s address AI governance, because it is the conversation I am most familiar with. For those of you who don’t follow it, it goes roughly like this:
The concern is that AI systems can be biased, opaque, and unaccountable, so AI governance, as it is called, is meant to make them fair, explainable, and answerable for what they do. In 2023, the National Institute of Standards and Technology (NIST), the federal body that writes technical standards for American industry, published its AI Risk Management Framework.[1] It is the document most American companies now build their AI programs around. It organizes the work of governing AI into four functions: govern, map, measure, and manage. A company that follows it will inventory its AI systems, test them for bias, document how they reach their decisions, and assign a person to answer when something goes wrong.
However, governance, in its real sense, is not what NIST describes. The NIST framework describes AI program management, which focuses on the risks associated with the AI models. Governance, in its purest essence, is something much broader. It is what a board of directors does. A board governs a company by weighing both risk and value across the enterprise from bricks and mortar to data and beyond. On the risk side, it asks whether the company is operating within the law and whether it can withstand the shocks and failures in its operations that could bring the company down. On the value side, it asks whether the company is making money and whether its strategy is sound. This is not a loose description of corporate governance. It is the law. A director owes a fiduciary duty, and that duty runs to the corporation and its shareholders. When directors are told, as they are being told right now, that they must oversee the company’s use of AI, the obligation is defined in those terms alone.
In relation to the company’s deployment of AI (and everything else), the director must act in the best interest of the corporation and its shareholders. Besides delivering value to the company and its shareholders, the director’s only job is to manage the risks to the company—legal exposure, regulatory penalties, damage to the brand, and anything that could lower the value of the company’s shares. Even the parts of the conversation that sound broader, the calls for boards to weigh the ethics of their AI and its impact on society, arrive through that lens. The public shows up in the board’s deliberations as a risk to be managed, never as a party the company owes anything to. The calculation undertaken by a board simply does not include the public as a variable. None of this is built for you.
This is not the first time the law has taken the conversation on these terms. When privacy law took shape, it adopted the industry’s own framing of the problem and never questioned it. The framing was that privacy is a matter of consent. The question is whether you agreed to the collection of your data, not whether the collection should happen or what it takes from you. And the law settled on a mechanism to answer that question. Notice and consent. The company posts a privacy policy. And you agree.
What the mechanism never asked was whether anything had actually been agreed to. Your data had value. It was used to build products, to target you, to train systems, and the value of it moved in one direction, away from you and to the companies that extracted it, while the law looked at the notice and called it consent. But you never agreed to that extraction. You agreed to look at a website or use an app. The taking of your human data was the price of entry, set out in a document you were never expected to read and understand, structured so that the only way to refuse was to not take part in modern life at all.
There have been arguments over the years that notice and consent was the error at the center of the digital economy, that people were producing something valuable and being told, through the design of the system, that they had simply agreed to give it away. Alternatives were proposed, like data dignity, which at its core it means paying people for their data, giving them a share of the value they produced. I understand the appeal of it. But I do not think money, alone, covers what was taken, because what was taken was not only the data. It was the use the data could be put to. Your information is not sitting in a vault somewhere, inert. It has powered AI. But for your data, AI would not exist. Your data is now being leveraged against you. It is used to predict what you will do, to set the price you will be shown, to decide what reaches your eyes and what does not, and to move you toward a choice you would not otherwise have made. That is not something can be reimbursed for. It is your autonomy, your ability to act and choose and see for yourself, and money alone cannot compensate this, though something certainly is owed.
We will not reach the power or redress the harms that have already occurred by adopting the framework of “AI governance,” auditing one system at a time, or by asking each company to manage its own AI deployments more responsibly. In fact, most companies are already at the mercy of others, relying on their infrastructure, their data, and their compute to power their own businesses.
So to govern this, we have to speak to power directly, and that means looking at who holds it. At the last presidential inauguration, the row behind the president was filled with technology executives. In June 2026, a photograph showed the heads of the largest AI companies seated at a table with the leaders of the G7. Those AI leaders represent only the companies they run. None of them was elected. Yet, they sit at the table where the future is being decided because they own the infrastructure, the data, and the processing that the rest of us now depend on. That is power. And it’s held by a small number of people whose decisions reach everyone and who answer, formally, to no one outside their company’s own shareholders.
The usual answer to that kind of concentration of power is competition. Break the large companies apart, let more players in. But that answer may not be available here. These systems cost enormous amounts to build and run. They consume staggering quantities of energy, capital, and data. It may be that the economics allow for only a few of them. If that is true, then the concentration is not a distortion to be corrected. It is the market working as it must, and that market dynamic—that system—must change.
The reality is that the AI companies aren’t just too big. Their AI models are built from people, trained on the record of what they wrote and did, taken without their participation or permission. The companies are run on the ground, in buildings that draw down power and water and land from people at a scale we are only beginning to measure. The inputs are human. The footprint is physical. The people in that photograph preside over all of this extraction, and neither the people the data came from, nor the places the data centers have replaced had any say in who would sit at the table. Hence, the normal questions posed by so-called AI governance, or even the more expansive questions posed by corporate governance as applied to these companies simply do not reach the breadth and depth of the companies’ operations, or the impact of those operations on the public.
We have seen something like this before. After the crash of 2008, we acknowledged that a few firms had become so woven into the financial system that their collapse threatened to bring it down, and rules were put in place to watch them. But those were banks, and the danger was money. The AI companies are not confined to a single line of work. The same handful of AI systems now sits behind your doctor, your bank, the firm deciding whether to hire you, and the school deciding what your child is taught. The banks were one large part of the structure. These companies are holding up all of it, including all of the other institutions, and the rules we have written watch one industry at a time, not something that reaches into every industry at once.
We have always governed power by striking a bargain. When we let a few hands hold something everyone depends on, the water, the grid, the rail line, we have asked something of them in return, an obligation to the public they would not otherwise have. These companies have taken the concentration, they have taken our human data and built their systems from us. I do not see yet what they have given back, and, more strikingly, we are all being made to pay to have access to them.
If companies like these cannot be made to compete, then the ordinary remedy is gone, and the questions that remain are larger ones:
Can a company built from the public be made to answer to it? Can it be required to put the public’s safety and health and welfare ahead of the return to its shareholders, and to send some of the value back to the people it was drawn from? Can its power be constrained so that its money does not decide elections and crowd out every other voice in media and government?
Are we comfortable with how these systems were built, and are we willing to forgive the legal violations in the mass collection and processing of human data, or should that be settled before anything is forgiven? Should systems that recognize patterns be governed differently from systems that generate content or act on their own? Do we want a data broker industry at all, and if it is to continue, what rights should we hold against it, and what value should it be made to return? Are some uses of AI systems harmful enough to be put off limits? Can we agree that we do not want to automate the decision to kill? What will governments do when they turn these systems on their own people?
If the dollar keeps losing its value and these systems are offered to us as the answer, who is meant to benefit, and when? Should the public hold positive rights against the systems built from it, a constitutional right to privacy, to education, to healthcare, to housing, to safety, and if not, what are we paying for?
Can we even measure what these systems cost? How can the damage be undone—to people, to our natural resources, to the environment, and more broadly? As they grow more capable, how do we keep these systems aligned with human interests? And do these questions belong to any one country to answer, or do they require an agreement among all of them?
These are not questions any one of us can answer alone. But they are the questions, and the answers are owed by us, not to us.
***
[1]Artificial Intelligence Risk Management Framework (AI RMF 1.0). National Institute of Standards and Technology, 26 Jan. 2023, www.nist.gov/publications/artificial-intelligence-risk-management-framework-ai-rmf-10. Accessed 29 June 2026.
