Illinois has chosen to move before Washington. Gov. JB Pritzker signed the Artificial Intelligence Safety Measures Act on July 6, 2026, adding transparency, accountability and audit requirements for the largest artificial intelligence developers. The law applies to companies generating more than $500 million in annual revenue and building models trained with massive computing power. It also requires annual independent audits, which gives the law its biggest assertion of accountability.
The Illinois legislation follows similar laws in California and New York, with lawmakers hoping that the combined market power of those states will create a de facto national standard in the absence of federal action. OpenAI and Anthropic supported the Illinois bill, while some industry representatives raised concerns about requiring private auditors to make subjective safety determinations before national standards, certifications or clear regulatory guardrails fully exist.
While the law attempts to force accountability into a market that has moved faster than institutions can govern, it also presages a larger problem: AI regulation may be necessary, but much of it will likely look naive in hindsight. Not because legislators are unserious or because industry should be trusted to govern itself. The problem is deeper. General-purpose AI is creating a reality modern regulation was not built to govern.
AI Is Not a Product Category
Most regulation assumes a bounded artifact. A medical device has an intended function. A drug is profiled as a specific compound, with dosage guidelines, a target patient population, approved uses, and a range of known risks. Off-label use complicates the picture, but it does not make a drug a universal treatment for all imagined ailments.
AI has no comparable boundary.
A frontier model can become a tutor, companion, fraud assistant, coding partner, legal drafter, medical explainer, hiring screener, synthetic media engine, customer service agent, workflow orchestrator, memory system, search interface or enterprise knowledge layer. Even that long list understates the scope. The same model can move from expression to analysis to advice to action without becoming a different regulated object.
That is not a product category. It is becoming capability infrastructure.
AI is not hard to regulate only because it is opaque. A regulatory challenge also arises because AI systems are general. A law can regulate a medical device because the device has an intended function. A law can regulate a drug because the drug has a defined use context. General-purpose AI has no natural use boundary. Its uses are discovered through interaction. Its risks emerge through combination. Its failures depend on context.
The Illinois law focuses on catastrophic risks, including the possibility that powerful models could assist with chemical, biological or nuclear weapons or enable serious cyberattacks. That is a legitimate concern. The state is right to ask whether model developers have safety processes, reporting practices and accountability mechanisms around high-consequence harms.
Most people, however, will encounter AI not as a catastrophic threat, but as an administrative, emotional, educational or economic intermediary. Most of those risks will not look catastrophic in the statutory sense. They will appear as denied opportunity, eroded judgment, dependency, institutional laziness, manipulated trust, invisible exclusion or work redesigned around systems no one can fully validate.
Testing such systems requires more than a checklist. It requires imagining a near-infinite range of use cases. The word “infinite” is not mathematically precise, but it is operationally accurate. The test surface expands across model capability, user intent, domain, interface, connected tools, data sources, organizational workflows, cultural context and time.
Enterprises deploying AI can test implementations against intent. Regulators can test known use cases and threats. Neither can exhaustively test the interaction between a general model and the human imagination.
That is the generality paradox. AI’s value comes from the same properties that make it difficult to govern. It can transfer patterns from one domain to another. It can recombine concepts. It can move from language to code to image to action. It can be embedded in products never imagined by its original developers. It can answer questions its creators did not anticipate.
As AI increases in value and footprint, it becomes less governable through narrow rules. Legislators will define harms around the cases visible today. Industry will comply against those definitions. Users will discover new behaviors. Developers will release new model classes. Agents will gain new permissions. Interfaces will shift from chat to voice to ambient computing. The risk will migrate.
A law that looks sophisticated in 2026 may look like an obsolete content moderation rule by 2028.
Opacity Compounds the Scope Problem
The popular critique says that AI is hard to regulate because even its inventors do not fully know how it works. AI developers do understand the architecture, training methods, optimization processes, safety tuning, evaluation techniques and deployment constraints behind their models. What they do not fully understand is why a large model produces a specific answer in a specific context, or how it will behave across every plausible interaction once released into the world, or even what those interactions might be.
That distinction is important. AI is engineered, but its scale, complexity, and learned internal representations make it impossible to fully comprehend in the way regulators understand conventional software. Anthropic, for instance, recently described Claude’s “internal thoughts” as occurring in what it calls the J-space, which activates as the model reasons about concepts. (For more on the J-space, see Anthropic’s post: “A global workspace in language models.”) The J-space is an emergent feature, not one that was designed into the system. It is likely not the last emergent feature to be discovered.
The opacity problem makes inspection difficult. A regulator cannot read a neural network the way an engineer might inspect a traditional software codebase. Explanations often come after the fact. Benchmarks test selected behaviors. Red teams probe anticipated harms. Safety frameworks describe processes. Audit logs preserve fragments of interaction. None of those mechanisms fully explain what the system “knows,” what it may infer or how it may respond when a user, application, data source, tool and institutional incentive collide.
Opacity makes AI difficult to inspect. Scope makes it difficult to bound.
Child Safety Shows the Problem
Child safety is one of the clearer areas for AI regulation. Legislators can prohibit sexual exploitation, manipulative design, certain forms of data collection, deceptive companion behavior, harmful synthetic media and inappropriate uses in schools. Those efforts should proceed.
But children will not encounter AI in neat categories. They will encounter it through toys, tutors, search, games, phones, social platforms, classroom tools, creator apps, household devices, chatbots, companions and augmented reality. They will also encounter AI through other children using AI.
The harms will not be limited to the obvious cases. They may include dependency, emotional manipulation, identity confusion, bullying through synthetic media, automated social exclusion, inappropriate personalization, distorted learning, and the normalization of machine companionship as an always-available authority.
No legislative imagination will keep pace with childhood experimentation, commercial design and peer culture. Child safety rules, while necessary, will also prove perpetually incomplete.
The Regulated Object Keeps Moving
Another problem sits beneath the surface of every AI law: what exactly is being regulated?
Is it the foundation model? The fine-tuned model? The application? The prompt layer? The retrieval system? The memory store? The user interface? The agent framework? The API integration? The data pipeline? The deploying organization? The employee who approved the output? The vendor that updated the model on Thursday night?
The answer is usually “some combination of those.” That is not a satisfying legal target.
AI risk often emerges from assembly. A model may be acceptable. A calendar request may be harmless. A CRM interaction may be routine. A payment system may be compliant. A workflow automation layer may pass review. Combine them, and an organization may have created an agent with authority to contact customers, move data, schedule meetings, trigger refunds, escalate complaints, draft contracts or make recommendations that humans rubber-stamp because the system appears competent.
Most regulation assesses components. AI risk often appears in the combinations.
Audits Will Help, But They Will Not Solve the Problem
Illinois deserves attention because it requires annual independent audits for covered frontier developers. That is more serious than asking companies to grade their own homework. It imposes external pressure and creates a record. It may also help build an ecosystem of AI assurance firms, audit practices and professional norms.
But audits work best when the audited object is stable. AI systems are not stable in the way regulators might prefer. Between audits, the target may change in several ways:
- Model updates and version swaps.
- System prompt changes.
- Retrieval sources shift.
- Tool permissions expand.
- Fine-tuning alters behavior.
- Users discover workarounds.
- Agents get connected to new systems.
- Data drifts.
- Institutional incentives change.
While an audit can certify a configuration, it cannot certify the living environment into which AI is released. Further, an audit assumes an understanding. If the thing being audited is not well understood, it is likely that the audit will, at minimum, be incomplete—and more likely, dangerously insufficient as a protective mechanism.
That does not make audits useless. It makes them provisional. AI compliance should be treated less like a building inspection and more like continuous monitoring of a changing operating environment.
Transparency Is Not Understanding
Transparency has become the default regulatory remedy for AI. Model cards, safety reports, incident disclosures, system frameworks, benchmark results and audit summaries create the appearance of accountability. They may also improve behavior. They force companies to document assumptions, define risks, disclose processes and prepare for external scrutiny.
But transparency is not understanding.
A report can say that a model was tested for a class of harmful behavior. It cannot prove the model will not produce adjacent harm in a new context. A safety framework can describe how a company assesses catastrophic risk. It cannot make a teacher, parent, employer, physician, claims processor or procurement officer understand the limits of the system in front of them. An audit can verify whether a company followed its stated process. It cannot guarantee that the process imagined the right world.
Transparency can become ritualized accountability: visible, procedural, but ultimately, inadequate.
Regulation Will Leak
Another regulatory challenge stems from the proliferation of open and distilled models, with large numbers already available through open repositories and model-sharing platforms. Regulated foundation models may offer some comfort to government or business as a compliance perimeter, but easy access to models that will likely remain outside of regulatory boundaries will give bad actors access to AI that does not require the same scrutiny as commercial models, while leaving vulnerable populations exposed to systems with fewer safeguards.
Safety Is Not Suitability
AI regulation often treats safety as the threshold question. Did the system avoid obvious harm? Did it refuse prohibited requests? Did it pass benchmark tests? Did the developer document safety practices? Was there a human in the loop?
Those are useful questions, but they are not enough.
A model may be safe in a general evaluation and still unsuitable for a particular use. A system that performs adequately as a writing assistant may be inappropriate as a benefits advisor. A chatbot that avoids explicit self-harm instructions may still be dangerous as a companion for vulnerable teenagers. A model that summarizes medical information may be unacceptable as a triage layer. A system that ranks job applicants may be statistically impressive and still institutionally unfair.
As for the “human in the loop,” safety definitions must avoid liability laundering. Humans without time, authority, expertise or access to evidence cannot act as mediators in an AI workflow.
Safety asks whether the system avoids harm in tested conditions. Suitability asks whether the system belongs in the work at all.
Regulators should ask not only whether an AI system can be made safe, but whether it belongs in the decision, relationship or workflow at all.
Liability May Become the Real Regulator
When technical rules fail, liability becomes the backstop. AI regulation will eventually have to answer basic questions of responsibility. Who is accountable when AI causes harm? The model developer? The deployer? The systems integrator? The employer? The public agency? The auditor? The vendor that supplied the retrieval data? The manager who trusted the output?
The Illinois bill gives enforcement authority to the attorney general and avoids creating a private right of action. That may make the law more politically viable, but it also reveals the difficulty of redress. People harmed by AI may not be harmed by catastrophic failures. They may be harmed by everyday administrative decisions that are hard to see, hard to contest and hard to trace.
The future of AI accountability may depend less on whether a model was certified and more on whether an organization can prove it had a defensible governance process, understood the risks, monitored failures, preserved records, offered appeal mechanisms and assigned accountable humans to consequential decisions.
Incumbents Will Benefit
AI regulation may protect the public. It may also protect incumbents.
Large firms can hire compliance teams, lobby statehouses, negotiate definitions, shape standards, pay auditors, publish reports and absorb penalties as a cost of operating. Smaller firms may struggle to interpret obligations, secure audit support or compete in procurement environments that treat compliance documentation as a proxy for trust.
The irony is obvious. Laws designed to constrain the largest AI developers may also reinforce their legitimacy. A company that can survive the audit regime becomes safer in the eyes of the market, even if the deeper limits of AI governance remain unresolved.
Regulation can become both a guardrail and a moat.
What Better AI Regulation Would Look Like
The argument against naive AI regulation is not an argument for no regulation. The market will not self-correct fast enough. Companies will not voluntarily internalize all social risk. Consumers cannot meaningfully evaluate the systems being placed in front of them. Public agencies cannot outsource judgment to vendors and call the result innovation. Court cases brought against AI vendors may move too slowly to impact model releases.
So what kind of regulation can survive contact with the reality of AI?
Better AI regulation would focus less on static categories and more on operating conditions. It would regulate uses before models. It would regulate claims before capabilities. It would require incident reporting, audit trails, update notices, data provenance, human accountability, appeal rights and clear liability chains. It would treat AI systems as changing environments rather than fixed products. It would distinguish safety from suitability. It would ask whether an institution should use AI for a purpose, not just whether the model passed a benchmark.
Regulation should also recognize that some uses should be off-limits, not because every possible harm has been proven, but because the combination of power, opacity, dependency and asymmetry is unacceptable.
At the practical level, procurement policy should be a key focus for states. While they may struggle to regulate AI, they can regulate what public agencies buy. Procurement rules may prove less visible than headline legislation, but more enforceable and less contested as a lever for shaping public-sector AI use.
The Illinois Lesson
Illinois has taken a serious step. The state is trying to impose accountability on frontier AI developers before the next major failure forces action under worse conditions.
But the law also shows the limits of the current regulatory imagination. It focuses on the largest developers. It emphasizes catastrophic risk. It requires transparency frameworks and third-party audits. Those are reasonable tools, but they are also incomplete solutions.
AI will humble regulators because it is not just another technology awaiting rules. It is a general-purpose capability that will be woven into other technologies, institutions, decisions and relationships. It will not sit still long enough for conventional regulation to wrap around it, discover its edges and apply constraint.
The danger is not that AI will remain entirely unregulated. The danger is that regulation will become precise where AI is narrow and vague where AI is transformative. Legislatures will prohibit the harms they can name. The market will create harms they did not imagine.
That is the hard truth behind the Illinois law. The task is not to create a complete AI regulatory framework. Lawmakers should focus on building regulatory systems that expect incompleteness, monitor for surprise and retain the authority to intervene when general-purpose systems become sources of specific harms.
For more serious insights from Dan on AI, visit: https://www.seriousinsights.net/serious-insights-on-ai/
His work has been featured in Harvard Business Review, Fast Company, Wired, NASA Ask, and dozens of other magazines and websites. His analysis on the future of work can be found at SeriousInsights.net. He is also the author of Understanding Artificial Intelligence, Cyberlife, Rethinking Smart Objects, and Empower Business with Generative AI.
Rasmus regularly speaks on the future of work at events such as Comic-Con International, Wondercon, Impact, Enterprise AI World, Worktech, CLO Symposium, KMWorld, AAAI, Computers in Libraries, Microsoft’s Building the Future, Educause, Expo Capital Humano, Devlearn, Internet Librarian, CAMEX, EduComm, and Sourcing Summit Europe.
As an Affiliate Instructor, Rasmus teaches scenario planning at the University of Washington and is an Associate Adjunct Professor at Bellevue College.