Why California’s AI transparency law for state government use was destined to fail

After announcing under a new law that California’s state government uses no so-called high-risk AI systems, the state has discovered — to no one’s surprise — that state agencies actually have been using at least six automated systems to make consequential decisions about the lives of Californians.

Systems like these mine sensitive personal data to automate what can be life-and-death government decisions for people, such as whether they will be granted cash assistance to feed their families, qualify for housing or be able to access needed medical care.

This episode exemplifies something many have long argued: Regulating technology through transparency will fail not only to spark significant change but even to meet the already low threshold of letting the public know what the government is up to.

Assembly Bill 302, which was signed three years ago, requires the state’s Department of Technology to conduct a “comprehensive inventory of all high-risk automated decision systems” proposed or used by “any state agency” and to publish a yearly report on its findings.

In 2025, following the department’s first report (finding no high-risk systems), we made a public records request for the department’s data. What we received back was laughable: a single spreadsheet with a column labeled “Is ADS used” and the word ‘no’ listed for each agency.

There was no evidence of any further technology department inquiry.

This year, the department interviewed a few agencies that finally raised their hands to say they used high-risk systems. That means AB 302 depends wholly on agencies to self-report, and there’s no process to verify an agency’s claims — nor any repercussions for agencies that fail. 

Worse, it’s unclear what systems the state believes it is responsible for disclosing. The technology department’s report implies that the decision on whether a system is “high risk” and reportable relies on how state agencies define their own tools.

AB 302 vaguely states that high-risk systems “assist or replace human discretionary decisions that have a legal or similarly significant effect, including decisions that materially impact access to, or approval for, housing or accommodations, education, employment, credit, health care, and criminal justice.” But known systems like the Uniformity Assessment System, for example, which has led to the reduction of In-Home Supportive Services for disabled Californians, and the Risk Segmentation, Stratification, and Tier model used to predict Medi-Cal recipients’ “risk” and “service underutilization,” were not reported.

Transparency bills nationwide have similarly failed. New York’s Public Oversight of Surveillance Technology Act was “meant to provide the public with a better picture into the NYPD’s unchecked use of spying technologies,” according to the Brennan Center for Justice. But NYPD’s Inspector General and outside groups have shown that the police department exploits legal loopholes to avoid scrutiny of technologies, like those creepy robot dogs, and describes its own policies and capabilities in vague terms that undermine any meaningful oversight.

Many jurisdictions have also tried “community control over police surveillance” bills which — like AB 302 — require police to disclose information about their own tech, to similar frustration. Community control strategies allow police to frame surveillance in favorable terms. That can have problematic anchoring effects as policymakers “fixate on whatever instantiation of technology” proponents “happen to put in front of them,” as University of Washington law professor Ryan Calo succinctly put it last year.

But perhaps the most fundamental problem with trying to regulate automated systems through transparency is that this approach starts by conceding that state agencies may adopt such systems and then invests in their continued use by building expensive public bureaucracies around them. Far from serving an oversight function, this normalizes and entrenches the use of automated systems as a regular part of the way government agencies do business.

If California wants to regulate high risk government uses of technology, the conversation can’t end at transparency. AB 302 was an interesting, if failed, experiment. Now it’s time for lawmakers to get serious about protecting Californians.