By Khari Johnson

WHAT THE BILL WOULD DO

SB 1047 by Senator Scott Wiener, a Democrat from San Francisco, requires the makers of advanced artificial intelligence models to test their likelihood to cause critical harm to society. The bill also protects the rights of whistleblowers to raise concerns to the attorney general. The bill attempts to codify, oversee, and enforce safety testing similar to what major companies voluntarily agreed to in deals with the White House and governments in the United Kingdom and South Korea. The bill also takes steps toward establishing a public cloud computing cluster known as CalCompute to advance AI development.

WHO SUPPORTS IT

In the weeks leading up to passage, X CEO Elon Musk and prominent researchers Yoshua Bengio and Geoffrey Hinton voiced support for the bill. Other supporters include AI company Anthropic, youth AI nonprofit organization Encode Justice, the Economic Security Project California, and experts including Lt. Gen. Jack Shanahan, the first director of the Joint AI Center at the Pentagon; Dan Hendrycks, director of the Center for AI Safety; and Daniel Kokotajlo and William Saunders, former OpenAI employees and whistleblowers. 

WHO IS OPPOSED

Major AI developers including Google, Meta, and OpenAI strongly oppose the bill, arguing that it will stifle innovation and the availability of open source software. Eight members of Congress who represent California districts, all Democrats, took the unusual step of urging Gov. Newsom to veto the bill. Former House Speaker Nancy Pelosi of San Francisco also opposes the bill, saying in a letter that it does more harm than good.

WHY IT MATTERS

California is home to the majority of the top AI companies in the world, so tough regulation of them could set a standard for the entire industry and for Congress. Supporters of the bill say now is the time to act because they don’t trust the makers of AI to self regulate particularly as profit pressures mount. Kokotajlo told CalMatters that if SB 1047 were in effect when he worked at OpenAI, it would have prevented or at least helped expose a violation of internal safety protocols he witnessed there.

GOVERNOR’S CALL ❌

Newsom announced Sept. 29 that he vetoed the bill, saying that a broader approach is needed and announcing a series of related initiatives. “By focusing only on the most expensive and large-scale models, SB 1047 establishes a regulatory framework that could give the public a false sense of security about controlling this fast-moving technology.”