ITHACA, N.Y. – A new modeling study finds that weak AI regulation may be worse than no regulation at all when it comes to the safety of AI products and services.
In the absence of strong federal AI regulation, states have attempted to take up the slack, but it's unclear how this patchwork of laws will alter incentives for companies to invest in the safety of their products. To better understand the implications, researchers at Cornell University and Carnegie Mellon University developed a theoretical model to estimate the effects of AI regulation, both for companies that produce general-purpose AI models – like the ones behind popular chatbots – and downstream companies that apply those models, such as for customer service chatbots or medical diagnostic systems.
"The goal of regulation should be the mutual benefit of everybody in society, and this can include those developing the technology, but also end users and the public," said Benjamin Laufer, first author of " The Backfiring Effect of Weak AI Safety Regulation ," which published July 20 in Proceedings of the National Academy of Sciences.
Laufer developed the model with his adviser, Jon Kleinberg , professor of computer science and information science at Cornell, and Hoda Heidari, assistant professor at Carnegie Mellon. In the model, they can set a minimum safety requirement – either for the general AI company, the downstream company or both – and then estimate the safety and performance of their products.
"There isn't much AI safety regulation, and so a lot of possible regulations are just proposals at this stage," said Laufer. "To some extent, regulation is poking in the dark, so it's worth reasoning through what effects these regulations might have on incentives."
They were surprised to see that when regulations targeted only the downstream companies, the resulting products were predicted to be less safe than if there was no regulation at all. This type of regulation could create an environment where general AI producers can skimp on safety investments, like third-party safety audits, knowing that downstream developers are still on the hook to ensure the safety of the final product.
"There's a free-riding behavior that occurs," Laufer said. "The regulation acts as a tool for the general provider to offload the safety burden onto the downstream specialist."
In another surprising finding, a sweet spot of regulation of both types of companies has the potential to yield safer products for consumers – as well as greater profits. When general AI producers and downstream companies must each reach a specific safety target, it reduces risk for both companies, because they don't have to take the other at their word that certain safety investments will be made.
The current study is a simplified model, the researchers said, but they hope to extend this work by looking at actual impacts of real-world regulation on AI model development and safety.
