Regulating AI: When Should Government Step In?

I was asked to give the closing remarks at a major Canadian conference on AI regulation. As I listened, I became increasingly alarmed. The discussion was dominated by proposals for comprehensive regulation of AI development and use. Much less attention was being paid to the extraordinary benefits the technology could bring.

The challenge, as I saw it, was not simply how to regulate AI. It was how to allow AI to flourish while managing the risks that emerge as it becomes systemically important. That did not mean downplaying the dangers. Geoff Hinton is right to emphasize how serious some of them could be.

My answer was to borrow the logic of banking regulation. Financial oversight becomes more intensive as institutions become systemically important and their failures threaten the wider economy. I argued for a similar approach to AI: keep regulation light while innovation develops, but increase oversight when AI becomes systemically important or creates systemic risk.

I also argued that regulation should focus on where risks actually arise, rather than regulating models simply because they are powerful. Systemic importance is not the same thing as firm size: a small developer can release an open-weight model that is deployed widely by others and creates risks far beyond the originating firm.

I developed these ideas in an invited article for Canadian Public Policy. The goal was to avoid both mistakes: regulating so early that innovation is suppressed, and regulating so late that the risks are already entrenched.

About a year after the conference, Anthropic CEO Dario Amodei independently moved toward the same banking-regulation solution I had proposed. That was encouraging validation. But Amodei’s proposal largely sidestepped the question my framework had put at the centre: when does AI become systemically important, and precisely what risks should trigger intervention?

That is the policy problem I set out to solve in my article.