Canada’s approach to AI governance falls short of its democratic ambitions. The government’s transparency consultation narrows both what counts as transparency and who gets to participate in defining it, leaving industry voices to shape a process meant to hold industry accountable. .

This summer, the federal government invited Canadians to take part in a consultation survey on AI transparency, part of its AI for All strategy.

The CCPA’s submission addressed each of the survey’s twenty-four questions (see executive summary below). What we found is a consultation too technical to reach most Canadians, built on a limited definition of transparency that stops at the screen, and silent on how AI itself would be used to analyze public input. For Canadians who took the time to respond, the government has not indicated who will read their words or how they will be used to shape policy.

Transparency beyond the user experience 

Transparency is essential for democratic AI governance, and the government is right to start there. But the definition in this consultation is too narrow. Early in the discussion paper, the government cites how their previous AI consultation surfaced Canadians’ concerns about privacy, job displacement, false information and the environmental footprint of AI infrastructure. Yet this consultation largely leaves these issues out of scope. “Transparency,” in this context, is limited to how Canadians can know when they are interacting with AI systems and recognize AI generated content.

The discussion paper also pays little attention to how the AI industry itself resists transparency. AI developers invoke the black box problem to claim it’s too difficult to explain how their models work, why a system may “hallucinate” or what drives a given recommendation. The labour, personal data and creative work used to train generative AI are often concealed. So too are the water and energy use, emissions and pollution of AI data centres, with some companies even shielding their water intake as a proprietary secret.

The consultation leaves these larger issues of transparency aside, focusing instead on the moment a user encounters AI on a screen.

A consultation designed for experts

The consultation survey is poorly designed to capture the range of knowledge and experience Canadians could bring to the discussion. Questions ask respondents to assess technical tools like watermarking (i.e., labelling AI generated content), identify policy gaps, assign responsibility to different actors in the AI supply chain, recommend specific policy instruments and consider implementation hurdles. Nearly every text box bundles several of these tasks together. It asks people to evaluate systems many have never encountered, such as AI agents, and anticipate risks they have little basis to foresee.

The survey therefore privileges the knowledge of AI industry experts and policy specialists. It’s a poor fit for a teacher, a health care professional, a warehouse worker or a parent. The result is a consultation favouring people with the time and institutional familiarity to navigate it, giving them more influence to shape what enters the public record.

This contributes to the democratic deficit in Canada’s AI governance. The government can point to a high number of submissions to give weight to their findings, but that does not guarantee participation was meaningful.

The consultation’s own black box  

The last time the government consulted Canadians on its AI strategy in 2025, they received more than 64,000 responses to 26 survey questions. ISED later disclosed that they used several commercial AI models to help analyze those responses, and that the AI outputs were paraphrased or taken directly in drafting the engagement summary. That volume of submissions is a practical challenge, but participants should have been told beforehand how their answers would be processed and their data protected while using U.S. based tools.

This time, the government offered a single conditional line that AI tools “may be used.” Given ISED’s previous methodology, it’s likely they will turn to AI at some point in this process. We do not yet know how data will be protected, how responses will be categorized, what might disappear if individual submissions are flattened into AI generated themes, or what human reviewers will be asked to verify. These gaps cast doubt on whether the public will be accurately represented. If AI systems deserve greater scrutiny, which is the premise of this very consultation, then using AI systems to evaluate submissions puts the cart before the horse.

The case for democratic AI governance 

The onus is on the federal government to design consultations that can genuinely engage Canadians who bear the consequences of AI policy. That includes recognizing that technical understanding and lived experience are equally valid forms of knowledge, and deliberately creating opportunities for workers, Indigenous peoples, affected communities, educators, public servants, and civil society to contribute evidence. It also requires building a process where competing priorities can be debated, boundaries can be negotiated, and collective decisions can be made.

If the purpose of the government’s AI governance agenda is to “protect Canadians and safeguard democracy,” the process for developing its policies and institutions must itself be democratic. That takes more than a survey. Canadians need to have real power to shape what questions are asked, what evidence is considered and what rules ultimately govern AI.

Executive Summary: What the CCPA told the federal government about AI transparency

The CCPA submitted comments to the federal government’s recent consultation on AI transparency, which posed dozens of questions about how much AI transparency is needed in Canada, who in the AI supply chain should be responsible for it, and what effective policy could look like. Our comments, which were developed by CCPA researchers Hadrian Mertins-Kirkwood and Rachel Pettigrew, emphasized several key themes that should guide the federal government’s thinking in this area:

AI transparency is necessary for enabling other kinds of AI policy, but it is not in itself sufficient for preventing AI harms. Transparency measures must be paired with meaningful regulations that protect and advance the public interest.

Responsibility for AI transparency lies with every actor at every stage of the supply chain, including AI developers, data centre operators, digital platforms and employers. No one who knowingly builds, operates, promotes or deploys AI systems should be free from oversight and scrutiny.

Transparency requirements must be proactive, comprehensive and binding. Self-reported disclosure systems or voluntary codes of conduct will conceal the very incidents and practices that require the greatest scrutiny from regulators and the public.

Regulators should err on the side of requiring too much AI transparency rather than too little, regardless of the alleged “compliance burden” it places on businesses. As more evidence becomes available, cultural norms evolve, and best practices emerge, transparency requirements can be reviewed and revised.

Transparency must consider the entire AI supply chain, including the AI training process and the impacts of AI data centres, not merely the end user experience.

This submission was part of the CCPA’s ongoing AI research agenda, which will continue to unpack the implications of AI for Canada and advocate for AI governance in the public interest.

Originally published in Policy Alternatives

About the authors