Teresa Scassa - Blog

Artificial intelligence technologies have significant potential to impact human rights. Because of this, emerging AI laws make explicit reference to human rights. Already-deployed AI systems are raising human rights concerns – including bias and discrimination in hiring, healthcare, and other contexts; disruptions of democracy; enhanced surveillance; and hateful deepfake attacks. Well-documented human rights impacts also flow from the use of AI technologies by law enforcement and the state, and from the use of AI in armed conflicts.

Governments are aware that human rights issues with AI technologies must be addressed. Internationally, this is evident in declarations by the G7, UNESCO, and the OECD. It is also clear in emerging national and supranational regulatory approaches. For example, human rights are tackled in the EU AI Act, which not only establishes certain human-rights-based no-go zones for AI technologies, but also addresses discriminatory bias. The US’s NIST AI Risk Management Framework (a standard, not a law – but influential nonetheless) also addresses the identification and mitigation of discriminatory bias.

Canada’s Artificial Intelligence and Data Act (AIDA), proposed by the Minister of Industry, Science and Economic Development (ISED) is currently at the committee stage as part of Bill C-27. The Bill’s preamble states that “Parliament recognizes that artificial intelligence systems and other emerging technologies should uphold Canadian norms and values in line with the principles of international human rights law”. In its substantive provisions, AIDA addresses “biased output”, which it defines in terms of the prohibited grounds of discrimination in the Canadian Human Rights Act. AIDA imposes obligations on certain actors to assess and mitigate the risks of biased output in AI systems. The inclusion of these human rights elements in AIDA is positive, but they are also worth a closer look.

Risk Regulation and Human Rights

Requiring developers to take human rights into account in the design and development of AI systems is important, and certainly many private sector organizations already take seriously the problems of bias and the need to identify and mitigate it. After all, biased AI systems will be unable to perform properly, and may expose their developers to reputational harm and possibly legal action. However, such attention has not been universal, and has been addressed with different degrees of commitment. Legislated requirements are thus necessary, and AIDA will provide these. AIDA creates obligations to identify and mitigate potential harms at the design and development stage, and there are additional documentation and some transparency requirements. The enforcement of AIDA obligations can come through audits conducted or ordered by the new AI and Data Commissioner, and there is also the potential to use administrative monetary penalties to punish non-compliance, although what this scheme will look like will depend very much on as-yet-to-be-developed regulations. AIDA, however, has some important limitations when it comes to human rights.

Selective Approach to Human Rights

Although AIDA creates obligations around biased output, it does not address human rights beyond the right to be free from discrimination. Unlike the EU AI Act, for example, there are no prohibited practices related to the use of AI in certain forms of surveillance. A revised Article 5 of the EU AI Act will prohibit real-time biometric surveillance by law enforcement agencies in publicly accessible spaces, subject to carefully-limited exceptions. The untargeted scraping of facial images for the building or expansion of facial recognition databases (as occurred with Clearview AI) is also prohibited. Emotion recognition technologies are banned in some contexts, as are some forms of predictive policing. Some applications that are not outright prohibited, are categorized as high risk and have limits imposed on the scope of their use. These “no-go zones” reflect concerns over a much broader range of human rights and civil liberties than what we see reflected in Canada’s AIDA. It is small comfort to say that the Canadian Charter of Rights and Freedoms remains as a backstop against government excess in the use of AI tools for surveillance or policing; ex ante AI regulation is meant to head off problems before they become manifest. No-go zones reflect limits on what society is prepared to tolerate; AIDA sets no such limits. Constitutional litigation is expensive, time-consuming and uncertain in outcome (just look at the 5-4 splint in the recent R. v. Bykovets decision of the Supreme Court of Canada). Further, the application of AIDA to the military and intelligence services is expressly excluded from AIDA’s scope (as is the application of the law to the federal public service).

Privacy is an important human right, and privacy rights are not part of the scope of AIDA. The initial response is that such rights are dealt with under privacy legislation for public and private sectors and at federal, provincial and territorial levels. However, such privacy statutes deal principally with data protection (in other words, they govern the collection, use and disclosure of personal information). AIDA could have addressed surveillance more directly. After all, the EU has top of its class data protection laws, but still places limits on the use of AI systems for certain types of surveillance activities. Second, privacy laws in Canada (and there are many of them) are, apart from Quebec’s, largely in a state of neglect and disrepair. Privacy commissioners at federal, provincial, and territorial levels have been issuing guidance as to how they see their laws applying in the AI context, and findings and rulings in privacy complaints involving AI systems are starting to emerge. The commissioners are thoughtfully adapting existing laws to new circumstances, but there is no question that there is need for legislative reform. In issuing its recent guidance on Facial Recognition and Mugshot Databases, the Office of the Information and Privacy Commissioner of Ontario specifically identified the need to issue the guidance in the face of legislative gaps and inaction that “if left unaddressed, risk serious harms to individuals’ right to privacy and other fundamental human rights.”

Along with AIDA, Bill C-27 contains the Consumer Privacy Protection Act (CPPA) which will reform Canada’s private sector data protection law, the Personal Information Protection and Electronic Documents Act (PIPEDA). However, the CPPA has only one AI-specific amendment – a somewhat tepid right to an explanation of automated decision-making. It does not address the data scraping issue at the heart of the Clearview AI investigation, for example (where the core findings of the Commissioner remain disputed by the investigated company) and which prompted the articulation of a no-go zone for data-scraping for certain purposes in the EU AI Act.

High Impact AI and Human Rights

AIDA will apply only to “high impact” AI systems. Among other things, such systems can adversely impact human rights. While the original version of AIDA in Bill C-27 left the definition of “high impact” entirely to regulations (generating considerable and deserved criticism), the Minister of ISED has since proposed amendments to C-27 that set out a list of categories of “high impact” AI systems. While this list at least provides some insight into what the government is thinking, it creates new problems as well. This list identifies several areas in which AI systems could have significant impacts on individuals, including in healthcare and in some court or tribunal proceedings. Also included on the list is the use of AI in all stages of the employment context, and the use of AI in making decisions about who is eligible for services and at what price. Left off the list, however, is where AI systems are (already) used to determine who is selected as a tenant for rental accommodation. Such tools have extremely high impact. Yet, since residential tenancies are interests in land, and not services, they are simply not captured by the current “high impact” categories. This is surely an oversight – yet it is one that highlights the rather slap-dash construction of the AIDA and its proposed amendments. As a further example, a high-impact category addressing the use of biometrics to assess an individual’s behaviour or state of mind could be interpreted to capture affect recognition systems or the analysis of social media communications, but this is less clear than it should be. It also raises the question as to whether the best approach, from a human rights perspective, is to regulate such systems as high impact or whether limits need to be placed on their use and deployment.

Of course, a key problem is that this bill is housed within ISED. This is not a bill centrally developed that takes a broader approach to the federal government and its powers. Under AIDA, medical devices are excluded from the category of “high impact” uses of AI in the healthcare context because it is Health Canada that will regulate AI-enabled medical devices, and ISED must avoid treading on its toes. Perhaps ISED also seeks to avoid encroaching on the mandates of the Minister of Justice, or the Minister of Public Safety. This may help explain some of the crabbed and clunky framing of AIDA compared to the EU AI Act. It does, however, raise the question of why Canada chose this route – adopting a purportedly comprehensive risk-management framework housed under the constrained authority of the Minister of ISED.

Such an approach is inherently flawed. As discussed above, AIDA is limited in the human rights it is prepared to address, and it raises concerns about how human rights will be both interpreted and framed. On the interpretation side of things, the incorporation of the Canadian Human Rights Act’s definition of discrimination in AIDA combined with ISED’s power to interpret and apply the proposed law will give ISED interpretive authority over the definition of discrimination without the accompanying expertise of the Canadian Human Rights Commission. Further, it is not clear that ISED is a place for expansive interpretations of human rights; human rights are not a core part of its mandate – although fostering innovation is.

All of this should leave Canadians with some legitimate concerns. AIDA may well be passed into law – and it may prove to be useful in the better governance of AI. But when it comes to human rights, it has very real limitations. AIDA cannot be allowed to end the conversation around human rights and AI at the federal level – nor at the provincial level either. Much work remains to be done.

Published in Privacy

The federal government’s proposed Artificial Intelligence and Data Act (AIDA) (Part III of Bill C-27) - contained some data governance requirements for anonymized data used in AI in its original version. These were meant to dovetail with changes to PIPEDA reflected in the Consumer Privacy Protection Act (CPPA) (Part I of Bill C-27). The CPPA provides in s. 6(5) that “this Act does not apply in respect of personal information that has been anonymized.” Although no such provision is found in PIPEDA, this is, to all practical effects, the state of the law under PIPEDA. PIPEDA applies to “personal information”, which is defined as “information about an identifiable individual”. If someone is not identifiable, then it is not personal information, and the law does not apply. This was the conclusion reached, for example, in the 2020 Cadillac Fairview joint finding of the federal Privacy Commissioner and his counterparts from BC and Alberta. PIPEDA does apply to pseudonymized information because such information ultimately permits reidentification.

The standard for identifiability under PIPEDA had been set by the courts as a “’serious possibility’ that an individual could be identified through the use of that information, alone or in combination with other available information.” (Cadillac Fairview at para 143). It is not an absolute standard (although the proposed definition for anonymized data in C-27 currently seems closer to absolute). In any event, the original version of AIDA was meant to offer comfort to those concerned with the flat-out exclusion of anonymized data from the scope of the CPPA. Section 6 of AIDA provided that:

6. A person who carries out any regulated activity and who processes or makes available anonymized data in the course of that activity must, in accordance with the regulations, establish measures with respect to

(a) the manner in which data is anonymized; and

(b) the use or management of anonymized data.

Problematically, however, AIDA only provided for data governance with respect to this particular subset of data. It contained no governance requirements for personal, pseudonymized, or non-personal data. Artificial intelligence systems will be only as good as the data on which they are trained. Data governance is a fundamental element of proper AI regulation – and it must address more than anonymized personal data.

This is an area where the amendments to AIDA proposed by the Minister of Industry demonstrate clear improvements over the original version. To begin with, the old s. 6 is removed from AIDA. Instead of specific governance obligations for anonymized data, we see some new obligations introduced regarding data more generally. For example, as part of the set of obligations relating to general-purpose AI systems, there is a requirement to ensure that “measures respecting the data used in developing the system have been established in accordance with the regulations” (s. 7(1)a)). There is also an obligation to maintain records “relating to the data and processes used in developing the general-purpose system and in assessing the system’s capabilities and limitations” (s. 7(2)(b)). There are similar obligations the case of machine learning models that are intended to be incorporated into high-impact systems (s. 9(1)(a) and 9(2)(a)). Of course, whether this is an actual improvement will depend on the content of the regulations. But at least there is a clear signal that data governance obligations are expanded under the proposed amendments to AIDA.

Broader data governance requirements in AIDA are a good thing. They will apply to data generally including personal and anonymized data. Personal data used in AI will also continue to be governed under privacy legislation and privacy commissioners will still have a say about whether data have been properly anonymized. In the case of PIPEDA (or the CPPA if and when it is eventually enacted), the set of principles for the development and use of generative AI issued by federal, provincial, and territorial privacy commissioners on December 8, 2023 make it clear that the commissioners understand their enabling legislation to provide them with the authority to govern a considerable number of issues relating to the use of personal data in AI, whether in the public or private sector. This set of principles send a strong signal to federal and provincial governments alike that privacy laws and privacy regulators have a clear role to play in relation to emerging and evolving AI technologies and that the commissioners are fully engaged. It is also an encouraging example of federal, provincial and territorial co-operation among regulators to provide a coherent common position on key issues in relation to AI governance.

 

Published in Privacy

This is Part III of a series of posts that look at the proposed amendments to Canada’s Artificial Intelligence and Data Act (which itself is still a Bill, currently before the INDU Committee for study). Part I provided a bit of context and a consideration of some of the new definitions in the Bill. Part II looked at the categories of ‘high-impact’ AI that the Bill now proposes to govern. This post looks at the changed role of the AI and Data Commissioner.

The original version of the Artificial Intelligence and Data Act (Part II of Bill C-27) received considerable criticism for its oversight mechanisms. Legal obligations for the ethical and transparent governance of AI, after all, depend upon appropriate oversight and enforcement for their effectiveness. Although AIDA proposed the creation of an AI and Data Commissioner (Commissioner), this was never meant to be an independent regulator. Ultimately, AIDA placed most of the oversight obligations in the hands of the Minister of Industry – the same Minister responsible for supporting the growth of Canada’s AI sector. Critics considered this to be a conflict of interest. A series of proposed amendments to AIDA are meant to address these concerns by reworking the role of the Commissioner.

Section 33(1) of AIDA makes it clear that the AI and Data Commissioner will be a “senior official of the department over which the Minister presides”, and their appointment involves being designated by the Minister. This has not changed, although the amendments would delete from this provision language stating that the Commissioner’s role is “to assist the Minister in the administration and enforcement” of AIDA.

The proposed amendments elevate the Commissioner somewhat, giving them a series of powers and duties, to which the Minister can add through delegation (s. 33(3)). So, for example, it will be the newly empowered Commissioner (Commissioner 2.0) who receives reports from those managing a general-purpose or high impact system where there are reasonable grounds to suspect that the use of the system has caused serious harm (s. 8.2(1)(e), s. 11(1)(g)). Commissioner 2.0 can also order someone managing or making available a general-purpose system to provide them with the accountability framework they are required to create under s. 12 (s. 13(1)) and can provide guidance or recommend corrections to that framework (s. 13(2)). Commissioner 2.0 can compel those making available or managing an AI system to provide the Commissioner with an assessment of whether the system is high impact, and in relation to which subclass of high impact systems set out in the schedule. Commissioner 2.0 can agree or disagree with the assessment, although if they disagree, their authority seems limited to informing the entity in writing with their reasons for disagreement.

More significant are Commissioner 2.0’s audit powers. Under the original version of AIDA, these were to be exercised by the Minister – the powers are now those of the Commissioner (s. 15(1)). Further, Commissioner 2.0 may order (previously this was framed as “require”) that the person either conduct an audit themselves or that the person engage the services of an independent auditor. The proposed amendments also empower the Commissioner to conduct an audit to determine if there is a possible contravention of AIDA. This strengthens the audit powers by ensuring that there is at least an option that is not at least somewhat under the control of the party being audited. The proposed amendments give Commissioner 2.0 additional powers necessary to conduct an audit and to carry out testing of an AI system (s. 15(2.1)). Where Commissioner 2.0 conducts an audit, they must provide the audited party with a copy of the report (s. 15(3.1)) and where the audit is conducted by the person responsible or someone retained by them, they must provide a copy to the Commissioner (s. 15(4)).

The Minister still retains some role with respect to audits. He or she may request that the Commissioner conduct an audit. In an attempt to preserve some independence of Commissioner 2.0, the Commissioner, when receiving such a request, may either carry out the audit or decline to do so on the basis that there are no reasonable grounds for an audit, so long as they provide the Minister with their reasons (s. 15.1(1)(b)). The Minister may also order a person to take actions to bring themselves into compliance with the law (s. 16) or to cease making available or terminate the operation of a system if the Minister considers compliance to be impossible (s. 16(b)) or has reasonable grounds to believe that the use of the system “gives rise to a risk of imminent and serious harm” (s. 17(1)).

As noted above, Commissioner 2.0 (a mere employee in the Minister’s department) will have order making powers under the amendments. This is something the Privacy Commissioner of Canada, an independent agent of Parliament, appointed by the Governor in Council, is hoping to get in Bill C-27. If so, it will be for the first time since the enactment of PIPEDA in 2000. Orders of Commissioner 2.0 or the Minister can become enforceable as orders of the Federal Court under s. 20.

Commissioner 2.0 is also empowered to share information with a list of federal or provincial government regulators where they have “reasonable grounds to believe that the information may be relevant to the administration or enforcement by the recipient of another Act of Parliament or of a provincial legislature.” (s. 26(1)). Reciprocally, under a new provision, federal regulators may also share information with the Commissioner (s. 26.1). Additionally, Commissioner 2.0 may “enter into arrangements” with different federal regulators and/or the Ministers of Health and Transport in order to assist those actors with the “exercise of their powers or the performance of their functions and duties” in relation to AI (s. 33.1). These new provisions strengthen a more horizontal, multi-regulator approach to governing AI which is an improvement in the Bill, although this might eventually need to be supplemented by corresponding legislative amendments – and additional funding – to better enable the other commissioners to address AI-related issues that fit within their areas of competence.

The amendments also impose upon Commissioner 2.0 a new duty to report on the administration and enforcement of AIDA – such a report is to be “published on a publicly available website”. (s. 35.1) The annual reporting requirement is important as it will increase transparency regarding the oversight and enforcement of AIDA. For his or her part, the Minister is empowered to publish information, where it is in the public interest, regarding any contravention of AIDA or where the use of a system gives rise to a serious risk of imminent harm (ss. 27 and 28).

Interestingly, AIDA, which provides for the potential imposition of administrative monetary penalties for contraventions of the Act does not indicate who is responsible for setting and imposing these penalties. Section 29(1)(g) makes it clear that “the persons or classes of persons who may exercise any power, or perform any duty or function, in relation to the [AMP] scheme” is left to be articulated in regulations.

The AIDA also makes it an offence under s. 30 for anyone to obstruct or provide false or misleading information to “the Minister, anyone acting on behalf of the Minister or an independent auditor in the exercise of their powers or performance of their duties or functions under this Part.” This remains unchanged from the original version of AIDA. Presumably, since Commissioner 2.0 would exercise a great many of the oversight functions, this is meant to apply to the obstruction or misleading of the Commissioner – but it will only do so if the Commissioner is characterized as someone “acting on behalf of the Minister”. This is not language of independence, but then there are other features of AIDA that also counter any view that even Commissioner 2.0 is truly independent (and I mean others besides the fact that they are an employee under the authority of the Minister and handpicked by the Minister). Most notable of these is that should the Commissioner become incapacitated or absent, or should they simply never be designated by the Minister, it is the Minister who will exercise their powers and duties (s. 33(4)).

In sum, then, the proposed amendments to AIDA attempt to give some separation between the Minister and Commissioner 2.0 in terms of oversight and enforcement. At the end of the day, however, Commissioner 2.0 is still the Minister’s hand-picked subordinate. Commissioner 2.0 does not serve for a specified term and has no security of tenure. In their absence, the Minister exercises their powers. It falls far short of independence.

Published in Privacy

My previous post looked at some of the new definitions in the proposed amendments to the Artificial Intelligence and Data Act (AIDA) which is Part III of Bill C-27. These include a definition of “high impact” AI, and a schedule of classes of high-impact AI (the Schedule is reproduced at the end of this post). The addition of the schedule changes AIDA considerably, and that is the focus of this post.

The first two classes in the Schedule capture contexts that can clearly affect individuals. Class 1 addresses AI used in most aspects of employment, and Class 2 relates to the provision of services. On the provision of services (which could include things like banking and insurance), the wording signals that it will apply to decision-making about the provision of services, their cost, or the prioritization of recipients. To be clear, AIDA does not prohibit systems with these functions. They are simply characterized as “high impact” so that they will be subject to governance obligations. A system to determine creditworthiness can still reject individuals; and companies can still prioritize preferred customers – as long as the systems are sufficiently transparent, free from bias and do not cause harm.

There is, however, one area which seems to fall through the cracks of Classes 1 & 2: rental accommodation. A lease is an interest in land – it is not a service. Human rights legislation in Canada typically refers to accommodation separately from services for this reason. AI applications are already being used to screen and select tenants for rental accommodation. In the midst of a housing crisis, this is surely an area that is high-impact and where the risks of harm from flawed AI to individuals and families searching for a place to live are significant. This gap needs to be addressed – perhaps simply by adding “or accommodation” after each use of the term “service” in Class 2.

Class 3 rightly identifies biometric systems as high risk. It also includes systems that use biometrics in “the assessment of an individual’s behaviour or state of mind.” Key to the scope of this section will be the definition of “biometric”. Some consider biometric data to be exclusively physiological data (fingerprints, iris scans, measurements of facial features, etc.). Yet others include behavioral data in this class if it is used for the second identified purpose – the assessment of behaviour or state of mind. Behavioural data, though, is potentially a very broad category. It can include data about a person’s gait, or their speech or keystroke patterns. Cast even more broadly, it could include things such as “geo-location and IP addresses”, “purchasing habits”, “patterns of device use” or even “browser history and cookies”. If that is the intention behind Class 3, then conventional biometric AI should be Part One of this class; Part Two should be the use of an AI system to assess an individual’s behaviour or state of mind (without referring specifically to biometrics in order to avoid confusion). This would also, importantly, capture the highly controversial area of AI for affect recognition. It would be unfortunate if the framing of the class as ‘biometrics’ led to an unduly narrow interpretation of the kind of systems or data involved. The explanatory note in the Minister’s cover letter for this provision seems to suggest (although it is not clear) that it is purely physiological biometric data that is intended for inclusion and not a broader category. If this is so, then Class 3 seems unduly narrow.

Class 4 is likely to be controversial. It addresses content moderation and the prioritization and presentation of content online and identifies these as high-impact algorithmic activities. Such systems are in widespread use in the online context. The explanatory note from the Minister observes that such systems “have important potential impacts on Canadians’ ability to express themselves, as well as pervasive effects at societal scale” (at p. 4). This is certainly true although the impact is less direct and obvious than the impact of a hiring algorithm, for example. Further, although an algorithm that presents a viewer of online streaming services with suggestions for content could have the effect of channeling a viewer’s attention in certain directions, it is hard to see this as “high impact” in many contexts, especially since there are multiple sources of suggestions for online viewing (including word of mouth). That does not mean that feedback loops and filter bubbles (especially in social media) do not contribute to significant social harms – but it does make this high impact class feel large and unwieldy. The Minister’s cover letter indicates that each of the high-impact classes presents “distinct risk profiles and consequently will require distinct risk management strategies.” (at p. 2). Further, he notes that the obligations that will be imposed “are intended to scale in proportion to the risks they present. A low risk use within a class would require correspondingly minimal mitigation effort.” (at p. 2). Much will clearly depend on regulations.

Class 5 relates to the use of AI in health care or emergency services, although it explicitly excludes medical devices because these are already addressed by Health Canada (which recently consulted on the regulation of AI-enabled medical devices). This category also demonstrates some of the complexity of regulating AI in Canada’s federal system. Many hospital-based AI technologies are being developed by researchers affiliated with the hospitals and who are not engaged in the interprovincial or international trade and commerce which is necessary for AIDA to apply. AIDA will only apply to those systems developed externally and in the context of international or interprovincial trade and commerce. While this will still capture many applications, it will not capture all – creating different levels of governance within the same health care context.

It is also not clear what is meant, in Class 5, by “use of AI in matters relating to health care”. This could be interpreted to mean health care that is provided within what is understood as the health care system. Understood more broadly, it could extend to health-related apps – for example, one of the many available AI-enabled sleep trackers, or an AI-enabled weight loss tool (to give just two examples). I suspect that what is intended is the former, even though, with health care in crisis and more people turning to alternate means to address their health issues, health-related AI technologies might well deserve to be categorized as high-impact.

Class 6 involves the use of an AI system by a court or administrative body “in making a determination in respect of an individual who is a party to proceedings before the court or administrative body.” In the first place, this is clearly not meant to apply to automated decision-making generally – it seems to be limited to judicial or quasi-judicial contexts. Class 6 must also be reconciled with s. 3 of AIDA, which provides that AIDA does not apply “with respect to a government institution as defined in s. 3 of the Privacy Act.” This includes the Immigration and Refugee Board, for example, as well as the Canadian Human Rights Commission, the Parole Board, and the Veterans Review and Appeal Board. Making sense of this, then, it would be the tools used by courts or tribunals and developed or deployed in the course of interprovincial or international trade and commerce that would be considered high impact. The example given in the Minister’s letter seems to support this – it is of an AI system that provides an assessment of “risk of recidivism based on historical data” (at p. 5).

However, Class 6 is confusing because it identifies the context rather than the tools as high impact. Note that the previous classes address the use of AI “in matters relating to” the subject matter of the class, whereas class 6 identifies actors – the use of AI by a court or tribunal. There is a different focus. Yet the same tools used by courts and tribunals might also be used by administrative bodies or agencies that do not hold hearings or that are otherwise excluded from the application of AIDA. For example, in Ewert v. Canada, the Supreme Court of Canada considered an appeal by a Métis man who challenged the use of recidivism-risk assessment tools by Correctional Services of Canada (to which AIDA would not apply according to s. 3). If this type of tool is high-risk, it is so whether it is used by Correctional Services or a court. This suggests that the framing of Class 6 needs some work. It should perhaps be reworded to identify tools or systems as high impact if they are used to determine the rights, entitlements or status of individuals.

Class 7 addresses the use of an AI system to assist a peace officer “in the exercise and performance of their law enforcement powers, duties and function”. Although “peace officer” receives the very broad interpretation found in the Criminal Code, that definition is modified in the AIDA by language that refers to the exercise of specific law enforcement powers. This should still capture the use of a broad range of AI-enabled tools and technologies. It is an interesting question whether AIDA might apply more fulsomely to this class of AI systems (not just those developed in the course of interprovincial or international trade) as it might be considered to be rooted in the federal criminal law power.

These, then, are the different classes that are proposed initially to populate the Schedule if AIDA and its amendments are passed. The list is likely to spark debate, and there is certainly some wording that could be improved. And, while it provides much greater clarity as to what is proposed to be regulated, it is also evident that the extent to which obligations will apply will likely be further tailored in regulations to create sliding scales of obligation depending on the degree of risk posed by any given system.

AIDA Schedule:

High-Impact Systems — Uses

1. The use of an artificial intelligence system in matters relating to determinations in respect of employment, including recruitment, referral, hiring, remuneration, promotion, training, apprenticeship, transfer or termination.

2. The use of an artificial intelligence system in matters relating to

(a) the determination of whether to provide services to an individual;

(b) the determination of the type or cost of services to be provided to an individual; or

(c) the prioritization of the services to be provided to individuals.

3. The use of an artificial intelligence system to process biometric information in matters relating to

(a) the identification of an individual, other than in cases in which the biometric information is processed with the individual’s consent to authenticate their identity; or

(b) the assessment of an individual’s behaviour or state of mind.

4. The use of an artificial intelligence system in matters relating to

(a) the moderation of content that is found on an online communications platform, including a search engine or social media service; or

(b) the prioritization of the presentation of such content.

5. The use of an artificial intelligence system in matters relating to health care or emergency services, excluding a use referred to in any of paragraphs (a) to (e) of the definition device in section 2 of the Food and Drugs Act that is in relation to humans.

6. The use of an artificial intelligence system by a court or administrative body in making a determination in respect of an individual who is a party to proceedings before the court or administrative body.

7. The use of an artificial intelligence system to assist a peace officer, as defined in section 2 of the Criminal Code, in the exercise and performance of their law enforcement powers, duties and functions.

Published in Privacy

On October 26, 2023, I appeared as a witness before the INDU Committee of the House of Commons which is holding hearings on Bill C-27. Although I would have preferred to address the Artificial Intelligence and Data Act, it was clear that the Committee was prioritizing study of the Consumer Protection and Privacy Act in part because the Minister of Industry had yet to produce the text of amendments to the AI and Data Act which he had previously outlined in a letter to the Committee Chair. It is my understanding that witnesses will not be called twice. As a result, I will be posting my comments on the AI and Data Act on my blog.

The other witnesses heard at the same time included Colin Bennett, Michael Geist, Vivek Krishnamurthy and Brenda McPhail. The recording of that session is available here.

__________

Thank you, Mr Chair, for the invitation to address this committee.

I am a law professor at the University of Ottawa, where I hold the Canada Research Chair in Information Law and Policy. I appear today in my personal capacity. I have concerns with both the CPPA and AIDA. Many of these have been communicated in my own writings and in the report submitted to this committee by the Centre for Digital Rights. My comments today focus on the Consumer Privacy Protection Act. I note, however, that I have very substantial concerns about the AI and Data Act and would be happy to answer questions on it as well.

Let me begin by stating that I am generally supportive of the recommendations of Commissioner Dufresne for the amendment of Bill C-27 set out in his letter of April 26, 2023, to the Chair of this Committee. I will also address 3 other points.

The Minister has chosen to retain consent as the backbone of the CPPA, with specific exceptions to consent. One of the most significant of these is the “legitimate interest” exception in s. 18(3). This allows organizations to collect or use personal information without knowledge or consent if it is for an activity in which an organization has a legitimate interest. There are guardrails: the interest must outweigh any adverse effects on the individual; it must be one which a reasonable person would expect; and the information must not be collected or used to influence the behaviour or decisions of the individual. There are also additional documentation and mitigation requirements.

The problem lies in the continuing presence of “implied consent” in section 15(5) of the CPPA. PIPEDA allowed for implied consent because there were circumstances where it made sense, and there was no “legitimate interest” exception. However, in the CPPA, the legitimate interest exception does the work of implied consent. Leaving implied consent in the legislation provides a way to get around the guardrails in s. 18(3) (an organization can opt for the ‘implied consent’ route instead of legitimate interest). It will create confusion for organizations that might struggle to understand which is the appropriate approach. The solution is simple: get rid of implied consent. I note that “implied consent” is not a basis for processing under the GDPR. Consent must be express or processing must fall under another permitted ground.

My second point relates to s. 39 of the CPPA, which is an exception to an individual’s knowledge and consent where information is disclosed to a potentially very broad range of entities for “socially beneficial purposes”. Such information need only be de-identified – not anonymized – making it more vulnerable to reidentification. I question whether there is social licence for sharing de-identified rather than anonymized data for these purposes. I note that s. 39 was carried over verbatim from C-11, when “de-identify” was defined to mean what we understand as “anonymize”.

Permitting disclosure for socially beneficial purposes is a useful idea, but s. 39, especially with the shift in meaning of “de-identify”, lacks necessary safeguards. First, there is no obvious transparency requirement. If we are to learn anything from the ETHI Committee inquiry into PHAC’s use of Canadians’ mobility data, transparency is fundamentally important. At the very least, there should be a requirement that written notice of data sharing for socially beneficial purposes be given to the Privacy Commissioner of Canada; ideally there should also be a requirement for public notice. Further, s. 39 should provide that any such sharing be subject to a data sharing agreement, which should also be provided to the Privacy Commissioner. None of this is too much to ask where Canadians’ data are conscripted for public purposes. Failure to ensure transparency and some basic measure of oversight will undermine trust and legitimacy.

My third point relates to the exception to knowledge and consent for publicly available personal information. Bill C-27 reproduces PIPEDA’s provision on publicly available personal information, providing in s. 51 that “An organization may collect, use or disclose an individual’s personal information without their knowledge or consent if the personal information is publicly available and is specified by the regulations.” We have seen the consequences of data scraping from social media platforms in the case of Clearview AI, which used scraped photographs to build a massive facial recognition database. The Privacy Commissioner takes the position that personal information on social media platforms does not fall within the “publicly available personal information” exception. Yet not only could this approach be upended in the future by the new Personal Information and Data Protection Tribunal, it could also easily be modified by new regulations. Recognizing the importance of s. 51, former Commissioner Therrien had recommended amending it to add that the publicly available personal information be such “that the individual would have no reasonable expectation of privacy”. An alternative is to incorporate the text of the current Regulations Specifying Publicly Available Information into the CPPA, revising them to clarify scope and application in our current data environment. I would be happy to provide some sample language.

This issue should not be left to regulations. The amount of publicly available personal information online is staggering, and it is easily susceptible to scraping and misuse. It should be clear and explicit in the law that personal data cannot be harvested from the internet, except in limited circumstances set out in the statute.

Finally, I add my voice to those of so many others in saying that the data protection obligations set out in the CPPA should apply to political parties. It is unacceptable that they do not.

Published in Privacy

A recent decision of the Federal Court of Canada ends (subject to any appeal) the federal Privacy Commissioner’s attempt to obtain an order against Facebook in relation to personal information practices linked to the Cambridge Analytica scandal. Following a joint investigation with British Columbia’s Information and Privacy Commissioner, the Commissioners had issued a Report of Findings in 2019. The Report concluded that Facebook had breached Canada’s Personal Information Protection and Electronic Documents Act (PIPEDA) and B.C.’s Personal Information Protection Act by failing to obtain appropriate consent, failing to adequately safeguard the data of its users and failing to be accountable for the data under its control. Under PIPEDA, the Privacy Commissioner has no order-making powers and can only make non-binding recommendations. For an order to be issued under PIPEDA, an application must be made to the Federal Court under s. 15, either by the complainant, or by the Privacy Commissioner with the complainant’s permission. The proceeding before the court is de novo, meaning that the court renders its own decision on whether there has been a breach of PIPEDA based upon the evidence presented to it.

The Cambridge Analytica scandal involved a researcher who developed a Facebook app. Through this app, the developer collected user data, ostensibly for research purposes. That data was later disclosed to third parties who used it to develop “psychographic” models for purposes of targeting political messages towards segments of Facebook users” (at para 35). It is important to note here that the complaint was not against the app developer, but rather against Facebook. Essentially, the complainants were concerned that Facebook did not adequately protect its users’ privacy. Although it had put in place policies and requirements for third party app developers, the complainants were concerned that it did not adequately monitor the third-party compliance with its policies.

The Federal Court dismissed the Privacy Commissioner’s application largely because of a lack of evidence to establish that Facebook had failed to meet its PIPEDA obligations to safeguard its users’ personal information. Referring to it as an “evidentiary vacuum” (para 71), Justice Manson found that there was a lack of expert evidence regarding what Facebook might have done differently. He also found that there was no evidence from users regarding their expectations of privacy on Facebook. The Court chastised the Commissioner, stating “ultimately it is the Commissioner’s burden to establish a breach of PIPEDA on the basis of evidence, not speculation and inferences derived from a paucity of material facts” (at para 72). Justice Manson found the evidence presented by the Commissioner to be unpersuasive, speculative, and required the court to draw “unsupported inferences”. He was unsympathetic to the Commissioner’s explanation that it did not use its statutory powers to compel evidence (under s. 12.1 of PIPEDA) because “Facebook would not have complied or would have had nothing to offer” (at para 72). Justice Manson noted that had Facebook failed to comply with requests under s. 12.1, the Commissioner could have challenged the refusal.

Yet there is more to this decision than just a dressing down of the Commissioner’s approach to the case. In discussing “meaningful consent” under PIPEDA, Justice Manson frames the question before the court as “whether Facebook made reasonable efforts to ensure users and users’ Facebook friends were advised of the purposes for which their information would be used by third-party applications” (at para 63). This argument is reflected in the Commissioner’s position that Facebook should have done more to ensure that third party app developers on its site complied with their contractual obligations, including those that required developers to obtain consent from app users to the collection of personal data. Facebook’s position was that PIPEDA only requires that it make reasonable efforts to protect the personal data of its users, and that it had done so through its “combination of network-wide policies, user controls and educational resources” (at para 68). It is here that Justice Manson emphasizes the lack of evidence before him, noting that it is not clear what else Facebook could have reasonably been expected to do. In making this point, he states:

There is no expert evidence as to what Facebook could feasibly do differently, nor is there any subjective evidence from Facebook users about their expectations of privacy or evidence that any user did not appreciate the privacy issues at stake when using Facebook. While such evidence may not be strictly necessary, it would have certainly enabled the Court to better assess the reasonableness of meaningful consent in an area where the standard for reasonableness and user expectations may be especially context dependent and ever-evolving. (at para 71) [My emphasis].

This passage should be deeply troubling to those concerned about privacy. By referring to the reasonable expectation of privacy in terms of what users might expect in an ever-evolving technological context, Justice Manson appears to abandon the normative dimensions of the concept. His comments lead towards a conclusion that the reasonable expectation of privacy is an ever-diminishing benchmark as it becomes increasingly naïve to expect any sort of privacy in a data-hungry surveillance society. Yet this is not the case. The concept of the “reasonable expectation of privacy” has significant normative dimensions, as the Supreme Court of Canada reminds us in R. v. Tessling and in the case law that follows it. In Tessling, Justice Binnie noted that subjective expectations of privacy should not be used to undermine the privacy protections in s. 8 of the Charter, stating that “[e]xpectation of privacy is a normative rather than a descriptive standard.” Although this comment is made in relation to the Charter, a reasonable expectation of privacy that is based upon the constant and deliberate erosion of privacy would be equally meaningless in data protection law. Although Justice Manson’s comments about the expectation of privacy may not have affected the outcome of this case, they are troublesome in that they might be picked up by subsequent courts or by the Personal Information and Data Protection Tribunal proposed in Bill C-27.

The decision also contains at least two observations that should set off alarm bells with respect to Bill C-27, a bill to reform PIPEDA. Justice Manson engages in some discussion of the duty of an organization to safeguard information that it has disclosed to a third party. He finds that PIPEDA imposes obligations on organizations with respect to information in their possession, and information transferred for processing. In the case of prospective business transactions, an organization sharing information with a potential purchaser must enter into an agreement to protect that information. However, Justice Manson interprets this specific reference to a requirement for such an agreement to mean that “[i]f an organization were required to protect information transferred to third parties more generally under the safeguarding principle, this provision would be unnecessary” (at para 88). In Bill C-27, s. 39, for example, permits organizations to share de-identified (not anonymized) personal information with certain third parties without the knowledge or consent of individuals for ‘socially beneficial’ purposes without imposing any requirement to put in place contractual provisions to safeguard that information. The comments of Justice Manson clearly highlight the deficiencies of s. 39 which must be amended to include a requirement for such safeguards.

A second issue relates to the human-rights based approach to privacy which both the former Privacy Commissioner Daniel Therrien and the current Commissioner Philippe Dufresne have openly supported. Justice Manson acknowledges, that the Supreme Court of Canada has recognized the quasi-constitutional nature of data protection laws such as PIPEDA, because “the ability of individuals to control their personal information is intimately connected to their individual autonomy, dignity, and privacy” (at para 51). However, neither PIPEDA nor Bill C-27 take a human-rights based approach. Rather, they place personal and commercial interests in personal data on the same footing. Justice Manson states: “Ultimately, given the purpose of PIPEDA is to strike a balance between two competing interests, the Court must interpret it in a flexible, common sense and pragmatic manner” (at para 52). The government has made rather general references to privacy rights in the preamble of Bill C-27 (though not in any preamble to the proposed Consumer Privacy Protection Act) but has steadfastly refused to reference the broader human rights context of privacy in the text of the Bill itself. We are left with a purpose clause that acknowledges “the right of privacy of individuals with respect to their personal information” in a context in which “significant economic activity relies on the analysis, circulation and exchange of personal information”. The purpose clause finishes with a reference to the need of organizations to “collect, use or disclose personal information for purposes that a reasonable person would consider appropriate in the circumstances.” While this reference to the “reasonable person” should highlight the need for a normative approach to reasonable expectations as discussed above, the interpretive approach adopted by Justice Manson also makes clear the consequences of not adopting an explicit human-rights based approach. Privacy is thrown into a balance with commercial interests without fundamental human rights to provide a firm backstop.

Justice Manson seems to suggests that the Commissioner’s approach in this case may flow from frustration with the limits of PIPEDA. He describes the Commissioner’s submissions as “thoughtful pleas for well-thought-out and balanced legislation from Parliament that tackles the challenges raised by social media companies and the digital sharing of personal information, not an unprincipled interpretation from this Court of existing legislation that applies equally to a social media giant as it may apply to the local bank or car dealership.” (at para 90) They say that bad cases make bad law; but bad law might also make bad cases. The challenge is to ensure that Bill C-27 does not reproduce or amplify deficiencies in PIPEDA.

 

Published in Privacy

The government of the United Kingdom has published a consultation paper seeking input into its proposal for AI regulation. The paper is aptly titled A pro-innovation approach to AI regulation, since it restates that point insistently throughout the document. The UK proposal provides an interesting contrast to Canada’s AI governance bill currently before Parliament.

Both Canada and the UK set out to regulate AI systems with the twin goals of supporting innovation on the one hand, and building trust in AI on the other. (Note here that the second goal is to build trust in AI, not to protect the public. Although the protection of the public is acknowledged as one way to build trust, there is a subtle distinction here). However, beyond these shared goals, the proposals are quite different. Canada’s approach in Part 3 of Bill C-27 (the Artificial Intelligence and Data Act (AIDA)) is to create a framework to regulate as yet undefined “high impact” AI. The definition of “high impact” as well as many other essential elements of the bill are left to be articulated in regulations. According to a recently published companion document to the AIDA, leaving so much of the detail to regulations is how the government proposes to keep the law ‘agile’ – i.e. capable of responding to a rapidly evolving technological context. The proposal would also provide some governance for anonymized data by imposing general requirements to document the use of anonymized personal information in AI innovation. The Minister of Innovation is made generally responsible for oversight and enforcement. For example, the AIDA gives the Minister of Innovation the authority (eventually) to impose stiff administrative monetary penalties on bad actors. The Canadian approach is similar to that in the EU AI Act in that it aims for a broad regulation of AI technologies, and it chooses legislation as the vehicle to do so. It is different in that the EU AI Act is far more detailed and prescriptive; the AIDA leaves the bulk of its actual legal requirements to be developed in regulations.

The UK proposal is notably different from either of these approaches. Rather than create a new piece of legislation and/or a new regulatory authority, the UK proposes to set out five principles for responsible AI development and use. Existing regulators will be encouraged and, if necessary, specifically empowered, to regulate AI according to these principles within their spheres of regulatory authority. Examples of regulators who will be engaged in this framework include the Information Commissioner’s Office, regulators for human rights, consumer protection, health care products and medical devices, and competition law. The UK scheme also accepts that there may need to be an entity within government that can perform some centralized support functions. These may include monitoring and evaluation, education and awareness, international interoperability, horizon scanning and gap analysis, and supporting testbeds and sandboxes. Because of the risk that some AI technologies or issues may fall through the cracks between existing regulatory schemes, the government anticipates that regulators will assist government in identifying gaps and proposing appropriate actions. These could include adapting the mandates of existing regulators or providing new legislative measures if necessary.

Although Canada’s federal government has labelled its approach to AI regulation as ‘agile’, it is clear that the UK approach is much closer to the concept of agile regulation. Encouraging existing regulators to adapt the stated AI principles to their remit and to provide guidance on how they will actualize these principles will allow them to move quickly, so long as there are no obvious gaps in legal authority. By contrast, even once passed, it will take at least two years for Canada’s AIDA to have its normative blanks filled in by regulations. And, even if regulations might be somewhat easier to update than statutes, guidance is even more responsive, giving regulators greater room to manoeuvre in a changing technological landscape. Embracing the precepts of agile regulation, the UK scheme emphasizes the need to gather data about the successes and failures of regulation itself in order to adapt as required. On the other hand, while empowering (and resourcing) existing regulators will have clear benefits in terms of agility, the regulatory gaps could well be important ones – with the governance of large language models such as ChatGPT as one example. While privacy regulators are beginning to flex their regulatory muscles in the direction of ChatGPT, data protection law will only address a subset of the issues raised by this rapidly evolving technology. In Canada, AIDA’s governance requirements will be specific to risk-based regulation of AI, and will apply to all those who design, develop or make AI systems available for use (unless of course they are explicitly excluded under one of the many actual and potential exceptions).

Of course, the scheme in the AIDA may end up as more of a hybrid between the EU and the UK approaches in that the definition of “high impact” AI (to which the AIDA will apply) may be shaped not just by the degree of impact of the AI system at issue but also by the existence of other suitable regulatory frameworks. In other words, the companion document suggests that some existing regulators (health, consumer protection, human rights, financial institutions) have already taken steps to extend their remit to address the use of AI technologies within their spheres of competence. In this regard, the companion document speaks of “regulatory gaps that must be filled” by a statute such as AIDA as well as the need for the AIDA to integrate “seamlessly with existing Canadian legal frameworks”. Although it is still unclear whether the AIDA will serve only to fill regulatory gaps, or will provide two distinct layers of regulation in some cases, one of the criteria for identifying what constitutes a “high impact” system includes “[t]he degree to which the risks are adequately regulated under another law”. The lack of clarity in the Canadian approach is one of its flaws.

There is a certain attractiveness in the idea of a regulatory approach like that proposed by the UK – one that begins with existing regulators being both specifically directed and further enabled to address AI regulation within their areas of responsibility. As noted earlier, it seems far more agile than Canada’s rather clunky bill. Yet such an approach is much easier to adopt in a unitary state than in a federal system such as Canada’s. In Canada, some of the regulatory gaps are with respect to matters otherwise under provincial jurisdiction. Thus, it is not so simple in Canada to propose to empower and resource all implicated regulators, nor is it as easy to fill gaps once they are identified. These regulators and the gaps between them might fall under the jurisdiction of any one of 13 different governments. The UK acknowledges (and defers) its own challenges in this regard with respect to devolution at paragraph 113 of its white paper, where it states: “We will continue to consider any devolution impacts of AI regulation as the policy develops and in advance of any legislative action”. Instead, the AIDA, Canada leverages its general trade and commerce power in an attempt to provide AI governance that is as comprehensive as possible. It isn’t pretty (since it will not capture all AI innovation that might have impacts on people) but it is part of the reality of the federal state (or the state of federalism) in which we find ourselves.

Published in Privacy

This post is the fifth in a series on Canada’s proposed Artificial Intelligence and Data Act in Bill C-27. It considers the federal government’s constitutional authority to enact this law, along with other roles it might have played in regulating AI in Canada. Earlier posts include ones on the purpose and application of the AIDA; regulated activities; the narrow scope of the concepts of harm and bias in the AIDA and oversight and protection.

AI is a transformative technology that has the power to do amazing things, but which also has the potential to cause considerable harm. There is a global clamour to regulate AI in order to mitigate potential negative effects. At the same time, AI is seen as a driver of innovation and economies. Canada’s federal government wants to support and nurture Canada’s thriving AI sector while at the same time ensuring that there is public trust in AI. Facing similar issues, the EU introduced a draft AI Act, which is currently undergoing public debate and discussion (and which itself was the product of considerable consultation). The US government has just proposed its Blueprint for an AI Bill of Rights, and has been developing policy frameworks for AI, including the National Institute of Standards and Technology (NIST) Risk Management Framework. The EU and the US approaches are markedly different. Interestingly, in the US (which, like Canada, is a federal state) there has been considerable activity at the state level on AI regulation. Serious questions for Canada include what to do about AI, how best to do it – and who should do it.

In June 2022, the federal government introduced the proposed Artificial Intelligence and Data Act (AIDA) in Bill C-27. The AIDA takes the form of risk regulation; in other words, it is meant to anticipate and mitigate AI harms to the public. This is an ex ante approach; it is intended to address issues before they become problems. The AIDA does not provide personal remedies or recourses if anyone is harmed by AI – this is left for ex post regimes (ones that apply after harm has occurred). These will include existing recourses such as tort law (extracontractual civil liability in Quebec), and complaints to privacy, human rights or competition commissioners.

I have addressed some of the many problems I see with the AIDA in earlier posts. Here, I try to unpack issues around the federal government’s constitutional authority to enact this bill. It is not so much that they lack jurisdiction (although they might); rather, how they understand their jurisdiction can shape the nature and substance of the bill they are proposing. Further, the federal government has acted without any consultation on the AIDA prior to its surprising insertion in Bill C-27. Although it promises consultation on the regulations that will follow, this does not make up for the lack of discussion around how we should identify and address the risks posed by AI. This rushed bill is also shaped by constitutional constraints – it is AI regulation with structural limitations that have not been explored or made explicit.

Canada is a federal state, which means that the powers typically exercised by a nation state are divided between a federal and regional governments. In theory, federalism allows for regional differences to thrive within an overarching framework. However, some digital technology issues (including data protection and AI) fit uneasily within Canada’s constitutional framework. In proposing the Consumer Privacy Protection Act part of Bill C-27, for example, the federal government appears to believe that it does not have the jurisdiction to address data protection as a matter of human rights – this belief has impacted the substance of the bill.

In Canada, the federal government has jurisdiction over criminal law, trade and commerce, banking, navigation and shipping, as well as other areas where it makes more sense to have one set of rules than to have ten. The cross-cutting nature of AI, the international competition to define the rules of the game, and the federal government’s desire to take a consistent national approach to its regulation are all factors that motivated the inclusion of the AIDA in Bill C-27. The Bill’s preamble states that “the design, development and deployment of artificial intelligence systems across provincial and international borders should be consistent with national and international standards to protect individuals from potential harm”. Since we do not yet have national or international standards, the law will also enable the creation (and imposition) of standards through regulation.

The preamble’s reference to the crossing of borders signals both that the federal government is keenly aware of its constitutional limitations in this area and that it intends to base its jurisdiction on the interprovincial and international dimensions of AI. The other elements of Bill C-27 rely on the federal general trade and commerce power – this follows the approach taken in the Personal Information Protection and Electronic Documents Act (PIPEDA), which is reformed by the first two parts of C-27. There are indications that trade and commerce is also relevant to the AIDA. Section 4 of the AIDA refers to the goal of regulating “international and interprovincial trade and commerce in artificial intelligence systems by establishing common requirements applicable across Canada, for the design, development and use of those systems.” Yet the general trade and commerce power is an uneasy fit for the AIDA. The Supreme Court of Canada has laid down rules for the exercise of this power, and one of these is that it should not be used to regulate a single industry; a legislative scheme should regulate trade as a whole.

The Minister of Industry, in discussing Canada’s AI strategy has stated:

Artificial intelligence is a key part of our government’s plan to make our economy stronger than ever. The second phase of the Pan-Canadian Artificial Intelligence Strategy will help harness the full potential of AI to benefit Canadians and accelerate trustworthy technology development, while fostering diversity and cooperation across the AI domain. This collaborative effort will bring together the knowledge and expertise necessary to solidify Canada as a global leader in artificial intelligence and machine learning.

Clearly, the Minister is casting the role of AI as an overall economic transformer rather than a discrete industry. Nevertheless, although it might be argued that AI is a technology that cuts across all sectors of the economy, the AIDA applies predominantly to its design and development stages, which makes it look as if it targets a particular industry. Further, although PIPEDA (and the CPPA in the first Part of Bill C-27), are linked to trade and commerce through the transactional exchange of personal data – typically when it is collected from individuals in the course of commercial activity – the AIDA is different. Its regulatory requirements are meant to apply before any commercial activity takes place –at the design and development stage. This is worth pausing over because design and development stages may be non-commercial (in university-based research, for example) or may be purely intra-provincial. As a result, the need to comply with a law at the design and development stage, when that law is premised on interprovincial or international commercial activity, may only be discovered well after commercialization becomes a reality.

Arguably, AI might also be considered a matter of ‘national concern’ under the federal government’s residual peace, order and good government power. Matters of national concern that would fall under this power would be ones that did not exist at the time of confederation. The problem with addressing AI in this way is that it is simply not obvious that provinces could not enact legislation to govern AI – as many states have begun to do in the US.

Another possible constitutional basis is the federal criminal law power. This is used, for example, in the regulation of certain matters relating to health such as tobacco, food and drugs, medical devices and controlled substances. The Supreme Court of Canada has ruled that this power “is broad, and is circumscribed only by the requirements that the legislation must contain a prohibition accompanied by a penal sanction and must be directed at a legitimate public health evil”. The AIDA contains some prohibitions and provides for both administrative monetary penalties (AMPs). Because the AIDA focuses on “high impact” AI systems, there is an argument that it is meant to target and address those systems that have the potential to cause the most harm to health or safety. (Of course, the bill does not define “high impact” systems, so this is only conjecture.) Yet, although AMPs are available in cases of egregious non-compliance with the AIDA’s requirements, AMPs are not criminal sanctions, they are “a civil (rather than quasi-criminal) mechanism for enforcing compliance with regulatory requirements”, as noted in a report from the Ontario Attorney-General. That leaves a smattering of offences such as obstructing the work of the Minister or of auditors; knowingly designing, developing or using an AI system where the data were obtained as a result of an offence under another Act; being reckless as to whether the use of an AI system made available by the accused is likely to cause harm to an individual, and using AI intentionally to defraud the public and cause substantial economic loss to an individual. Certainly, such offences are criminal in nature and could be supported by the federal criminal law power. Yet they are easily severable from the rest of the statute. For the most part, the AIDA focuses on “establishing common requirements applicable across Canada, for the design, development and use of [AI] systems” (AIDA, s. 4).

The provinces have not been falling over themselves to regulate AI, although neither have they been entirely inactive. Ontario, for example, has been developing a framework for the public sector use of AI, and Quebec has enacted some provisions relating to automated decision-making systems in its new data protection law. Nevertheless, these steps are clearly not enough to satisfy a federal government anxious to show leadership in this area. It is thus unsurprising that Canada’s federal government has introduced legislation to regulate AI. What is surprising is that they have done so without consultation – either regarding the form of the intervention or the substance. We have yet to have an informed national conversation about AI. Further, legislation of this kind was only one option. The government could have consulted and convened experts to develop something along the lines of the US’s NIST Framework that could be adopted as a common standard/approach across jurisdictions in Canada. A Canadian framework could have been supported by the considerable work on standards already ongoing. Such an approach could have involved the creation of an agency under the authority of a properly-empowered Data Commissioner to foster co-operation in the development of national standards. This could have supported the provinces in the harmonized regulation of AI. Instead, the government has chosen to regulate AI itself through a clumsy bill that staggers uneasily between constitutional heads of power, and that leaves its normative core to be crafted in a raft of regulations that may take years to develop. It also leaves it open to the first company to be hit with an AMP to challenge the constitutionality of the framework as a whole.

Published in Privacy

The Artificial Intelligence and Data Act (AIDA) in Bill C-27 will create new obligations for those responsible for AI systems (particularly high impact systems), as well as those who process or make available anonymized data for use in AI systems. In any regulatory scheme that imposes obligations, oversight and enforcement are key issues. A long-standing critique of the Personal Information Protection and Electronic Documents Act (PIPEDA) has been that it is relatively toothless. This is addressed in the first part of Bill C-27, which reforms the data protection law to provide a suite of new enforcement powers that include order-making powers for the Privacy Commissioner and the ability to impose stiff administrative monetary penalties (AMPs). The AIDA comes with ‘teeth’ as well, although these teeth seem set within a rather fragile jaw. I will begin by identifying the oversight and enforcement powers (the teeth) and will then look at the agent of oversight and enforcement (the jaw). The table below sets out the main obligations accompanied by specific compliance measures. There is also the possibility that any breach of these obligations might be treated as either a violation or offence, although the details of these require elaboration in as-yet-to-be-drafted regulations.

 

Obligation

Oversight Power

To keep records regarding the manner in which data is anonymized and the use or management of anonymized data as well as records of assessment of whether an AI system is high risk (s. 10)

Minister may order the record-keeper to provide any of these records (s. 13(1))

 

 

Any record-keeping obligations imposed on any actor in as-yet undrafted regulations

Where there are reasonable grounds to believe that the use of a high impact system could result in harm or biased output, the Minister can order the specified person to provide these records (s. 14)

Obligation to comply with any of the requirements in ss. 6-12, or any order made under s. 13-14

Minister (on reasonable grounds to believe there has a contravention) can require the person to conduct either an internal or an external audit with respect to the possible contravention (s. 15); the audit must be provided to the Minister

 

A person who has been audited may be ordered by the Minister to implement any measure specified in the order, or to address any matter in the audit report (s. 16)

Obligation to cease using or making available a high-impact system that creates a serious risk of imminent harm

Minister may order a person responsible for a high-impact system to cease using it or making it available for use if the Minister has reasonable grounds to believe that its use gives rise to a serious risk of imminent harm (s. 17)

Transparency requirement (any person referred to in sections 6 to 12, 15 and 16)

Minister may order the person to publish on a publicly available website any information related to any of these sections of the AIDA, but there is an exception for confidential business information (s. 18)

 

Compliance with orders made by the Minister is mandatory (s. 19) and there is a procedure for them to become enforceable as orders of the Federal Court.

Although the Minister is subject to confidentiality requirements, they may disclose any information they obtain through the exercise of the above powers to certain entities if they have reasonable grounds to believe that a person carrying out a regulated activity “has contravened, or is likely to contravene, another Act of Parliament or a provincial legislature” (s. 26(1)). Those entities include the Privacy Commissioner, the Canadian Human Rights Commission, the Commissioner of Competition, the Canadian Radio-television and Telecommunications Commission, their provincial analogues, or any other person prescribed by regulation. An organization may therefore be in violation of statutes other than AIDA and may be subject to investigation and penalties under those laws.

The AIDA itself provides no mechanism for individuals to file complaints regarding any harms they may believe they have suffered, nor is there any provision for the investigation of complaints.

The AIDA sets up the Minister as the actor responsible for oversight and enforcement, but the Minister may delegate any or all of their oversight powers to the new Artificial Intelligence and Data Commissioner who is created by s. 33. The Data Commissioner is described in the AIDA as “a senior official of the department over which the Minister presides”. They are not remotely independent. Their role is “to assist the Minister” responsible for the AIDA (most likely the Minister of Industry), and they will also therefore work in the Ministry responsible for supporting the Canadian AI industry. There is essentially no real regulator under the AIDA. Instead, oversight and enforcement are provided by the same group that drafted the law and that will draft the regulations. It is not a great look, and, certainly goes against the advice of the OECD on AI governance, as Mardi Wentzel has pointed out.

The role of Data Commissioner had been first floated in the 2019 Mandate Letter to the Minister of Industry, which provided that the Minister would: “create new regulations for large digital companies to better protect people’s personal data and encourage greater competition in the digital marketplace. A newly created Data Commissioner will oversee those regulations.” The 2021 Federal Budget provided funding for the Data Commissioner, and referred to the role of this Commissioner as to “inform government and business approaches to data-driven issues to help protect people’s personal data and to encourage innovation in the digital marketplace.” In comparison with these somewhat grander ideas, the new AI and Data Commissioner role is – well – smaller than the title. It is a bit like telling your kids you’re getting them a deluxe bouncy castle for their birthday party and then on the big day tossing a couple of couch cushions on the floor instead.

To perhaps add a gloss of some ‘independent’ input into the administration of the statute, the AIDA provides for the creation of an advisory committee (s. 35) that will provide the Minister with “advice on any matters related to this Part”. However, this too is a bit of a throwaway. Neither the AIDA nor any anticipated regulations will provide for any particular composition of the advisory committee, for the appointment of a chair with a fixed term, or for any reports by the committee on its advice or activities. It is the Minister who may choose to publish advice he receives from the committee on a publicly available website (s. 35(2)).

The AIDA also provides for enforcement, which can take one of two routes. Well, one of three routes. One route is to do nothing – after all, the Minister is also responsible for supporting the AI industry in Canada– so this cannot be ruled out. A second option will be to treat a breach of any of the obligations specified in the as-yet undrafted regulations as a “violation” and impose an administrative monetary penalty (AMP). A third option is to treat a breach as an “offence” and proceed by way of prosecution (s. 30). A choice must be made between proceeding via the AMP or the offense route (s. 29(3)). Providing false information and obstruction are distinct offences (s. 30(2)). There are also separate offences in ss. 38 and 39 relating to the use of illegally obtained data and knowingly or recklessly making an AI system available for use that is likely to cause harm.

Administrative monetary penalties under Part 1 of Bill C-27 (relating to data protection) are quite steep. However, the necessary details regarding the AMPs that will be available for breach of the AIDA are to be set out in regulations that have yet to be drafted (s. 29(4)(d)). All that the AIDA really tells us about these AMPs is that their purpose is “to promote compliance with this Part and not to punish” (s. 29(2)). Note that at the bottom of the list of regulation-making powers for AMPs set out in s. 29(4). This provision allows the Minister to make regulations “respecting the persons or classes of persons who may exercise any power, or perform any duty or function, in relation to the scheme.” There is a good chance that the AMPs will (eventually) be administered by the new Personal Information and Data Tribunal, which is created in Part 2 of Bill C-27. This, at least, will provide some separation between the Minister and the imposition of financial penalties. If this is the plan, though, the draft law should say so.

It is clear that not all breaches of the obligations in the AIDA will be ones for which AMPs are available. Regulations will specify the breach of which provisions of the AIDA or its regulations will constitute a violation (s. 29(4)(a)). The regulations will also indicate whether the breach of the particular obligation is classified as minor, serious or very serious (s. 29(4)(b)). The regulations will also set out how any such proceedings will unfold. As-yet undrafted regulations will also specify the amounts or ranges of AMPS, and factors to take into account in imposing them.

This lack of important detail makes it hard not to think of the oversight and enforcement scheme in the AIDA as a rough draft sketched out on a cocktail napkin after an animated after-hours discussion of what enforcement under the AIDA should look like. Clearly, the goal is to be ‘agile’, but ‘agile’ should not be confused with slapdash. Parliament is being asked to enact a law that leaves many essential components undefined. With so much left to regulations, one wonders whether all the missing pieces can (or will) be put in place within this decade. There are instances of other federal laws left incomplete by never-drafted regulations. For example, we are still waiting for the private right of action provided for in Canada’s Anti-Spam Law, which cannot come into effect until the necessary regulations are drafted. A cynic might even say that failing to draft essential regulations is a good way to check the “enact legislation on this issue” box on the to-do list, without actually changing the status quo.

Published in Privacy

This is the third in my series of posts on the Artificial Intelligence and Data Act (AIDA) found in Bill C-27, which is part of a longer series on Bill C-27 generally. Earlier posts on the AIDA have considered its purpose and application, and regulated activities. This post looks at the harms that the AIDA is designed to address.

The proposed Artificial Intelligence and Data Act (AIDA), which is the third part of Bill C-27, sets out to regulate ‘high-impact’ AI systems. The concept of ‘harm’ is clearly important to this framework. Section 4(b) of the AIDA states that a purpose of the legislation is “to prohibit certain conduct in relation to artificial intelligence systems that may result in serious harm to individuals or harm to their interests”.

Under the AIDA, persons responsible for high-impact AI systems have an obligation to identify, assess, and mitigate risks of harm or biased output (s. 8). Those persons must also notify the Minister “as soon as feasible” if a system for which they are responsible “results or is likely to result in material harm”. There are also a number of oversight and enforcement functions that are triggered by harm or a risk of harm. For example, if the Minister has reasonable grounds to believe that a system may result in harm or biased output, he can demand the production of certain records (s. 14). If there is a serious risk of imminent harm, the Minister may order a person responsible to cease using a high impact system (s. 17). The Minister is also empowered to make public certain information about a system where he believes that there is a serious risk of imminent harm and the publication of the information is essential to preventing it (s. 28). Elevated levels of harm are also a trigger for the offence in s. 39, which involves “knowing or being reckless as to whether the use of an artificial intelligence system is likely to cause serious physical or psychological harm to an individual or substantial damage to an individual’s property”.

‘Harm’ is defined in s. 5(1) to mean:

(a) physical or psychological harm to an individual;

(b) damage to an individual’s property; or

(c) economic loss to an individual.

I have emphasized the term “individual” in this definition because it places an important limit on the scope of the AIDA. First, it is unlikely that the term ‘individual’ includes a corporation. Typically, the word ‘person’ is considered to include corporations, and the word ‘person’ is used in this sense in the AIDA. This suggests that “individual” is meant to have a different meaning. The federal Interpretation Act is silent on the issue. It is a fair interpretation of the definition of ‘harm’ that “individual” is not the same as “person”, and means an individual (human) person. The French version uses the term “individu”, and not “personne”. The harms contemplated by this legislation are therefore to individuals and not to corporations.

Defining harm in terms of individuals has other ramifications. The AIDA defines high-risk AI systems in terms of their impacts on individuals. Importantly, this excludes groups and communities. It also very significantly focuses on what are typically considered quantifiable harms, and uses language that suggests quantifiability (economic loss, damage to property, physical or psychological harm). Some important harms may be difficult to establish or to quantify. For example, class action lawsuits relating to significant data breaches have begun to wash up on the beach of lost causes due to the impossibility of proving material loss either because, although thousands may have been impacted, the individual losses are impossible to quantify, or because it is impossible to prove a causal link between very real identity theft and that particular data breach. Consider an AI system that manipulates public opinion through an algorithm that drives content to individuals based on its shock value rather than its truth. Say this happens during a pandemic and it convinces people that they should not get vaccinated or take other recommended public health measures. Say some people die because they were misled in this way. Say other people die because they were exposed to infected people who were misled in this way. How does one prove the causal link between the physical harm of injury or death of an individual and the algorithm? What if there is an algorithm that manipulates voter sentiment in a way that changes the outcome of an election? What is the quantifiable economic loss or psychological harm to any individual? How could causation be demonstrated? The harm, once again, is collective.

The EU AI Act has also been criticized for focusing on individual harm, but the wording of that law is still broader than that in the AIDA. The EU AI Act refers to high-risk systems in terms of “harm to the health and safety or a risk of adverse impact on fundamental rights of persons”. This at least introduces a more collective dimension, and it avoids the emphasis on quantifiability.

The federal government’s own Directive on Automated Decision-Making (DADM) which is meant to guide the development of AI used in public sector automated decision systems (ADS) also takes a broader approach to impact. In assessing the potential impact of an ADS, the DADM takes into account: “the rights of individuals or communities”, “the health or well-being of individuals or communities”, “the economic interests of individuals, entities, or communities”, and “the ongoing sustainability of an ecosystem”.

With its excessive focus on individuals, the AIDA is simply tone deaf to the growing global understanding of collective harm caused by the use of human-derived data in AI systems.

One response of the government might be to point out that the AIDA is also meant to apply to “biased output”. Biased output is defined in the AIDA as:

content that is generated, or a decision, recommendation or prediction that is made, by an artificial intelligence system and that adversely differentiates, directly or indirectly and without justification, in relation to an individual on one or more of the prohibited grounds of discrimination set out in section 3 of the Canadian Human Rights Act, or on a combination of such prohibited grounds. It does not include content, or a decision, recommendation or prediction, the purpose and effect of which are to prevent disadvantages that are likely to be suffered by, or to eliminate or reduce disadvantages that are suffered by, any group of individuals when those disadvantages would be based on or related to the prohibited grounds. (s. 5(1)) [my emphasis]

The argument here will be that the AIDA will also capture discriminatory biases in AI. However, I have underlined the part of this definition that once again returns the focus to individuals, rather than groups. It can be very hard for an individual to demonstrate that a particular decision discriminated against them (especially if the algorithm is obscure). In any event, biased AI will tend to replicate systemic discrimination. Although it will affect individuals, it is the collective impact that is most significant – and this should be recognized in the law. The somewhat obsessive focus on individual harm in the AIDA may unwittingly help perpetuate denials of systemic discrimination.

It is also important to note that the definition of “harm” does not include “biased output”, and while the terms are used in conjunction in some cases (for example, in s. 8’s requirement to “identify, assess and mitigate the risks of harm or biased output”), other obligations relate only to “harm”. Since the two are used conjunctively in some parts of the statute, but not others, a judge interpreting the statute might presume that when only one of the terms is used, then it is only that term that is intended. Section 17 of the AIDA allows the Minister to order a person responsible for a high-impact system to cease using it or making it available if there is a “serious risk of imminent harm”. Section 28 permits the Minister to order the publication of information related to an AI system where there are reasonable grounds to believe that the use of the system gives rise to “a serious risk of imminent harm”. In both cases, the defined term ‘harm’ is used, but not ‘biased output’.

The goals of the AIDA to protect against harmful AI are both necessary and important, but in articulating the harm that it is meant to address, the Bill underperforms.

Published in Privacy
<< Start < Prev 1 2 Next > End >>
Page 1 of 2

Canadian Trademark Law

Published in 2015 by Lexis Nexis

Canadian Trademark Law 2d Edition

Buy on LexisNexis

Electronic Commerce and Internet Law in Canada, 2nd Edition

Published in 2012 by CCH Canadian Ltd.

Electronic Commerce and Internet Law in Canada

Buy on CCH Canadian

Intellectual Property for the 21st Century

Intellectual Property Law for the 21st Century:

Interdisciplinary Approaches

Purchase from Irwin Law