Teresa Scassa - Blog

Displaying items by tag: AI and Data Act

My previous post looked at some of the new definitions in the proposed amendments to the Artificial Intelligence and Data Act (AIDA) which is Part III of Bill C-27. These include a definition of “high impact” AI, and a schedule of classes of high-impact AI (the Schedule is reproduced at the end of this post). The addition of the schedule changes AIDA considerably, and that is the focus of this post.

The first two classes in the Schedule capture contexts that can clearly affect individuals. Class 1 addresses AI used in most aspects of employment, and Class 2 relates to the provision of services. On the provision of services (which could include things like banking and insurance), the wording signals that it will apply to decision-making about the provision of services, their cost, or the prioritization of recipients. To be clear, AIDA does not prohibit systems with these functions. They are simply characterized as “high impact” so that they will be subject to governance obligations. A system to determine creditworthiness can still reject individuals; and companies can still prioritize preferred customers – as long as the systems are sufficiently transparent, free from bias and do not cause harm.

There is, however, one area which seems to fall through the cracks of Classes 1 & 2: rental accommodation. A lease is an interest in land – it is not a service. Human rights legislation in Canada typically refers to accommodation separately from services for this reason. AI applications are already being used to screen and select tenants for rental accommodation. In the midst of a housing crisis, this is surely an area that is high-impact and where the risks of harm from flawed AI to individuals and families searching for a place to live are significant. This gap needs to be addressed – perhaps simply by adding “or accommodation” after each use of the term “service” in Class 2.

Class 3 rightly identifies biometric systems as high risk. It also includes systems that use biometrics in “the assessment of an individual’s behaviour or state of mind.” Key to the scope of this section will be the definition of “biometric”. Some consider biometric data to be exclusively physiological data (fingerprints, iris scans, measurements of facial features, etc.). Yet others include behavioral data in this class if it is used for the second identified purpose – the assessment of behaviour or state of mind. Behavioural data, though, is potentially a very broad category. It can include data about a person’s gait, or their speech or keystroke patterns. Cast even more broadly, it could include things such as “geo-location and IP addresses”, “purchasing habits”, “patterns of device use” or even “browser history and cookies”. If that is the intention behind Class 3, then conventional biometric AI should be Part One of this class; Part Two should be the use of an AI system to assess an individual’s behaviour or state of mind (without referring specifically to biometrics in order to avoid confusion). This would also, importantly, capture the highly controversial area of AI for affect recognition. It would be unfortunate if the framing of the class as ‘biometrics’ led to an unduly narrow interpretation of the kind of systems or data involved. The explanatory note in the Minister’s cover letter for this provision seems to suggest (although it is not clear) that it is purely physiological biometric data that is intended for inclusion and not a broader category. If this is so, then Class 3 seems unduly narrow.

Class 4 is likely to be controversial. It addresses content moderation and the prioritization and presentation of content online and identifies these as high-impact algorithmic activities. Such systems are in widespread use in the online context. The explanatory note from the Minister observes that such systems “have important potential impacts on Canadians’ ability to express themselves, as well as pervasive effects at societal scale” (at p. 4). This is certainly true although the impact is less direct and obvious than the impact of a hiring algorithm, for example. Further, although an algorithm that presents a viewer of online streaming services with suggestions for content could have the effect of channeling a viewer’s attention in certain directions, it is hard to see this as “high impact” in many contexts, especially since there are multiple sources of suggestions for online viewing (including word of mouth). That does not mean that feedback loops and filter bubbles (especially in social media) do not contribute to significant social harms – but it does make this high impact class feel large and unwieldy. The Minister’s cover letter indicates that each of the high-impact classes presents “distinct risk profiles and consequently will require distinct risk management strategies.” (at p. 2). Further, he notes that the obligations that will be imposed “are intended to scale in proportion to the risks they present. A low risk use within a class would require correspondingly minimal mitigation effort.” (at p. 2). Much will clearly depend on regulations.

Class 5 relates to the use of AI in health care or emergency services, although it explicitly excludes medical devices because these are already addressed by Health Canada (which recently consulted on the regulation of AI-enabled medical devices). This category also demonstrates some of the complexity of regulating AI in Canada’s federal system. Many hospital-based AI technologies are being developed by researchers affiliated with the hospitals and who are not engaged in the interprovincial or international trade and commerce which is necessary for AIDA to apply. AIDA will only apply to those systems developed externally and in the context of international or interprovincial trade and commerce. While this will still capture many applications, it will not capture all – creating different levels of governance within the same health care context.

It is also not clear what is meant, in Class 5, by “use of AI in matters relating to health care”. This could be interpreted to mean health care that is provided within what is understood as the health care system. Understood more broadly, it could extend to health-related apps – for example, one of the many available AI-enabled sleep trackers, or an AI-enabled weight loss tool (to give just two examples). I suspect that what is intended is the former, even though, with health care in crisis and more people turning to alternate means to address their health issues, health-related AI technologies might well deserve to be categorized as high-impact.

Class 6 involves the use of an AI system by a court or administrative body “in making a determination in respect of an individual who is a party to proceedings before the court or administrative body.” In the first place, this is clearly not meant to apply to automated decision-making generally – it seems to be limited to judicial or quasi-judicial contexts. Class 6 must also be reconciled with s. 3 of AIDA, which provides that AIDA does not apply “with respect to a government institution as defined in s. 3 of the Privacy Act.” This includes the Immigration and Refugee Board, for example, as well as the Canadian Human Rights Commission, the Parole Board, and the Veterans Review and Appeal Board. Making sense of this, then, it would be the tools used by courts or tribunals and developed or deployed in the course of interprovincial or international trade and commerce that would be considered high impact. The example given in the Minister’s letter seems to support this – it is of an AI system that provides an assessment of “risk of recidivism based on historical data” (at p. 5).

However, Class 6 is confusing because it identifies the context rather than the tools as high impact. Note that the previous classes address the use of AI “in matters relating to” the subject matter of the class, whereas class 6 identifies actors – the use of AI by a court or tribunal. There is a different focus. Yet the same tools used by courts and tribunals might also be used by administrative bodies or agencies that do not hold hearings or that are otherwise excluded from the application of AIDA. For example, in Ewert v. Canada, the Supreme Court of Canada considered an appeal by a Métis man who challenged the use of recidivism-risk assessment tools by Correctional Services of Canada (to which AIDA would not apply according to s. 3). If this type of tool is high-risk, it is so whether it is used by Correctional Services or a court. This suggests that the framing of Class 6 needs some work. It should perhaps be reworded to identify tools or systems as high impact if they are used to determine the rights, entitlements or status of individuals.

Class 7 addresses the use of an AI system to assist a peace officer “in the exercise and performance of their law enforcement powers, duties and function”. Although “peace officer” receives the very broad interpretation found in the Criminal Code, that definition is modified in the AIDA by language that refers to the exercise of specific law enforcement powers. This should still capture the use of a broad range of AI-enabled tools and technologies. It is an interesting question whether AIDA might apply more fulsomely to this class of AI systems (not just those developed in the course of interprovincial or international trade) as it might be considered to be rooted in the federal criminal law power.

These, then, are the different classes that are proposed initially to populate the Schedule if AIDA and its amendments are passed. The list is likely to spark debate, and there is certainly some wording that could be improved. And, while it provides much greater clarity as to what is proposed to be regulated, it is also evident that the extent to which obligations will apply will likely be further tailored in regulations to create sliding scales of obligation depending on the degree of risk posed by any given system.

AIDA Schedule:

High-Impact Systems — Uses

1. The use of an artificial intelligence system in matters relating to determinations in respect of employment, including recruitment, referral, hiring, remuneration, promotion, training, apprenticeship, transfer or termination.

2. The use of an artificial intelligence system in matters relating to

(a) the determination of whether to provide services to an individual;

(b) the determination of the type or cost of services to be provided to an individual; or

(c) the prioritization of the services to be provided to individuals.

3. The use of an artificial intelligence system to process biometric information in matters relating to

(a) the identification of an individual, other than in cases in which the biometric information is processed with the individual’s consent to authenticate their identity; or

(b) the assessment of an individual’s behaviour or state of mind.

4. The use of an artificial intelligence system in matters relating to

(a) the moderation of content that is found on an online communications platform, including a search engine or social media service; or

(b) the prioritization of the presentation of such content.

5. The use of an artificial intelligence system in matters relating to health care or emergency services, excluding a use referred to in any of paragraphs (a) to (e) of the definition device in section 2 of the Food and Drugs Act that is in relation to humans.

6. The use of an artificial intelligence system by a court or administrative body in making a determination in respect of an individual who is a party to proceedings before the court or administrative body.

7. The use of an artificial intelligence system to assist a peace officer, as defined in section 2 of the Criminal Code, in the exercise and performance of their law enforcement powers, duties and functions.

Published in Privacy

Note: This is the first in a series of posts that will look at the proposed amendments to Canada's Artificial Intelligence and Data Act, which is Part III of Bill C-27, currently before Parliament. The amendments are extensive and have only just been introduced. Please consider these assessments to be preliminary.

 

Canada’s Artificial Intelligence and Data Act (AIDA) (Part III of Bill C-27) has passed second reading and is currently before the INDU Committee for study. Early in this committee process, the Minister of Industry Philippe Champagne announced that his department was working on amendments to AIDA in response to considerable criticism. Those amendments have now been tabled for consideration by the committee.

One of the criticisms of the Bill was that it left almost all of its substance to be developed in regulations. It is unsurprising, then, that the amendments are almost as long as the original bill. While it is certainly the case that the amendments contain more detail than the original text, some of the additional length is attributable to new provisions intended to address generative AI systems. This highlights just how quickly things are moving in the AI space, as generative AI was not on anyone’s legislative radar when Bill C-27 was introduced in June 2022.

One of the criticisms of AIDA was the absence of any specific prior consultation before its appearance in Bill C-27. This, combined with its lack of substance on many issues, raised basic concerns about how it would apply and to what. For example, AIDA was to govern “high-impact” AI systems, but the definition of such systems was left to regulations. Concerns were also raised about oversight being largely in the hands of the Minister of Industry who is also responsible for supporting Canada’s AI sector.

The proposed amendments demonstrate that ISED has been listening to the feedback it has received since June 2022, just as it has been adapting to the challenges of generative AI, and engaging with its international partners on AI governance issues. The amendments, which include new definitions, more explicit obligations, and governance principles for generative AI, will make AIDA a better bill. They may be enough to garner sufficient support to pass it into law, something which the Minister describes as “pivotal”.

This is the first in a series of posts that will explore some of the changes proposed to AIDA – as well as some of the remaining issues. This post addresses some of the new definitions.

The proposed AIDA amendments propose a new definition of “artificial intelligence system” which would read: “a technological system that, using a model, makes inferences in order to generate output, including predictions, recommendations or decisions” (s. 2). This provides greater alignment with the OECD definition of an AI system (“An AI system is a machine-based system that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments. Different AI systems vary in their levels of autonomy and adaptiveness after deployment.”) It is an improvement over the previous definition, which was criticized for being too specific about the types of techniques used in AI. It is unclear, though, why the new AIDA definition does not include “content” as an output as is the case with the OECD definition. The AIDA definition is also supplemented by a separate definition for a “general-purpose system”, which is “an artificial intelligence system that is designed to be adapted for use, in many fields and for many purposes and activities, including fields, purposes and activities not contemplated during the system’s development” (s. 5(1)). There is a further definition for a “machine learning model”, which is “a digital representation of patterns identified in data through the automated processing of the data using an algorithm designed to enable the recognition or replication of those patterns”. A new s. 5(2) makes it clear that the definition of AI system includes general-purpose systems, and that general-purpose systems can also be high-impact. These new definitions reflect the major changes in both the technology and in the evolving regulatory context in the short time since AIDA was introduced. They also shape a new framework for obligations under the legislation.

The proposed amendments also contain a definition of “high-impact system”: “an artificial intelligence system of which at least one of the intended uses may reasonably be concluded to fall within a class of uses set out in the schedule”. (s. 5(1)). The previous version of AIDA left the articulation of “high impact” to future regulations. The schedule sets out a list of classes that describe certain uses. These are:

High-Impact Systems — Uses

1. The use of an artificial intelligence system in matters relating to determinations in respect of employment, including recruitment, referral, hiring, remuneration, promotion, training, apprenticeship, transfer or termination.

2. The use of an artificial intelligence system in matters relating to

(a) the determination of whether to provide services to an individual;

(b) the determination of the type or cost of services to be provided to an individual; or

(c) the prioritization of the services to be provided to individuals.

3. The use of an artificial intelligence system to process biometric information in matters relating to

(a) the identification of an individual, other than in cases in which the biometric information is processed with the individual’s consent to authenticate their identity; or

(b) the assessment of an individual’s behaviour or state of mind.

4. The use of an artificial intelligence system in matters relating to

(a) the moderation of content that is found on an online communications platform, including a search engine or social media service; or

(b) the prioritization of the presentation of such content.

5. The use of an artificial intelligence system in matters relating to health care or emergency services, excluding a use referred to in any of paragraphs (a) to (e) of the definition device in section 2 of the Food and Drugs Act that is in relation to humans.

6. The use of an artificial intelligence system by a court or administrative body in making a determination in respect of an individual who is a party to proceedings before the court or administrative body.

7. The use of an artificial intelligence system to assist a peace officer, as defined in section 2 of the Criminal Code, in the exercise and performance of their law enforcement powers, duties and functions.

(Note: the classes in this schedule will be the subject of the next blog post)

The list is not intended to be either closed or permanent. Under a proposed s. 36.1, the Governor in Council (GinC) can enact regulations amending the schedule by adding, modifying, or deleting a category of use. Any such decision by the GinC is to be guided by criteria set out in s. 36.1. These include the risks of adverse impact on “the economy or any other aspect of Canadian society and on individuals, including on individual’s health and safety and on their rights recognized in international human rights treaties to which Canada is a party”. The GinC must also consider the “severity and extent” of any adverse impacts, as well as the “social and economic circumstances of any individuals who may experience” such impacts. A final consideration is whether the uses in the category are adequately addressed under another Act of Parliament or of a provincial legislature.

The AIDA only applies to “high impact” systems, and since there is no screening or registration process, it is up to those who manage or make such systems available to identify them as such and to meet the obligations set out in the law. A proposed s. 14 would empower the AI and Data Commissioner to order a person who makes available or who manages an AI system to provide the Commissioner with their assessment of whether the system is a high impact system, a general purpose system (which can also be high impact), or a machine learning model intended to be incorporated into a high impact system.

My next post will look at the classes of “high-impact” AI as set out in the Schedule.

Published in Privacy

The government of the United Kingdom has published a consultation paper seeking input into its proposal for AI regulation. The paper is aptly titled A pro-innovation approach to AI regulation, since it restates that point insistently throughout the document. The UK proposal provides an interesting contrast to Canada’s AI governance bill currently before Parliament.

Both Canada and the UK set out to regulate AI systems with the twin goals of supporting innovation on the one hand, and building trust in AI on the other. (Note here that the second goal is to build trust in AI, not to protect the public. Although the protection of the public is acknowledged as one way to build trust, there is a subtle distinction here). However, beyond these shared goals, the proposals are quite different. Canada’s approach in Part 3 of Bill C-27 (the Artificial Intelligence and Data Act (AIDA)) is to create a framework to regulate as yet undefined “high impact” AI. The definition of “high impact” as well as many other essential elements of the bill are left to be articulated in regulations. According to a recently published companion document to the AIDA, leaving so much of the detail to regulations is how the government proposes to keep the law ‘agile’ – i.e. capable of responding to a rapidly evolving technological context. The proposal would also provide some governance for anonymized data by imposing general requirements to document the use of anonymized personal information in AI innovation. The Minister of Innovation is made generally responsible for oversight and enforcement. For example, the AIDA gives the Minister of Innovation the authority (eventually) to impose stiff administrative monetary penalties on bad actors. The Canadian approach is similar to that in the EU AI Act in that it aims for a broad regulation of AI technologies, and it chooses legislation as the vehicle to do so. It is different in that the EU AI Act is far more detailed and prescriptive; the AIDA leaves the bulk of its actual legal requirements to be developed in regulations.

The UK proposal is notably different from either of these approaches. Rather than create a new piece of legislation and/or a new regulatory authority, the UK proposes to set out five principles for responsible AI development and use. Existing regulators will be encouraged and, if necessary, specifically empowered, to regulate AI according to these principles within their spheres of regulatory authority. Examples of regulators who will be engaged in this framework include the Information Commissioner’s Office, regulators for human rights, consumer protection, health care products and medical devices, and competition law. The UK scheme also accepts that there may need to be an entity within government that can perform some centralized support functions. These may include monitoring and evaluation, education and awareness, international interoperability, horizon scanning and gap analysis, and supporting testbeds and sandboxes. Because of the risk that some AI technologies or issues may fall through the cracks between existing regulatory schemes, the government anticipates that regulators will assist government in identifying gaps and proposing appropriate actions. These could include adapting the mandates of existing regulators or providing new legislative measures if necessary.

Although Canada’s federal government has labelled its approach to AI regulation as ‘agile’, it is clear that the UK approach is much closer to the concept of agile regulation. Encouraging existing regulators to adapt the stated AI principles to their remit and to provide guidance on how they will actualize these principles will allow them to move quickly, so long as there are no obvious gaps in legal authority. By contrast, even once passed, it will take at least two years for Canada’s AIDA to have its normative blanks filled in by regulations. And, even if regulations might be somewhat easier to update than statutes, guidance is even more responsive, giving regulators greater room to manoeuvre in a changing technological landscape. Embracing the precepts of agile regulation, the UK scheme emphasizes the need to gather data about the successes and failures of regulation itself in order to adapt as required. On the other hand, while empowering (and resourcing) existing regulators will have clear benefits in terms of agility, the regulatory gaps could well be important ones – with the governance of large language models such as ChatGPT as one example. While privacy regulators are beginning to flex their regulatory muscles in the direction of ChatGPT, data protection law will only address a subset of the issues raised by this rapidly evolving technology. In Canada, AIDA’s governance requirements will be specific to risk-based regulation of AI, and will apply to all those who design, develop or make AI systems available for use (unless of course they are explicitly excluded under one of the many actual and potential exceptions).

Of course, the scheme in the AIDA may end up as more of a hybrid between the EU and the UK approaches in that the definition of “high impact” AI (to which the AIDA will apply) may be shaped not just by the degree of impact of the AI system at issue but also by the existence of other suitable regulatory frameworks. In other words, the companion document suggests that some existing regulators (health, consumer protection, human rights, financial institutions) have already taken steps to extend their remit to address the use of AI technologies within their spheres of competence. In this regard, the companion document speaks of “regulatory gaps that must be filled” by a statute such as AIDA as well as the need for the AIDA to integrate “seamlessly with existing Canadian legal frameworks”. Although it is still unclear whether the AIDA will serve only to fill regulatory gaps, or will provide two distinct layers of regulation in some cases, one of the criteria for identifying what constitutes a “high impact” system includes “[t]he degree to which the risks are adequately regulated under another law”. The lack of clarity in the Canadian approach is one of its flaws.

There is a certain attractiveness in the idea of a regulatory approach like that proposed by the UK – one that begins with existing regulators being both specifically directed and further enabled to address AI regulation within their areas of responsibility. As noted earlier, it seems far more agile than Canada’s rather clunky bill. Yet such an approach is much easier to adopt in a unitary state than in a federal system such as Canada’s. In Canada, some of the regulatory gaps are with respect to matters otherwise under provincial jurisdiction. Thus, it is not so simple in Canada to propose to empower and resource all implicated regulators, nor is it as easy to fill gaps once they are identified. These regulators and the gaps between them might fall under the jurisdiction of any one of 13 different governments. The UK acknowledges (and defers) its own challenges in this regard with respect to devolution at paragraph 113 of its white paper, where it states: “We will continue to consider any devolution impacts of AI regulation as the policy develops and in advance of any legislative action”. Instead, the AIDA, Canada leverages its general trade and commerce power in an attempt to provide AI governance that is as comprehensive as possible. It isn’t pretty (since it will not capture all AI innovation that might have impacts on people) but it is part of the reality of the federal state (or the state of federalism) in which we find ourselves.

Published in Privacy

Canadian Trademark Law

Published in 2015 by Lexis Nexis

Canadian Trademark Law 2d Edition

Buy on LexisNexis

Electronic Commerce and Internet Law in Canada, 2nd Edition

Published in 2012 by CCH Canadian Ltd.

Electronic Commerce and Internet Law in Canada

Buy on CCH Canadian

Intellectual Property for the 21st Century

Intellectual Property Law for the 21st Century:

Interdisciplinary Approaches

Purchase from Irwin Law