EU Regulation on Artificial Intelligence
On April 21, 2021 the European Commission published its Proposal for a Regulation on Artificial Intelligence, the first ever legal framework on AI. With its Proposal the European Commission addresses the risks of AI systems and positions Europe to play a leading role in setting standard for the regulation of AI globally.
The Regulation follows the well-known risk-based approach the European legislator already implemented for a range of product categories, such as those implemented in the Medical Devices Regulation, including differentiating between high-risk AI, limited-risk AI, and minimal-risk AI. Depending on the risk level involved, providers of AI systems must employ different safety measures to ensure the trustworthiness of their AI system.
High-Risk AI Systems
According to the proposal, AI systems that create a high risk to the health and safety or fundamental rights of persons will be permitted on the European market subject to an ex-ante conformity assessment. Such high-risk AI systems are:
- AI systems intended to be used as safety components of products that are subject to a third party ex-ante conformity assessment, such as medical devices, machinery, motor vehicles, radio equipment and personal protecting equipment (AI components);
- other stand-alone AI systems with mainly fundamental rights implications, such as remote biometric identification, educational and vocational training, employment and law enforcement.
High-risk AI systems will be subject to strict obligations before they can be put on the market:
- Adequate risk assessment and mitigation systems;
- High quality datasets feeding the system to minimize risks and discriminatory outcomes;
- Logging of activity to ensure traceability of results;
- Detailed documentation explaining the system’s purpose and providing all other information necessary for authorities to assess its compliance;
- Clear and adequate information provided to the user;
- Appropriate human oversight measures to minimize risk;
- High level of robustness, security and accuracy.
Providers of such high-risk AI systems will have to evaluate whether the AI system complies with these obligations prior to placing the AI system on the market. In case of AI components, conformity assessment of the AI system will be part of the conformity assessment of the overall product. The conformity assessment must be renewed, wherever AI-systems are substantially modified. For AI-systems that continue to learn after being placed on the market, changes to the high-risk AI and its performance that have been predetermined by the provider at the moment of the initial conformity assessment and are part of the information contained in the technical documentation do not require recertification.
Also, providers of high-risk AI systems must have a post-market monitoring system in place, where providers actively collect, document and analyze relevant data throughout their lifetime. Providers and users will also have to report serious incidents and malfunctioning. Stand-alone AI systems must be registered in an EU database.
Certain high-risk AI systems considered to be a clear threat to the safety, livelihoods and rights of people will be banned entirely. The prohibitions cover practices that have a significant potential to manipulate persons through subliminal techniques beyond their consciousness or exploit vulnerabilities of specific vulnerable groups such as children or persons with disabilities in order to materially distort their behavior in a manner that is likely to cause them or another person psychological or physical harm. The proposal also prohibits AI-based social scoring for general purposes done by public authorities. Finally, the use of ‘real time’ remote biometric identification systems in publicly accessible spaces for the purpose of law enforcement is also prohibited unless certain limited exceptions apply.
Limited-Risk AI Systems
Under the proposed Artificial Intelligence Act, AI systems with risk of manipulation will have to comply with specific transparency obligations. Such transparency obligations will apply for systems that:
- interact with humans,
- are used to detect emotions or determine association with (social) categories based on biometric data, or
- generate or manipulate content (‘deep fakes’).
When using such AI systems, generally people must be informed of in advance. This allows persons to make informed choices or step back from a given situation.
Minimal-Risk AI Systems
The proposed Artificial Intelligence Act allows the free use of all other applications such as AI-enabled video games or spam filters. Such minimal-risk AI systems can be developed and used subject to the existing legislation and without additional legal obligations. Voluntarily, providers of such minimal-risk AI systems may choose to apply the requirements for trustworthy AI and adhere to voluntary codes of conduct.
In the further legislative procedure, the European Parliament and the Member States will need to adopt the Commission’s proposal.
The implementation of the Artificial Intelligence Act will have a major impact on the market for AI systems. Companies will no longer be free in the assessment of safety of their products using AI software, but will have to observe the strict requirements of the new regulatory framework. Now is the time to get familiar with the proposed legislation, raise potential concerns to the attention of the lawmaking bodies and prepare for compliance with the new rules and obligation.
If you have questions about the Proposal Regulation on Artificial Intelligence, do not hesitate to reach out to our EU lawyers Elisabeth Kohoutek and Ulf Grundmann.