CDT Europe calls for revisions to the European Commission’s draft guidance on high-risk AI systems

The Centre for Democracy and Technology Europe (CDT Europe) has published its response to the European Commission’s draft Guidelines on the Classification of High-Risk AI Systems under Article 6 of the AI Act, calling for a number of revisions to improve legal certainty and ensure that high-risk AI systems are classified consistently with the objectives of the AI Act.

In its submission, CDT Europe argues that the Commission should maintain a careful interpretation of the high-risk use cases listed in Annex III of the AI Act while avoiding both overly broad and overly narrow readings. The organisation recommends clarifying when the filtering mechanism under Article 6(3) applies and stresses that AI systems which materially influence decision-making should not be excluded from the high-risk category, even where one of the filtering conditions appears to be met.

CDT Europe also criticises several examples in the draft guidance, arguing that some incorrectly suggest that human oversight can remove an AI system from the high-risk category. According to the organisation, the guidelines should place greater emphasis on the risk of automation bias, whereby human decision-makers may rely too heavily on AI-generated outputs despite formally retaining responsibility for the final decision.

Another area identified for clarification concerns substantial modification of AI systems. CDT Europe notes that general-purpose AI models may become high-risk when deployed, even if the original provider did not intend such use. The organisation recommends that the guidance specify when deployers assume the responsibilities of providers after modifying an AI system’s intended purpose.

The submission further argues that several exclusions in the draft guidelines interpret Annex III too narrowly. CDT Europe recommends expanding the scope of high-risk classification for AI systems used in biometric categorisation, educational assessment, welfare administration, and migration management. It argues, for example, that AI-powered formative assessments in education can significantly influence students’ learning outcomes and should therefore be treated as high-risk, on par with summative assessments. Similarly, it recommends that AI systems used to administer tax credits be considered within the scope of essential public assistance where they affect access to social support.

The organisation also opposes suggestions to exclude AI systems based on linear and logistic regression from the scope of Annex III. It argues that such systems can still have significant impacts on fundamental rights when used in areas such as credit scoring, insurance pricing, or welfare decisions, and therefore should remain subject to the AI Act’s high-risk requirements.

The recommendations were submitted as part of the European Commission’s public consultation on the draft guidelines, which are intended to help organisations determine whether AI systems fall within the AI Act’s high-risk classification and are therefore subject to the regulation’s stricter compliance obligations.