ITU publishes new standard for assessing AI foundation models
A new ITU technical recommendation introduces a framework to evaluate the performance, reliability and security of large AI models used across digital services and applications.
The International Telecommunication Union (ITU) approved Recommendation ITU-T F.748.77 on 14 December 2025, establishing a general framework for assessing AI foundation models. The recommendation was published on 11 March 2026, as part of the ITU-T standards series on multimedia services.
The document outlines criteria and procedures for evaluating large AI systems trained on broad datasets and used across multiple digital applications. These so-called foundation models underpin many emerging technologies, including generative AI tools, multimodal systems and automated decision-making services.
According to the recommendation, the rapid expansion of foundation models in sectors such as language processing, image generation and automated services has created the need for objective and transparent evaluation methods. The standard, therefore, proposes a structured framework to measure the capabilities and performance of such systems.
The framework defines a workflow for evaluating AI models, starting with analysing assessment requirements and preparing testing environments and datasets. The process then proceeds through model testing, result analysis and the potential publication of evaluation outcomes.
To assess digital systems built on foundation models, the recommendation introduces several key evaluation dimensions, including functionality, accuracy, reliability, security, interactivity and real-world applicability. These criteria aim to capture how AI models perform across different tasks, how robust they are to errors or adversarial inputs, and whether they can operate safely in practical deployment environments.
The document also highlights the importance of diverse datasets, transparent evaluation tools and reproducible testing methods. Assessment tools should produce consistent results, disclose evaluation metrics and avoid bias toward particular models or technologies.
By setting out a common assessment framework, ITU recommendation seeks to support comparability, accountability and responsible development of large AI systems, as governments, companies and researchers increasingly rely on foundation models in digital services and infrastructure.
