To broaden its support of collaborative research in artificial intelligence (AI), the National Institute of Standards and Technology (NIST) is extending the scope of an AI-focused consortium it founded two years ago and calling for new members.
Formerly known as the AI Safety Institute Consortium (AISIC), the newly renamed NIST Artificial Intelligence Consortium reflects the group's augmented goals. While the consortium will continue some of its previous work, it will concentrate on AI measurement, innovation and adoption. Among other efforts, the group will focus on building an AI evaluation ecosystem, investing in AI-enabled science and promoting the use of U.S.-developed AI technology and systems.
In addition to expanding the consortium's mission, NIST is seeking to expand its membership and inviting new members to join.
"We are inviting technically capable organizations to join the NIST AI Consortium to address the challenges associated with the development and deployment of AI," said Deputy NIST Director Craig Burkhardt. "To encourage more extraordinary AI technological innovations, NIST is seeking to expand its AI measurement efforts by harnessing the broader community's interests and capabilities."
NIST initially announced its establishment of the Artificial Intelligence Safety Institute Consortium in 2023, bringing together more than 280 organizations to develop science-based and empirically backed guidelines and standards for AI measurement and laying a foundation for global AI metrology.
Reorienting the consortium is a step toward implementing NIST's Strategy for American Technology Leadership in the 21st Century, which will accelerate the progress of critical and emerging technologies from development to adoption in close partnership with U.S. industry. In support of NIST's directives under the National Artificial Intelligence Initiative Act of 2020, in accordance with Executive Order 14179 in 2025, and as mandated by America's AI Action Plan, NIST and the AI Consortium will support the collaborative establishment of a new measurement science that will enable the identification of proven, scalable and interoperable techniques and metrics to promote the development and use of AI.
Six task groups will perform the work of the newly reorganized consortium:
- The AI Testing, Evaluation, Verification and Validation (AI TEVV) Zero Draft Task Group will provide the community with tools to determine whether an AI system meets its design requirements and is sufficient for its intended use. "Zero drafts" are preliminary, stakeholder-driven drafts of standards that are as thorough as possible and will be submitted into the private sector-led standardization process.
- The Annotation for AI Risks & Validity Task Group will support the development of a science-based tool kit for assessing the risks and impacts of AI for use with NIST's ARIA program.
- The AI Evaluation and Measurement Methods Task Group will identify and prioritize gaps, barriers and open questions across the field of AI evaluation science based on input from a range of organization types, sectors and stakeholder roles.
- The Bias Effects and Notable Generative AI Limitations (BENGAL) Group will team with IARPA to explore scalable solutions to misinformation, sensitive information leakage, flawed and inaccurate reasoning, and susceptibility to attack. The goal of BENGAL is to enable confident, effective use of large language models for intelligence analysis.
- The AI Documentation Cards Task Group will provide standardized, practical templates for documenting AI datasets, AI models, AI systems and AI TEVV, based on existing best practices and community input.
- NIST is also restarting the Chemical and Biological Security Task Group from the consortium's previous incarnation to share insights on emerging AI measurement and evaluation approaches relevant to chemical and biological security.
NIST is inviting letters of interest in the consortium from all organizations with relevant technical capabilities. The agency will select participants that have submitted complete letters of interest on a first-come, first-served basis. Existing members do not have to reapply but must sign an amendment agreeing to the changes that have been made to the consortium. Member organizations will enter into a Cooperative Research and Development Agreement (CRADA) with NIST. Details on how to submit a letter of interest are available at the project website. Additional information is provided in the Federal Register notice.
