The National Institute of Standards and Technology (NIST) will host a workshop on Mitigating AI Bias in Context on Wednesday, August 31, 2022 at the National Cybersecurity Center of Excellence (NCCoE). Managing bias in an artificial intelligence (AI)/machine learning (ML) system is critical to establishing and maintaining trust in its operation. Despite its importance, bias in AI systems remains endemic across many application domains and can lead to harmful impacts regardless of intent. Bias is also context-dependent.
To tackle this complex problem, we are preparing to launch a project adopting a comprehensive socio-technical approach to testing, evaluation, verification, and validation (TEVV) of AI systems in context for detecting bias. This approach will connect the technology to societal values in order to develop guidance for recommended practices in deploying and using automated decision-making supported by AI/ML systems in a specific context (consumer and small business credit underwriting). These practices will help promote fair and positive outcomes that benefit users of AI/ML services, the organizations that deploy them, and all of society. A small but novel part of this project will be to look at the interplay between bias and cybersecurity and how they interact with each other, with the goal of identifying approaches which might mitigate risks that exist across these two critical characteristics of trustworthy AI.
The workshop will bring together experts from academia, industry, and government to discuss bias in AI and identify existing tools and technologies which could help mitigate this challenge to improve trust in AI systems. The workshop will focus on operational, real-world decision automation, bias-detection, and bias-mitigation tools. Additionally, the use and application of the NIST Dioptra testbed to this area will be discussed with the potential for the addition of new extensions providing new insights into the properties of an AI system. The feedback from this workshop will inform the development of a potential NCCoE demonstration project that will result in a public NIST AI/ML Practice Guide.