PyRIT, which stands for Python Risk Identification Tool, is an open-access automation framework that Microsoft has provided to help generative artificial intelligence (AI) systems proactively identify risks.
According to Microsoft's AI red team lead, Ram Shankar Siva Kumar, the red teaming tool is intended to “enable every organization across the globe to innovate responsibly with the latest artificial intelligence advances.”

According to the business, PyRIT can be used to evaluate the resilience of large language model (LLM) endpoints against several forms of harm, including forbidden content (e.g., harassment), misuse (e.g., bias), and fabrication (e.g., hallucinations).

It can also be used to identify privacy damages such as identity theft and security harms like malware development and jailbreaking.

Five interfaces are included with PyRIT: target, datasets, scoring engine, support for numerous attack tactics, and memory component that stores intermediate input and output interactions in a database or JSON format.

Additionally, the scoring engine provides red teamers with two ways to score the target AI system's outputs: they can utilize an LLM endpoint for self-evaluation or a traditional machine learning classifier.

"The goal is to allow researchers to have a baseline of how well their model and entire inference pipeline is doing against different harm categories and to be able to compare that baseline to future iterations of their model," Microsoft stated.

“This allows them to have empirical data on how well their model is doing today, and detect any degradation of performance based on future improvements.”

Having said that, the tech behemoth is cautious to highlight that PyRIT is meant to supplement a red team's current subject experience and should not be used in place of manual red teaming of generative AI systems.

Stated differently, the tool's purpose is to identify risk “hot spots” by producing prompts that can be used to assess the AI system and identify areas that need more research.

Microsoft also pointed out the significant variations in generative AI system topologies and said that red-teaming generative AI systems necessitates probing for both security and responsible AI issues concurrently and that the practice is more probabilistic.

“Manual probing, though time-consuming, is often needed for identifying potential blind spots,” Siva Kumar stated. “Automation is needed for scaling but is not a replacement for manual probing.”

The announcement coincided with Protect AI disclosing many serious flaws that potentially allow for arbitrary code execution and the exposure of private data in well-known AI supply chain systems such as Hugging Face, MLflow, Triton Inference Server, and ClearML.