AIKIDO-2026-838069
inspect-ai is vulnerable to Remote Code Execution (RCE)
98
Critical Risk
This Affects:
inspect-aiTL;DR
The math() scorer passes normalized model answer text to SymPy's parse_expr() to check mathematical equivalence. Because parse_expr() evaluates its input, a model completion containing a crafted expression such as ().__class__.__base__.__subclasses__() runs arbitrary Python on the machine executing the evaluation. Untrusted model output therefore reaches code execution on the evaluator host, outside any sample sandbox. The fix parses answers with a non-evaluating grammar, enforces size and complexity limits, rejects code-shaped answers, and runs parsing in a time-bounded worker thread.
Who does this affect?
You are affected if you are using a version that falls within the vulnerable range and you score evaluations with the math() scorer against untrusted or adversarial model output.
Background info
inspect-ai is vulnerable to Remote Code Execution (RCE) in versions 0.3.184 - 0.3.255.
How to fix this
Upgrade the inspect-ai library to the patch version.
Are You Affected?
Connect your repositories to instantly see whether vulnerable or malicious packages exist in your codebase.
Free. No credit card required.
I consent to receiving marketing communications based on Aikido’s Privacy Policy.
SOC 2Compliant
ISO 27001CompliantISO 42001Compliant
FedRAMPImplementing© All Intel data is openly available and commercially licensed.