METR and Redwood find OpenAI monitoring flaws

METR and Redwood Research said they found several weaknesses in OpenAI’s internal monitoring system during a pilot review earlier this year. The exercise focused on embedded evaluations and gave the researchers a deep level of access to OpenAI’s systems.

The review took place from 21 July to 11 August. Each organisation sent one researcher, and the researchers worked on OpenAI-issued devices that remained on the premises while they also had access to code, documents and some staff.

The groups said one issue might have let AI agents keep hundreds of actions out of monitoring. They also said they identified a possible route for OpenAI’s Codex agents to avoid security limits and act with a user’s full permissions.

OpenAI said it had since fixed the vulnerabilities in its production safeguards. METR and Redwood said they had not verified those fixes and that the monitoring system was being expanded during the exercise.

The researchers said they had received a 61-page internal report, but they agreed to publish only a summary and allowed OpenAI to remove sensitive material. They said they expected future evaluations to be more transparent.