15 Research Lab Research San Francisco
John Kearney

The measurement layer.

John Kearney

Adversarial auditing of AI safety evaluation infrastructure

Authensor, Inc. (grantee), continuing at 15 Research Lab

01

Introduction

I audit the instruments the AI industry measures itself with, and file every fix upstream.

The research is conducted through Authensor, Inc., the AI-security startup and grantee, and is expanding to 15 Research Lab, its dedicated research home.

AI safety decisions run on evaluation tooling: benchmark scores, LLM-judge verdicts, red-team harnesses, guardrails. Almost nobody red-teams the tooling itself. That is the work: find where the measurement can be broken, prove it by execution, fix it upstream, and name the pattern so the field can see it.

02

The record (Feb-Sep 2026)

Defect reports filed228 across 125 open-source projects, each with a runnable proof
Fixes merged from our own pull requests15 (accounts: AUTHENSOR, john-kearney, JohnKearney15)
Fixes maintainers shipped from our reports7
Fixes shipped after our filings11
Fixes in production code, total33, across 18 organizations
Inside the UK AI Security Institute7 landings across 5 repos
Named and published the defect classThe Evaluator Trust Boundary (76 confirmed instances, 36 organizations)
Research papers4
Coordinated private disclosures16 (details withheld under responsible disclosure)
03

Selected findings

  • SWE-bench: models could forge PASSED lines in stdout and ace the canonical agentic-coding benchmark. Grading now verifies test exit codes. (merged)
  • UK AISI inspect/control stack: models under evaluation could steer the monitors watching them, through unsanitized chain-of-thought in monitor prompts, forgeable verdict extraction in the core scorer, and defeatable stealth evals. (7 landings across 5 repos)
  • Presidio, Microsoft's PII engine (10M+ installs a month): internationalized emails, six countries' IBANs, and lowercase Spanish national IDs passed through redaction unmasked. (4 fixes merged)
  • MLflow (34M downloads a month): DNS-rebinding SSRF reaching cloud metadata endpoints, fixed after the report.
  • The Evaluator Trust Boundary: 76 confirmed instances across 36 organizations: judges that trust first-match JSON from the model being graded, denominators that silently drop unscorable samples, guardrails that fail open. Taxonomized as ETB-01..10 and submitted to OWASP's GenAI LLM Top 10 and MITRE ATLAS.
  • Payment agents: unbounded USDC pulls, unsigned tool approvals, hot-wallet drains on backend-authored messages, across Stripe, PayPal, Coinbase, x402, Turnkey.
  • Open silicon: a parallel hardware-security track, with merges in Yosys and neorv32, OpenTitan root-of-trust findings, and rt-si, an open-source silicon security scanner.
04

The current question

The audit was done the hard way, by reading the code. The next phase asks the opposite question: can a gold-blind fuzzer, shown nothing about the known defects, rediscover the same failures in grading pipelines? Early answer: yes. And can RL training against ground-truth rewards drive a model's exploitation of its own graders from 50 percent to zero? Measured: yes. Qwen2.5 1.5B and 7B both reach 0.0%.

The method is now crossing from software into the lab itself. 15 Research Lab is a working fabrication and measurement shop in San Francisco, built for researchers and for the autonomous systems that increasingly run the experiments. The hardware track is already live: merges and root-of-trust findings in OpenTitan, Yosys, and neorv32, an open-source silicon security scanner (rt-si), and work on MHS hardware standards for autonomous labs. The question there is the same one that started this work, one layer down: when an AI agent runs the instruments, who audits the trust boundary between the agent and the physical lab?

05

Receipts

Every claim links to its merge or the maintainer's own words. Grouped by route.

Merged from our pull requests

Maintainers shipped from our reports

  • MLflow DNS-rebinding SSRF to cloud metadata; their fix cites our report. report / fix
  • instructor verdict injection; maintainer: "Thank you for surfacing this issue." report / fix
  • ComfyUI logging API keys in plaintext; maintainer: "excellent writeup." report / fix
  • Vercel AI SDK unsigned tool approvals; maintainer shipped signed approvals. report / fix
  • UK AISI inspect_ai core scorer; merge cites our report. report / fix
  • UK AISI control-arena monitor sanitization. report / fix

Fixed after our filings (selected)

  • NVIDIA garak: homoglyph Unicode evading toxicity detection. #1867
  • OpenCompass: models passing code-exec benchmarks via __eq__ override. #2535
  • x402: forged on-chain app attribution. #3299
  • Yosys: write_smt2 silently dropping assertions. #6108
  • neorv32: permanent CPU hang in default config, fixed same day. #1633
  • lm-evaluation-harness: denominator drop inflating scores; reporting half landed via community PR. issue / PR

The maintainer, on the SWE-bench fix: "Tysm for this fix! This is a really interesting potential hack I hadn't thought about." #620

06

Publications

Published pre-prints, open access (Zenodo, CC BY 4.0)

  1. The Evaluator Trust Boundary: A defect class in AI evaluation infrastructure, its prevalence across 36 organizations, and its removal by training. July 2026. https://doi.org/10.5281/zenodo.21633620
  2. The Verbosity Premium: What RLHF-Induced Token Inflation Costs the AI Industry. March 2026. https://doi.org/10.5281/zenodo.19346709
  3. Grokking Has Finite Capacity: Measuring and Overcoming Limits on Simultaneous Algorithmic Discovery. March 2026. https://doi.org/10.5281/zenodo.19346536
  4. Side-Channel Exfiltration and Narrative Erosion in Frontier Language Models. March 2026. https://doi.org/10.5281/zenodo.19346069

Published with code

  1. Beyond Prompt Injection: Offensive Security Tradecraft Exposes Architectural Blind Spot Classes in AI Control Monitors. Apart Research AI Control Hackathon, 2026. Paper, 22 attack transcripts, and the evaluation harness: https://github.com/15researchlab/ai-control-monitor-blind-spots

Intentionally closed

  1. How Graded Models Write Their Own Grades. Under review at USENIX Security '27. Manuscript, evidence whitelist, and attack battery withheld from public distribution until the review decision. The defense in it resists the paper's own attack corpus 11 for 11; the details stay sealed until the venue rules.
  2. The Hawthorne Protocol: Can AI systems detect when they're being evaluated? In preparation. 22 evaluation-detection indicators and a reference implementation. Release timing follows the disclosure lifecycle of the evaluation harnesses it measures.
07

Tools

  • etb-scan (pip install etb-scan): offline scanner for evaluator trust boundaries in your own judging pipeline.
  • rtsilicon: open-source silicon security scanner, 25 CWE classes across Verilog, SystemVerilog, VHDL, Chisel/Migen.
08

Elsewhere

AUTHENSOR on GitHub · john-kearney on GitHub · authensor.com/etb · hello@15researchlab.com