Agent Perimeter

The problem

Teams are wiring LLM agents into internal systems through the Model Context Protocol (MCP). Every MCP server an agent connects to hands it a set of tools — and the descriptions of those tools are attacker-authored text that the agent reads as instructions. A poisoned description, a shadowed tool name, a leaked secret in a config file, or a server that silently drifts after approval can all make the agent do something nobody authorised.

Why the obvious approach fails

• Static flaggers stop at suspicion: they mark a description as suspicious and stop — reachability is inferred, not proven. Nobody can tell an auditor what the agent would do.
• Census publishing is unreproducible: large “N% of MCP servers are vulnerable” numbers circulate, but the corpus and scripts never ship, so the number cannot be checked by anyone.
• Findings lack taxonomy and reproduction: a finding that doesn’t cite a CWE, a published taxonomy entry, and a reproduction command is an opinion, not a report.

Architecture

Two pipelines share one model layer and one database. The scan pipeline points at a single server: discover tools / resources / prompts, build the capability graph, run checks, emit SARIF/HTML with a reproduction for every finding. The census pipeline passively crawls the official MCP Registry, detects artifacts, draws a seeded random sample, and produces aggregate statistics only.

1. MCP server
2. Discovery
3. Capability graph
4. 34 checks · 8 groups
5. Findings + reproductions
6. SARIF / HTML / CI drift gate

agent_perimeter/
  transport/   stdio (containerised launcher + seccomp.json), streamable_http,
               legacy HTTP+SSE, MCP protocol-revision negotiation
  discover/    enumerate tools / resources / prompts
  model/       ServerProfile, Tool, CapabilityEdge, ScopeFile, Finding
  checks/      revision (12) · static (5) · descriptions (5) · secrets (3)
               · active (4, scope-gated) · injection (2) · drift (1) · policy (2)
  graph/       capability graph + policy evaluation
  report/      sarif.py, html.py, census_report.py
  census/      registry fetch, artifact detection, seeded sampling (passive only)
  api/         FastAPI
web/           Next.js: scan → findings / graph / drift pages
action.yml     composite GitHub Action: drift gate + SARIF upload
tests/fixtures/servers/   vulnerable-server fleet (Python + Node)

Threat model

• Attacker-authored tool descriptions. The agent treats tool descriptions as instructions; a poisoned description is a prompt-injection delivery channel into internal systems.
• Untrusted binary execution. Scanning a stdio MCP server *is* executing an untrusted binary — so stdio targets run inside a locked-down container by construction.
• Secret leakage through the scanner itself. Discovered secrets must never be validated live or stored raw, or the scanner becomes the leak.
• Analysis-content exfiltration. Content under analysis must never reach a tool-capable context, or the target can weaponise the judge.
• Silent drift after approval. A server approved once can change its tool descriptions afterwards; drift must be detectable and CI-enforceable.
• Naming third parties. Publishing findings about other people’s servers carries legal and ethical obligations that must be handled before publication, not after.

Decisions

Drive a live agent through the injection path

The injection check instruments the target and drives a bundled minimal agent harness through it, so a finding says “an agent did follow this instruction to this effect” — demonstrated impact, not inferred reachability. Proves reachability and stops; nothing is weaponised.

Explicit protocol-revision negotiation

MCP’s 2026-07-28 revision moved to a stateless architecture and is not backward compatible. Checks declare which revisions they apply to; a per-revision check matrix governs what runs, and a deprecated HTTP+SSE transport is kept for legacy targets.

Drift detection is the subscription

description_hash and drift_event landed in the v1 schema. Every scan is diffed against the previous scan of the same target, rendered as a word-level LCS diff in the UI, with --fail-on-drift (exit 3) for CI.

A determinism budget of at least 90%

test_degraded_mode_still_produces_findings proves that at least 90% of finding classes survive with every model provider disabled. The LLM judge is the only model-gated check; the rules-based detectors stand alone.

Fail-closed census pagination

A registry page that returns a full page with no cursor is recorded as a suspected pagination bug, never as a completed population. The methodology doc documents the exact nextCursor trap that would otherwise silently truncate a 30k-entry census to ~2%.

Drop a feature rather than carve out a safety rule

Code review of the census’s Tier 3 found it sent a live, unauthenticated discovery probe to each sampled third-party host with no scope-file check, the one active-probe path that skipped require_scope(). A scope file records one authorising party consenting to one named target; nobody can sign one for a random sample of unrelated servers. Fixing it meant giving up either random sampling or per-target consent, so Tier 3 was deleted (tier3.py and its tests) on 23 Sep 2026 and the ruling recorded in docs/census/CHANGELOG.md.

Honest re-scoping, recorded in writing

A competitive inventory led to dropping “enterprise deployment posture” as a differentiator when the evidence didn’t support it, and narrowing “evidence-graded reporting” to reproducibility rather than volume — recorded in docs/methodology.md with dated rationale. 14 open design decisions are closed in docs/open-decisions.md.

Implementation

• Python 3.12, FastAPI, Pydantic v2, SQLAlchemy 2 + Alembic (8 migrations), Postgres 16, Typer CLI. Next.js 15.5 / React 19 / TypeScript strict frontend with scan → findings / graph / drift pages.
• A vulnerable-server fixture fleet (one flaw each, plus clean controls) drives golden SARIF files and the precision/recall table CI rebuilds on every commit.
• Every finding cites a CWE plus at least one published taxonomy entry (OWASP LLM Top 10, OWASP MCP Top 10, CoSAI, NSA CSI, MITRE ATLAS) and ships with a reproduction command.
• Output: SARIF 2.1.0 (schema-validated, renders in GitHub code scanning), an HTML report, and a one-pager.
• Ships as a composite GitHub Action (uses: Berakhah/agent-perimeter@v1): scans a server in CI, uploads SARIF, and fails the job on drift. A dedicated action-selftest workflow exercises it.
• Ops-console web redesign (Sep 2026): type-scale and elevation tokens, stat-tile grids on the findings and live-scan screens, drift sparklines that never fabricate a zero line, and motion that respects reduced-motion and print.
• ~13,800 LOC of non-test Python across 123 files; ~4,800 LOC TypeScript with 80+ Playwright test cases in 9 spec files; mypy --strict, ruff, gitleaks pre-commit, hypothesis.

Evidence

• Live-agent injection harness — the differentiator no inventoried competitor (Snyk agent-scan, Cisco, Akto, Enkrypt, Trend Micro, four arXiv tools) ships.
• First public census report (2026-09-14): 31,953 registry entries, full pagination; 0 of 174 npm and 0 of 128 PyPI artifacts show published evidence of supporting the 2026-07-28 revision — reported with Wilson 95% intervals (0.0–2.2% and 0.0–2.9%) so a zero is never reported as “none”. 377 artifacts fetched, 23 fetch failures disclosed by cause.
• A real detection bug found during verification, documented: @modelcontextprotocol/sdk never shipped a 2.x (the SDK split into /server, /client, …), so the artifact detector could never have matched a v2 npm pin. Fixed and written up rather than buried.
• Reproducibility: analysis/census_analysis.py recomputes every published figure from records.csv alone. Two earlier same-day runs were discarded for detector bugs and recorded in the changelog.
• Clean-machine proof: 82-second green quickstart on a fresh ubuntu-latest VM (docs/evidence/clean-machine.md).

Security controls

• No active probe without a scope file (target, authorising party, date, attestation). CLI and HTTP API refuse through the same authorisation function; the API returns 422 authorization_required, never a silent skip.
• stdio servers run inside a locked-down container — non-root, read-only rootfs, seccomp profile, no network unless the check needs it, tmpfs scratch, CPU/memory caps, hard timeout.
• Discovered secrets are never validated live and never stored raw — SHA-256 fingerprint, entropy, prefix, last 4 characters, file/line only.
• Content under analysis never reaches a tool-capable context — the LLM judge has no tools, no network, constrained-enum output only.
• No third-party server is ever named in a public report — aggregate statistics only, with a 90-day coordinated-disclosure embargo, right-of-reply, and versioned results with changelog.

Trade-offs

• Deterministic-first over model-first: rules-based detectors make results reproducible and cheap, at the cost of missing patterns only a model might catch — mitigated by the (currently disabled) LLM-judge escalation lane.
• Container-per-stdio-scan costs startup time on every scan, buying the property that scanning is never raw untrusted execution on the host.
• Strict scope-gating means active checks do nothing without written authorisation — deliberately inconvenient for casual use, deliberately safe for real use.

Caveats & status

The LLM-judge check is registered but runs in its disabled lane — no paid provider account is provisioned yet. The census is passive-only: Tier 3 (live discovery against third-party servers) was dropped permanently on 23 Sep 2026 because it could not satisfy the scope-file rule.

• The competitive table was compiled via web search after the build, not primary-source repo reads; the methodology page says so.
• The census is passive-only by ruling: with Tier 3 dropped, it measures what registries and published artifacts declare, not how live servers behave.
• Coverage floor is 75% — enforced, but not the 85%+ this codebase’s sibling projects hold themselves to.

Repository

Public repository with a clean-machine reproduction. Clone it, run the README quickstart, and every published figure can be regenerated from the repo itself.