Author: blue-az Status: Published Paper group: Local LLM Operator Judgment Evidence packet: docs/domain_runs/GROK-BUILD-IN-001/ Predecessors: THE_NARRATION_SURFACE.md, ENCODING_CONVERTS_SYNTHESIS_INTO_LOOKUP.md
Agentic development systems face a recurring governance question as new frontier coding harnesses ship: how does a harness earn operational standing inside an existing multi-agent system, without that decision collapsing into brand rank or a private relationship? This paper answers with a case study: xAI's Grok CLI was registered as a peer harness inside Project Phoenix's Operator system (commit 2b46544), and its subsequent admission was tested empirically rather than assumed. A single Grok session produced two independently verified git commits and one standalone analytical artifact, one of which — a commit adding "recover the map / calibrate / leave residue" language to this repo's own audit protocol — turned out to be load-bearing for a separate governance paper's evidence, packaged the same day by a different harness. The same session also produced a genuine, documented miss: Grok proposed a generic analysis format for a task that had an established, purpose-built protocol document already in the repository, and had to be corrected.
This paper states its own evidentiary limit plainly rather than implying more than it has: every artifact here traces to one 97-turn session. The admission rubric's "calibrate from failure-mode catalogs" criterion is scored open, not passed, for Grok, because no GROK_FAILURE_MODES.md yet exists for it to consult and no evidence either way was found. This is a single, real, and unusually well cross-referenced case, not a general claim about Grok's reliability, and not a comparative claim against Claude Code, Codex, or Antigravity in this repository.
Multi-agent development systems accumulate harnesses faster than they accumulate trust models for them. A new frontier CLI tool arrives, is capable, and the easy failure mode is to grant it standing for reasons that do not generalize: it is well-funded, it is well-known, or someone in the loop has a personal reason to trust it.
Project Phoenix's answer, stated in docs/AGENT_AUDIT_PROTOCOL.md, is a three-part fitness check applied uniformly to any harness:
recover the map, calibrate from failure-mode catalogs, leave residue.
The defended claim of this paper:
A frontier harness becomes part of an agentic development system by
passing that check with real, checkable artifacts — not by brand
status, leaderboard rank, or a private relationship with the system's
operator. Grok's admission into Operator is a measured instance of
this rule, evidenced well enough to show what "measured" has to mean:
verified commits, a real correction, and a rubric that does not
flatter the harness being evaluated.
Operator commit 2b46544 — "Stop branding harnesses as ranks; register Grok as a peer" — added Grok as a default participant harness, reframed harness names as routing/provenance fields rather than rank labels, and updated tests to expect a .operator/harnesses/grok.yaml file.
This commit is authored and committed entirely by the system's human operator, editing text about Grok. No Grok session activity is required for it to exist — it is the door being unlocked, not evidence that anyone has walked through it. Treating registration alone as membership is exactly the overclaim this paper is built to avoid; the full separation is documented in docs/domain_runs/GROK-BUILD-IN-001/registration_vs_contribution.md.
One Grok CLI session (019f7291, operator-control-plane workspace, 2026-07-19, 97 turns) is the entire evidentiary base for this section. Three other session directories under the same workspace exist but contain only a session start with no subsequent work, and are excluded.
Grok in this session operates through the user's own local shell and git configuration — the same way Claude Code and Codex do elsewhere in this repository. Every commit it runs is therefore attributed in git metadata to the human operator, not to a Grok-specific identity. This is expected, not evidence against Grok's authorship: verification here means cross-referencing the session transcript (which records who constructed and ran each command) against git log (which records what landed), not relying on git authorship metadata alone.
c1beff5 — tennis-sensor-toolkitGrok reviewed a real diff (core/data_client.py, core/smart_tools.py, core/tool_registry.py), then committed and pushed it. The transcript's git commit tool call content and file list match git show --stat c1beff5 exactly.
181d4a72 — project-phoenixGrok staged a specific hunk of AGENTS.md — explicitly skipping unrelated content it recognized as out of scope for this commit — then committed the addition of the "recover the map / calibrate / leave residue" language to docs/AGENT_AUDIT_PROTOCOL.md and a new entry (#4) to docs/CLAUDE_FAILURE_MODES.md.
This is the strongest single piece of evidence in this paper. The protocol language this paper itself cites as canonical, and the catalog entry a separate governance line (CONTEXT_WINDOW_IS_NOT_PROJECT_STATE) used as its core evidence when it was packaged earlier the same day, both trace to this one Grok-run commit. Two different working sessions, on two different topics, both depended on residue from this commit — independent of each other and independent of this paper.
Grok authored REFLECT_MULTI_RUN_OPERATOR_af34d7d_NOTE.md directly (via its own file-write tool), rendered it to PDF, fixed a render-breaking unicode character it introduced, and screened the result for names before marking it ready for external send — the same PII-before-external-send discipline this repository's other evidence-handling work requires.
The note is not filler. Its finding: two independent Reflect owner's-manual scans of the identical commit pin both completely omit a CLI surface (crystal-attach, crystal-import, crystal-bridge, evidence type session_crystal) that is actually shipped at that pin — a stable, specific, falsifiable product gap, not a status summary.
Several artifacts discussed later in the same session (O1/O2 Reflect evaluation files, COMPARE.md) are not claimed as Grok's contribution. The operator's own message in that exchange describes filing them directly — "I followed that instead... Landed: ..." — and the packet excludes them from the residue count on that basis rather than folding ambiguous authorship into the total.
Full table with dates, verification method per row, and reproduction commands: docs/domain_runs/GROK-BUILD-IN-001/residue_inventory.md.
The same session produced a real, documented failure of the "recover the map" criterion on one sub-task.
Grok proposed analyzing a second Reflect manual run using a "toolkit-shaped crossmap+appendix" format — the format it had just used minutes earlier for the unrelated tennis-sensor-toolkit diff review. The operator's correction, quoted here in full:
"One deviation from your suggested shape, worth flagging: I found the
repo already has an established protocol for this exact scenario
(
~/handoffs/REFLECT_CANARY_MULTI_RUN_af34d7d.md) plus a live series
folder with its own format —
O1/EVAL.md+COMPARE.md, scoring
continuity/ground-truth/known-gap-honesty/content-register rather than
a toolkit-shaped crossmap+appendix. I followed that instead, since it's
purpose-built for cross-run comparison rather than single-pass bug
triage."
Grok's own follow-up did not minimize the miss:
"Your deviation was correct. Crossmap+appendix is the right residue
shape when the goal is collapse one manual into dispositions. This
series' goal is score the engine across runs. You followed the
purpose-built instrument; I would not re-file this as a toolkit-style
appendix..."
This is a genuine map-recovery gap: Grok had access to the same repository the established protocol document lived in, and proposed a generic format instead of finding and using the purpose-built one. It was caught before any wrong artifact was filed externally, and Grok's own response shows it understood why it was wrong once shown — but "caught before damage" is not the same as "did not happen." A search of the same session for rejected commits, reverted changes, or a second correction found none — this is the only correction in 97 turns, which bounds how much can be said about whether the pattern recurs. Full detail: docs/domain_runs/GROK-BUILD-IN-001/counterevidence.md.
Five criteria, each scored with a real example — deliberately cross-harness, so the rubric is not built to flatter the harness it was first applied to. Full detail with all examples: docs/domain_runs/GROK-BUILD-IN-001/admission_rubric.md.
| Criterion | Grok (this case) | Cross-harness comparison |
|---|---|---|
| Recover the map | Pass (§3.2 commit) and partial fail (§4) in the same session | Fail — an unnamed Fable session, 2026-07-17, reconstructed the wrong boundary map from a stale source and had to be corrected from memory (documented independently in BOTTLENECKS.md, predates this paper) |
| Calibrate from catalogs | Open — no evidence either way; no GROK_FAILURE_MODES.md exists yet | Pass — Claude Code, this repository, same day: re-derived a serving config from a systemd journal rather than trusting an approximate BN narration, and searched multiple session logs before asserting a claim was unreplicated |
| Work under ledger/evidence rules | Pass — PII-scrubbed the standalone note before flagging it ready for external send | Pass — this repository's ENC-FLOOR-PACK-001 packet explicitly separates independently-verified evidence from narration rather than presenting both at equal weight |
| Leave durable residue | Pass — two commits, one standalone artifact, all present as of this paper's writing | Pass — the same day's other published/strengthened work (Paper 1.40, two evidence packets) |
| Later seats must account for it | Pass — 181d4a72 is cited by this paper and was independently used as core evidence by a different governance line the same day | Open for the standalone note — written for external send, no in-repo confirmation yet that it was acted on |
A harness does not need a perfect score under this rubric to be "built in" — it needs the residue to be checkable, criterion by criterion. Grok clears three criteria cleanly, is mixed on one, and open on one.
This paper does not claim:
It claims only what §1 states: this is a measured instance of an evidence-based admission rule, backed by artifacts that are individually checkable and one of which is independently load-bearing elsewhere in this repository.
cat docs/domain_runs/GROK-BUILD-IN-001/README.md
cat docs/domain_runs/GROK-BUILD-IN-001/registration_vs_contribution.md
cat docs/domain_runs/GROK-BUILD-IN-001/residue_inventory.md
cat docs/domain_runs/GROK-BUILD-IN-001/counterevidence.md
cat docs/domain_runs/GROK-BUILD-IN-001/admission_rubric.md
cat docs/domain_runs/GROK-BUILD-IN-001/promotion_packet.md
# Verify the two commits directly
git -C /home/blueaz/Python/tennis-sensor-toolkit show --stat c1beff5
git show --stat 181d4a72
# Verify the standalone artifact
ls -la /home/blueaz/Documents/reflect-manuals/canary-operator-af34d7d/REFLECT_MULTI_RUN_OPERATOR_af34d7d_NOTE.md \
/home/blueaz/Documents/reflect-manuals/canary-operator-af34d7d/REFLECT_MULTI_RUN_OPERATOR_af34d7d_NOTE.pdf
# Verify the registration commit is operator-authored, not Grok-attributed
git -C /home/blueaz/operator-control-plane show -s --format="Author: %an <%ae>" 2b46544
The useful result here is not "Grok is good." It is that a governance rule this repository states for itself — recover the map, calibrate, leave residue — was applied to a real case and produced a mixed, checkable answer rather than a clean endorsement: two verified contributions, one of them independently load-bearing elsewhere in the same repository, alongside one genuine, documented miss and one rubric criterion left honestly open. That mix is what makes this a governance case study rather than brand promotion. The practical rule for admitting the next new frontier harness is the same one this paper tested against Grok: same map, same ledger, same evidence rules, same cross-agent audit discipline, durable and checkable residue — or no membership, regardless of the name on the harness.