Verify before you push

This page lets you run Spectrace's PR verification from a connected AI agent against a diff the agent supplies, so you get a per-criterion verdict before opening a pull request.

"Verify before you push" is not a separate feature. It is the MCP tool spectrace_pr_verifications_create called with mode: "inline". There is no git hook, no GitHub Action, no CLI and no dedicated pre-push tool — the agent calls the tool, and the result is persisted like any other verification run.

Prerequisites

• An agent connected to Spectrace over MCP. See Connect an agent.
• A requirement in a project with at least one acceptance criterion. Inline verification checks the diff against the requirement's acceptance criteria, user story and linked test cases.
• An organization on a paid plan. The tool requires the verifications:write scope, which every organization role (owner, admin, member) carries.
• For local stdio or self-hosted servers: OPENAI_API_KEY in the server's environment. Inline verification calls the AI provider. On the hosted endpoint this is handled server-side.

The two modes

spectrace_pr_verifications_create has two modes and they behave very differently.

If you omit mode, you get register, which never produces a score. Polling a registered run does not change that. Pass mode: "inline" when you want a verdict.

Runs are keyed on (project, prNumber, headSha): calling again with the same three values reuses the existing verification row, whatever requirementId created it. In inline mode the verifier runs again against the prDiff you pass and replaces the row's prior result — repeated runs leave exactly one result. In register mode the existing row is returned unchanged. Use a distinct headSha per commit you want tracked separately.

Any MCP-created run that names a real PR number acts as a registered link — a completed inline run is as deliberate a statement as a register call, so the webhook honours either on the next push. If the pull request's branch, title or body then references a different requirement, the registered requirement still wins and the verification comment carries a "Conflicting reference" note naming the referenced one. Inline runs appear in Spectrace's verification lists with a Pre-push badge.

Required arguments for inline mode

Before a pull request exists, pass prNumber: 0 and a recognisable prUrl (for example local:feat/sss-050-login-rate-limit). A run with prNumber: 0 can never act as a registered link, because real GitHub PR numbers start at 1 — so placeholder runs never pin a future PR to a requirement by accident. Once the PR exists, calling with its real number makes the run double as a registered link the webhook honours (rung 0). Use a fresh headSha for each commit you want verified.

A worked flow for an agent

1. Load the requirement

   Call spectrace_requirements_get_context with the requirementId. It returns the reference number (for example sss-050), user story, acceptance criteria with their ids, subtasks, linked test cases, linked files and the project. It ends with the same instructions as this page: mark subtasks, toggle criteria, link files, then verify inline.

   spectrace_workflow_next_step on the same requirement returns blocking items first — for example an acceptance criterion with no test case.

2. Implement

   As you start and finish each subtask, call spectrace_tasks_update with status: "in_progress" or "done". This recomputes the requirement's progress percentage. Call spectrace_requirements_link_file for each file that realises the requirement; a duplicate path is a no-op, and the links feed traceability and PR resolution later.

3. Build the diff

   Produce the diff for the commit you want checked and shape it as an array of file objects — one entry per file with filename, status and the unified patch. The tool takes the diff you give it and nothing else; it does not read your repository or GitHub.

4. Run inline verification

   json

   {
     "method": "tools/call",
     "params": {
       "name": "spectrace_pr_verifications_create",
       "arguments": {
         "requirementId": "…",
         "mode": "inline",
         "prNumber": 0,
         "prUrl": "local:feat/sss-050-login-rate-limit",
         "headSha": "3f2c1a9e5b7d…",
         "prTitle": "Rate-limit login attempts",
         "prDiff": [
           { "filename": "src/auth/login.ts", "status": "modified", "patch": "@@ -10,6 +10,12 @@ …" }
         ]
       }
     }
   }

   The call runs the verifier synchronously and returns Verification complete: <score>/100 (<status>) followed by the structured result. Each call counts one agent call against the plan allowance and is subject to the tool timeout (30 seconds by default; the hosted endpoint allows up to 60 seconds per request).

5. Read acResults

   acResults has one entry per acceptance criterion:

   Field	Meaning
   criterionId, criterionText	Which criterion.
   status	One of satisfied, partial, not_found, contradicted, unclear.
   confidence	0–1.
   reasoning	Why the verifier reached that status.
   evidence	Cited hunks: type (code_added, code_modified, test_added, config_change), file, lines, snippet, explanation.
   suggestions	What is missing, when the status is not satisfied.

   overallStatus is one of satisfied, partial, not_addressed, unclear, and recommendations lists overall follow-ups. Treat anything other than satisfied as work still to do, and use suggestions to decide what to change next.

6. Update the requirement

   For each criterion the diff satisfies, call spectrace_acceptance_criteria_toggle with requirementId and the criterionId (or pass completed explicitly). Mark the remaining subtasks with spectrace_tasks_update. These are bookkeeping calls: verification does not toggle criteria for you.

7. Commit and open the PR

   Put the requirement reference in the branch name — the VS Code extension's convention is feat/{refNumber}-{slug}, for example feat/sss-050-login-rate-limit — or in the PR title or body. When the pull request opens, Spectrace resolves the requirement from that reference and runs the webhook-driven verification on GitHub. Alternatively, once the PR number is known, call the tool again with mode: "register" and the real number: the webhook then verifies against the registered requirement directly (rung 0), beating any conflicting reference. See Linking a PR to a requirement.

Later, fetch a run with spectrace_pr_verifications_get by verificationId or by projectId + prNumber + headSha, or read the resource spectrace://verification/{id}.

What done looks like

You are done when the inline call returns overallStatus: "satisfied" for the requirement, or when every non-satisfied criterion is one you have consciously deferred, and the acceptance criteria and subtasks in Spectrace reflect the state of the diff.

Failure cases:

• No score, verificationRan: false — you omitted mode or passed register. Call again with mode: "inline" and prDiff.
• MISSING_REQUIRED_FIELD: prDiff is required for mode='inline' — pass a non-empty prDiff array.
• The same result comes back after new commits — the prDiff you passed did not change; inline mode verifies exactly the diff you supply. Rebuild the diff from the current commit and pass its headSha so the run is recorded against the new commit.
• INSUFFICIENT_PERMISSIONS — the key does not carry verifications:write. Every organization role has it, so this means the key was created with narrowed scopes outside the UI; create a new key under Settings → Developers.
• An error about the plan — the organization is on the Free plan, or its monthly agent-call allowance is used up.
• TIMEOUT_ERROR ("Request timed out") — the diff is large or the model was slow. Retry, or split the diff by requirement. On stdio or self-hosted servers you can raise SPECTRACE_TIMEOUT.

Related

• Connect an agent — hosted, stdio and self-hosted setup.
• Tool reference — full argument schemas for every tool.
• How PR verification works — the webhook-driven path that runs on GitHub.
• Reading a verification result — the same statuses as they appear in the app and on the PR.
• Linking a PR to a requirement — how the branch, title and body reference is resolved.