> ## Documentation Index > Fetch the complete documentation index at: https://mcpjam-mintlify-docs-update-pr-4053-1786992934526.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Get run status > Run status, result, and summary. Poll until `status` is terminal (`completed`, `failed`, or `cancelled`). ## OpenAPI ````yaml /reference/openapi.json get /projects/{projectId}/eval-runs/{runId} openapi: 3.1.0 info: title: MCPJam API version: 1.0.0-preview description: >- Programmatic access to MCP servers saved in your MCPJam projects — live diagnostics (validate, inspect, export) and operations: call tools, render prompts, run eval suites asynchronously and poll their results, and import OAuth tokens. **The API is in preview**: the surface may change while we finish the design. Error `code` values are stable; error `message` strings are not. Write clients that ignore unknown response fields. contact: name: MCPJam url: https://github.com/MCPJam/inspector/issues servers: - url: https://app.mcpjam.com/api/v1 description: Hosted MCPJam security: - bearerAuth: [] tags: - name: Hosts description: >- Project hosts: named model + capability profiles you run chats and eval suites against. - name: Environments description: >- Project environments: named, live-editable execution bundles (one host, an optional standalone server group, optionally pinned skills and plugin versions) that eval suites and journeys run against. Distinct from Sandbox images, which are Computer base images. Reads require project membership; every write requires project admin. - name: Plugins description: >- Agent Plugins imported into a project — read-only inventory and version detail. - name: Sandbox images description: >- Custom Computer images: a digest-pinned Dockerfile built into an immutable image your project's computers boot from. - name: Server diagnostics description: Connect-level health checks against a saved MCP server. - name: Primitives description: 'The server''s MCP primitives: tools, prompts, and resources.' - name: Export description: Full-server snapshots for diffing and CI. - name: Execution description: 'Run the server''s primitives: call tools, render prompts.' - name: Eval runs description: >- Asynchronous eval suite runs: create with 202, poll status, iterations, and traces. - name: Conformance runs description: >- Ingest MCP spec-conformance results from the SDK/CLI into project-owned history. Distinct from Eval runs (authored LLM cases) and from directory readiness. - name: Server connections description: >- Connect an MCP server URL to a project, authorizing in a browser when the server requires it. - name: OAuth description: 'Bring-your-own OAuth: import externally obtained tokens for a server.' - name: Scenarios description: >- Read-only access to the scenarios published from a project: listing, settings, attached servers, and share links. - name: Catalog description: >- Discover the resources the other routes operate on: your account, projects, servers, eval suites, and chat sessions. - name: Tunnels description: >- Relay tunnels that expose local MCP servers through a public URL, registered as first-class project servers (the `mcpjam cloud tunnel` CLI flow). - name: Agent description: >- Headless agent turns over the public API: send a message history, the server runs one assistant turn with project-scoped workspace tools (eval reads + suite creation) on a pinned hosted model, and returns the reply plus created-resource references. - name: Swarms description: >- Personas, journeys and swarm containers — the authoring half of Swarms — plus the model-backed generation that drafts them. - name: Swarm runs description: >- Launching journeys and reading what they produced. Launching SPENDS — see the per-operation notes. - name: Swarm insights description: >- What a swarm run revealed. The scorecard and findings are deterministic and free; requesting wave insights runs models and draws on your shared daily ledger. - name: User testing description: >- Publishing an environment for real visitors, and controlling who can reach it. Several of these NARROW access and take effect immediately. - name: Directory readiness description: >- Grade a saved server against a publisher's listing requirements: Anthropic's connector directory or OpenAI's plugin directory. Reported as lane status and coverage, never as a numeric score, and excluded from `pooledConformanceScore`. Deterministic grading is free; model-backed experience observations are an explicit opt-in that consumes MCPJam credits and can never decide a verdict. - name: Registry description: >- Search the scraped MCP directories (Claude, ChatGPT, and any future source), list curated/org registry cards, and install them into a project. Install writes a `servers` row and provenance — it does not open a live session. There is no catalog-uninstall route: delete the project server instead. Directory reads require a bearer (including minted guest tokens) but do not materialize a user. Card/connection reads and all writes are authed-non-guest. paths: /projects/{projectId}/eval-runs/{runId}: get: tags: - Eval runs summary: Get run status description: >- Run status, result, and summary. Poll until `status` is terminal (`completed`, `failed`, or `cancelled`). operationId: getEvalRun parameters: - $ref: '#/components/parameters/projectId' - $ref: '#/components/parameters/runId' responses: '200': description: The run. content: application/json: schema: $ref: '#/components/schemas/EvalRun' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '429': $ref: '#/components/responses/RateLimited' '500': $ref: '#/components/responses/InternalError' components: parameters: projectId: name: projectId in: path required: true description: ID of the hosted project that contains the server. schema: type: string runId: name: runId in: path required: true description: Eval run ID, as returned by `POST /eval-runs`. schema: type: string schemas: EvalRun: type: object required: - id - suiteId - status - source - createdAt properties: id: type: string suiteId: type: string runNumber: type: - integer - 'null' status: type: string description: 'Poll until terminal: `completed`, `failed`, or `cancelled`.' enum: - pending - running - completed - failed - cancelled result: type: - string - 'null' description: Pass/fail verdict once terminal. enum: - passed - failed - null summary: type: - object - 'null' properties: total: type: integer passed: type: integer failed: type: integer passRate: type: number source: type: string description: Run origin. API-created runs are `api`. enum: - ui - api - sdk notes: type: - string - 'null' createdAt: type: number description: Epoch milliseconds. completedAt: type: - number - 'null' description: Epoch milliseconds, `null` until terminal. scoreIntegrity: type: - string - 'null' description: >- Whether the run's score evidence verified at ingest. TRI-STATE, and the third state matters: `valid` means the backend checked and definitions and results agree; `invalid` means they do not; `null` (or absent) means NO VERDICT was produced, on a deployment that predates integrity checking. A score gate must treat `null` exactly like `invalid` — absent evidence is not valid evidence. enum: - valid - invalid - null environment: description: >- The environment revision this run is pinned to. `null` on a legacy run that recorded none — always present, so a caller never has to distinguish absent from unpinned. oneOf: - $ref: '#/components/schemas/EvalRunEnvironment' - type: 'null' runGroupId: type: string description: >- Shared by every per-target run from the same fan-out launch. Absent on a single-target launch and on rows created before run groups. effectiveModelId: type: string description: >- Model the run actually executed with. Absent on pre-attribution rows. modelSource: type: string enum: - client_default - override description: >- `client_default` inherited the host model; `override` used the environment's `modelId`. executionEngine: type: string description: >- Which engine executed the run: `emulated` (the platform's own turn loop) or `harness:` (a real agent runtime such as Claude Code). ABSENT means the run recorded no engine — a run created before the platform attributed one. Treat that as UNKNOWN, never as `emulated`: those are different claims, and the runs whose engine was never recorded are exactly the ones a reader must not vouch for. insights: allOf: - $ref: '#/components/schemas/InsightsEnvelope' description: >- The common actionable-insights envelope. Present on the DETAIL response only — lists stay compact — and absent when the caller may not have it or the deployment cannot produce one. Treat absence exactly like `status: "not_available"`. judges: allOf: - $ref: '#/components/schemas/EvalRunJudges' description: >- Advisory LLM graders on this run. Present on the DETAIL response only — lists stay compact — and absent on deployments that predate the envelope. EvalRunEnvironment: type: - object - 'null' description: >- The project environment a run is pinned to, at the revision resolved when it launched. `null` for a legacy run that used the suite's saved server selection. required: - id properties: id: type: string name: type: - string - 'null' revision: type: - integer - 'null' description: The environment revision the run executed against. InsightsEnvelope: type: object required: - schemaVersion - scope - status - reasonCode - retryable - error - generatedAt - updatedAt - summary - coverage - findings - truncation description: >- The common insights envelope, shared by eval runs, swarm waves and user-testing windows. One shape for three producers, so a caller writes the reading code once. An ABSENT envelope and `status: "not_available"` mean the same thing and both are normal: the field is an enrichment, and a caller who may not have it gets the resource without it rather than an error. properties: schemaVersion: type: integer enum: - 1 scope: $ref: '#/components/schemas/InsightScope' status: type: string description: >- `not_available` means this deployment cannot produce insights at all — treat an ABSENT envelope the same way. `not_requested` means nobody has asked. `pending` means one is running: poll, do not re-request. enum: - not_available - not_requested - pending - completed - failed reasonCode: type: - string - 'null' retryable: type: boolean description: >- Whether asking again could produce a different answer. False on a `failed` envelope means the input, not the attempt, was the problem. error: oneOf: - type: object required: - code - message properties: code: type: string message: type: string - type: 'null' generatedAt: type: - integer - 'null' updatedAt: type: - integer - 'null' summary: type: - string - 'null' coverage: type: object required: - unit - analyzed - total - truncated - lowConfidence description: >- READ THIS BEFORE QUOTING ANY FINDING. `truncated` and `lowConfidence` are the difference between "this happens" and "this happened in the part we looked at". properties: unit: type: string enum: - iterations - sessions analyzed: type: integer total: type: integer gradedCount: type: integer feedbackCount: type: integer truncated: type: boolean description: The analysis saw `analyzed` of `total`, not all of it. lowConfidence: type: boolean description: >- Too little was analyzed to generalize. Findings still stand as observations of what WAS seen. findings: type: array items: $ref: '#/components/schemas/ActionableFinding' runHealth: type: object required: - targets description: >- Swarm only. Launch outcomes never appear as findings — a run that could not start is an operational fact, not something the server under test did. properties: targets: type: array items: type: object required: - subjectKind - subjectId - subjectLabel - attempted - succeeded - failed - rateLimited properties: subjectKind: type: string enum: - environment - host subjectId: type: string subjectLabel: type: string attempted: type: integer succeeded: type: integer failed: type: integer rateLimited: type: integer truncation: type: object required: - truncated - omittedFindings - omittedEvidence - contractTruncated description: >- What this RESPONSE dropped to stay a sane size, as distinct from what the ANALYSIS did not look at (`coverage`). properties: truncated: type: boolean omittedFindings: type: integer omittedEvidence: type: integer contractTruncated: type: boolean EvalRunJudges: type: object description: >- Advisory graders on a run, keyed by judge. An envelope rather than a bare `judge` field because goal completion is one of several; a future judge is a new key here, not a reshaped response. properties: goalCompletion: $ref: '#/components/schemas/EvalRunGoalCompletionJudge' groundedness: $ref: '#/components/schemas/EvalRunGroundednessJudge' Error: type: object required: - code - message properties: code: type: string description: >- Stable, machine-readable error code. New codes may be added over time; treat unknown codes as non-retryable failures unless the HTTP status says otherwise. enum: - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - CONFLICT - VALIDATION_ERROR - RATE_LIMITED - FEATURE_NOT_SUPPORTED - SERVER_UNREACHABLE - TIMEOUT - OAUTH_REQUIRED - INTERNAL_ERROR message: type: string description: >- Human-readable description. May change between releases — don't match on it. details: type: object description: Optional, unstructured context bag. additionalProperties: true InsightScope: type: object required: - kind - id description: What this envelope is about. The extra fields depend on `kind`. properties: kind: type: string enum: - eval_run - swarm_wave - user_testing_window id: type: string runId: type: string description: '`swarm_wave` only.' scenarioId: type: string description: '`user_testing_window` only.' windowStartAt: type: integer description: '`user_testing_window` only.' windowEndAt: type: integer description: '`user_testing_window` only.' ActionableFinding: type: object required: - id - signalFingerprint - title - category - attribution - actionTarget - actionability - severity - confidence - observed - recommendation - acceptanceCriteria - affected - evidence description: >- One actionable finding. The fields are ordered from cheapest to trust to most: `observed` is deterministic, `recommendation` is the model's, and `evidence` lets you check both. properties: id: type: string description: >- Stable remediation id (`rf_<16 hex>`). Survives dynamic error values, so the same problem keeps the same id across runs — dismiss it once and it stays dismissed. signalFingerprint: type: string description: >- The registry signal this derives from. Several findings can share one. title: type: string category: type: string enum: - unknown - tool_contract - tool_runtime - capability_gap - workflow - agent_behavior - test_design - environment attribution: type: string description: >- WHOSE problem this is. `server_*` points at the MCP server; `agent_or_prompt` and `test_design` point back at the caller. enum: - unknown - server_contract - server_runtime - server_capability - agent_or_prompt - test_design - environment actionTarget: type: string description: What you would change to fix it. enum: - investigate - mcp_server - agent_configuration - eval_case - environment actionability: type: string description: >- `ready` means the finding names a specific target and change. `investigate` means it does not yet. `informational` means there is nothing to do. enum: - informational - investigate - ready severity: type: string enum: - info - low - medium - high confidence: type: string enum: - low - medium - high observed: type: string description: >- DETERMINISTIC observation — counts and identities, never model prose. This is the part you can verify yourself. rootCause: type: string recommendation: type: string acceptanceCriteria: type: array description: How you would know the fix worked. items: type: string affected: type: object required: - count - total - unit description: >- How much of the analyzed population hit this. Read it as a ratio — `1/40` and `38/40` are different problems. properties: count: type: integer total: type: integer unit: type: string enum: - iterations - sessions patternSlug: type: string target: type: object required: - serverId - surface - snapshotHash description: >- Present only when a server (and, for tool surfaces, a tool) resolved against the pinned snapshot. Required for `mcp_server` / `ready`. properties: serverId: type: string toolName: type: string surface: type: string enum: - description - input_schema - output_schema - handler - server_instructions - capability fieldPath: type: string snapshotHash: type: string description: >- The pinned snapshot the target resolved against, so a finding cannot silently re-point at a definition that changed after it was written. currentDefinition: type: object required: - truncated properties: description: type: string inputSchemaJson: type: string outputSchemaJson: type: string truncated: type: boolean evidence: type: array items: $ref: '#/components/schemas/ActionableFindingEvidence' EvalRunGoalCompletionJudge: type: object description: >- Goal completion: grades each case's final answer against its expected output. required: - status - errorCode - summary - generatedAt - modelUsed - threshold - cases properties: status: type: - string - 'null' enum: - pending - completed - failed - null description: >- `null` means this judge was NEVER requested for the run — a different answer from "requested and graded nothing". Poll rather than re-requesting while `pending`. errorCode: type: - string - 'null' description: 'Machine-readable failure reason, set alongside `status: "failed"`.' summary: type: - string - 'null' generatedAt: type: - number - 'null' description: Epoch milliseconds. modelUsed: type: - string - 'null' threshold: type: - number - 'null' description: >- The threshold these results were scored against (`passed = score >= threshold`). cases: type: array items: $ref: '#/components/schemas/EvalRunGoalCompletionCase' description: >- Per-case grades. EMPTY unless `status` is `completed` — a pending or failed judge carries no cases, and `status` is what says which. EvalRunGroundednessJudge: type: object description: >- Groundedness: grades whether each answer is SUPPORTED by its tool trajectory. required: - status - errorCode - summary - generatedAt - modelUsed - threshold - cases properties: status: type: - string - 'null' enum: - pending - completed - failed - null description: >- `null` means this judge was NEVER requested for the run — a different answer from "requested and graded nothing". Poll rather than re-requesting while `pending`. errorCode: type: - string - 'null' description: 'Machine-readable failure reason, set alongside `status: "failed"`.' summary: type: - string - 'null' generatedAt: type: - number - 'null' description: Epoch milliseconds. modelUsed: type: - string - 'null' threshold: type: - number - 'null' description: >- The threshold these results were scored against (`passed = score >= threshold`). cases: type: array items: $ref: '#/components/schemas/EvalRunGroundednessCase' description: Per-case grades. EMPTY unless `status` is `completed`. ActionableFindingEvidence: type: object required: - kind - excerpt description: >- One citation behind a finding. Evidence is what makes a finding checkable rather than an assertion. properties: sessionId: type: string iterationId: type: string kind: type: string enum: - tool_error - transcript - feedback - judge - contrast excerpt: type: string description: Scrubbed and clipped at the producer. Never a full transcript. toolName: type: string errorCode: type: string EvalRunGoalCompletionCase: type: object required: - caseKey - score - passed - reason - rubricHits properties: caseKey: type: string description: >- The stable AUTHORED-case identity, as persisted. NOT a case row id — do not join it against the ids the per-case routes take. score: type: - number - 'null' passed: type: boolean reason: type: - string - 'null' rubricHits: type: array items: type: string description: Rubric criteria the answer satisfied. EvalRunGroundednessCase: type: object required: - caseKey - score - passed - reason - unsupportedClaims properties: caseKey: type: string description: >- The stable AUTHORED-case identity, as persisted. NOT a case row id — do not join it against the ids the per-case routes take. score: type: - number - 'null' passed: type: boolean reason: type: - string - 'null' unsupportedClaims: type: array items: type: string description: Claims the tool trajectory does not support. responses: Unauthorized: description: >- Missing, invalid, revoked, or orphaned key (`UNAUTHORIZED`) — or the **target MCP server** needs an OAuth grant (`OAUTH_REQUIRED`), which is a property of the server, not your key. content: application/json: schema: $ref: '#/components/schemas/Error' examples: badKey: summary: Invalid or revoked key value: code: UNAUTHORIZED message: Invalid API key oauthRequired: summary: Target server needs an OAuth grant value: code: OAUTH_REQUIRED message: Server requires OAuth authorization Forbidden: description: Key is valid but not allowed to do this. content: application/json: schema: $ref: '#/components/schemas/Error' example: code: FORBIDDEN message: You do not have access to this project NotFound: description: Unknown project, server, or resource. content: application/json: schema: $ref: '#/components/schemas/Error' example: code: NOT_FOUND message: Server not found RateLimited: description: >- Per-key rate limit exceeded (60 requests/minute sustained, bursts up to 10). Honor `Retry-After` and back off with jitter. headers: Retry-After: description: Seconds to wait before retrying. schema: type: integer content: application/json: schema: $ref: '#/components/schemas/Error' example: code: RATE_LIMITED message: API key rate limit exceeded. Slow down and retry. InternalError: description: Something failed on MCPJam's side. content: application/json: schema: $ref: '#/components/schemas/Error' example: code: INTERNAL_ERROR message: Unexpected internal error securitySchemes: bearerAuth: type: http scheme: bearer description: >- MCPJam API key (`sk_…`). Create one at [Settings → API keys](https://app.mcpjam.com/settings/api-keys). Guest sessions cannot use the API, and API keys cannot manage other API keys. ````