# The model’s word is never evidence

Every autonomous coding system faces the same uncomfortable fact: language models report success optimistically. A system that trusts those reports ships confident regressions. CrewWork is engineered around the opposite assumption. A model’s claim of success is structurally inadmissible; completion must be earned with evidence the platform produced itself, and everything that enforces this fails closed. This article walks through the actual mechanisms, including what they do not guarantee.

Evidence-based completionProvenance-tagged checksAttested test resultsFail-closed boundaries

## Completion depends on independently produced evidence

When a delivery step finishes, the platform checks what it can verify independently: commands it executed, the actual Git diff, and configured acceptance probes against the running preview. It can use metadata from the model’s result to locate and inspect the work, but the model’s own success claim is not completion evidence.

-   Every verification check is stamped with its provenance at a single splice point: platform-sourced for checks the deterministic validator ran, model-sourced for checks copied from the model’s own report. The stamp is written after the model’s payload is merged, so a model that tries to forge its own provenance is overridden.
-   Only platform-sourced checks are admissible, and a check with unknown provenance is treated as untrusted.
-   Grounding runs in both directions. A model that declares a program complete while platform-sourced checks are failing is overridden to failed. A model that declares failure while the evidence is clean is redirected to repair work rather than accepted. The acceptance-item check is unconditional: in the code’s own words, a disabled safety gate refuses; it never bypasses.
-   The completion check reads only from platform-executed validation results. Model-reported success cannot satisfy that requirement; the platform must produce its own admissible evidence.
-   Implementation and repair steps must produce a real, non-empty change set: at least one added or removed line of substance in the actual git diff. Blank lines and comment-only changes do not count.
-   Missing evidence defaults to not-success. If the reviewer that confirms completion is degraded or unavailable, the step cannot complete on the model’s verdict alone; a program that reaches its end without the enforced validation floor is marked failed, never quietly finished.
-   HTTP and browser-flow acceptance probes issue real requests against the live previewed application through a signed internal proxy. A planner cannot raise the acceptable status ceiling above 400 to make a failing endpoint pass.

## Numbers you can trace back to the tool that produced them

A quality gate is only as honest as its inputs, so CrewWork treats measurement provenance as a first-class problem: where a number came from, what produced it, and whether the thing being measured was actually the thing under review.

-   Test containers never receive a Docker socket or mounted volumes, and the test command only executes after the container has been disconnected from every network. The disconnection is programmatically re-verified; if any network is still attached, the run stops with an explicit isolation failure instead of running the tests anyway.
-   Coverage numbers are read out of the coverage tool’s own native report and tagged with the tool version that produced them, retained down to the file level. Only Python and JavaScript/TypeScript have this measured path today; the other stacks report pass and fail counts and no coverage number is invented for them.
-   A coverage-improvement run re-checks its target file against the exact measured revision before it is allowed to start, and its result is only ever reported from a later measured run.
-   Preview quality evidence is recorded only when the workspace is a clean committed revision and the running container’s reported source identity matches it exactly, checked before and after the probe. If anything drifts mid-measurement, the evidence is voided rather than recorded.
-   Flaky-test detection is automatic: a test that both passed and failed at the identical revision gets flagged with no human input. Quarantining that test is the opposite: it requires an authenticated human, a written reason, and an expiry capped at 90 days. A service or agent actor is refused outright.

The full gate and test-engine model lives on [Testing & Quality Gates](https://crewwork.ai/quality/).

## The hardest trust problem: a platform that edits its own code

Platform Self-Repair is where trust engineering gets adversarial, because the code under test could, in principle, tamper with the tests. CrewWork’s answer is that a candidate never grades itself: everything that judges a candidate belongs to the trusted incumbent, and everything the candidate produces is bound to exactly what ran.

-   The compose files, Dockerfiles, and quality-gate configuration that grade a candidate are the trusted orchestrator’s own read-only copies, overlaid onto every candidate before validation runs. The overlay itself fails closed: a writable or symlinked control file aborts the run.
-   Every matrix test result carries an attestation binding 12 fields together, including the exact run, candidate tree hash, command line, exit code, and a recomputed hash of the actual test report. Any mismatch invalidates the whole result and blocks promotion.
-   The quality gate diffs the candidate line by line against its exact trusted parent. One new lint, type-check, or security suppression comment fails the gate outright, and skipping the parent comparison is itself treated as a failure.
-   Well over 100 platform-authority files and path prefixes short-circuit straight to mandatory human review before any test matrix runs. There is no tier override and no bypass.
-   Every candidate’s commit is durably checkpointed before its branch ref is published, and the checkpoint refuses to advance toward a commit, push, or pull request unless an identity-matched assurance record accompanies that exact candidate.
-   A PR observed as merged without an identifiable merging GitHub account is treated as unresolved, never silently accepted as a promotion.

The complete lifecycle, tiers, and guardrails are on [Platform Self-Repair](https://crewwork.ai/self-modification/).

## When a check cannot run, the answer is no

Most trust failures in automated systems are not broken checks. They are checks that silently did not run: the scanner that crashed and let the push through, the sandbox that fell back to the host runtime, the session that outlived the permission behind it. CrewWork’s boundaries are built so that the absent-check case is a refusal.

-   gVisor has no fallback: every untrusted container is created with the runsc runtime explicitly, there is no code path that retries on the host runtime, and a project’s own compose file is rejected if it tries to override the runtime.
-   Writes are denied before they happen: every workspace write is checked against the plan’s declared allowed paths plus protected and suspicious path rules, and a violation refuses the tool call before any file changes.
-   Pushes are scanned or stopped: every CrewWork-initiated push runs a pinned Gitleaks scan over the exact base-to-head range in a staged clone, the scanned commit hash is bound into the push itself, and a scan that fails to complete blocks the push rather than defaulting to allow.
-   Repeated identical failures stop themselves: a database-backed breaker halts a step after three consecutive same-reason failures, and the count survives restarts, crashes, and step replacement, so a stuck loop cannot outlast the process that noticed it.
-   Access dies with membership: every backend replica re-checks a per-user authorization revision on a two-second cadence, and a membership change closes that user’s live sessions with an explicit revocation reason. If the revision check itself fails, everyone is re-fenced rather than trusted on cache.

The sandboxing model behind these boundaries is on [Runtime Isolation](https://crewwork.ai/isolation/), and the wider protections are on [Security](https://crewwork.ai/security/).

## Built to be honest on a 27B model, not a frontier one

None of the mechanisms above depend on a smarter model. That is the point. CrewWork’s default generative model is the same revision-pinned Qwen3.8 27B open-weight artifact across planning, implementation, review, structured extraction, and chat. Workloads use separately configured generation profiles and can run on separate model hosts. Semantic search uses a separate embedding model. Local hosting is the default; other OpenAI-compatible endpoints can be configured explicitly.

-   Extended reasoning is switched on per lane only where the model registry marks the model as reasoning-capable: on for review and agentic tool turns, off for planning and structured extraction. A directive the model cannot honor is never sent.
-   Structured output is validated against the caller’s JSON schema and rejected with a typed reason rather than coerced into something that parses. Typed reasons include invalid JSON, wrong shape, empty content, and schema failure.
-   Each lane has an explicit output token budget. Each program also has a persisted total budget, checked against exact recorded usage, so a small model working in long loops is bounded by accounting, not by hope.
-   Platform Self-Repair programs run entirely on this model, from plan through agentic implementation, isolated validation, final review, and the promotion matrix to a draft pull request on the platform repository. That capability is described without a success rate; see Honest Limits below.

## What we deliberately do not claim

A trust story that overclaims is worse than no trust story. These are the limits of the mechanisms above, stated the way we hold ourselves to them internally.

-   We do not publish accuracy, completion, or success percentages for the model lane, because we have not measured them at a scale worth quoting.
-   The write-scope gate is a runtime policy default. An operator can configure an unbounded write scope for a run; the protected-path and suspicious-path checks stay active regardless, but scope enforcement is a default, not a physical impossibility.
-   The nine-dimension assurance record on self-repair candidates is recorded evidence, not a gate. Its enforcement field is literally the string “shadow”; only three of the nine dimensions are live per-run diffs, and its own aggregate decision is never consulted to stop a promotion. The blocking comes from the attestation, quality gate, authority paths, and checkpoint described above.
-   The record’s hash proves it has not been altered since it was written. That is self-consistency, not a third-party signature, and we deliberately avoid stronger words for it.
-   Secret scanning is a pinned detection tool matched against known secret shapes. It is a strong gate, not a proof that no secret exists.
-   Prompt trust boundaries neutralize known override patterns, fence untrusted content behind reserved delimiters that injected text cannot forge, and trace every line to its source. That is compiler-level hygiene, not a formally verified defense against every possible injection phrasing.
-   The protected platform-authority list is a maintained inventory of known-sensitive paths, not dynamic taint tracking. Changes that reach sensitive behavior through unlisted paths rely on the other, independent gates.
-   Measured coverage provenance covers Python and JavaScript/TypeScript today. The full breakdown of what is measured versus reported lives on Testing & Quality Gates.

## And after all of it, a person decides

Every mechanism on this page exists to make the change that reaches you worth your review time, never to remove the review. Project fixes land on isolated local delivery branches with push disabled by default; platform self-repair opens draft PRs, and nothing merges without a human. Evidence narrows the question; you still answer it.

[Security Model](https://crewwork.ai/security/)

[Platform Self-Repair](https://crewwork.ai/self-modification/)

## Bring your next piece of work.

Discuss what you want to build, repair, or improve, along with your deployment needs and access to CrewWork.

[Discuss your project](https://crewwork.ai/about/#access)

[View the Architecture](https://crewwork.ai/architecture/)

Or [see how it works](https://crewwork.ai/how-it-works/) for the full technical picture.

---

Canonical page: https://crewwork.ai/trust-engineering/