A deployment is not successful until the running workload changed

We built Deployment Proof because our old definition of success was too easy to satisfy.

A deploy command could finish. The deployment record could say succeeded. The health endpoint could return HTTP 200. Yet the machine might still be serving the previous container. That is a particularly bad failure mode for an AI agent: every familiar signal looks green, so the agent confidently reports a result that is not true.

Deployment Proof is a read-only Appaloft query that compares the deployment’s recorded intent with runtime evidence. It returns one verdict: verified, partially-verified, unverified, stale, or failed. Only verified is permission to say the deployment succeeded.

The false positive we wanted to catch

Consider a routine redeploy. Version 1 is healthy. We change APP_VERSION to v2, redeploy, and get another healthy response. If the old container was never replaced, a health-only check passes anyway.

The proof needs to answer harder questions:

• Did the resolved artifact match the requested artifact?
• Did the workload identity or generation change?
• Did the runtime configuration fingerprint change when the Resource configuration changed?
• Does the access route point at the current workload?
• Is there a usable recovery candidate?

None of those checks requires returning a raw secret. Configuration is represented by a SHA-256 fingerprint. The proof schema contains expected effects, observed evidence, mismatches, and safe follow-up actions, but no secret values.

One query, five verdicts

The query is available through the same operation catalog as the rest of Appaloft:

appaloft deployments proof <deploymentId>

The HTTP route is GET /api/deployments/{deploymentId}/proof. The generated SDK and MCP tool expose the same deployments.proof operation, and the Web console renders the same DTO on deployment detail pages.

The verdict deliberately avoids turning missing evidence into success:

• verified means the required artifact, workload, configuration, health, access, and recovery evidence agrees with the deployment intent.
• partially-verified means some evidence agrees, but a required readback is unavailable.
• unverified means there is not enough runtime evidence to make the claim.
• stale means the runtime is healthy but no longer represents this deployment.
• failed means observed evidence directly contradicts the intended result, such as failed health or an access route attached to the wrong workload.

This distinction matters. “We cannot read the artifact digest” and “we read a different digest” are different operational problems.

What the Docker smoke actually did

We tested the feature against a real local Docker daemon, not a mocked success response. The smoke built a tiny fixture whose health endpoint returns its APP_VERSION.

The sequence was:

1. Deploy the fixture with APP_VERSION=v1 through Appaloft.
2. Change the Resource configuration to v2 through the formal configuration operation.
3. Redeploy through Appaloft and query Deployment Proof.
4. Replace the current container externally with a healthy container serving v1 and carrying the old deployment/config labels.
5. Query the same proof again.

After the formal redeploy, the result was verified. The workload identity changed, the configuration fingerprint changed, health passed, and access passed. The runtime reported the immutable image digest it had resolved.

Then came the useful part. The externally replaced container still returned a healthy JSON response:

{
  "healthStatus": "ok",
  "servedVersion": "v1",
  "proofVerdict": "stale",
  "mismatches": [
    "workload_generation_mismatch",
    "configuration_fingerprint_mismatch",
    "public_access_failed",
    "access_route_workload_mismatch"
  ]
}

HTTP 200 did not rescue the verdict. The running workload was healthy, but it was not the workload created by the deployment under review.

Where the evidence comes from

For local Docker and Docker Compose deployments, Appaloft finds the container by its Resource label and inspects its image identity, workload labels, start time, health, route ownership, and safe configuration fingerprint. Docker Swarm uses the corresponding service identity and update generation.

The application layer does not know Docker commands. It asks a runtime evidence port for normalized observations, then evaluates those observations against deployment state. This keeps the verdict logic testable and lets another runtime adapter provide equivalent evidence without changing the query contract.

Generic SSH deployments currently report an explicit evidence gap when no capable runtime readback adapter is present. That produces a non-verified verdict. We prefer an honest gap over inferring success from an execution record.

AI and GitHub checks use the same rule

The Appaloft skill now tells agents to read deployments.proof after observing deployment progress. An agent may describe a terminal state, but it may only claim success when the verdict is verified.

Cloud’s GitHub-check summary follows the same mapping. A green workflow is input evidence, not the final judgment. The control plane still has to reconcile what was requested with what is running.

That gives CLI users, the Web console, an MCP client, and an automated check one answer instead of four loosely related interpretations.

Read the deployment lifecycle documentation for the command and verdict model. For the provenance side of the problem, see what evidence a GitHub Actions deployment should leave behind.

The next adapters will have to earn verified with their own runtime evidence. That is the point of the verdict: unsupported readback remains visible instead of disappearing behind a green status.