Approvals
A rule with an approval block holds matching calls until enough of the named people approve the exact change.
How a held call flows
- The agent calls a tool. Diffhold runs the dry-run, sees that a matched rule needs approval and records a pending request with the diff.
- The agent gets a denial that includes the call ID and the proposed change, and is told to retry after approval. For workloads and pods the change ends with its blast radius, for example
! Deployment team-a/web blast radius: 1 of 1 pods terminated or replaced (100%), replicas 1 → 0; PDB web allows 0 disruption(s). - An approver lists pending calls, reads the diff and approves with their own token.
- The agent retries the same call. Diffhold runs the dry-run again. If the result matches what was approved, the approval is used up and the call goes through with a per-call token.
The call ID is a hash of the agent, the tool and its arguments, so a retry with different arguments is a different call that needs its own approval.
Rules that are enforced
- Who. Approvers are listed as
user:<name>orgroup:<name>, exactly as your API server authenticates them: OIDC users and groups, orgroup:system:serviceaccounts:<namespace>for ServiceAccounts. - How many.
countdistinct people must approve (default 1). When several rules apply to one call, each rule's requirement must be met. - Not the agent. The agent's own ServiceAccount, and any ServiceAccount in an agent namespace, can never approve.
- How many open. One agent can have at most 20 open requests (
--max-pending-per-agent); more are refused until some are approved or expire. - How long. A request stays open for the rule's
timeout(default 1 hour). Once approved, it must be used within 15 minutes (--approved-ttl). - Once, for this diff. An approval is consumed by the first allowed retry. If the dry-run result changes before then, including values hidden by redaction, approval is requested again.
Approving from the CLI
# kubectl -n diffhold-system port-forward svc/diffhold 9001 # the admin port serves HTTPS with the chart's certificate kubectl -n diffhold-system get secret diffhold-gateway-client -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt export DIFFHOLD_ADMIN=https://localhost:9001 DIFFHOLD_CA_FILE=ca.crt TLS="--tls-server-name diffhold.diffhold-system.svc" # list pending calls (admin token) diffhold approvals list $TLS --token-file admin.token # approve as yourself: a Kubernetes token with audience "diffhold" kubectl create token alice -n oncall --audience diffhold > me.token diffhold approvals approve 6663a7ce3960 $TLS --token-file me.token
Diffhold checks the approver's token with a TokenReview and requires the diffhold audience (--approver-audience), so a token meant for the API server can't be replayed here. With OIDC, set the audience to your client ID and pass your ID token.
| Response | Meaning |
|---|---|
200 | Approval recorded. The body says whether the call is approved or still needs more approvers. |
401 | Token missing, invalid or for another audience. |
403 | Authenticated, but not a listed approver, the agent itself, or already counted. |
404 | No pending call with that ID, or it expired. |
HTTP API
On the admin port (9001, HTTPS): GET /approvals lists pending calls with their diffs and structured changes (admin token). POST /approvals/{callId} approves (approver's own token). GET /leader shows whether this replica is the leader.
In the audit log
The allowed retry's audit record carries approvedBy with each approver's user, groups and time, and the decision reason names them.
Approving from Slack
With Slack enabled, Diffhold posts each new approval request to a channel with the diff and an Approve button. A retry of a call that is still waiting doesn't post again, and approvals already given are kept.
- Create a Slack app with bot scopes
chat:write,users:read,users:read.emailandusergroups:read, and invite the bot to the channel. - Turn on interactivity and set the request URL to your ingress for the
diffholdService, port 9002, path/slack/interactions. - Store the bot token and signing secret, then enable it in the chart:
kubectl -n diffhold-system create secret generic diffhold-slack \ --from-literal=botToken=xoxb-... --from-literal=signingSecret=... helm upgrade diffhold charts/diffhold -n diffhold-system --reuse-values \ --set slack.enabled=true --set slack.existingSecret=diffhold-slack --set slack.channel=C0123456789
A click is accepted only if Slack's signature is valid and less than 5 minutes old. The person who clicked is identified as user:<their Slack email>, with groups group:slack:<user group handle>. Name them in the policy:
approval: approvers: [group:slack:sre-oncall, user:alice@example.com]