Diffhold pre-alpha

Approvals

A rule with an approval block holds matching calls until enough of the named people approve the exact change.

How a held call flows

  1. The agent calls a tool. Diffhold runs the dry-run, sees that a matched rule needs approval and records a pending request with the diff.
  2. The agent gets a denial that includes the call ID and the proposed change, and is told to retry after approval. For workloads and pods the change ends with its blast radius, for example ! Deployment team-a/web blast radius: 1 of 1 pods terminated or replaced (100%), replicas 1 → 0; PDB web allows 0 disruption(s).
  3. An approver lists pending calls, reads the diff and approves with their own token.
  4. The agent retries the same call. Diffhold runs the dry-run again. If the result matches what was approved, the approval is used up and the call goes through with a per-call token.

The call ID is a hash of the agent, the tool and its arguments, so a retry with different arguments is a different call that needs its own approval.

Rules that are enforced

Approving from the CLI

# kubectl -n diffhold-system port-forward svc/diffhold 9001
# the admin port serves HTTPS with the chart's certificate
kubectl -n diffhold-system get secret diffhold-gateway-client -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt
export DIFFHOLD_ADMIN=https://localhost:9001 DIFFHOLD_CA_FILE=ca.crt
TLS="--tls-server-name diffhold.diffhold-system.svc"

# list pending calls (admin token)
diffhold approvals list $TLS --token-file admin.token

# approve as yourself: a Kubernetes token with audience "diffhold"
kubectl create token alice -n oncall --audience diffhold > me.token
diffhold approvals approve 6663a7ce3960 $TLS --token-file me.token

Diffhold checks the approver's token with a TokenReview and requires the diffhold audience (--approver-audience), so a token meant for the API server can't be replayed here. With OIDC, set the audience to your client ID and pass your ID token.

ResponseMeaning
200Approval recorded. The body says whether the call is approved or still needs more approvers.
401Token missing, invalid or for another audience.
403Authenticated, but not a listed approver, the agent itself, or already counted.
404No pending call with that ID, or it expired.

HTTP API

On the admin port (9001, HTTPS): GET /approvals lists pending calls with their diffs and structured changes (admin token). POST /approvals/{callId} approves (approver's own token). GET /leader shows whether this replica is the leader.

In the audit log

The allowed retry's audit record carries approvedBy with each approver's user, groups and time, and the decision reason names them.

Approving from Slack

With Slack enabled, Diffhold posts each new approval request to a channel with the diff and an Approve button. A retry of a call that is still waiting doesn't post again, and approvals already given are kept.

  1. Create a Slack app with bot scopes chat:write, users:read, users:read.email and usergroups:read, and invite the bot to the channel.
  2. Turn on interactivity and set the request URL to your ingress for the diffhold Service, port 9002, path /slack/interactions.
  3. Store the bot token and signing secret, then enable it in the chart:
kubectl -n diffhold-system create secret generic diffhold-slack \
  --from-literal=botToken=xoxb-... --from-literal=signingSecret=...
helm upgrade diffhold charts/diffhold -n diffhold-system --reuse-values \
  --set slack.enabled=true --set slack.existingSecret=diffhold-slack --set slack.channel=C0123456789

A click is accepted only if Slack's signature is valid and less than 5 minutes old. The person who clicked is identified as user:<their Slack email>, with groups group:slack:<user group handle>. Name them in the policy:

approval:
  approvers: [group:slack:sre-oncall, user:alice@example.com]
Slack approvals trust your Slack workspace's identities. Use them only where Slack sign-in goes through your SSO and people can't change their own email. The Slack app has been tested against a simulated Slack API, not a real workspace yet.