- CRM create/update
- Outbound messaging or notifications
- Code, configuration or deployment changes
- Ticket, record or document mutation
- Agent-to-tool workflows with approvals
- Multi-step workflows with retry/recovery logic
Primary engineering engagement
Decide how much authority this workflow has earned.
The audit answers one operational question: does this action-capable workflow have enough evidence, confirmation and recovery control for the authority it already holds — or the authority you are about to give it?
Object of the audit
We audit the action chain, not the model in isolation.
The risk-bearing object is the workflow that connects a model to data, permissions, tools and an external effect. A model answer may be imperfect without causing an incident; a write-capable action chain can turn ambiguity into a real change.
- Pure text-generation quality review
- General model benchmarking
- Certification requests
- Unbounded production exploitation
- Legal compliance opinions
- Requests that require public disclosure of private systems
Decision model
Four planes. One authority decision.
The workflow should gain authority only when all four planes tell a coherent story. The audit looks for places where capability has moved ahead of proof or recovery control.
Authority plane
What can act, on which object, with whose approval, under which scope, and which transitions must remain impossible.
Evidence plane
What evidence must be present and fresh at decision time: inputs, provenance, approvals, tool state, timestamps and version context.
Effect plane
How the intended external change is independently confirmed instead of inferring success from internal completion or a tool response.
Recovery plane
How uncertainty is contained: fail closed, retry safely, compensate, roll back, resume from a known state or escalate to a human owner.
BitEvo Doctrine in practice
The vocabulary maps directly to audit tests.
These are not slogans layered on top of the method. Each term points to a concrete question the audit must answer.
Authority Budget
Measure the permission surface created by actions, objects, integrations, approvals and retry paths. Then ask whether the evidence chain is strong enough to justify that surface.
Evidence Before Effect
For critical actions, identify the evidence that must be present and fresh before execution. Post-hoc observability is useful, but it is not an authorization gate.
False Green
Test for states that appear healthy or successful while required evidence, freshness or external confirmation has diverged from the decision state.
Failure plan
Test the assumptions that justify authority.
The 10–20 scenarios are agreed during scope. The point is not random breakage; it is to remove one assumption at a time and observe whether the workflow still knows enough to act.
01Duplicate or retry execution
02Stale or missing source evidence
03Ambiguous approval state
04Tool response ≠ external effect
05Version or configuration drift
06Interrupted run and resume
07Authority applied to the wrong object
08Evidence unavailable at decision time
Finding anatomy
A finding is complete only when it changes the owner decision.
Accepted findings are written so another engineer can challenge the assumptions, reproduce the path and see why the evidence supports a specific operational decision.
Trigger
The exact condition that makes the failure path reachable.
Authority
What the workflow was allowed to do, to which object, under whose approval and with which prohibited transitions.
Evidence
What the workflow knew at decision time — including freshness, provenance, approvals and state required to justify action.
Effect
What actually changed outside the orchestration layer, including partial, missing and duplicate effects.
Recovery
What the system does when the action cannot be trusted: stop, retry, compensate, rollback or escalate.
Owner decision
The bounded next action supported by the evidence: expand authority, constrain it, repair the workflow or retest.
Evidence discipline
Severity is not confidence, and confidence is not authority.
Observed consequence, reachability, reproducibility, evidence quality and control leverage are kept distinct. A dramatic theory with weak evidence does not outrank a narrower failure that is repeatable and tied to a real external effect.
Decision package
The output tells an owner what the workflow may do next.
The deliverables are organized around action, not theatre: what authority exists, what evidence supports it, what failed, what must change and what can be retested.
AAuthority Ledger + Evidence Contract
Critical actions, objects, approvals, integrations, retry paths, prohibited transitions and the evidence required to justify action in the agreed scope.
BReproducible Finding Records
Accepted findings with trigger, Evidence Before Effect state, observed external effect and recovery behavior.
CDecision Memo + owner backlog
Bounded owner decision plus prioritized expand / constrain / repair actions tied directly to the evidence and consequence.
DOne retest
Re-run against the same agreed finding criteria after relevant remediation is available.
False Green: capability exceeded fresh evidence.
A service surface appeared healthy while the underlying source freshness had stopped advancing. The operational issue is not a universal claim about the agent. It is narrower: the workflow could still appear ready to act after one piece of evidence required to justify that action had become stale.
The owner decision is to constrain the Authority Budget until freshness and external confirmation are explicit gates rather than implicit assumptions.
“False Green” is BitEvo terminology for an operational evidence mismatch. This public example contains no customer information, credentials, private infrastructure identifiers or secret-bearing logs. It is an internal lab example, not independent certification or customer validation.
Rules of Engagement
Authorization is part of the evidence chain.
Before testing: written scope, allowed tests, prohibited actions, access approver, data classification, minimum necessary data and safe replay conditions.
Public intake: prepares a scope brief only. It does not authorize testing and must not contain API keys, passwords, tokens, private keys, wallet seeds, production credentials or customer secrets.
Default boundary: staging/test workflow. Production penetration testing is not included by default.
Map one workflow