An agent being able to act is not evidence that it should act. This experiment tests which quality actions can safely become autonomous and which should remain human-controlled.

The hypothesis

Quality autonomy should increase only when evidence strength, action reversibility, consequence and policy permit it—not simply because an agent is technically capable of acting.

The question

Take a set of common quality actions and ask:

Which of these actions should an intelligent quality system be allowed to perform without approval, and what evidence should be required first?

The autonomy ladder

Level 0 — Observe

Collect and correlate evidence. No action.

Level 1 — Recommend

Suggest a test, investigation, or risk response. A human chooses.

Level 2 — Execute reversible actions

Rerun tests, gather diagnostics, expand a test subset or create a draft defect.

Level 3 — Act within policy

Change test prioritization, quarantine a known unstable scenario or trigger a pre-approved remediation workflow.

Level 4 — Governed autonomy

Coordinate several actions while continuously evaluating confidence, policy and consequence.

The experiment matrix

Select quality actions with different levels of consequence and reversibility:

  • rerun a failed automated test;
  • collect additional logs;
  • expand targeted regression;
  • create a defect draft;
  • quarantine a flaky test;
  • change release risk classification;
  • block a deployment;
  • trigger remediation.

For each action, define the minimum evidence, confidence threshold, policy rule and escalation path.

Four variables determine autonomy

Evidence strength

How complete, fresh and consistent is the evidence?

Consequence

What happens if the action is wrong?

Reversibility

Can the action be safely undone?

Novelty

Is the situation covered by known policy, or is the system extrapolating into unfamiliar territory?

Inject disagreement

Create scenarios where the evidence conflicts: high predicted risk but strong coverage, a failure with known flakiness but elevated business consequence, or good pre-release evidence with abnormal runtime telemetry.

The system should lower autonomy when uncertainty rises.

What we measure

  • percentage of autonomous actions taken correctly without human intervention;
  • unsafe or unnecessary autonomous actions;
  • human overrides;
  • time saved on reversible low-risk work;
  • quality of escalation when evidence conflicts;
  • and whether policy violations are prevented rather than merely logged.

What would falsify the hypothesis?

The experiment fails if confidence thresholds are unstable, if humans routinely override autonomous actions, if policy becomes too complex to understand, or if the cost of governing autonomy exceeds the value of the actions automated.

Stop conditions

The system should automatically stop and escalate when:

  • required evidence is missing;
  • provenance cannot be established;
  • the action is irreversible and consequential;
  • agents materially disagree;
  • confidence falls below the defined threshold;
  • or the situation is outside known policy.

The next experiment

Allow the system to choose between several assurance actions based on expected information value. Instead of simply automating a predetermined workflow, it would decide which next action reduces the most meaningful uncertainty.

The frontier of Autonomous Quality is not “no humans.” It is autonomy that knows its authority, its evidence and when to stop.