Predictive Quality is appealing because it promises earlier risk visibility. The experiment is only useful, however, if prediction changes where teams test, investigate or seek more evidence.
The hypothesis
Quality risk can become visible before regression starts if we combine change, dependency, historical failure and business-criticality signals.
The value of prediction is not a better dashboard. The value is changing what the team does before expensive execution begins.
What would we try to predict?
Not “Will this release fail?” as a single binary outcome. That is too coarse.
More useful targets might include:
- which changed components deserve deeper validation;
- which business journeys have elevated exposure;
- where regression failure probability has increased;
- which dependencies are likely to create cascading risk;
- and where current assurance evidence is unusually weak.
Candidate signals
A practical experiment could start with signals already available in many engineering organizations:
- change size and churn;
- number and criticality of dependencies;
- historical defect density;
- production incident history;
- test failure history;
- ownership changes;
- release frequency;
- business journey criticality;
- observability anomalies;
- and coverage around the changed area.
The first trap: correlation dressed as intelligence
A model may learn that large changes fail more often, or that one component historically creates incidents. That can be useful, but it can also become lazy scoring.
The prediction has to remain explainable enough for an engineer to challenge it. “High risk” is not sufficient. We need to know which signals contributed and whether they are still relevant.
The second trap: optimizing the score instead of the decision
Predictive quality succeeds only if teams act differently because of it.
Did the model cause a targeted test to be added? Did it redirect exploratory testing? Did it trigger a dependency review? Did it prevent unnecessary full regression in a low-risk area?
The experiment should therefore measure decision impact, not just prediction accuracy.
What evidence would make this useful?
I would look for four outcomes:
Earlier signal
Can material risk be identified before traditional test execution reveals it?
Better prioritization
Does the model cause assurance effort to move toward areas with higher information value?
Explainability
Can engineers understand and challenge the risk classification?
Calibration
Do high-risk predictions fail more often than low-risk predictions in a stable, measurable way?
A deliberately small first experiment
Start with one application and one release history. Build a simple risk model using only a handful of explainable signals. Compare the predictions against actual defects, incidents and regression failures. Then ask the harder question: if the team had seen this score before testing, what would they have done differently?
If the answer is “nothing,” the model offers interesting analytics but weak Quality Intelligence.
Prediction becomes Quality Intelligence only when it changes the assurance decision.