The Quality Graph should prove that connected evidence improves impact prediction. This experiment compares graph reasoning with simpler dependency approaches.
The hypothesis
A relationship-aware Quality Graph can predict release blast radius more accurately than static dependency mapping alone.
The comparison
Run the same set of historical changes through three approaches:
- repository/file-level change analysis;
- static service dependency map;
- Quality Graph using dependencies, business journeys, test evidence, defects, and incidents.
What counts as blast radius?
The eventual affected set may include:
- services;
- customer journeys;
- tests that failed;
- production incidents;
- AI evaluations that regressed;
- downstream operational signals.
What we measure
- precision of predicted impact;
- recall of material affected areas;
- time to produce the prediction;
- explainability of the relationship path;
- false expansion of the blast radius.
The main risk
A graph can appear sophisticated while simply encoding more historical bias.
The experiment should include changes that create genuinely new interaction patterns.
What would falsify the hypothesis?
If the Quality Graph adds complexity but does not improve material impact prediction over a well-maintained dependency map, the graph should not be treated as automatically superior.
Next experiment
Use the predicted blast radius to choose assurance actions and test whether the improved relationship model leads to better release decisions.
A graph earns its place in QI only if relationships improve the decision—not because graphs look intelligent.
The data set
Use a series of releases where the eventual affected areas are known from test failures, incidents, defect records, and operational telemetry.
Freeze the evidence at the point immediately after the code change so the experiment does not benefit from hindsight.
Graph relationships under test
The Quality Graph should include only relationships that could reasonably exist before the outcome:
- component dependencies;
- business journey mappings;
- test-to-component relationships;
- historical defect relationships;
- historical incident patterns;
- ownership and change history.
Precision and recall both matter
A graph that predicts everything as potentially affected will have high recall and little operational value.
A useful blast-radius model must remain selective enough to guide assurance effort.
Explainability test
For every predicted affected area, require a path explanation.
For example:
Changed API → downstream service → checkout journey → historical incident cluster.
Engineers should be able to reject incorrect relationships and see how the prediction changes.
Human comparison
Ask experienced engineers to predict impact using the same pre-release information.
The graph does not need to outperform experts everywhere. It needs to provide consistent, scalable reasoning and reveal relationships experts may miss.