Explain a product recommendation
What supports this product for this customer? Start with the actual purchase records, inspect the model’s explanation, then test whether that explanation distinguishes the product you are investigating. An explanation can identify an important purchase for a customer’s list while providing weak evidence for one particular recommendation.
This example follows the saved v4 run of a synthetic retail experiment using Kumo Engine 1.0.1. The data contains 1,890 customers, 80 products, and 72,258 purchases. It includes known starter-to-accessory patterns, giving us an answer key for evaluating explanations after scoring. The personalized recommendation walkthrough covers the broader workflow and business checks.
Trace one recommendation to the source history
For customer 1790, inspect the recommendation for baking accessory 73 at October 1, 2024, 12:00. The retained explanation matches six source purchase records, with no grouped purchases hidden from this example.
The model scores rank products; they are not purchase probabilities. The local purchase score is a separate, unsigned sensitivity measure, normalized within the retained example. A value of 1.000 identifies its largest local purchase score, rather than one unit of contribution or certainty.
The starter looks like a plausible explanation for accessory 73. However, the same local purchase scores accompany every product in this customer’s list, including camping product 28. They cannot establish that the starter increased this accessory’s score specifically.
Only after recording that interpretation, inspect the next 30 days. Customer 1790 bought baking accessories 74 and 75, which ranked third and fifth, and did not buy accessory 73 again. The known purchasing pattern helps evaluate the story; future purchases must never become inputs to its explanation.
Test the explanation against a shortcut
Use the same frozen model and remove purchases in three orders: highest local score first, newest first, and random. Rescore after each removal and measure how quickly the chosen product’s score falls. If the explanation identifies evidence for that product, its ordering should outperform a simple recency shortcut as well as random removal.
In the v4 experiment’s 30 retained examples with a recent starter, score-ordered removal beat random removal for the matching accessory. It did not establish an advantage over newest-first removal. The difference in area under the flipping curve was -0.029, with a 95% paired customer bootstrap interval of -0.078 to +0.022; lower values favor score order, and this interval includes zero. These are removal-curve units, rather than probabilities or logit contributions.
The top-scored purchase identified the starter in 22 of those 30 examples, compared with 14 for newest-first. The paired McNemar test gave p = 0.096, which does not meet the experiment’s threshold of 0.05. For the model’s top-ranked product, removal in score order also showed no advantage over random removal. The result supports investigating list-level sensitivity and leaves the stronger product-specific explanation unsupported by these checks.
Separate Engine artifacts from research calculations
The notebook’s walk method is a research helper, separate from Engine’s released explanation API and from the experimental Engine walk-attribution path described in Relevance scores and graph walks. Its accessory ranking correlated more strongly with single-purchase removal effects in these 30 examples: 0.74, compared with 0.26 for the Engine local scores. That comparison is evidence from this trained model and selected examples, rather than a general guarantee about either method.
Carry a bounded answer to the business
You can say that the explanation matches this customer’s records and highlights a recent starter that matters to the list. The checks do not justify saying, “This customer receives accessory 73 because of purchase 70743.”
Retain the model identity, cutoff, source records, candidate product, and removal-test configuration with the review. The saved notebook checked the removal run against the v4 model hash and reproduced its unchanged-history scores exactly. All retained explanations came from customers included in fitting, at an earlier cutoff than the main ranking evaluation. Repeat the checks after retraining and for customer groups outside that coverage before reusing the conclusion. For artifact retrieval and eligibility, see Model Explainability.