Auditing Return Conditioning as a Control Knob: An Offline Diagnostic for Decision Transformer Recommendation
2026-08-25 • Information Retrieval
Information Retrieval
AI summaryⓘ
The authors study how changing the expected future reward (called return-to-go or RTG) in a movie recommendation system affects its output. They compare making changes to just the current moment versus the whole history of previous actions, finding that larger changes lead to bigger shifts in genre predictions on one dataset but not another. They also test variations where the RTG signal is randomized or removed and find little difference in prediction accuracy. The authors suggest using four specific tests to better understand if these models truly respond to reward signals. Overall, their findings show mixed evidence about whether the recommender can be controlled by manipulating RTG.
return-to-gorecommender systemDecision Transformeroffline evaluationintervention localityMovieLens datasetMyAnimeList datasetgenre predictionreward controlablation study
Authors
Jingyu Wang
Abstract
Offline return-to-go (RTG) sweeps can test whether a recommender conditioned on return is controllable, but the intervention is rarely audited. Rewriting every historical RTG token creates an increasingly synthetic context, while rewriting only the current token is more local. We test this distinction in an offline setting with a fixed window. On MovieLens 25M and MyAnimeList 2020 (MAL), we evaluate a Decision Transformer using an RTG locality ladder, a control without RTG, a logged match and score reward check, and a within-trajectory shuffled RTG ablation. On MovieLens, a $K=20$ intervention that covers the full context, applied only to real context positions, shifts the share of Crime predictions by $+23.61 \pm 2.96$ percentage points from the validation 5th to 95th percentile, whereas changing only the current slot shifts it by $+1.77 \pm 1.17$ points. The shuffled RTG model largely removes this response ($+2.08 \pm 1.20$ points at $K=20$). On MAL, the same protocol does not produce a Drama response: $K=20$ changes Drama by $-0.03 \pm 0.07$ points, and $K=1$ by $-0.01 \pm 0.01$. Genre prediction accuracy is numerically close across real RTG, no RTG, and shuffled RTG, and at $K=1$ logged match rates and matched ratings change little. Because dataset and focus-genre selection were exploratory, these magnitudes are descriptive; the cross-diagnostic pattern across locality, shuffled RTG, and the null result on MAL does not establish reward control. We propose four checks: intervention locality, a no-RTG baseline, a reward check, and RTG-content ablation.