Event-Time Confounding Under Bursty Human Dynamics
2026-08-21 • Human-Computer Interaction
Human-Computer Interaction
AI summaryⓘ
The authors explain that when researchers study how people behave online by looking at activities around specific events (like opening an AI assistant), they might mistake ongoing activity for effects caused by the event itself. They found that user activity often rises before these events, which means the event is happening during an already active period, not necessarily causing more activity afterward. They prove it's hard to separate true event effects from this bias without extra data and provide tools to diagnose and reduce this problem. Their work suggests that just looking at more activity after an event isn’t enough to say the event caused the change.
endogenous time zeroepisode-selection biasuser fixed effectscross-surface web logsevent alignmentplacebo testtime-varying confounddigital behavior analysiszero-effect simulationsactivity matching
Authors
Michael Iannelli, Alan Ai
Abstract
Studies of digital behavior often align users at moments they choose, such as opening an AI assistant, clicking a recommendation, or visiting a product page, and interpret higher activity afterward as an event effect. We show how this creates an endogenous time zero: the event occurs during an ongoing task episode, so the aligned curve can trace episode continuation rather than a response to the event. In same-user, cross-surface web logs, AI, shopping, news, coding, and reference events are all preceded by broad activity increases that peak before time zero. Our strongest test uses known-null timestamps that cause nothing. Among the 5.8% of AI responses meeting strict pre-event activity and washout criteria, these timestamps show 3.42 times the post-event search activity of a within-user placebo, compared with 4.32 times for real events. The fraction of excess reproduced by the known null falls from 0.56 at detectably active moments to -0.04 at quiet moments, where the design detects none. We formalize this episode-selection bias, prove that a single-surface event window cannot separate it from a genuine effect without additional assumptions, and show in zero-effect simulations why user fixed effects and coarse activity matching can fail: the confound is within-user and time-varying. We provide a diagnostic protocol, public-data benchmarks, and burstcheck, a lightweight audit tool. User-timed events may have real effects, but post-event volume does not identify them by default; studies should compare similar episodes with and without the event.