Echos of the Ping: How Push Notifications Shape App Engagement and its Implications for Targeting

Echos of the Ping: How Push Notifications Shape App Engagement and its Implications for Targeting

For many marketing interventions, businesses can observe customers’ direct responses, such as coupon redemptions, clicks on links in email newsletters, or taps on mobile push notifications. Leveraging these observable outcomes, businesses often use machine learning-powered artificial intelligence (AI) to predict such responses for targeting purposes. When businesses allocate interventions to those most likely to respond, following the assumption that this maximizes business value, they overlook that the total causal effect typically includes additional “hidden” effects. For example, even an ignored push notification may act as a reminder, increasing future engagement. We report a preregistered field experiment that disentangles and quantifies these causal pathways in the context of mobile push notifications. We find that immediate reactions (clicks) explain about one-third (≈34%) of the total effect on app engagement, while the larger share (≈66%) stems from a hidden “memory effect,” where users (later) open the app without clicking the notification. Despite this hidden mechanism, directly observable responses remain valuable for targeting, as users who click also tend to show stronger indirect effects. Using off-policy evaluation, we demonstrate how standard supervised machine learning can leverage this correlation to enable effective targeting, providing a practical alternative to more complex causal methods. These findings contribute to the ongoing academic debate on predictive vs. causal targeting and have important implications for managerial use of AI in business decision-making.