TV Ad Events and Digital Search: On the Selection of Outcome Measures

Prior research has shown that TV content affects what people do on the web, particularly in the minutes after a TV ad airs, when online searches for the advertised product spike. To study this, researchers have typically focused on the total volume of search queries that include any and all keywords associated with the brands and products in TV ads. We argue that a granular consideration of search queries would be beneficial for two reasons. First, focusing on relevant keywords reduces measurement error, which can hinder identification of a significant effect from TV ads. Second, by grouping queries into themes based on semantic similarity, marketers can, for example, explore consumer intent in searches, or whether response manifests in queries with objective 'value'. Our nuanced proposed approach considers query activity more broadly. Leveraging data from Bing and iSpotTV for 12 product campaigns, we first demonstrate that the outcome measure (i.e., response variable) can significantly influence conclusions about whether an ad has influenced search behavior. Second, we present a theme-based difference-in-differences method to determine the associations between TV ads and individual queries and query groups. We do this by first exploring the effects of TV ads on distinct, individual queries, and then aggregating queries into themes reflecting customer intentions, instead of aggregating all queries under a product or brand name umbrella. Our approach has implications for researchers who can improve measurement in search response; for marketers who can evaluate whether and how marketing messages resonate with consumers; and for sponsored search advertisers who can determine which keywords and queries they should bid on in the moments after their TV ads air. In addition, our method can be applied outside of the advertising context to understand how user generated content is impacted by events in general. Finally, we have developed a standalone tool that has been deployed to by Marketers internal to our company to generate reports to assess the success of their TV ad campaigns and has been deployed to our advertising customers by the Bing Ads Sales team to provide business intelligence¹.¹The dashboards in production can be demonstrated at the conference.