Incrementality Bidding via Reinforcement Learning under Mixed and Delayed Rewards
Nov 28, 2022
Speakers
Ashwinkumar Badanidiyuru
Speaker · 0 followers
Haifeng Xu
Speaker · 0 followers
About
Incrementality, which is used to measure the causal effect of showing an ad to a potential customer (e.g. a user in an internet platform) versus not, is a central object for advertisers in online advertising platforms. This paper investigates the problem of how an advertiser can learn to optimize the bidding sequence in an online manner without knowing the incrementality parameters in advance. We formulate the offline version of this problem as a specially structured episodic Markov Decision Process (MDP) and then, for its online learning counterpart, propose a novel reinforcement learning (RL) algorithm with regret at most O(H^2√(T)), which depends on the number of rounds H and number of episodes T, but does not depend on the number of actions (i.e., possible bids). A fundamental difference between our learning problem from standard RL problems is that the realized reward feedback from conversion incrementality is mixed and delayed. To handle this difficulty we propose and analyze a novel pairwise moment-matching algorithm to learn the conversion incrementality, which we believe is of independent of interest.Incrementality, which is used to measure the causal effect of showing an ad to a potential customer (e.g. a user in an internet platform) versus not, is a central object for advertisers in online advertising platforms. This paper investigates the problem of how an advertiser can learn to optimize the bidding sequence in an online manner without knowing the incrementality parameters in advance. We formulate the offline version of this problem as a specially structured episodic Markov Decision Pro…
Organizer
NeurIPS 2022
Account · 961 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
IMED-RL: Regret optimal learning of ergodic Markov decision processes
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
EcoFormer: Energy-Saving Attention with Linear Complexity
Watch later
Jing Liu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Optimal Transport-based Identity Matching for Identity-invariant Facial Expression Recognition
Watch later
Daeha Kim, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Learning dexterous robotic manipulation from pre-collected data
Watch later
Stefan Bauer, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
MWP-BERT: A Numeracy-augmented Pre-trained Encoder for Math Word Problems
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
BiT: Robustly Binarized Multi-distilled Transformer
Watch later
Zechun Liu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%