A/B Testing for Recommender Systems in a Two-sided Marketplace
6. prosince 2021
Řečníci
Preetam Nandy
Řečník · 0 sledujících
Divya Venugopalan
Řečník · 0 sledujících
Shaunak Chatterjee
Řečník · 0 sledujících
O prezentaci
Two-sided marketplaces are standard business models of many online platforms (e.g., Amazon, Facebook, LinkedIn), wherein the platforms have consumers, buyers or content viewers on one side and producers, sellers or content-creators on the other. Consumer side measurement of the impact of a treatment variant can be done via simple online A/B testing. Producer side measurement is more challenging because the producer experience depends on the treatment assignment of the consumers. Existing approaches for producer side measurement are either based on graph cluster-based randomization or on certain treatment propagation assumptions. The former approach results in low-powered experiments as the producer-consumer network density increases and the latter approach lacks a strict notion of error control. In this paper, we propose (i) a quantification of the quality of a producer side experiment, and (ii) a new experiment design mechanism that generates high quality experiments based on this quantification. Our approach, called UniCoRn (Unifying Counterfactual Rankings), provides explicit control over the quality of the experiment and its computation cost. Further, we prove that our experiment design is optimal. Our approach is agnostic to the density of the producer-consumer network and does not rely on any treatment propagation assumption. Moreover, unlike the existing approaches, we do not need to know the underlying network in advance, making this widely applicable to the industrial setting where the underlying network is unknown and challenging to predict a priori due to its dynamic nature. We use simulations to thoroughly validate our approach and compare it against existing methods. We also implement UniCoRn in an edge recommendation application that serves tens of millions of members and billions of edge recommendations daily.Two-sided marketplaces are standard business models of many online platforms (e.g., Amazon, Facebook, LinkedIn), wherein the platforms have consumers, buyers or content viewers on one side and producers, sellers or content-creators on the other. Consumer side measurement of the impact of a treatment variant can be done via simple online A/B testing. Producer side measurement is more challenging because the producer experience depends on the treatment assignment of the consumers. Existing approac…
Organizátor
NeurIPS 2021
Účet · 1,9k sledujících
O organizátorovi (NeurIPS 2021)
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Baví vás formát? Nechte SlidesLive zachytit svou akci!
Profesionální natáčení a streamování po celém světě.
Sdílení
Doporučená videa
Prezentace na podobné téma, kategorii nebo přednášejícího
Open Rule Induction
Zhlédnout později
Wanyun Cui, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Meta-Adaptive Nonlinear Control: Theory and Algorithms
Zhlédnout později
Guanya Shi, …
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Optimal Order Simple Regret for Gaussian Process Bandits
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
DEBAGREEMENT: A comment-reply dataset for (dis)agreement detection in online debates
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Machine Learning Speeding Up the Development of Portfolio of New Crop Varieties to Adapt to and Mitigate Climate Change
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %
Novel Upper Bounds for the Constrained Most Probable Explanation Task
Zhlédnout později
Pro uložení prezentace do věčného trezoru hlasovalo 0 diváků, což je 0.0 %