Towards unbiased and robust causal ranking for recommender systems

We study the problem of optimizing ranking metrics with unbiased and robust causal estimation for recommender systems. A user may click/purchase an item regardless of whether the item is recommended or not. Thus, it is important to estimate the causal effect of recommendation and rank items higher with a larger causal effect. However, most existing works focused on improving the accuracy of recommendations, which usually have large bias and variance. Therefore, in this paper, we provide a general and theoretically rigorous framework for causal recommender systems, which enables unbiased evaluation and learning for the ranking metrics with confounding bias. We first propose a robust estimator for unbiased ranking evaluation and theoretically show that this estimator has a smaller bias and variance. We then propose a deep variational information bottleneck (IB) approach to exploit the sufficiency of the propensity score for estimation adjustment and better generalization. We also provide the learning bound and develop an unbiased learning algorithm to optimize the causal metric. Results on semi-synthetic and real-world datasets show that our evaluation and learning algorithms significantly outperform existing methods.