Cost-Aware Cascading Bandits

Chao Gan, Ruida Zhou, Jing Yang, Cong Shen

Research output: Contribution to journalArticlepeer-review

Abstract

In this paper, we propose a cost-aware cascading bandits model, a new variant of multi-armed bandits with cascading feedback, by considering the random cost of pulling arms. In each step, the learning agent chooses an ordered list of items and examines them sequentially, until certain stopping condition is satisfied. Our objective is then to maximize the expected net reward in each step, i.e., the reward obtained in each step minus the total cost incurred in examining the items, by deciding the ordered list of items, as well as when to stop examination. We first consider the setting where the instantaneous cost of pulling an arm is unknown to the learner until it has been pulled. We study both the offline and online settings, depending on whether the state and cost statistics of the items are known beforehand. For the offline setting, we show that the Unit Cost Ranking with Threshold 1 (UCR-T1) policy is optimal. For the online setting, we propose a Cost-aware Cascading Upper Confidence Bound (CC-UCB) algorithm, and show that the cumulative regret scales in O(\log T). We also provide a lower bound for all \alpha-consistent policies, which scales in \Omega (\log T) and matches our upper bound. We then investigate the setting where the instantaneous cost of pulling each arm is available to the learner for its decision-making, and show that a slight modification of the CC-UCB algorithm, termed as CC-UCB2, is order-optimal. The performances of the algorithms are evaluated with both synthetic and real-world data.

Original languageEnglish (US)
Article number9113431
Pages (from-to)3692-3706
Number of pages15
JournalIEEE Transactions on Signal Processing
Volume68
DOIs
StatePublished - 2020

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Cost-Aware Cascading Bandits'. Together they form a unique fingerprint.

Cite this