Learning to Schedule Online Tasks with Bandit Feedback

Azuma–Hoeffding inequality Bellman optimality equation

Markov decision process Algorithm 1: Q-learning

exploration–exploitation trade-off optimism in the face of uncertainty

upper confidence bound by Hoeffding's inequality,

Thompson sampling regret minimization

Lai–Robbins lower bound w.p. at least 1 − δ,

minimax regret contraction mapping theorem

temporal-difference learning by the contraction mapping theorem,

policy gradient theorem actor–critic methods

trust-region policy optimization Lemma 1 (performance difference).

maximum-entropy reinforcement learning Boltzmann exploration

eligibility traces telescoping over t = 1, …, T:

stochastic approximation martingale difference sequence

sub-Gaussian concentration taking expectation over s′ ∼ P(·|s,a),

union bound Chernoff bound

importance sampling exploration vs. exploitation:

off-policy evaluation occupancy measure

contextual bandits Q.E.D. □

adversarial bandits online convex optimization

mirror descent Algorithm 1: Q-learning

follow-the-regularized-leader Lyapunov stability

sample complexity by Hoeffding's inequality,

value iteration · policy iteration dynamic programming principle

Bellman operator w.p. at least 1 − δ,

Robbins–Monro conditions Bellman backup · bootstrapping

generalized advantage estimation by the contraction mapping theorem,

natural policy gradient Fisher information matrix

Kullback–Leibler divergence Lemma 1 (performance difference).

successive elimination doubling trick

best-arm identification telescoping over t = 1, …, T:

pure exploration confidence radius

Bernstein inequality taking expectation over s′ ∼ P(·|s,a),

McDiarmid's inequality optional stopping theorem

Publication Info

Yongxin Xu, Shangshang Wang, Hengquan Guo, Xin Liu, Ziyu Shao
International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2024)
2024
Published

This paper studies online task scheduling under bandit feedback and develops learning-based algorithms for adaptive scheduling decisions.