GRB: A Generative Reinforcement Bidding Framework for Multi-Channel Online Advertising

Azuma–Hoeffding inequality Bellman optimality equation

Markov decision process Algorithm 1: Q-learning

exploration–exploitation trade-off optimism in the face of uncertainty

upper confidence bound by Hoeffding's inequality,

Thompson sampling regret minimization

Lai–Robbins lower bound w.p. at least 1 − δ,

minimax regret contraction mapping theorem

temporal-difference learning by the contraction mapping theorem,

policy gradient theorem actor–critic methods

trust-region policy optimization Lemma 1 (performance difference).

maximum-entropy reinforcement learning Boltzmann exploration

eligibility traces telescoping over t = 1, …, T:

stochastic approximation martingale difference sequence

sub-Gaussian concentration taking expectation over s′ ∼ P(·|s,a),

union bound Chernoff bound

importance sampling exploration vs. exploitation:

off-policy evaluation occupancy measure

contextual bandits Q.E.D. □

adversarial bandits online convex optimization

mirror descent Algorithm 1: Q-learning

follow-the-regularized-leader Lyapunov stability

sample complexity by Hoeffding's inequality,

value iteration · policy iteration dynamic programming principle

Bellman operator w.p. at least 1 − δ,

Robbins–Monro conditions Bellman backup · bootstrapping

generalized advantage estimation by the contraction mapping theorem,

natural policy gradient Fisher information matrix

Kullback–Leibler divergence Lemma 1 (performance difference).

successive elimination doubling trick

best-arm identification telescoping over t = 1, …, T:

pure exploration confidence radius

Bernstein inequality taking expectation over s′ ∼ P(·|s,a),

McDiarmid's inequality optional stopping theorem

Publication Info

Hongchang Wu*, Weitong Ou*, Hengquan Guo*, Zixin Shao*, Hongyan Xue, Junwei Pan, Shudong Huang, Zhangbin Zhu, Xin Liu, Nianhua Xie, Lei Xiao, Haijie Gu
ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD 2026, Applied Data Science Track)
2026
Published

This paper introduces a generative reinforcement bidding framework for multi-channel online advertising systems.