Towards Temporal Interest Modeling in Recommendation via Reinforcement Learning

Azuma–Hoeffding inequality Bellman optimality equation

Markov decision process Algorithm 1: Q-learning

exploration–exploitation trade-off optimism in the face of uncertainty

upper confidence bound by Hoeffding's inequality,

Thompson sampling regret minimization

Lai–Robbins lower bound w.p. at least 1 − δ,

minimax regret contraction mapping theorem

temporal-difference learning by the contraction mapping theorem,

policy gradient theorem actor–critic methods

trust-region policy optimization Lemma 1 (performance difference).

maximum-entropy reinforcement learning Boltzmann exploration

eligibility traces telescoping over t = 1, …, T:

stochastic approximation martingale difference sequence

sub-Gaussian concentration taking expectation over s′ ∼ P(·|s,a),

union bound Chernoff bound

importance sampling exploration vs. exploitation:

off-policy evaluation occupancy measure

contextual bandits Q.E.D. □

adversarial bandits online convex optimization

mirror descent Algorithm 1: Q-learning

follow-the-regularized-leader Lyapunov stability

sample complexity by Hoeffding's inequality,

value iteration · policy iteration dynamic programming principle

Bellman operator w.p. at least 1 − δ,

Robbins–Monro conditions Bellman backup · bootstrapping

generalized advantage estimation by the contraction mapping theorem,

natural policy gradient Fisher information matrix

Kullback–Leibler divergence Lemma 1 (performance difference).

successive elimination doubling trick

best-arm identification telescoping over t = 1, …, T:

pure exploration confidence radius

Bernstein inequality taking expectation over s′ ∼ P(·|s,a),

McDiarmid's inequality optional stopping theorem

Publication Info

Hengquan Guo, Haobo Zhang, Junwei Pan, Wentao Ning, Xiaotian Li, Zhixiang Feng, Shoujun Liu, Gong Chen, Shudong Huang, Haijie Gu, Xin Liu
In Submission
2026
Submitted

This work studies temporal interest modeling in recommendation systems through reinforcement learning-based decision making.