POBO: Safe and Optimal Resource Management for Cloud Microservices

Azuma–Hoeffding inequality Bellman optimality equation

Markov decision process Algorithm 1: Q-learning

exploration–exploitation trade-off optimism in the face of uncertainty

upper confidence bound by Hoeffding's inequality,

Thompson sampling regret minimization

Lai–Robbins lower bound w.p. at least 1 − δ,

minimax regret contraction mapping theorem

temporal-difference learning by the contraction mapping theorem,

policy gradient theorem actor–critic methods

trust-region policy optimization Lemma 1 (performance difference).

maximum-entropy reinforcement learning Boltzmann exploration

eligibility traces telescoping over t = 1, …, T:

stochastic approximation martingale difference sequence

sub-Gaussian concentration taking expectation over s′ ∼ P(·|s,a),

union bound Chernoff bound

importance sampling exploration vs. exploitation:

off-policy evaluation occupancy measure

contextual bandits Q.E.D. □

adversarial bandits online convex optimization

mirror descent Algorithm 1: Q-learning

follow-the-regularized-leader Lyapunov stability

sample complexity by Hoeffding's inequality,

value iteration · policy iteration dynamic programming principle

Bellman operator w.p. at least 1 − δ,

Robbins–Monro conditions Bellman backup · bootstrapping

generalized advantage estimation by the contraction mapping theorem,

natural policy gradient Fisher information matrix

Kullback–Leibler divergence Lemma 1 (performance difference).

successive elimination doubling trick

best-arm identification telescoping over t = 1, …, T:

pure exploration confidence radius

Bernstein inequality taking expectation over s′ ∼ P(·|s,a),

McDiarmid's inequality optional stopping theorem

Publication Info

Hengquan Guo, Hongchen Cao, Jingzhu He, Xin Liu, Yuanming Shi
Performance Evaluation 162
2023
Journal

This journal paper studies safe and efficient resource configuration for cloud microservices under uncertain latency-resource relationships and strict SLA constraints.

← Back to publications