Abstract: We extend anytime constraints to the Markov game setting and the corresponding solution concept of anytime-constrained equilibrium (ACE). Then, we present a comprehensive theory of anytime-constrained equilibria that includes (1) a computational characterization of feasible policies, (2) a fixed-parameter tractable algorithm for computing ACE, and (3) a polynomial-time algorithm for approximately computing ACE. Since computing a feasible policy is NP-hard even for two-player zero-sum games, our approximation guarantees are the best possible so long as $P \neq NP$. We also develop the first theory of efficient computation for action-constrained Markov games, which may be of independent interest.

Lay Summary:

Fully autonomous vehicles are a core goal of many machine learning researchers. However, most standard frameworks for computing effective routes ignore two key factors: (1) routes must obey strict constraints including safety considerations, (2) routes must take into account the behavior of other vehicles on the road. We handle both considerations by adding strict "anytime" constraints and multiple agents on top of the usual Markov Decision Making Process (MDP) model often used for autonomous vehicles. Our contributions revolve around developing a rigorous, mathematical theory of multi-agent MDPs, also called Markov games (MG), under these anytime constraints. A significant component of our theory is the design of efficient approximation algorithms for the solution of such a constrained MG.