We study reinforcement learning (RL) in a setting with a network of agents whose states and actions interact in a local manner where the objective is to find policies such that the (discounted) global reward is maximized. A fundamental challenge in this setting is that the state-action space size scales exponentially in the number of agents, rendering the problem intractable for large networks. In this paper, we present our framework that exploits the network structure to conduct reinforcement learning in a scalable manner. The key feature in our framework is that we prove spatial decay properties for the Q function and the policy, meaning their dependence on faraway agents decays when the distance increases. Such spatial decay properties enable approximations by truncating the Q functions and policies to local neighborhoods, hence drastically reducing the dimension and avoiding the exponential blow-up in the number of agents.

The speaker will be in-person.

https://engineering.cmu.edu/directory/bios/qu-guannan.html

Math, 501 and Zoom https://arizona.zoom.us/j/81337180102 Password:  applied