PRISMA: a packet routing simulator for multi-agent reinforcement learning

Abstract

One challenging problem in networking is the Distributed Packet Routing (DPR), where the nodes have to define a routing interface for an incoming packet. The traditional routing methods present some limitations when there is no complete view about the network topology or about the traffic (e.g. multi domain systems or wireless ad-hoc networks). One good candidate for packet routing is the usage of Multi-Agent Reinforcement Learning (MARL), a distributed approach, where the agents are located at each node and learn a routing policy in a collaborative way. Some works were done in this direction. However, they don’t use a realistic and standard simulation framework. Also, they don’t consider the control signalling messages between the agents. Then, we propose PRISMA [1], a packet level network simulator based on the network library NS3 [2]. PRISMA is capable of integrating with a MARL algorithm. We also integrated to PRISMA the control signalling mechanisms and evaluated the performance and the communication overhead. For our knowledge, this is the first tool designed for solving DPR using MARL, and this is the first work that evaluates the communication overhead caused by control signalling. The results indicate a tradeoff between the performance and the communication overhead, where the agent, which outperforms Shortest Path and approximates to the Oracle Routing, presented an overhead ratio of 150%.