Flatland AMLD 2021 competition

The Flatland challenge is a multi-agent simulation problem, in which the actions of a certain number of trains are coordinated, with the goal of minimizing the total travel time for all trains combined. The trains operate in a 2d grid world.

The Flatland organizers specifically encourage the usage of reinforcement learning, a technique that uses a reward structure to learn itself the best policy, rather than the absolute truth of a situation. For example, a train making a decision that leads to it getting stuck behind another train, will quickly learn to avoid this decision. One of the problems faced in the Flatland challenge is which information should be provided to the agents to base their actions on. The competition provides three observation methods, in which either the entire grid, a local part of the grid or a tree-like structure is passed to the agent.

The approach for our baseline uses a combination of the global and tree observation models to transform the grid into a directed graph, after which we use Dijkstra to compute efficient paths for each agent to their respective target. Our next steps will be aimed at using reinforcement learning to solve problems such as malfunctioning trains or situations in which trains collide, preventing them both from reaching their target.

After submitting the baseline, we are currently in a shared third place, as we aim to improve this score by introducing our planning algorithm! If you want to know more about the competition and our approach, please visit us at the Software Engineering channel!

We are proud to announce that we won this competition!