Multi Agent Behavior Challenge

We as humans have the ability to recognize the behaviors of objects that we come across in our lives relatively easily. If it looks like a duck, swims like a duck, and quacks like a duck then it probably is a duck. However, replicating or at least imitating the complex structure of our visual system as well as our cognitive ability is a difficult task. As we have seen from our previous competitions, the recognition of an object from a video through frames is feasible by machine learning standards. Recognizing the behavioral patterns of "agents" or the interaction between these agents, on the other hand, necessitates a different approach. We need the algorithm to automatically observe and detect the characteristics of these agents (not just "what are they?" but also "how do they behave?"). This technique is known as the Representation/Feature Learning.

AI Crowd's Multi Agent Behaviour Challenge, sponsored by Google, attempts to solve this problem as part of the CVPR 2022. The competition's goal is to develop a model that can not only learn these agents' behavioral representations but also apply to a variety of downstream behavior analysis tasks. This means that the algorithms must not only successfully demonstrate their ability to learn the features, but they must also be able to generalize this method to other cases where it can be applied. It is difficult to understand an individual's behavior without considering the interactions of the group, especially given the plethora of interaction possibilities among the multiple agents. As a result, submissions must take into account behavior on both an individual and group level.

An overview of what’s expected from the contestants. Source

There are two rounds to the challenge. The competitors are given a dataset of tracking data/videos of socially interacting animals in the first round. Rather than recognizing a specific behavior, competitors must submit a frame-by-frame representation of the dataset. This requires a low-dimensional mapping of the animals' movements through time in the submissions. The contestants must examine the interactions between trios of mice and groups of flies, and train their algorithms to distinguish between distinct behaviors. The host will employ these provided representations as input to train their own single-layer neural networks for detecting "hidden tasks" in order to evaluate these learned representations. This will allow them to practically test the generality of the submissions directly, and the efficiency of the submissions on any downstream task.

Would you like to learn more? Check out the competition website or contact us directly for any questions you have!