Introduction

Firstly, we would like to thank Zindi and the Lacuna fund for hosting this competition. We had a lot of fun doing this, and learned a lot. We hope our model has a positive impact on the world.

Despite modern medical advances, malaria diagnostics in resource-constrained regions still heavily rely on manual microscopic examination - a process bottlenecked by the scarcity of trained technicians. The Lacuna Malaria Detection Challenge, hosted on Zindi, aimed to address this problem. The practical implications to this are substantial: this technology can serve as a screening tool in high-volume settings, potentially enabling earlier intervention in regions with limited access to trained microscopists.

Our solution achieved strong and consistent performance, with scores of 0.928 and 0.925 on the public and private leaderboards respectively.

In this post, we'll detail our technical approach, from initial data preprocessing challenges through model architecture decisions to our final ensemble strategy. We'll cover:

• An overview of the problem and evaluation metric

• Our analysis into the provided image dataset

• Our model architecture choices and their empirical justification

• Key optimizations that improved model performance

• Observations and methods that gave us a winning edge.

• Technical limitations and areas for future improvement

Problem Overview

Prior to understanding the evaluation metric of this challenge, it is vital to understand the concept of bounding boxes, what they are, and how they relate to this problem. 

Bounding boxes are rectangular regions defined by four coordinates: (x_min, y_min) for the top-left corner and (x_max, y_max) for the bottom-right corner. Think of them as drawing a rectangle around an object of interest - in our case, either a malaria parasite (trophozoite) or a healthy blood cell - that completely encloses the object while staying as tight as possible to its boundaries. An example of bounding boxes are presented below:

Zindi provided us with mean Average Precision (mAP) as our evaluation metric, specifically mAP@0.5. This metric is the standard for object detection tasks, and understanding it was crucial for optimizing our solution.

Mean Average Precision combines two critical aspects of object detection:

1. How good we are at locating objects (the bounding box precision)

2. How accurate we are at classifying what we've found (infected vs. healthy cells)

The "@0.5" in mAP@0.5 refers to the Intersection over Union (IoU) threshold of 0.5. IoU measures how well our predicted bounding boxes align with the ground truth boxes - imagine sliding two rectangles over each other and calculating what percentage they overlap. An IoU of 0.5 means our predicted box needs to overlap with at least 50% of the ground truth box (and vice versa) to be considered a correct detection.

Digging deeper, for each bounding box we predict, we determine whether it's a true positive (TP, a correct prediction), false positive (FP, we predicted something not in the image), or false negative (FN, we don't predict an object) based on two criteria: our IoU is greater than 0.5, and our class prediction is correct. 

We now calculate precision and recall, two very important terms in machine learning and computer vision. 

Precision and recall are two fundamental metrics used to evaluate the performance of predictive models. 

Precision measures how accurate our positive predictions are by calculating the ratio of correct positive predictions (true positives) to all positive predictions made (true positives + false positives). The formula is given by:

Recall measures how well we identify all relevant cases by calculating the ratio of correct positive predictions (true positives) to all actual positive cases (true positives + false negatives). The formula is given by:

There exists an important inverse relationship between precision and recall, known as the precision-recall trade-off. When we adjust our model to achieve higher precision, we see a decrease in recall. Conversely, when we optimize for higher recall, precision tends to decrease. Making a system more sensitive to catch all true cases (avoiding false negatives) leads to more false alarms (false positives) and vice versa. This relationship demonstrates that improving one metric often comes at the expense of the other, requiring careful consideration of which metric is more important for your specific use case.

The mAP equation is given by:

This can actually be visualised as the area under the precision recall curve! This is vitally important in understanding what matters in this competition, and how confidence values and thresholds affect our score. 

Consequently, the objective of this competition can be thought of as having the biggest area under this precision recall graph, while maintaining the highest accuracy in identifying the classes within the image.