Jigsaw Rate Severity of Toxic Comments
With our society’s shift to social media in the 21st century, our ways of communication have been flooded with profanity. As with all things context matters, and sometimes you might feel like expletive words can be more effective to relay your feelings on a particular topic (like when you are trying to explain how you felt when you hit your toe on the corner of the couch). But with the addition of partial anonymity of being in an online community brings, people are more willing to use severely toxic comments and also more likely to be subjected to online abuse and harassment. There are types and levels of profanity, and different communities draw the line on different points about what can be considered toxic(rude, disrespectful, stop people from participating further) and what’s not. Because of the subjectivity of this issue, it’s difficult to pinpoint each word from a person’s perspective, but is it possible to teach a computer to differentiate between these? If input is given from a group of moderator's point of view, would an algorithm be able to reflect the severity of comments without any context from their view?
This is what Google’s Jigsaw is trying to answer through their competition on Kaggle. As ranking the severity of comments directly without any context with multiple people is a very difficult task to do, they decided to choose pairs of comments and ask their expert raters to decide on which one is more harmful according to their own understanding of toxicity. This means that multiple raters might have rated the same pairs, and even answer the same pair differently. The goal of the competition is to build a model by taking this input data from expert raters and try to predict the ranking of new pairs of comments in the same way these experts did. By achieving this, the Jigsaw is aiming to create a spectrum of profanity in which it is more feasible to draw the line. This way, artificial intelligence can use the advantage of still reflecting the raters’ perspectives but will aim to replace the personal bias of individual ratings. The submissions will be evaluated on the average agreement with the annotators with approximately 200,000 pair ratings in the provided data. Especially with the increased importance of the internet these days, we hope that our submission can pave the wave for reducing online toxicity.
