Vox Blames the AI
The algorithms that detect hate speech online are biased against black people
But two new studies show that AI trained to identify hate speech may actually end up amplifying racial bias. In one study, researchers found that leading AI models for processing hate speech were one-and-a-half times more likely to flag tweets as offensive or hateful when they were written by African Americans, and 2.2 times more likely to flag tweets written in African American English (which is commonly spoken by black people in the US). Another study found similar widespread evidence of racial bias against black speech in five widely used academic data sets for studying hate speech that totaled around 155,800 Twitter posts.
This is in large part because what is considered offensive depends on social context. Terms that are slurs when used in some settings — like the “n-word” or “queer” — may not be in others. But algorithms — and content moderators who grade the test data that teaches these algorithms how to do their job — don’t usually know the context of the comments they’re reviewing.
Both papers, presented at a recent prestigious annual conference for computational linguistics, show how natural language processing AI — which is often proposed as a tool to objectively identify offensive language — can amplify the same biases that human beings have. They also prove how the test data that feeds these algorithms have baked-in bias from the start.
No one knows for sure if the content moderation systems that Facebook, Twitter, and Google use show exactly the same biases in these studies; the technology these companies use to moderate content is proprietary. But the tech giants often turn to academics for guidance on how to better enforce standards around hate speech. So, if top researchers are finding flaws in widely used academic data sets, that presents a significant problem for the tech industry at large.
The algorithms that detect hate speech online are biased against black people
But two new studies show that AI trained to identify hate speech may actually end up amplifying racial bias. In one study, researchers found that leading AI models for processing hate speech were one-and-a-half times more likely to flag tweets as offensive or hateful when they were written by African Americans, and 2.2 times more likely to flag tweets written in African American English (which is commonly spoken by black people in the US). Another study found similar widespread evidence of racial bias against black speech in five widely used academic data sets for studying hate speech that totaled around 155,800 Twitter posts.
This is in large part because what is considered offensive depends on social context. Terms that are slurs when used in some settings — like the “n-word” or “queer” — may not be in others. But algorithms — and content moderators who grade the test data that teaches these algorithms how to do their job — don’t usually know the context of the comments they’re reviewing.
Both papers, presented at a recent prestigious annual conference for computational linguistics, show how natural language processing AI — which is often proposed as a tool to objectively identify offensive language — can amplify the same biases that human beings have. They also prove how the test data that feeds these algorithms have baked-in bias from the start.
No one knows for sure if the content moderation systems that Facebook, Twitter, and Google use show exactly the same biases in these studies; the technology these companies use to moderate content is proprietary. But the tech giants often turn to academics for guidance on how to better enforce standards around hate speech. So, if top researchers are finding flaws in widely used academic data sets, that presents a significant problem for the tech industry at large.
