This post is part 2 in a series about automated content moderation. Read the first post
here
.
When whistleblower Frances Haugen leaked a set of documents from Meta in 2020, among the revelations was a jarring statistic: The company’s algorithms designed to detect terrorist content incorrectly
deleted nonviolent Arabic-language content
77 percent of the time, while
failing to detect hate speech
under the company’s own policies in many instances. Meta’s own transparency report released later that year
demonstrated similar findings
. Five years later, researchers in the region report that
overzealous moderation remains a problem
, while paths to remedy have all but collapsed.
Where these systems are faltering in Arabic, they’re positively failing in less-resourced languages. As a
2025 report
from the Center for Democracy and Technology found, labeled datasets in certain languages and dialects such as Maghrebi Arabic and Kiswahili contain inconsistencies, bias, and inaccuracies due to the limited hiring of annotators who actually speak the languages as well as shifts in the languages themselves. An
investigation
into ChatGPT’s outputs in several low-resource languages demonstrates the depth of problem.
But language disparities are just one of several concerns as automated moderation becomes more widespread. From the
systemic suppression of content from Palestine
to the repeated
misclassification of LGBTQ+ content as adult or explicit material
, these varied examples demonstrate the risks of overreliance on automated moderation—and the need for stronger safeguards.
Transparency, Cultural Competence, Appeals
As we discussed in
Part 1
of this series, automated systems can process content at a scale that humans never could, potentially enabling better moderation at scale and alleviating the psychological load on ill-paid moderators whose jobs require them to view incredibly disturbing content. But automated systems also reproduce existing biases,
