This blog post is part 1 of a 2-part series. The second part will set out recommendations for companies and policymakers.
Six years ago—one month into a global pandemic—we
argued
that the automated moderation processes many platforms were rapidly adopting should be highly transparent, easily appealable, and temporary. We warned that "protocols adopted in times of crisis often persist when the crisis is over."
That warning proved prescient. The use of automation and artificial intelligence (AI) to identify, flag, and moderate content has become the new norm—a permanent feature of how platforms govern speech online. In this two part series, we’re take stock of this new norm, and considering what platforms can and should do to ensure that AI serves online expression rather than stifling it.
A brief history of automated content moderation
From spam filtering and keyword blacklists to the hash-matching technologies used to identify child sexual abuse material and terrorist content, automated technologies have been used in commercial content moderation for many years. While these tools have long posed risks to freedom of expression, their use was, for quite some time, relatively limited in scope.
Then, in 2017, a
blog post
published by Facebook (now Meta) described the company's "fairly recent" use of artificial intelligence to identify, classify, and remove violent extremist content. At the same time, Facebook emphasized caution, noting that it did not want to suggest there was "any easy technical fix."
Just one year later, Mark Zuckerberg appeared before the U.S. Senate's Commerce and Judiciary Committees and
disclosed
that "99 percent of the ISIS and Al Qaida content" removed by Facebook was flagged by AI "before any human sees it." He also stated that Facebook was "developing A.I. tools that can identify certain classes of bad activity proactively and flag it for our team at Facebook." At the time, we
raised concerns
about the ethical implications of
… [more]
