Overselling AI: Facebook's Content Moderation Issues in Ethiopia

Leading social media companies, such as Facebook and X, are promoting Artificial Intelligence (AI) as a solution to manage problematic content. But can AI address the dual challenges of scale and inflammatory nature of content on social media? The 2024 Milton Wolf seminar has explored the implications of AI for global conflicts and human rights. In this blog, I examine the case of Ethiopia's civil war to shed light on the role of social media discourse in the conflict and the effectiveness of AI-assisted content moderation on Facebook.

Facebook's Ethiopia problems

While Ethiopians and rights groups have long expressed their concerns about the role of Facebook in Ethiopia's civil war, a former Facebook employee turned whistleblower, Frances Haugen, brought the charge to the forefront through her testimony before U.S. Congress in 2021. "Facebook is fueling ethnic violence in Ethiopia's civil war," Haugen told American lawmakers. The so-called Facebook papers Haugen turned to regulators have linked Facebook posts with deadly violence in Ethiopian villages.

Despite broad concerns, Facebook's role in fueling Ethiopia's civil war is yet to be systematically studied. Nevertheless, several media reports, which rely on experts and journalists who follow the country closely (1; 2, for example), documented several cases in which posts that were unambiguously calling for violence against minorities had free rein on the site. According to one of the reports, a known TV personality who makes regular appearances on state-run broadcasters posted to more than 120,000 of his followers on Facebook that the war "is with those you grew up with, your neighbor. If you can rid your forest of these thorns … victory will be yours." The post was in Amharic, a Semitic language that serves as the official language of Ethiopia and is widely used by most ethnic groups. Timnit Gebru, an Ethiopian-born scholar whose research focuses on AI ethics, said such posts are particularly concerning because they were being pushed out by influential figures who give out clear directions to commit violent acts. "It was not one random person with 10 or 100 or 1,000 followers. It was a group of leaders, with hundreds of thousands of followers, clearly instructing people what to do," Gebru said, adding that "the most shocking part was the urgency, and the horrifying way in which the words were designed to make people act now."

The reports about social media's role in Ethiopia's conflict note that Facebook has often failed to take down inciting posts for several hours and even days. These posts go against the company's own "Community Standards," including policies against "Hate Speech" and "Violence and Incitement," and should not have been on the platform in the first place or be removed quickly.

Social media's content moderation practices have evolved in the last decade. These practices of "screening user-generated data" and removing objectionable content had to evolve due to increased scrutiny from the public and regulators and the increasing scale of content users post on these sites.

Facebook now uses AI to review posts for potential violations of its "Community Standards." It has deployed AI technology to proactively check if a newly uploaded post matches previously banned content. A positive match with previously removed content means the post would be taken down before anyone sees it. The technology refers the post to a human reviewer when further evaluation is needed.

Facebook touts its AI-assisted content moderation, which it claims dramatically increased its capacity to remove objectionable content. Guy Rosen, vice president of Integrity at Facebook, stated in 2021 that AI had helped them proactively remove 97% of hate speech. As recently as in 2016, most of these removal processes took a long time to accomplish as reviews were initiated only after a user complained about a post. A human moderator had to review the problematic post to determine if it was in violation. By then, the post likely could have reached millions of people, but now AI is helping prevent its distribution before anyone sees it. Rosen said the prevalence of hate speech on the platform has dramatically dropped as a result, pointing to internal metrics that show it has been halved.

In 2021, when Facebook's Rosen touted the company's AI system success, Ethiopia's civil war raged. From its onset in November 2020, the war was fought along ethnic lines, once influential Tigray minorities of the country's northern region versus the federal government, including the majority Amhara and Oromo groups in the diverse population of over 120 million. The war was not just fought on the ground. Online hate and inciting speech by the warring sides were part of the conflict, according to Agence France-Presse (AFP), which reported that "digital activists have been engaged in a fierce battle to discredit their opponents, from pro-government sites claiming to promote independent fact-checking to opponents sharing doctored content of alleged attacks." Facebook's "Community Standards," which the company says are in place to prevent "potential offline harm that may be related to content on Facebook," were being violated by thousands of daily posts. However, the company's failure to enforce its own standards has precisely resulted in the feared outcome- offline harm-.

Facebook posts linked to violence

The fate of a village in Ethiopia's northern region, Amhara, could be a telling example of violence inspired by Facebook posts. An Ethiopian freelance journalist, Zecharias Zelalem, who has extensively reported about his country's civil war to several global news sites, told NPR that an ''inflammatory post has accused ethnic minorities in the village of being responsible for recent murders and kidnappings.'' The accusations were baseless. Nevertheless, ".. this Facebook post got hundreds of shares, hundreds of likes, all sorts of reactions. And a day later (after the accusation was posted on Facebook) … the village cited in the Facebook post was ransacked, burnt to the ground, inhabitants murdered," explained Zecharias. Two weeks after the violence, Facebook had still not taken down the post.

Whistleblower Haugen said Facebook's executives knew the site's role in Ethiopia. They still watched as the social media giant ''fans ethnic violence'' and served as a tool to coordinate attacks against civilians. Facebook has pushed back on the claim, but Facebook's own Oversight Board, a 22-member body advising the company about its content moderating policies, has implored the executives to investigate the matter further and conduct a formal investigation. Facebook did not commit to conducting the investigation and only committed to "assess the feasibility of a focused human rights due diligence project." While Facebook has long struggled to rein in threatening speech globally, Ethiopia's case shows the failure of its much-touted AI CM to handle posts in non-English languages and the harm such failures lead to.

Revelations from ''Facebook Papers''

Social media services like Facebook have a pronounced role in Ethiopia's current conflict despite the country's small number of social media users. Citing researchers, AFP reported that members of Ethiopia's diaspora, including those in the U.S., were instrumental in spreading disinformation about both sides of Ethiopia’s political conflict.

Low Internet penetration and regular Internet shutdowns mean Ethiopia is not a lucrative market for Facebook. Therefore, even when faced with increasingly widespread inciting discussions and posts, Facebook executives ignored the country. CNN reported that ''Facebook Papers,'' most of which are internal research, warned the platform was ''ill-equipped to address issues such as hate speech and misinformation in languages other than English, potentially making users in some of the most politically unstable countries more vulnerable to real-world violence.''

Facebook has been defending its handling of Ethiopia. The civil war lasted about two years, giving the multi-billion-dollar company, which has the resources to allocate for the effort if it chooses to, plenty of space to catch up. In November 2021, Facebook said the country had been put among those considered at the highest risk for conflict and violence. As a result, the company has been implementing ''both proactive solutions and a long-term strategy to keep people safe.'' The company claimed it has expanded the number of moderators to include the country’s top four languages spoken by the parties in the conflict. The company also noted that it is putting these resources for the country ''despite the fact that the country's lower internet adoption means that less than 10% of the population uses Facebook.'' But many, including its own Oversight Board, are skeptical. As part of its ruling over content concerning the country's conflict, the board recommended that the company carry out a formal assessment to ''review the success of measures Facebook took to prevent the misuse of its products and services in Ethiopia'' and its ''language capabilities in Ethiopia and if they are adequate to protect the rights of its users. '' With the PR fallout from Haugen's testimony behind it, Facebook would not be interested in exploring the issue further.

How intelligent is Facebook's Artificial Intelligence?

The assessment of Facebook's automated system efficacy in Ethiopia, which the Oversight Board recommended in December 2021, was not done. The board wanted the assessments completed within six months and made public. However, the board's recommendations were just that: recommendations. While Facebook promised to abide by the panel's ruling regarding a particular content, it has never agreed to accept or implement policy recommendations, including about Ethiopia, which would have ensured its much-touted AI CM systems were working as the company's executives said they were.

When independent researchers stepped in to review Ethiopia's case Facebook bucked looking, the results contradicted the company's claims. The advocacy group Global Witness teamed up with an independent researcher to test if the site's AI would block hateful content if it were submitted for review. They selected a dozen ''worst examples of Amharic-language hate speech Facebook posts'' that had previously been reported, including most that were taken down by the platform itself. If Facebook's AI-assisted content moderation technology works, it will flag the posts as previously banned and swiftly block their publication without a human moderator's intervention. They submitted the posts for Facebook to publish as ads. They reported that ''all 12 of the ads were accepted by Facebook for publication.'' The researchers alerted Facebook and halted the ads' publication. The researchers explained that in conversations with them, Facebook admitted its systems failed and the posts should have been blocked. However, when the researchers submitted another two hate speech ads a week later, they were again approved for publication.

We do not know why Facebook did not want to conduct the assessment in Ethiopia's case. Maybe it is because it already knew its failures, as the whistleblower Haugen claimed. Perhaps it did the evaluation and did not want to publish it to avoid another PR crisis. The experiment by Global Witness and its partnering researcher suggests a look into Ethiopia's case would not be flattering for the social media giant. Maybe Facebook does not care and does not want to do the study. We do not know. What is certain is that its much-touted AI-content moderation is not working as the company complains, and the system's failures lead to violence, exacerbating Ethiopia's conflict that has already been linked to genocide.