The same suspicions apply to other platforms: Even though President Trump arguably violates Twitter’s content policies, he has been allowed to stay up. Imagine the outcry if he was blocked.
This gets to the larger issue of transparency. Ms. Bickert is messianic about openness — she points out that Facebook was the first large platform to publish its entire community standard rule book.
That’s salutary. But if Facebook’s written policies are clear, how it implements them is less so. Little is known, for example, about the army of contract workers the company hires to review content that has been flagged — in other words, the people who actually make the decisions. (Facebook says they are extensively trained and their actions audited.) And because much of Facebook is personalized and many of its rules are enforced through slight tweaks in its ranking algorithm, the overall effect of its content policies may be very difficult for outsiders to determine.
That problem also plagues other platforms. Twitter, for instance, has a content filter that governs which tweets are displayed in parts of your feed and search results. But the filter’s priorities are necessarily secret, because if Twitter tells you what signals it looks for in ranking tweets, people will simply game it.
“People try to game every system there is,” David Gasca, a Twitter product manager, told me.
None of these problems are impossible to solve. Tech companies are spending huge sums to improve themselves, and over time they may well come up with innovative new ideas for policing content. For example, Facebook’s recent decision to release its data to a group of academic researchers may allow us to one day determine, empirically, what effects its content policies are really having on the world.
Still, in the end we will all be left with a paradox. Even if they’re working with outsiders to craft these policies, the more these companies do to moderate what happens on their sites, the more important their own policies become to the global discourse.
A lot of people are worried that Mark Zuckerberg is already too powerful. The danger is that we ain’t seen nothing yet.
