How do you solve a problem like Bluesky?
Third-party labeling of posts and posters can be used to trade off between speech and reach. But this model has run into problems on Bluesky. The solution might be old-fashioned standardization.
In my last post, I pointed out that the problem with social media—what made it difficult to regulate—was its scale, how its sociality was a consequence of a collapse of contexts, and the unusual way social media posts could become “viral.” All of the attendant problems with social media that have been diagnosed such as “misinformation” or harassment or even the ubiquitous focus on “algorithms” can be traced to these three things.
Typically, social media platforms have mostly focused on trying to fix the symptoms (removing “misinformation”; banning “hate speech”) rather than the underlying causes (scale, context collapse, and virality). Their main solution has been a standardized labeling regime produced through commercial content moderation. Platforms have developed community guidelines for posting content; and then employed armies of content moderators to figure out if individual posts (often flagged by users) violate those guidelines and then either taking down the post or leaving it up with a label (such as “misleading content” or “strong language” or “adult”). In recent years, they have often used these labels to demote content in their newsfeeds, a technique that has often been called “shadowbanning.”
But this method of moderation has always left people unsatisfied. As the journalist Mike Masnick has written, “content moderation at scale is impossible to do well.” Partly, because platforms end up having to make rules on their own about all the content that people post about; this is extremely difficult given the different types of posts. And second, moderation is a lose-lose situation especially for content that sits at the boundaries of many different contexts. Platforms need to make decisions about what label to slap on the content or whether to even leave it up. And whatever they do, at least one group of users gets pissed off. Moreover, it grants a great deal of power to platforms in terms of getting to shape the public sphere.
Recently, the social network Bluesky has become a destination for many left and center-left social media posters who left Twitter after it was taken over by Elon Musk.1 Bluesky’s USP is its “stackable moderation regime” In other words, Bluesky is trying to decentralize the act of labeling. Rather than the platform itself creating labels for posts and for posters, third-party providers can create labelers and consumers can subscribe to those labeling regimes; the labeler acts like a filter so that users can get the content they are most interested in and not see content (such as harassing posts or posts from certain posters) they do not like.
Decentralized labeling regimes promise to both guard the freedom of speech while restricting the reach of inappropriate speech. But recent experiences on Bluesky suggest that this sort decentralized labeling may not actually satisfy the most vocal users of social media who actively want to deplatform content and posters that they do not agree with.
I want to suggest an option in this post that we have never tried: creating a labeling regime for social media that is modeled on a process we can see actively at work in the history of technology: standardization by committee. This might help in coming up with a set of centralized labels that appeal to most stakeholders rather than just relying on the judgements of platforms and/or the most vocal social media users.
The centralized labeling regime through which platforms regulate content on social media
How do platforms currently regulate social media content? They do it by labeling content and posters. The goal is not to solve the big-picture problem like context collapse. Instead, platforms have constructed “community guidelines,” i.e., policies about certain kinds of content which are deemed inappropriate. They then employ content moderators to label the problematic content.
But the definition of problematic content remains very very specific. For instance, in the initial phase of content moderation, platforms were most interested in segmenting “adult” content (nudity, sex, violence, swearing) from content suitable for children. When they did find such content, they labeled it with tags like “adult” or “violence” or “restricted.” After the 2016 election and intensifying during the pandemic, they become much more concerned with “misinformation,” information campaigns, conspiracies, rumors, and the like. If they found such content, they labeled it as “misleading” and sometimes would put it behind a warning (if they didn’t remove it altogether). This labeling regime is designed to solve very particular problems. E.g., what content should be restricted to adults? What content should be deemed inflammatory? What content should be considered "misinformation"?
These are hard, thorny problems and they require platforms to put themselves into untenable positions that may not be so easy to defend. Because platforms had been scalded by the accusations that they did not do a good job figuring out foreign information campaigns, Twitter put a notice—at least for a day—on anyone sharing a link of the New York Post story on Hunter Biden's laptop. They had to remove the notice a day later but the damage was done. Right-wing influencers understandably pointed to this as evidence of the "media's" corruptness and its bias towards left-wing narratives.
Platforms have tried to mitigate some of this reputational loss they suffer by trying to reduce the centralization of their labeling regime. They now at least partly make the community guidelines public (this was not true in the earliest days). They have systems through which social media users can appeal their suspensions (again, this only works spottily). They have also published at least some information on their recommendation systems (but obviously, they cannot be fully transparent about this if they don't want to get gamed). Perhaps the most ambitious undertaking has been Facebook's Oversight Board, which aims to be a "Supreme Court" of sorts; but the Oversight Board takes very few cases and takes even longer to render decisions.
So, to conclude this section, the current moderation regime does not (a) try to solve the root of the problem, i.e. context collapse, but (b) instead, tries to solve piecemeal problems like trying to determine particular pieces of "misinformation" which, ultimately, does no favors to their reputation because (c) they are either suppressing someone's speech or allowing speech that someone somewhere does not approve of.
The promise of decentralized labeling … and its problems
The journalist Mike Masnick has made a case for a very different kind of content regime that happens through user-driven protocols. A protocol is essentially a labeling algorithm, implemented in code, that classifies platform posts and creators into different kinds. The key thing though is that this protocol is decentralized; it can be offered by platform companies that host the content (Facebook, Twitter) but also by users themselves and third parties (who may charge for this service).
A protocol-driven regime is thus a marketplace of labeling schemes which consumers of social media can opt into if they so wish and which they can use to structure their social media feeds. Masnick argues that a protocol of marketplaces can offer better solutions to the problem of regulating speech on social media while still remaining true to the ideal of a public sphere with freedom of expression for all. A protocol-driven regime means that platforms do not need to be heavy-handed about suppressing certain kinds of content (obviously, they would still have to look for illegal content like CSAM); instead, their users are free to decide what kind of content is acceptable to them.
For example, consider a hypothetical user who does not care for Twitter shitposts but is instead interested in finding a variety of interesting analyses of American politics across the web on Twitter.2 This user, if he has programming skills, might create a hand-crafted list of political commentators and other posters on Twitter; he might then write an algorithm that separates shit-posts from posts with links to interesting commentary and ranks the links in descending order of interestingness. This hand-crafted list + algorithm can then become the basis of a Twitter filter (a protocol-based algorithm) that can be overlaid on that user's Twitter timeline.
The hypothetical user, if he likes his filter a lot, might also make it available to others like him who don't care for shitposts, perhaps even sell it at a price. Lots of Twitter users might then use this filter as their interface to Twitter; this way, they will be exposed to posts that they like and not to those that they don't care for, all without having some centralized content moderation regime do this for them. And of course, this does not have to be the only filter they use; they could have a filter for posts about movies or music; they can also have a filter that only shows them the choicest shitposts around Twitter, and so on, and they can switch between them when they feel like it.
It should be noted that a protocol-based filter like this is actually participating in the two parts of social media that platform companies have typically kept separate: the moderation and the recommendation systems. In that sense, then, the protocol-based filter system is genuinely radical. It partakes both in the content moderation and recommendation system regimes; and it genuinely takes control away from platform companies. This should not be unwelcome to the platform companies either. This is because centralized content moderation, as it is currently practiced, is a lose-lose game. As Mike Masnick writes in his original brief for a decentralized protocol-based labeling, when platforms take down content, some people are unhappy, and when they keep the content up, another group of people is unhappy.
When I first read about Masnick's proposal for a protocol-based moderation regime, I will say that while I understood it in the abstract, I still had a hard time figuring out how it would work in practice. But my experience with the new Twitter replacement Bluesky has made me understand it much more concretely and I have to say: I now like this idea very much and I wish it can be taken forward.
Bluesky is explicitly based on the protocol framework; it allows third-parties to construct "feeds" that Bluesky users can use as filters—which then allows them to decide what kind of content they would like to consume.
In the beginning, Bluesky would only show me two timelines both of which I hated. The first one was "Discover" which I interpreted as Bluesky's answer to Twitter's "For you" feed (which I never cared for either and rarely click on). But then Bluesky's "Following" feed was useless as well because it was filled with reposts of outraged content that I did not care for.
But since then—as more and more (left-wing identified) users have flown to Bluesky from Twitter after Trump's re-election—Bluesky has added many more "feeds" and these are genuinely useful. They have performance issues and I think their ability to calculate and order the posts in the light of how much I might "like" them is on the low end. But these feeds are great! Right now, I use "OnlyPosts" to see the chronological posts on my timeline because it filters out reposts which takes care of eliminating the reposting of outrage-bait, "Catchup" to see non-reposted posts that have garnered some engagement, "Quiet Posters" to see original posts from people who don't post a lot (I love this idea), and "Mutuals" to see posts from people who also follow me. Sometimes, the feeds don't refresh so well but that seems more of a computational problem as Bluesky is struggling to ramp up its servers to increased traffic. But overall, I really like this system and I try to look for new feeds that might fit my tastes.
So, then, protocol-based decentralized labeling works as a content moderation tool, right? By letting users take control of their feeds, does it lead to less dissatisfaction with the current centralized moderation regimes of platforms? Well, not so fast—it turns out that some users, at least, are not very happy with it.
Nothing has illustrated this better than the recent kerfuffle on Bluesky about whether the journalist Jesse Singal should be banned from it. Singal, as some of you might know, wrote an influential piece in 2018 that described some of the ongoing conflicts among the experts about the efficacy of gender-affirmative medical treatments for minors. This, and later online spats, have meant that he is persona non grata for an influential section of the online left.
When Singal recently joined Bluesky, he quickly became the most blocked person on Bluesky. Partly, this was because of yet another innovation from the protocol-based labeling system that emerged on Bluesky. Enterprising Bluesky users created lists of people to block that included Singal and people who followed him. These lists, in turn, were used by other users who did not want to read Singal’s content—or others who associated with his content—to block this content from reaching their feeds.
Overall, this seems like success. But that’s not how things played out. There was a vocal segment of the Bluesky online left that pushed hard to get Singal banned from Bluesky.3 Which is to say: they were not just satisfied with this decentralized mode of content curation that Bluesky was built for but instead, they wanted to go back to old-school centralized regime where content and creators were either labeled or banned outright (i.e., “deplatformed”). When the Bluesky Trust and Safety Team decided against banning Singal, this group was bitterly disappointed.
In a wise Bluesky thread on this controversy, the content moderation scholar Joseph Seering argued that this kerfuffle tells us something about decentralized labeling/moderation. It is a nuanced argument so I will lay out the main tweets:
Seering is arguing here that the “core idea of delegating T&S [Trust and Safety] power to users can only go so far” because “users [editorial note: or at least some influential users] don’t see filters as a direct substitute for centralized T&S actions. They still care if offensive/problematic content exists on the site, even if they personally never see that content.” Seering goes a step further and points out that “they should care” and that platforms benefit from having a set of engaged users like these.
Overall, Seering ends up making both a normative and empirical argument. He argues that “Bluesky should ditch the user control angle” which should “end up being like 30% of the solution, rather than the 90% that was initially hoped for” both because its the right thing for platforms to accede to its users wishes and because this is the “vision” that its users seem to be advocating for.
Centralized labeling, redux: Learning from the history of technical standards
So what’s to be done? We seem to have arrived at a bind. Social media started off with centralized labeling which left many, if not most, users unhappy. But Bluesky’s experiment with user-driven moderation through the creation of third-party lists—which was promised as a solution to this problem—seems to have made many (influential) users unhappy as well.4
So should we go back to the old model of centralized content moderation?5 To some extent, yes, but I would argue that we have been doing the centralized model all wrong. It grants power to the two entities who have the most twisted incentives when it comes to labeling: platform companies who provide the infrastructure of platforms (and make profits based on the content) and extremely political, highly vocal social media users who organize when they see stuff they don’t like—and it leaves out other, more moderate parties, who post on social media but are not the most vocal. If all of the parties could be gotten to the table, it is possible that we could arrive at a much better system of labeling that is not driven by either the most ideological people on social media or the ones who control the infrastructure.
In other words, we need labeling standards that are created collaboratively by all parties rather than those who tend to be most motivated to organize.
The criticisms of the older model of centralized labeling that I have made here should not be new. Lately, many people (who sometimes call themselves “supply-side progressives”) have argued that the current system of permitting in the US, which is based on the idea of “community participation,” favors a certain kind of highly politically engaged resident who are able to use the threat of lawsuits to make sure that new construction projects either take too long or just get cancelled. As the Atlantic’s Jerusalem Demsas has written:
Key to understanding the undemocratic nature of “community participation” is defining who is actually meant by “community.” First, the types of people who have the time and money to sue developers under federal environmental statutes are not representative of the broader community. Second, the costs of construction (noise, a disrupted view) are localized, whereas the benefits of renewable energy are large and diffuse. That means if the process for green-lighting a project prioritizes local voices, it will miss a much larger piece of the picture: all of the millions of people who will bene t from a greener future. The environmental-justice movement’s response to this problem has been to propose expanding opportunities for litigation for marginalized communities. But research has shown that even when community leaders reduce the barriers to entry, input meetings remain just as unrepresentative as before.
Demsas’ point is not that community input is unnecessary but that we need to expand the notion of community beyond those who show up in town-halls, typically wealthier people who oppose a development for the costs it poses for them.6
In the same way, while user input should be an essential part of the labeling process in social media, it needs to accommodate stakeholders beyond the most vocal segment of social media posters.
But what would be the way to incorporate those that have a stake in the outcome (in a less immediate way) while also allowing all the parties concerned to discuss and make decisions about the trade-offs involved?
Here, we might look to one of the most salient processes in the history of technological development: committee-driven standards. In their book Engineering Rules, the historians Joanne Yates and Craig Murphy describe how much of our vast technological infrastructure has been created through standards set by committees of experts who often represented all the different interest-groups working with the technology. The goal was to reach a consensus, or if you prefer, a compromise, such that no one group could benefit from the result. As Yates and Murphy write:
From the beginning, at the heart of the process have been technical committees that worked to reach consensus on documents—the documents are the standards—that defined specific qualities of products, technical processes, or (more recently) organizational practices. At the national level, committees were typically designed to balance membership of engineers from producer companies, user or consumer companies, and those unaffiliated with either category. No category of participant was allowed to dominate. Committees typically worked on a given standard or set of standards over many years, by correspondence and in periodic meetings. They exchanged results of technical studies relevant to the standardization task, proposed potential specifications for the standard, discussed and deliberated over the proposals, tried to reach unanimous consensus, and often voted on them in the final stages.
From the inside, the decision-making process in standard-setting committees has looked a lot like deliberative democracy—a careful process that allows all voices to be heard and all positions to be considered before working toward a consensus. [my emphasis]
Often these standards-setting bodies consisted of representatives of private actors but governments played an enabling role in bringing all of them together. Yates and Murphy write that “although the standards developed this way may not always be optimal technically, reflecting conflicts and compromises among engineers representing different firms and interests, much of what these private processes have achieved has been for the good.”
The process of creating a set of labels for social media, I would argue, would benefit from being modeled on the kind of standards-setting process that enables our technological world, a process that does not easily let any one actor dominate but also allows powerful actors to reach some kind of consensus/compromise.
Obviously, the labeling system that will result cannot be predicted in advance since it will reflect the compromise between the various groups. And we would still need to think clearly about how various groups that work with social media (consumers, content creators, platforms, other third-parties) should be represented in a committee that will set labeling standards. Since this post
has become too long, I will make that the topic of a separate (and hopefully much smaller!) future post.
Musk fired most of Twitter’s Trust and Safety team. Anecdotally, it seems that the new moderation regime on Twitter is friendlier to ultra-rightwing content; however, Twitter does take down harassment and threats.
This is me, in case you haven't guessed. Though I would not care to use a filter that showed me only shitposts; I consider the shitpost to be the worst thing that came out of social media.
Predictably, this led to some vicious posts and a whole round of flame wars.
I should say that it is still very early in Bluesky’s tenure to hand down this judgement.
I’m using “go back” for rhetorical effect; the centralized model is still the most used model by big platforms like Facebook, YouTube, Twitter, and TikTok.
A reform movement devoted to “deliberative democracy” has tried to create new ways of deriving community input like “deliberative polls” and “mini-publics” that don’t just empower the most ideological members of the community.