How Shadow Banning Can Silently Shift Opinion Online

Legislation forcing TikTok to sell or shut down its U.S. operations is now law, the result of concerns that the app could be providing data about Americans to the Chinese government. But Yale SOM’s Tauhid Zaman says there is a less abstract threat from TikTok—and it’s one that applies to all of the social media platforms where hot takes are shared and opinions are shaped.

“TikTok and these other platforms select the content they show you,” Zaman says. “They can promote anything, demote anything. That means they can shift opinions any way they want.”

As far as most social media users know, the most powerful tool through which platforms steer public opinion is through the outright removal of objectionable content or users. But Zaman argues that there’s a more potent means through which social media platforms can control collective opinions over time, called “shadow banning.” Part of this tool’s power derives from the fact that it’s currently near-impossible to uncover, even by policymakers or software engineering experts.

More clandestine than a straightforward ban from a platform, shadow banning limits the broader visibility of a user’s content without their knowledge. A Facebook or Instagram post that’s been subjected to shadow banning would remain on the original poster’s profile page, but it would appear less, or not at all, in the timelines of other users.

In a new paper co-authored with Yale SOM PhD student Yen-Shao Chen, Zaman takes up this phenomenon—not to determine whether it’s currently happening but instead to lay out exactly how it can be done and how powerful it can be.

For the study, the researchers built a simulation of a real social network, and then succeeded in using shadow banning to shift simulated users’ opinions as well as increasing and decreasing polarization. Even when the goal was to use shadow banning to move collective sentiment to the right or left, Zaman says, the content moderation policy appeared neutral from an outside perspective. That’s because it’s possible, he discovered, to shift opinions by turning down the volume on accounts on both sides of a debate at the same time.

“It’s like a frog sitting in a pot of water; the frog’s relaxing, and suddenly, he’s cooked,” Zaman said. “A network could, in fact, be driving people towards one point of view, but if someone tries to call them out on it—like a regulatory body—they're going to see the network censoring both sides equally,” Zaman says. “It looks like there's nothing untoward happening, so they leave the network alone—and suddenly everybody thinks the earth's flat. That's what we find you could do with our technique, which is a little scary.”

If a large-scale shadow banning strategy could prove dangerous, why would Zaman set out to reverse-engineer the best way to do it? Like any powerful tool, Zaman says, shadow banning can be used for good or ill—and what’s urgently necessary is developing a detailed understanding of how it operates in the first place.

For example, a better understanding of shadow banning could help regulators recognize any bad actors out to shape network opinions. And it could help social media platforms improve their content-recommendation algorithms so as to stringently avoid inadvertent pushes into polarization.

“Using this paper, policymakers can articulate what kinds of recommendation systems to allow, and what kinds of shadow bans to allow,” Zaman says. Suppose a regulator tells platforms, “Your system should not be polarization-enhancing,” he says. “How do you define that? Well, we show in the paper what that means.”

In understanding how content regulation online can affect users’ opinions, the researchers’ first step was to establish a model of opinion dynamics, which they based on widely accepted findings on persuasion. According to their model, a user can be shifted slightly in one way or another by the opinions pronounced by their online connections—but only if those stances are relatively close to where they are already positioned. An opinion that falls too far outside of their narrow opinion window won’t budge them.

With this opinion-changeability model in place, the researchers’ next task was to simulate a large-scale social network conversation focused on specific hashtags. Zaman and Chen built two such simulations, using tweets Zaman had gathered for previous research. One of these simulated conversations drew from real tweets about the 2016 U.S. Presidential election (2.4 million tweets from seventy-eight thousand users were collected on the topic between January 2016 and November 2016) and the other drew from tweets about France’s Yellow Vest protests (2.3 million tweets from forty thousand users were collected on this topic between January 2019 and April 2019). Using a neural network, the researchers measured the sentiment of each tweet.

Next, they built a follower graph for each topic, to map out who was following whom in the large-scale conversation. Within the map, particularly important were each of the “edges” – that is, the link between any two users.

Then came the simulation stage: could they move a user to the right by carefully muting all of the connections falling slightly to their left? By turning down the volume on the most extreme opinions on either side, could they reduce overall polarization within the conversation? And by silencing more moderate positions, could they amp up the conversation’s polarization? To each of these questions, the answer was yes.

Shadow banning is hard to spot because the opinions that are muted depend on their stance relative to other users—resulting in a mix of shadow-banned and amplified users, without any obvious rhyme or reason. If, for example, a network’s goal is to move the collective sentiment to the left, the network might choose to show the content of a moderate user to a relatively right-leaning connection (to pull that connection leftward)— but block that same content from the timeline of a left-leaning connection (to keep that connection from moving even slightly toward the right). At first blush, the banning appears to impact every user more or less equally.

But Zaman and Chen say it is possible to spot whether shadow banning is present: first, by applying a negative score to the edges that pull sentiment in one direction, and a positive score to those edges that pull in the other direction. “Then you quantify the scores of the edges that are blocked,” Zaman explains. “To catch the biased shadow-banning policy, you have to think about things differently. It’s not about the people you’re blocking; it’s the connections between people you have to look at.”

Zaman plans to share his research with policymakers. “I want to show them, ‘Here’s what this network can do; this is the danger,’” he says. “If you’re not going to ban them but want to regulate them, this is how to do it—by quantifying their content algorithm. This is how we should be regulating all of the networks—X, Meta, Instagram, YouTube, all of them.’”