UPDATE: See a discussion of this article on Hacker News.

Introduction

Bridging-Based Ranking is a way of scoring and ranking online content that bridges divides.

The term “Bridging-Based Ranking” was introduced in this essay by Aviv Ovadya of the Harvard Kennedy School Belfer Center. In his essay Ovadya explains how social media algorithms today tend to promote polarization and division. But it doesn’t have to be this way. Instead of promoting divisive comment that triggers people’s tribal instincts, couldn’t the algorithms help find and promote areas of common ground?

There are at least two examples of social platforms that have successfully implemented ranking algorithms that intentionally bridge divides by finding areas of common ground. The first was pol.is which was used successfully by the vTaiwan open citizenship consultation process. But probably the most successful implementation is X’s Community Notes.

Diversity of Perspectives

The Community Notes Documentation explains that the algorithm favors notes that are rated highly by users across a “diversity of perspectives”. But as I show in this article, it is somewhat more accurate to say that the note must be highly rated regardless of diversity of perspective.

The algorithm works by attempting to explain why a post receives the ratings it does: how many upvotes are due to users left-wing or right-wing biases, and how many are due to other factors. For example, in an online forum dominated by right-wing users, posts that support a right-wing worldview will obviously tend to get more upvotes. The algorithm estimates how many of those extra upvotes are a consequence of users’ right-wing bias, and how many upvotes a post would receive if that bias didn’t exist.

Extracting Information

Now why would we want to do this? Why would a predominantly right-wing community want to artificially give left-wing opinions more weight, or vice versa?

For a fact-checking product like Community Notes, plausible political neutrality may be necessary for public acceptance. But the algorithm has advantages beyond political neutrality: it actually allows us to extract more information from users.

If Community Notes was dominated by right-wing users, what would we learn by the fact that a note received a lot of “Helpful” votes? That it was helpful? Or that it supported a right-wing worldview? Or both? We can’t tell just from the raw vote counts how “helpful” the note is.

The community notes algorithm on the other hand let’s us break down the vote counts, attributing some to whatever users think “helpfulness” means and others to users’ biases. So it is not about giving “both sides” equal weight; by cancelling out the effect of political bias, we can actually extract more interesting information from the users’ votes.

Projection in Opinion Space

The chart below illustrates how this works. This charts shows a subset of notes from the Community Notes public data set, run through my own implementation of the algorithm. The horizontal axis shows the note’s “polarity” – e.g. +1 for right-wing and -1 for left wing – and the vertical axis shows its “helpfulness” (which I have labeled “common ground factor” in my chart). The note’s final score is its vertical component, or its projection on the helpfulness/common ground axis. The colors of the dots indicate their actual status in Community Notes.

Notice how there is a large spread along not just the the vertical axis, but also the horizontal axis. If want we want to know how helpful a note is, the horizontal axis is just noise. But there is a lot of information along the vertical axis. The vertical component tells us something about the post that users are trying to express through their votes. Separating this component from the horizontal component lets us extract this information, in a sense separating signal from noise.

But what is this information? It is a measure of some feature of a post that users think deserves an upvote independently of their political biases. We pinpoint or describe exactly what this feature is, but presumably it reveals how users interpret the idea of “helpfulness”.

To get an idea of what notes are “helpful”, I include a section with example notes at the end of this article. This section includes examples with a variety of combinations of factors (Helpful+Left, Unhelpful+Right, Helpful+Neutral, etc).

Why it Works

People are politically biased, but they have other tendencies, such as the tendency to upvote interesting, accurate, entertaining, or helpful information. They may mostly upvote things that support their political perspective but they will especially upvote things that support their perspective and are actually relevant and factually accurate. And they will tend to downvote notes that support opposing perspectives, but will downvote even more zealously when those notes use false or misleading information.

When the Community Notes algorithm dissects users voting behavior and factors out the polarity component, it finds that most users are at least somewhat “biased” towards helpfulness! You can see this in the plot of a sample of Community Notes users below.

There are more users with a positive common-ground factor than a negative one. There is clump of users in the upper-right quadrant because community notes users are overall right-leaning. But notice also that the helpfulness factor for these users is mostly above zero. They are also mostly biased towards helpfulness. These users are more likely to upvote posts that support a right-wing worldview, and also more likely to upvote posts that are helpful.

Common Ground

This vertical component in these plots represents what I think of as common ground. It is something users tend to agree on independently of their politics.

In the case of Community Notes, this is presumably some common idea of what constitutes “helpfulness”. But in general what exactly the common ground is depends on the community. Suppose for example there is a forum for Harry Potter fan fiction that unfortunately in recent years it has been overwhelmed by debates about whether J.K. Rowling is transphobic. There is still a lot of good fan-fiction being posted, but the home page is dominated by posts about the controversy.

In this case, the horizontal axis would likely represent the pro- and anti- J.K. Rowling factions, and the vertical axis would represent the common ground of the community: quality Harry Potter fan fiction. The algorithm lets us in a sense de-polarize the forum, factoring out the effect of polarization and getting back to community’s original essence.

Politics is not the only factor that can divide a forum. Suppose there is a popular forum for posting ridiculously cute pet pics. Sadly, in recent years, two factions have formed: the cat faction and the dog faction. The more extreme cat people mercilessly downvote pictures of dogs (regardless of how cute they are), and the dog people vice versa. Recently, the dog faction has gained the upper hand, and a cat-picture has little chance of making the front page, no matter how adorably it is.

Again, by separating the dog-cat factor from the common ground factor, we can estimate how users would vote if there were no dog-cat controversy, re-focusing the community on it’s original purpose: raw frigging cuteness.

Understanding the Algorithm

But how does the algorithm actually work? How does it determine the polarization factor and common ground factor for each user and post?

It actually works using a fairly simple algorithm called Matrix Factorization. Below I will explain how the Matrix Factorization algorithm works, starting with the version implemented by Community Notes and described in the Birdwatch Paper. There is also a good writeup by Vitalik Buterin. In my next post describe my variation of the algorithm that uses multi-factor matrix factorization.

A good way of understanding Matrix Factorization is that it is like running a bunch of linear regressions: one for each user and each item.

For example, suppose we have already discovered the polarity factor for each user, and we want to find the polarity factor for each post. A linear regression predicts users’ votes on the post as a function of the users’ polarity factors.

For a highly polarizing right-wing post, the regression line might look like this.

Highly Polarizing Right-Wing Post

       Vote 
        +1   ✕ ✕ ✕ ✕ 
         |    ↗
         |  ↗ 
-1 ______|↗______ +1  User's Polarity Factor
        ↗|
      ↗  |
    ↗    |
  ✕ ✕   -1

In this chart upvotes have a value of +1 and downvotes have a value of -1. All the right-wing users upvoted and all the left-wing users downvoted (as shown by the little ✕s). So the best fit is a line with a slope of approximately +1: the more right-wing the user, the higher the probability of an upvote, and the closer the predicted vote value is to 1. The more left-wing, the higher the probability of a downvote, and the closer the predicted vote value is to -1.

Note that there are more right-wing users than left wing users, but it doesn’t make a difference. Even if there were 100 right-wing users and only a handful of left-wing users, the slope of the best fit would be approximately the same. The algorithm does not favor the majority.

A very polarizing lift-wing post might have a negative slope:

A Highly Polarizing Left-Wing Post

       Vote    
  ✕ ✕   +1    
    ↘    |     
      ↘  |    
-1 _____↘|________ +1  User's Polarity Factor
         |↘
         |  ↘
         |    ↘
            ✕ ✕ ✕ ✕

For a completely non-polarazing post, on the other hand, the slope would be zero:

A Non Polarizing, “Good” Post

       Vote    
✕ ✕     +1     ✕ ✕ ✕ 
  →  →  →|→  →  →  
         |  
-1 ______|________ +1  User's Polarity Factor
         | 
         |   
       ✕ |       

This is a good post. Not just because the upvote probability is independent of the user’s politics, but because this post receives mostly upvotes – the intercept is above zero. This post has some quality that users of this forum are looking for that is independent of political bias.

Now, suppose there is a post that looks like this:

A “Good” but Polarizing Post

       Vote
   ✕    +1   ✕ ✕
         |  ↗ 
         |↗
        ↗|
-1 __ ↗__|________ +1  User's Polarity Factor
    ↗    |
  ↗      |
✕ ✕ ✕ ✕ 

This post has a positive slope, so it is clearly very polarizing. But the positive intercept means that voting behavior for this post cannot be explained entirely by politics. There is also a component that makes users more likely to upvote it independently of politics.

The Intercept is Common Ground

The intercept represent some kind of “common ground”: something about the post that causes users to upvote the post that cannot be explained entirely by users’ polarity factors.

The Intercept is not the Average

We might suppose that the last post above will receive more upvotes than downvotes because it has a positive intercept. But this is not necessarily the case. It depends on how many left-wing and right-wing users there are. The intercept is not the average: a post can have a positive intercept even though it receives more downvotes than upvotes, or it can have a negative intercept even though it receives more upvotes than downvotes.

What a positive intercept does tell us is that this post would receive more upvotes than downvotes if there was an equal balance of left and right-wing users.

It also tells us how users would hypothetically vote if they were all totally neutral. In such a hypothetical world, the only thing influencing users’ votes would be some common-ground factor that aligned with the intent of this particular community, attracting upvotes independently of politics.

Matrix Factorization

Okay, so we have used regression analysis to find the polarity factor for each post (the slopes of the regression lines). But in order to do these regressions, we first need to know the polarity factors for the users.

But how do we find these?

Well, if we knew all the posts’ polarity factors, we could use regression analysis to estimate the probability that a user upvotes a post as a function of the polarity factors of the posts. The slope of the regression line would then be the user’s polarity factor. The regression line for a very right-wing user, for example, might look similar to that for a very right-wing post.

A Right-Wing User

       Vote 
        +1   ✕ ✕ ✕ ✕ 
         |    ↗
         |  ↗ 
-1 ______|↗______ +1  Post's Polarity Factor
        ↗|
      ↗  |
    ↗    |
  ✕ ✕   -1

But we seem to have a chicken-and-egg problem, where we can’t find the polarity factors of users unless we know the polarity factors for posts, and vice versa.

However, the Matrix Factorization algorithm solves this by discovering the polarity factors (and intercepts) for every user and every post all in one go.

It does this by using a single equation to estimate the probability that user $i$ upvotes post $j$:

$$ ŷ_{ij} = w_{i}x_{j} + b_i + c_j $$

Here $w_i$ is the user’s polarity factor, $x_i$ is the post’s polarity factor, $b_i$ is the user’s intercept, and $c_j$ is the post’s intercept.

It then simply finds a combination of values for every $w_i$, $x_j$, $b_i$, and $c_j$ that best fits the data – that produce estimates for $ŷ_{ij}$ that are closet to the actual values $y_{ij}$. This is usually done using a variant of the standard gradient descent algorithm.

The polarity factor the algorithm discovers doesn’t necessarily correspond exactly to politics, or cat-dog preferences, or any measurable quantity. It may be a linear combination of factors. But whatever it is, it represents some latent factor of users and posts that divides the community and does a good job explaining their votes.

Conclusion

One of the reasons for my interest in Community Notes is that the same basic algorithm may be a critical part of better social protocols for improving the quality of discussion and decision-making online communities. Without it, user polarization will tend to lead to either suffocating uniformity or least-common-denominator mediocrity. The Community Notes algorithm can be used in any forum with high entropy (lots of downvotes) as a way to identify posts with posts with high Information Value based on the common-ground factor.

In my next article, I discuss ways that this algorithm can fail, and introduce an improved implementation of the algorithm that users multi-factor matrix factorization.

Example Notes

The tables below include examples of notes with various combinations of factors (Helpful + Left, Unhelpful+Left, etc.).

Helpful + Left

Helpful + Neutral

Helpful + Right

Unhelpful + Left

Unhelpful + Neutral

Unhelpful + Right