Valuing a Revolution

effective altruism

Effective altruists [1] have long sounded the alarm on the dangers of artificial super-intelligence. Their mission seems noble - to quantitatively reason about which problems are the most pressing and the best strategy for tackling them. Based on their models, they speculate there will come a point of no return - a world where humans are completely reliant on AI, where AI drives its own research and advancement. But does their math check out [2], [3]?

In the past couple years, the EA community has been plagued with poor public perception, notably surrounding their emergence as the “cult” running frontier AI labs, as well as the financial crimes of former advocate Sam Bankman-Fried [4]. As with any ideological group, there will be strong adherents, possibly to an extreme, as well as the average follower. Also, importantly, they were right - AI risk is very real! This is an extremely good reason to take their modeling methodology seriously.

I’m also personally invested in EA since I’m aligned with them on the front of AI safety, so I met up with a buddy on the inside and investigated some my qualms with the philosophy. Philosophy Tube has a great video on the common criticisms of EA [5], but I focused on one particular point - Does optimizing for the “most good” bias their selection of problems and solutions toward only the most measurable?

the yardstick

EA preaches quantification of actions in terms of the amount of “good” that ensues, proposed through probabilistic measurement systems [6].

Let’s characterize this with the simple metric of QALYs and DALYs - Quality-Adjusted Life Years, and Disability-Adjusted Life Years. Essentially, QALYs are “years of life gained” and DALYs are “years of life lost”:

QALY = \(\sum\)(quality of life improvement) \(\times\) (how long is your life improved for)

DALY = YLL + YLD where YLL = years of life lost from early death and YLD = (years of life disabled) \(\times\) (disability weight)

Kind of makes sense, but again, this is a simple example. A recent scheme that I’ve come across is closer to assigning a “moral weight”, computing welfare rather than health:

where the value \(V\) of the intervention \(I\) is computed by examining all outcomes \(j\). Note the two components - \(E[\Delta O_j \mid I]\) is what the outcome caused, while \(w_j\) is how we morally weight it.

The first term actually makes a lot of sense for short horizon events in my opinion - take malaria nets. We can factor in the number of people reached, baseline malaria risk and reduced malaria risk based on epidemiological statistics about the usage of nets, and actual usage percentage given deployment.

The moral weight of the outcome is more suspicious, but we can still work with it. For example, we can perform Discrete Choice Experiments, which ask questions like “Would you rather receive $50 in cash, or receive an intervention that reduced your child’s malaria risk by \(x\)?” repeatedly, varying the cash value and \(x\) across numerous iterations [7]. Given a large enough sample size, this should approximately converge to true value of one unit of intervention [8].

Before I get too far into the weeds of their modeling approach, I return to my original question - does optimizing for the “most good” lead to selection bias around easily measurable tasks? I asked my EA friend, “How do you know less easily-measurable actions, like influencing politics [9], won’t result in more good?” And his answer, to my disappointment, was that he doesn’t see an alternative. He challenged me - “If you think it’s more important, then just measure it”

measure it

Now, I will (goofily) attempt to measure the value of the original tool for enacting change upon those in power: revolution. I have two reasons for picking this - one is that this is a great adversarial argument for EA’s measurement system since it has several downstream effects, long-tail timelines, and sparse experimental evidence. The second reason, as I will go on to explain, is its relevance to the power dynamics inherent to the rise of super-intelligent AI.

Generally, I believe there is merit to capitalism’s efficiency; markets reveal lots of information, and information results in more certainty about the value of a good, service, or action that one can take.

However, I am also an advocate of the attributing value to those that produced it, and this is not inherent in raw, unfettered capitalism; workers are not compensated the value that they have produced, while the capitalist extracts much of the wealth [10], [11]. This is already exacerbated by AI - models are trained at internet-scale, and compress the data of all who have ever lived into their latent representations, without compensation or even permission.

This is partially due to the intrinsic incentives of capitalism - once innovations stop flowing, then in a poorly regulated system, the capitalist cuts corners in order to make the business more efficient. An obvious path to protect your own profit is just to reduce the wages of your workers. But independently of capitalism’s nature, AI can still make this untenably bad. Imagine - the capitalist class creates a means of extracting value without the need for a working class; what’s left is a moat between the masses and the elite.

So is the solution here really to throw our resources at AI safety? I truly believe that some of the best advice for a Machine Learning or Software Engineer is to devote their attention towards aligning the behavior of LLMs to be “ethical”, writing robust model evaluation code, etc. But for the vast majority of Americans, there is a world where AI does not result in a utopian outcome; perhaps, the best path forward is to stop AI in its tracks. And what better way than to stop funneling our value as workers into the AI slop machine than to revolt?

So let’s play the EA game - let’s value how “good” [allocating resources towards] a worker revolution would be.

valuing a revolution

The question I’m truly answering here is: How much “welfare” does $1 million of investment towards the chosen intervention produce, where the intervention is worker organization. I want to model the difference of welfare between whether the intervention happened or not:

I’ve added an annual discount rate \(r\) too (thanks high school Econ).

For worker organization, I want to separate compensation gains from changes in political power:

For the direct welfare term:

• \(N_{\mathrm{additional}}\) are the new workers because of the grant
• \(L\) is the effective number of years of the benefit
• \(\Delta u\) is the annual welfare change per additional worker that’s covered by the benefits For the political term:
• the \(s\) in the probability \(P_s\) is a possible political trajectory, like lasting reform, transformation towards more democracy/authority, etc
• \(B_s\) is the welfare from the new trajectory

Here was how I computed each term:

Here, \(G\) is the grant, \(a\) is the fraction that makes it to actual organizing expenditure, \(c\) is the cost per additional member, and \(\tau\) is an adjustment of the cost to the recipient’s circumstances. Voos estimated costs equating to around $2,208-$5,969 in 2024 dollars, which means that $1 million would support 168-453 members at those historical productivities [11], [12].

Since \(L\) is how long the welfare applies for, I made it include the discounting:

For the income component, I use log-consumption welfare - a person’s annual gain is \(log(c_{\mathrm{after}} / c_{\mathrm{before}})\), so doubling consumption for one year contributes \(log2=0.693\) units:

The political welfare term is a bit freaky since it also changes with scale, duration, and consequences. Instead of modeling it like:

I actually modeled variable intra-trajectory welfare too:

This just leaves the \(N_{\mathrm{additional}}\rightarrow\Delta P_s\) relationship - how does membership affect the probability of a consequential political trajectory? New workers aren’t just units that build up toward a political outcome, and theres lots of intermediate effects that are difficult to account for. Take, for example, whether the organization does coalition building and the political stances of the collaborating orgs. For this reason, I created an intermediate variable \(C\) to represent organization capacity. Essentially, I’m trying to separate how much capacity a grant creates and how capacity impacts political outcomes:

One way to express this is:

where \(X\) describes the political and economic context. For the \(C\rightarrow P\) relationship, or the question of political outcomes given membership, there is actually some data - based on a reanalysis. I did of some international campaign data, national union participation was correlated with a 34.8 percentage points higher probability of democracy 5 years later [12]:

To connect this with funding, let’s use a narrower model where the grant only changes whether national unions participate in a consequential campaign. Let \(q\) be the probability that this campaign happens, and \(\delta_U(G)\) be the grand-induced increased probability of national union participation. Then:

Note, \(\theta_{\mathrm{target}}\) is the actual causal association, and \(\widehat{\theta}_{\mathrm{association}}\) is just our benchmark of it based on the historical data.

The intermediate organization capacity model makes the remaining funding relationship explicit; if \(h(c, X)\) is the probability of national union participation given capacity \(c\) and context \(X\), then:

For the outcome’s value, I used Acemoglu and colleagues’ estimated democratization-GDP trajectory, where the data in question implies a 25 year log GDP effect of ~0.169 with standard error 0.0530 [13]. Summing reconstructed annual trajectory over 25 years at a 3% discount rate gives:

So far, the calculation values a political outcome; it doesn’t establish how much the grant actually changes its probability. Under the assumption that GDP gains translate proportionally to consumption gains, the restricted democratization channel \(G \longrightarrow N_{\mathrm{additional}} \longrightarrow C \longrightarrow P_s\) is:

I’ve added \(A\), the welfare associated with an AI safety intervention, as an example. \(D\) is the net direct welfare from the organizing grant and \(H\) is the total number of people affected by the modeled democratization. So we arrive at:

Or - how much would the organizing grant need to increase the probability of democratization to match the AI safety grant, in terms of welfare? I see two interesting cases here:

• \(\Delta P^* < 0\): organizing’s direct benefits already exceed AI’s estimated value
• \(0<\Delta P^* < 1-p_0\): the required probability increase of whether the population transitions to democracy due to the grant is possible mathematically For illustration, suppose \(A=20,000\), \(D=5,000\), and \(H=1,000,000\). Then:

So AI Safety starts with a 15,000 welfare point advantage, but the worker organization would move the probability of democratization from 10% to 10.9% here.

ineffective altruism

Well, that was a fun exercise, but it points at some of the limitations of EA - data for a lot of this work was few and far between, struggling with either recency bias or historical effects not mapping to a modern context. This does make me wonder how difficult it must’ve been to perform this computation for AI trajectories, and how we arrived in this situation where AI is as powerful as predicted. Perhaps there were many predicted trajectories, some correct and far more incorrect, and there exists a selection bias of the issues that actually occurred…

The truth is that AI can do an immense amount of good, yet result in an immense amount of bad. Part of the “bad” is that the current American economic and political systems will leave the workers whose effort surmounted in this God technology in the ashes, as the capitalists in control watch from far beyond the flames. And even with the power of math, it remains unclear how we will escape this hell.