Charity Cost-Effectiveness in an Uncertain World

First written: 28 Oct. 2013; last update: 4 Dec. 2015

Summary

Evaluating the effectiveness of our actions, or even just whether they're positive or negative by our values, is very difficult. One approach is to focus on clear, quantifiable metrics and assume that the larger, indirect considerations just kind of work out. Another way to deal with uncertainty is to focus on actions that seem likely to have generally positive effects across many scenarios, and often this approach amounts to meta-level activities like encouraging positive-sum institutions, philosophical inquiry, and effective altruism in general. When we consider flow-through effects of our actions, the seemingly vast gaps in cost-effectiveness among charities are humbled to more modest differences, and we begin to find more worth in the diversity of activities that different people are pursuing. Those who have abnormal values may be more wary of a general "promote wisdom" approach to shaping the future, but it seems plausible that all value systems will ultimately benefit in expectation from a more cooperative and reflective future populace.

Epigraphs

To teach how to live without certainty, and yet without being paralyzed by hesitation, is perhaps the chief thing that philosophy, in our age, can still do for those who study it. --Bertrand Russell, History of Western Philosophy

If something sounds too good to be true, it probably is. --proverb

Introductory dialogue

In fall 2005, I was talking with a friend about utilitarianism:

Friend: "I see where you're coming from, but utilitarianism doesn't work."

Me_2005: "Why not?"

Friend: "You have no way of assessing all the effects of your actions. Everything you do has a million consequences."

Me_2005: "We do the best we can to estimate them."

Friend: "The error bars are infinite in both directions."

Me_2005: "Hmm, yes, the error bars are infinite, but we can assume the unknowns cancel out. For example, consider building safer seat belts. In the short term, we know it prevents injuries, which is good. There may be spill-over longer-term effects, but those could equally be good or bad, so they cancel out in the calculations. Hence, the expected value is still positive."

If I were to continue the conversation now, talking with my past self, here's how it might go:

Me_2013: "Ok, let's consider the example of seat-belt safety. One effect is to prevent injuries and tragic deaths, which is good. But another effect is to increase the human population, which means more meat consumption. Since people eat over a thousand chickens and an order of magnitude more fish in their lifetimes, this seems net bad on balance."

Me_2005: "Hmm, ok. Then maybe seat belts are net negative.... Those two factors you enumerated are pretty concrete, and everything else is fuzzy, so we can assume everything else cancels out. Hence the overall balance is bad."

Me_2013: "Well, there might be other good side effects of seat belts. Better car safety means more people driving, which means more inclination to build parking lots. And parking lots are one very clear way to prevent vast amounts of suffering by wildlife that would have otherwise lived short lives and endured painful deaths on the land had it not been paved. Maybe there are more insects whose painful deaths are spared by parking lots than there are additional chickens and fish killed by the drivers who remain alive."

Me_2005: "Ah, good point. Parking lots are quite beneficial, so on balance, it may be that seat belts are net positive."

Me_2013: "On the other hand, more driving of cars means more greenhouse-gas emissions, which means more climate change, which means more global political instability, which may worsen prospects for compromise in the far future. The amount at stake there could swamp everything else."

Me_2005: "Oh, yes, you're right. So then seat belts may be net bad on balance."

Me_2013: "On the other hand, fewer premature deaths may allow people to feel less threatened and more far-sighted in their actions...."

[...and so on]

Nick Bostrom offers more examples of this type in his 2014 talk, "Crucial considerations and wise philanthropy".

Should we only use rigorous data?

This idea of focusing on the effects you can see and ignoring those you can't, hoping they all kind of cancel out, is tempting. It's one way to feel like "I'm actually making a difference" by doing a given action, rather than being paralyzed by uncertainty, which can sometimes be depressing.

Some in the effective-altruism movement adopt this kind of approach. Basically, "any cause that's uncertain, far off, and speculative isn't 'rigorous,' so we should ignore it. Instead, we should focus on clearly proven interventions with hard scientific evidence behind them, like malaria prevention or veg outreach, because at least here we know we're doing something good rather than floating off into space."

I think there's some value to this perspective, especially insofar as digging deep into scientific details can teach you important lessons about the world that are transferable elsewhere. However, where it falls down as a philosophical stance is with this question: How do you know malaria prevention or veg outreach is good on balance? How can you know the sign of the activity (much less the effectiveness) without considering the longer-range and speculative implications? You could try to assume that everything you can't see "just cancels out," but as we saw in the introductory dialogue, this approach can yield arbitrarily flip-flopping conclusions depending on where the boundary of "known facts" meets the boundary of "everything else we assume away." Ultimately we have no choice but to look at the whole picture and grapple with it as best we can.

Alternate approach: Cause robustness

Another way to pick a cause to work on is to look for actions that have broadly positive effects across a wide range of scenarios. For example, in general:

• More cooperation, democracy, and effective institutions for positive-sum compromise are better.
• More philosophical reflectiveness and discourse are better.
• Meta-level activities to support the above (like the effective-altruism movement, advising people on career choice for making a difference, and so on) are probably good.

Note that I didn't include on this list technology, economic growth, and other trends generally seen as positive, because both negative-leaning and positive-leaning utilitarians have concerns about whether faster technology is net good or bad. Of course, particular kinds of technologies, like those that advance wisdom faster than raw power, are safer and often clearly beneficial.

Punting to the future

Focusing on the very robust projects often amounts to punting the hard questions to future generations who will be better equipped to solve them. In general, we have basically no other choice. There are known puzzles and unknown future discoveries in physics, neuroscience, anthropics, infinity, general epistemology, human (and nonhuman) values, and many other areas of life that are likely to radically transform our views on ethics and how we should act in the world. Taking object-level actions based only on what we know now is sort of like Socrates trying to determine whether the Higgs boson exists. Socrates would have been at a complete loss to solve this question directly, but one thing he could have done would have been to encourage more thinking on intellectual topics in general, with eventual payoff in the future.

Gaverick on Felicifia proposed this basic idea to "get smarter" before we try to address object-level problems. The caution I would add is that we don't just need knowledge per se but specifically ethically reflective knowledge that can slowly, carefully, and circumspectly decide how to move ahead. We don't want a quick knowledge explosion in which one party runs roughshod over most things that most organisms care about in a winner-takes-all fashion. Thus, even though artificial general intelligence (AGI) would ultimately be needed to figure out these questions that are beyond human grasp, we need to develop AGI wisely, and fostering better social conditions within which that AGI development can proceed more prudently can make sense in the short term.

One aspect of preparing the ground for future generations to do good things is making sure our values are transferred, at least in broad outline. The biggest danger of the "punt to the future" strategy is not that the future won't be intelligent (selection pressure makes it likely that it will be) but that it may not care about what we care about. While we don't want to lock the future in to a moral view that would seem silly with greater understanding of the universe, we also don't want the future's values to float around arbitrarily. The space of possible values is huge, and the space of values that we might come to care about even upon further reflection is a tiny subset of the total. This is why our slogan shouldn't be just "more intelligence" but instead something like "more wisdom."

(I should mention in passing that it would probably be better from a suffering-focused perspective if humans don't develop AGI because of the extreme risks of future suffering that doing so entails. However, given that humanity will develop AGI whether I like it or not, my best prospects for making a difference seem to come from pushing it in more humane and positive-sum directions. I also recognize that other people care very much about the fruits that AGI could offer, and I give this some moral weight as well.)

Thinking meta doesn't always mean acting meta

It seems that the main way in which our actions make a difference is by how they prepare the ground for our successors to think more carefully about big questions in altruism and take good actions based on that thinking. However, this doesn't mean that the only way to affect the far future is to work directly on meta-level activities, like movement-building, career advising, or friendly AGI. Rather, all causes that we might work on have flow-through effects in other areas. For example, studying how to reduce wild-animal suffering in the near term can encourage people to recognize the importance of the issue more broadly, which might also make them think more cautiously before spreading wildlife to other planets or in simulations. Studying object-level crucial considerations in philosophy can motivate more people to follow suit, perhaps more effectively than by just preaching about the general importance of philosophical reflection without setting the example. In general, taking small, concrete steps alongside a bigger vision can potentially be more inspiring than just talking about the bigger vision, although exactly how to strike the balance between these two is up for debate.

An effective altruist in 1800

As one final example to drive home the point, imagine an effective altruist in the year 1800 trying to optimize his positive impact. He would not know most of modern economics, political science, game theory, physics, cosmology, biology, cognitive science, psychology, business, philosophy, probability theory, computation theory, or manifold other subjects that would have been crucial for him to consider. If he tried to place his bets on the most significant object-level issue that would be relevant centuries later, he'd almost certainly get it wrong. I doubt we would fare substantially better today at trying to guess a specific, concrete area of focus more than a few decades out. (Of course, if we can guess such an area with decent precision, we could achieve high leverage, as Nick Beckstead points out in the presentation discussed at the end of this essay.)

What this 1800s effective altruist might have guessed correctly would have been the importance of world peace, philosophical reflection, positive-sum social institutions, and wisdom. Promoting those in 1800 may have been close to the best thing this person could have done, and this suggests that these may remain among the best options for us today. We should of course be on the lookout for more specific, high-leverage options, but we ought also to remain humble about the extent of our abilities against the vast space of unknown unknowns as far as errors in our current beliefs and new domains of knowledge that we never even knew existed.

Nick Beckstead discusses a similar analogy in section 6.4.3 of On the Overwhelming Importance of Shaping the Far Future, giving the example of a person in the year 1500 who wanted to help humanity build telephones centuries later.

Against charity fanaticism

Treating other groups as competitors

In business, another company that performs a similar service to customers as yours is a competitor, and your goal is to steal as much market share as you can from that competitor. The only cost of marketing is the money that you spend on it, and if it draws away enough customers from the competitor, it's worth it.

Charities, both EA and otherwise, can adopt a similar mindset: They want more donors for their cause, without paying too much attention about what charities they're pulling donors away from. EAs are concerned with replaceability issues and recognize that pulling donors away from other issues might matter, but usually they feel that the charity they're promoting is vastly more effective than the one they're pulling people away from, so there's not much lost. It's possible this is true in some cases -- e.g., encouraging donors to fund HIV prevention instead of AIDS treatments, or recruiting donors who would have funded art galleries -- but in other cases it becomes much less clear. Especially in the realm of policy analysis and political advocacy, which arguably have some of the highest returns, it's more difficult to say that one charity is vastly more important than another, because the issues are complex and multifaceted.

So for altruists, the cost of marketing and fundraising is more than the time and money required to carry them out. Charity is not a competition.

After writing this piece, I came across a presentation that Nick Beckstead gave in July 2013. Nick explains several similar views to those expressed in this essay. For instance, his concluding slide (p. 39):

• There is an interesting question about where you want to be on the targeted vs. broad spectrum, and I think it is pretty unclear
• Lots of ordinary stuff done by people who aren't thinking about the far future at all may be valuable by far future standards
• Broad approaches (including general technological progress) look more robustly good, but some targeted approaches may lead to outsized returns if done properly
• There are many complicated questions, and putting it all together requires challenging big picture thinking. Studying targeted approaches stands out somewhat because it has the potential for outsized returns.

Nick proposes these as some robustly positive goals (p. 5):

• Coordination: How well-coordinated people are
• Capability: How capable individuals are at achieving their goals
• Motives: How well-motivated people are

I agree with Coordination and Motives. I'm less certain about Information, because this speeds up development of risks along with development of safety measures. The same is even more true for Capability. I would therefore favor differentially pushing on wisdom and compromise relatively more than economic and technological growth. Nick makes some recognition of this on p. 30:

• Broad approaches are more likely to enhance bad stuff as well as good stuff
• Increasing people's general capabilities/information makes people more able to do things that would be dangerous, offsetting some of the benefits of increased capabilities/information
• Improving coordination or motives may do this to a lesser extent

Nick himself argues that faster economic growth is very likely positive because it improves cooperation and tolerance. He quotes Benjamin Friedman's The Moral Consequences of Economic Growth: "Economic growth -- meaning a rising standard of living for the clear majority of citizens -- more often than not fosters greater opportunity, tolerance of diversity, social mobility, commitment to fairness, and dedication to democracy." I agree with this, but the question is not whether economic growth has good effects of this type but whether these effects can outpace the risks that it also accelerates. I feel that this question remains unresolved.

On pp. 18-19, Nick deflects the argument that faster technology is net bad by pointing out that it also means faster countermeasures, along with some other considerations that I think are minor. This point is relevant, but I maintain that it's not clear what the net balance is. In my view it's too early to say that faster technology is net good, much less sufficiently good that we should push on it compared with other things.

On p. 24, Nick echoes my point about deferring to the future on some questions:

• In some ways, trying to help future people navigate specific challenges better is like trying to help people from a different country solve their specific challenges, and to do so without intimate knowledge of the situation, and without the ability to travel to their country or talk to anyone who has been there at all recently.
• Sometimes, only we can work on the problem (this is true for climate change and people who will be alive in 100 years)
• It is less clearly true with risks from future technology

Nick concludes with some important research questions about historical examples of what interventions were most important and what current opportunities and funding/talent gaps look like.

Carl Shulman on flow-through effects

Carl Shulman has a piece, "What proxies to use for flow-through effects?," that suggests many possible metrics that are relevant for impacts on the far future, though he explains that not all of them are always obviously positive in sign. From Carl's list, these are some that I believe are pretty robustly positive:

• Education to increase "wisdom"
• International peace and cooperation, including all of Carl's sub-bullets there
• Institutional quality metrics

The sign of most other metrics is less clear to me, including economic growth, population, education in general, and especially technology. Carl cites the World Values Survey as an important demonstration of the impact of per-capita wealth on rationality and cosmopolitan perspective.

Within the "wisdom" category, I would include scientometrics that Carl mentions for natural sciences applied to social sciences and philosophy. For example, number of publications, number of web pages discussing those topics, number and length of Wikipedia articles on those topics, etc. Of course, the value of some of these domains is in the pudding -- insofar as they improve democracy, transparency, global cooperation, and so on.

In the comments, Nick Beckstead suggested inequality as another candidate metric. I haven't studied the literature extensively, but I have heard arguments about how it erodes many other relevant metrics, including trust, cooperation, mental health, and interpersonal kindness. For example, according to Richard Wilkinson: "Where there is more equality we use more cooperative social strategies, but where there is more inequality, people feel they have to fend for themselves and competition for status becomes more important."

Modeling unknown unknowns

It might seem as though we're helpless to respond to not-yet-discovered crucial considerations for how we should act. Is our only option to keep researching to find more crucial considerations and to move society toward a more cooperative and wise state in the meanwhile? Not necessarily. Maybe another possibility is to model the unknown crucial considerations.

Consider the following narrative. Andrew is a young boy who sees people going to a blood-donation drive. He doesn't know what they're doing, but he sees them being stuck with needles. He concludes that he wouldn't like to participate in a blood drive. Let's call this his "initial evaluation" (IE) and represent it by a number to indicate whether it favors or opposes the action. In this case, Andrew assumes he would not like to participate in the blood drive, so let's say IE = -1, where the negative number means "oppose".

A few years later, Andrew learns that blood drives are intended to save lives, which is a good thing. This crucial consideration is not something he anticipated earlier, which makes it an "unknown unknown" discovery. Since it's Andrew's first unknown-unknown insight, let's call it UU1. Since this consideration favors giving blood, and it does so more strongly than Andrew's initial evaluation opposed giving blood, let's say UU1 = 3. Since IE + UU1 = -1 + 3 > 0, Andrew now gives blood at drives.

However, one year later, Andrew becomes a deep ecologist. He feels that humans are ruining the Earth, and that nature preservation deserves more weight than human lives. Giving blood allows a person in a rich country to live perhaps an additional few years, during which time the person will ride in cars, eat farmed food, use electricity, and so on. Andrew judges that these environmental impacts are sufficiently bad they're not worth the benefit of saving the person's life. Let's say UU2 = -5, so that now IE + UU1 + UU2 = -1 + 3 + -5 = -3 < 0, and Andrew now stops giving blood again.

After another few months, Andrew reads Peter Singer and realizes that individual animals also matter. Since human activities like driving and farming food injure and kill lots of wild animals, Andrew concludes that this additional insight further argues against blood donation. Say UU3 = -2.

However, not long after, Andrew learns about wild-animal suffering and realizes that animals suffer immensely even when they aren't being harmed by humans. Because human activity seems to have on the whole reduced wild-animal populations, Andrew concludes that it's better if more humans exist, and this outweighs the harm they cause to wild animals and to the planet. Say UU4 = 10. Now IE + Σi=04 UUi = -1 + 3 + -5 + -2 + 10 = 5. Andrew donates blood once more.

Finally, Andrew realizes that donating blood takes time that he could otherwise spend on useful activities. This consideration is relevant but not dominating. UU5 = -1.

What about future crucial considerations that Andrew hasn't yet discovered? Can he make any statements about them? One way to do so would be to model unknown unknowns (UUs) as being sampled from some probability distribution P: UUi ~ P for all i. The distribution of UUs so far was {3, -5, -2, 10, -1}. The sample mean is 1, and the standard error is 2.6. The standard error is big enough that Andrew can't have much confidence about future UUs, though the sample mean very weakly suggests future UUs are more likely on average to be positive than negative.

If Andrew instead had 100 UU data points, the standard error would be much smaller, which would give more confidence. This illustrates one lesson when handling UUs: The more considerations you've already considered, the more confident you can be that the distribution of remaining UUs also has positive mean.

That we can anticipate something about UUs despite not knowing what they will be can be seen more clearly in a case where the current UUs are more lopsided. For example, suppose the action under consideration is "start fights with random people on the street". While this probably has a few considerations in its favor, almost all of the crucial considerations that one could think of argue against the idea, suggesting that most new UUs will point against it as well.

Modeling UUs in practice may be messier than what I've discussed here, because it's not clear how many UUs remain (although one could apply a prior probability distribution over the number remaining), nor is it clear that they all come from a fixed probability distribution. Perhaps future UUs tend to dominate past ones in size, leading to ever more unstable estimates; for example, if previous UUs tend to change the sign of lots of previous considerations at once, then the latest UU would have a bigger and bigger magnitude as time went on, since it would need to "undo" more and more past UUs. It's also not clear how to partition insights into UU buckets. For example, the insight that donating blood helps wild animals could be stated simply as a single UU with magnitude 10, or it could be broken down as "donating blood helps wild vertebrates" (magnitude 2) and "donating blood helps wild invertebrates" (magnitude 8). Different ways of partitioning UUs would lead to a different estimated probability distribution, although the sign of the sample mean would always remain the same.

There are problems with the approach I described. Magnus Vinding noted to me that

When it comes to UUs, a major problem is that pretty much our entire value system and worldview seem to be up for grabs, and, moreover, that the different UUs likely will be dependent in deep, complex ways that will make modelling of them very hard. Modelling the interrelations of the UUs in our sample and how they make each other change would also seem a necessary element to include in such an analysis.

Acknowledgments

My thoughts on these topics were influenced by many effective-altruist thinkers, including Holden Karnofsky, Jonah Sinick, and Nick Beckstead. See also Paul Christiano's "Beware brittle arguments."

Feedback

The discussion of this essay on Facebook's "Effective Altruists" forum includes a debate about whether flow-through effects are actually significant relative to first-order effects and how inevitable the future is likely to be.