FUTURE SUFFERING & MACROSTRATEGY

Risks of Astronomical Future Suffering

Space colonization would likely increase rather than decrease total suffering. Because many people care nonetheless about humanity’s spread into the cosmos, we should reduce risks of astronomical future suffering without opposing others’ spacefaring dreams. In general, we recommend to focus on making sure that an intergalactic future will be good if it happens rather than making sure there will be such a future.

Download Read online

How the Simulation Argument Dampens Future Fanaticism

The simulation argument suggests a non-trivial chance that most of the copies of ourselves are instantiated in relatively short-lived ancestor simulations run by superintelligent civilizations. If so, when we act to help others in the short run, our good deeds are duplicated many times over. This reasoning dramatically upshifts the relative importance of short-term helping over focusing on the far future.

Download Read online

Backup Utility Functions: A Fail-Safe AI Technique

Setting up the goal systems of advanced AIs in a way that results in benevolent behavior is expected to be difficult. We should account for the possibility that the goal systems of AIs fail to implement our values as originally intended. In this paper, we propose the idea of backup utility functions: Secondary utility functions that are used in case the primary ones “fail”.

Download Read online

Altruists Should Prioritize Artificial Intelligence

We can expect smarter-than-human artificial intelligence (AI) to be better than humans at self-preservation and goal preservation. If we want our actions to have an influence on the very long-term future, we should consider focusing on outcomes with AI.

Read more

Reducing Risks of Astronomical Suffering:
A Neglected Priority

Will we go extinct, or will we succeed in building a flourishing utopia? Discussions about the future trajectory of humanity often center around these two possibilities, which tends to ignore that survival does not always imply utopian outcomes, or that outcomes where humans go extinct could differ tremendously in how much suffering they contain.

Read more

Artificial Intelligence and Its Implications for Future Suffering

Artificial intelligence (AI) will likely transform the world later this century. Whether uncontrolled or controlled AIs would create more suffering in expectation is a question to explore further. Regardless, the field of AI safety and policy seems to be a very important space where altruists can make a positive-sum impact along many dimensions.

Read more

How Feasible Is the Rapid Development of Artificial Superintelligence?

Two crucial questions in discussions about the risks of artificial superintelligence are: 1) How much more powerful could an AI become relative to humans, and 2) how easily could superhuman capability be acquired? To answer these questions, this article reviews the literature on human expertise and intelligence and discusses its relevance for AI.

Read more

Charity Cost-Effectiveness in an Uncertain World

Evaluating the effectiveness of our actions, or even just whether they're beneficial or harmful, is very difficult. One way to deal with uncertainty is to focus on actions that likely have positive effects across many scenarios. This approach often amounts to meta-level activities like promoting positive-sum institutions, reflectiveness, and effective altruism in general.

Read more

Against Wishful Thinking

Some people hold more hopeful beliefs about the world than justified. These include the feeling that life for wild animals isn't so bad and the expectation that humanity's future will reduce more suffering than it creates. By feeding these dreams, optimistic visions of suffering reduction may in fact cause net harm. We should explore ways of increasing empathy that also expose the true extent of suffering in the world.

Read more
All content on Future Suffering & Macrostrategy

COOPERATION AND FORESIGHT

Gains from Trade through Compromise

When agents of differing values compete, they may often find it mutually advantageous to compromise rather than continuing to engage in zero-sum conflicts. Potential ways of encouraging cooperation include promoting democracy, tolerance and (moral) trade. Because a future without compromise could be many times worse than a future with it, advancing compromise seems an important undertaking.

Download Read online

International Cooperation vs. AI Arms Race

There's a decent chance that governments will be the first to build artificial general intelligence (AI). International hostility, especially an AI arms race, could exacerbate risk-taking, hostile motivations, and errors of judgment when creating AI. If so, then international cooperation could be an important factor to consider when evaluating the flow-through effects of charities.

Download Read online

Possible Ways to Promote Compromise

Compromise has the potential to jointly benefit many different individuals, organizations, countries, and value systems. This article enumerates ideas for how to encourage compromise, drawn from political science, international relations, moral psychology, game theory, sociology, and ethics.

Download Read online

Reasons to Be Nice to Other Value Systems

Several arguments support the heuristic that we should help groups holding different value systems from our own when doing so is cheap, unless those groups prove uncooperative to our values. This is true even if we don't directly care at all about other groups' value systems. Exactly how nice to be depends on the particulars of the situation.

Read more

How Would Catastrophic Risks Affect Prospects for Compromise?

Global catastrophic risks – such as biotech disasters or nuclear war – would cause major damage in the short run, but their effects on the long-run trajectory that humanity takes are also significant. In particular, to the extent these disasters increase risks of war, they seem likely to precipitate AI arms races between nations and worsen prospects for compromise.

Read more

Education Matters for Altruism

Learning is an extremely important activity for altruists. Learning can seem ineffective in the short run, but used properly, it can pay off more than most financial or single-domain-focused investments. It's important for young activists not to neglect learning in order to just "do more to help now."

Read more

ETHICS

Formalizing Preference Utilitarianism in Physical World Models

Most ethical work is done at a low level of formality which can lead to misunderstandings in ethical discussions. In this paper, we use Bayesian inference to introduce a formalization of preference utilitarianism in physical world models. Even though our formalization is not immediately applicable, it is a first step in providing ethical inquiry with a formal basis.

Download Read online

The Importance of Wild-Animal Suffering

The number of wild animals vastly exceeds that of animals on factory farms. Therefore, animal advocates should consider focusing their efforts to raise concern about the suffering that occurs in nature. In theory, engineering more humane ecological systems might be valuable. In practice, however, it seems more effective to promote the meme of caring about wild animals to other activists, academics and other sympathetic groups.

Download Read online

Measuring Happiness and Suffering

Is the balance of happiness versus suffering in the future net positive or net negative (in expectation)? Is the aggregate happiness and suffering in a group of individuals positive or negative? For such questions to have factual answers that are free from value judgements, happiness and suffering would need to be objectively measurable to a very high degree. However, such a degree of measurability is widely (although not universally) rejected.

Download Read online

What Is the Difference Between Weak Negative and Non-Negative Ethical Views?

Weak negative views in ethics, such as negative-leaning utilitarianism, are said to give more weight to reducing suffering than to promoting happiness. In contrast, non-negative views such as traditional utilitarianism are said to give equal weight to happiness and suffering. However, this way of distinguishing between the views rests on controversial assumptions about the measurability of happiness and suffering.

Download Read online

The Case for Suffering-Focused Ethics

“Suffering-focused ethics” is an umbrella term for moral views that place primary or particular importance on the prevention of suffering. Most views that fall into this category are pluralistic in that they hold that other things beside suffering reduction also matter morally. To illustrate the diversity within suffering-focused ethics as well as to present a convincing case for it, this article will introduce four separate motivating intuitions.

Read more

Hedonistic vs. Preference Utilitarianism

It's a classic debate among utilitarians: Should we care about an organism's happiness and suffering (hedonic wellbeing), or should we ultimately value fulfilling what it wants, whatever that may be (preferences)? This article discusses various intuitions on both sides and explores a hybrid view that gives greater weight to the hedonic subsystems of brains than to other overriding subsystems.

Read more

Value Lexicality

An example of value lexicality is that an outcome with both torture and happiness is bad, regardless of the amount of happiness. Value lexicality is important partly because it can lead to suffering-focused ethics. Key topics that this essay explains include strong versus weak lexicality, value aggregation, views on large numbers and sequence arguments.

Read more

Dealing with Moral Multiplicity

The ethical views we hold depend significantly on the network structures of our brains: which ideas are associated with which valences and how strongly. These feelings and weights are shaped by our genetic predispositions, cultural circumstances, and life experiences. Had you developed differently, your moral views would have been different. It's up to us whether…

Read more

The 'Asymmetry' and Extinction Thought Experiments

Someone who wants to do good is faced with the question how to prioritize preventing badness vs. bringing about more individuals with good lives. A relevant idea is the ‘Asymmetry,’ which roughly says that it is bad to bring into existence individuals with bad lives but not good to add individuals with good lives. One…

Read more

Should We Base Moral Judgments on Intentions or Outcomes?

Different ethical intuitions place different weight on the importance of intentions vs. outcomes in evaluating our actions. One might think that consequentialists would favor the outcome-based approach, and indeed, judging based on outcomes is sometimes the best way to optimize performance. However, in other circumstances – e.g., when you have strong prior knowledge or when…

Read more
All content on Ethics

CONSCIOUSNESS

Do Artificial Reinforcement-Learning Agents Matter Morally?

Artificial reinforcement learning (RL), a widely used training method in computer science, has striking parallels to reward and punishment learning in biological brains. Plausible theories of consciousness imply a non-zero probability that RL agents qualify as sentient and deserve our moral consideration, especially as AI research advances and RL agents become more sophisticated.

Download Read online

The Eliminativist Approach to Consciousness

This essay explains my version of an eliminativist approach to understanding consciousness. It suggests that we stop thinking in terms of "conscious" and "unconscious" and instead look at physical systems for what they are and what they can do. This perspective dissolves some biases in our usual perspective and shows us that the world is…

Download Read online

A Dialogue on Suffering Subroutines

This piece presents a hypothetical dialogue that explains why instrumental computational processes of a future superintelligence might evoke moral concern. Generally, agent-like components might emerge in many places, including the computing processes of a future civilization. Whether and how much these subroutines matter are questions for future generations to figure out, but it's good to keep an open mind to the possibility that our intuitions about what suffering is may change dramatically.

Read more

Flavors of Computation Are Flavors of Consciousness

If we don't understand why we're conscious, how come we're so sure that extremely simple minds are not? I propose to think of consciousness as intrinsic to computation, although different types of computation may have very different types of consciousness – some so alien that we can't imagine them. Since all physical processes are computations,…

Read more
Other research

GET INVOLVED