Multiverse-wide Cooperation via Correlated Decision Making

Some decision theorists argue that when playing a prisoner's dilemma-type game against a sufficiently similar opponent, we should cooperate to make it more likely that our opponent also cooperates. This idea, which Hofstadter calls superrationality, has strong implications when combined with the insight from modern physics that we live in a large universe or multiverse of some sort.

Read more

Tranquilism

What makes an experience valuable or disvaluable? In contrast to hedonism, which holds that pleasure is what is good and pain is what is bad, tranquilism is an “absence of desire” theory that counts pleasure as instrumentally valuable only. According to tranquilism, what matters is whether an experience is free from bothersome components. States of contentment such as flow or meditative tranquility also qualify.

Read more

Value Lexicality

An example of value lexicality is that an outcome with both torture and happiness is bad, regardless of the amount of happiness. Value lexicality is important partly because it can lead to suffering-focused ethics. Key topics that this essay explains include strong versus weak lexicality, value aggregation, views on large numbers and sequence arguments.

Read more

Backup Utility Functions: A Fail-Safe AI Technique

Setting up the goal systems of advanced AIs in a way that results in benevolent behavior is expected to be difficult. We should account for the possibility that the goal systems of AIs fail to implement our values as originally intended. In this paper, we propose the idea of backup utility functions: Secondary utility functions that are used in case the primary ones “fail”.

Read more

How Feasible Is the Rapid Development of Artificial Superintelligence?

Two crucial questions in discussions about the risks of artificial superintelligence are: 1) How much more powerful could an AI become relative to humans, and 2) how easily could superhuman capability be acquired? To answer these questions, this article reviews the literature on human expertise and intelligence and discusses its relevance for AI.

Read more

How the Simulation Argument Dampens Future Fanaticism

The simulation argument suggests a non-trivial chance that most of the copies of ourselves are instantiated in relatively short-lived ancestor simulations run by superintelligent civilizations. If so, when we act to help others in the short run, our good deeds are duplicated many times over. This reasoning dramatically upshifts the relative importance of short-term helping over focusing on the far future.

Read more

Identifying Plausible Paths to Impact and their Strategic Implications

FRI’s research seeks to identify the best intervention(s) for suffering reducers to work on. Rather than continuing our research indefinitely, we will eventually have to focus our efforts on an intervention directly targeted at improving the world. This report outlines plausible candidates for FRI’s “path to impact” and distills some advice on how current movement building efforts can best prepare for them.

Read more

What Is the Difference Between Weak Negative and Non-Negative Ethical Views?

Weak negative views in ethics, such as negative-leaning utilitarianism, are said to give more weight to reducing suffering than to promoting happiness. In contrast, non-negative views such as traditional utilitarianism are said to give equal weight to happiness and suffering. However, this way of distinguishing between the views rests on controversial assumptions about the measurability of happiness and suffering.

Read more

Bibliography of Suffering-Focused Views

A good bibliography encourages others to conduct research and write papers in the field. Thus, creating an up-to-date bibliography on suffering-focused views seems an important undertaking. This subpage of our open research questions page contains examples of sources to include in such a bibliography.

Read more

Formalizing Preference Utilitarianism in Physical World Models

Most ethical work is done at a low level of formality which can lead to misunderstandings in ethical discussions. In this paper, we use Bayesian inference to introduce a formalization of preference utilitarianism in physical world models. Even though our formalization is not immediately applicable, it is a first step in providing ethical inquiry with a formal basis.

Read more

Hedonistic vs. Preference Utilitarianism

It's a classic debate among utilitarians: Should we care about an organism's happiness and suffering (hedonic wellbeing), or should we ultimately value fulfilling what it wants, whatever that may be (preferences)? This article discusses various intuitions on both sides and explores a hybrid view that gives greater weight to the hedonic subsystems of brains than to other overriding subsystems.

Read more

Do Artificial Reinforcement-Learning Agents Matter Morally?

Artificial reinforcement learning (RL), a widely used training method in computer science, has striking parallels to reward and punishment learning in biological brains. Plausible theories of consciousness imply a non-zero probability that RL agents qualify as sentient and deserve our moral consideration, especially as AI research advances and RL agents become more sophisticated.

Read more

Measuring Happiness and Suffering

Is the balance of happiness versus suffering in the future net positive or net negative (in expectation)? Is the aggregate happiness and suffering in a group of individuals positive or negative? For such questions to have factual answers that are free from value judgements, happiness and suffering would need to be objectively measurable to a very high degree. However, such a degree of measurability is widely (although not universally) rejected.

Read more

How Could an Empty World Be Better than a Populated One?

Leslie (1998) writes that when “discussing whether the universe was created by a benevolent deity, philosophers regularly point out that our world might be considered an ethical disaster, something of negative value, because of all the misery it contains." This essay surveys different ways in which ethical views may come to conclude that a world’s value is overall negative.

Read more

Should We Base Moral Judgments on Intentions or Outcomes?

Different ethical intuitions place different weight on the importance of intentions vs. outcomes in evaluating our actions. One might think that consequentialists would favor the outcome-based approach, and indeed, judging based on outcomes is sometimes the best way to optimize performance. However, in other circumstances – e.g., when you have strong prior knowledge or when […]

Read more

How Would Catastrophic Risks Affect Prospects for Compromise?

Global catastrophic risks – such as biotech disasters or nuclear war – would cause major damage in the short run, but their effects on the long-run trajectory that humanity takes are also significant. In particular, to the extent these disasters increase risks of war, they seem likely to precipitate AI arms races between nations and worsen prospects for compromise.

Read more

Reasons to Be Nice to Other Value Systems

Several arguments support the heuristic that we should help groups holding different value systems from our own when doing so is cheap, unless those groups prove uncooperative to our values. This is true even if we don't directly care at all about other groups' value systems. Exactly how nice to be depends on the particulars of the situation.

Read more

The 'Asymmetry' and Extinction Thought Experiments

Someone who wants to do good is faced with the question how to prioritize preventing badness vs. bringing about more individuals with good lives. A relevant idea is the ‘Asymmetry,’ which roughly says that it is bad to bring into existence individuals with bad lives but not good to add individuals with good lives. One […]

Read more

Education Matters for Altruism

Learning is an extremely important activity for altruists. Learning can seem ineffective in the short run, but used properly, it can pay off more than most financial or single-domain-focused investments. It's important for young activists not to neglect learning in order to just "do more to help now."

Read more

Charity Cost-Effectiveness in an Uncertain World

Evaluating the effectiveness of our actions, or even just whether they're beneficial or harmful, is very difficult. One way to deal with uncertainty is to focus on actions that likely have positive effects across many scenarios. This approach often amounts to meta-level activities like promoting positive-sum institutions, reflectiveness, and effective altruism in general.

Read more

Differential Intellectual Progress as a Positive-Sum Project

Fast technological development carries a risk of creating extremely powerful tools, especially AI, before society has a chance to figure out how best to use those tools in positive ways for many value systems. Suffering reducers may want to help mitigate the arms race for AI so that AI developers take fewer risks and have […]

Read more

Against Wishful Thinking

Some people hold more hopeful beliefs about the world than justified. These include the feeling that life for wild animals isn't so bad and the expectation that humanity's future will reduce more suffering than it creates. By feeding these dreams, optimistic visions of suffering reduction may in fact cause net harm. We should explore ways of increasing empathy that also expose the true extent of suffering in the world.

Read more

A Lower Bound on the Importance of Promoting Cooperation

This article suggests a lower-bound Fermi calculation for the cost-effectiveness of promoting cooperation. The purpose of this exercise is to make our thinking more concrete about how cooperation might reduce suffering and to make its potential more tangible.

Read more

A Dialogue on Suffering Subroutines

This piece presents a hypothetical dialogue that explains why instrumental computational processes of a future superintelligence might evoke moral concern. Generally, agent-like components might emerge in many places, including the computing processes of a future civilization. Whether and how much these subroutines matter are questions for future generations to figure out, but it's good to keep an open mind to the possibility that our intuitions about what suffering is may change dramatically.

Read more

Dealing with Moral Multiplicity

The ethical views we hold depend significantly on the network structures of our brains: which ideas are associated with which valences and how strongly. These feelings and weights are shaped by our genetic predispositions, cultural circumstances, and life experiences. Had you developed differently, your moral views would have been different. It's up to us whether […]

Read more

The Eliminativist Approach to Consciousness

This essay explains my version of an eliminativist approach to understanding consciousness. It suggests that we stop thinking in terms of "conscious" and "unconscious" and instead look at physical systems for what they are and what they can do. This perspective dissolves some biases in our usual perspective and shows us that the world is […]

Read more

Flavors of Computation Are Flavors of Consciousness

If we don't understand why we're conscious, how come we're so sure that extremely simple minds are not? I propose to think of consciousness as intrinsic to computation, although different types of computation may have very different types of consciousness – some so alien that we can't imagine them. Since all physical processes are computations, […]

Read more

Artificial Intelligence and Its Implications for Future Suffering

Artificial intelligence (AI) will likely transform the world later this century. Whether uncontrolled or controlled AIs would create more suffering in expectation is a question to explore further. Regardless, the field of AI safety and policy seems to be a very important space where altruists can make a positive-sum impact along many dimensions.

Read more

Possible Ways to Promote Compromise

Compromise has the potential to jointly benefit many different individuals, organizations, countries, and value systems. This article enumerates ideas for how to encourage compromise, drawn from political science, international relations, moral psychology, game theory, sociology, and ethics.

Read more

Gains from Trade through Compromise

When agents of differing values compete, they may often find it mutually advantageous to compromise rather than continuing to engage in zero-sum conflicts. Potential ways of encouraging cooperation include promoting democracy, tolerance and (moral) trade. Because a future without compromise could be many times worse than a future with it, advancing compromise seems an important undertaking.

Read more

The Importance of Wild-Animal Suffering

The number of wild animals vastly exceeds that of animals on factory farms. Therefore, animal advocates should consider focusing their efforts to raise concern about the suffering that occurs in nature. In theory, engineering more humane ecological systems might be valuable. In practice, however, it seems more effective to promote the meme of caring about wild animals to other activists, academics and other sympathetic groups.

Read more

Risks of Astronomical Future Suffering

Space colonization would likely increase rather than decrease total suffering. Because many people care nonetheless about humanity’s spread into the cosmos, we should reduce risks of astronomical future suffering without opposing others’ spacefaring dreams. In general, we recommend to focus on making sure that an intergalactic future will be good if it happens rather than making sure there will be such a future.

Read more

International Cooperation vs. AI Arms Race

There's a decent chance that governments will be the first to build artificial general intelligence (AI). International hostility, especially an AI arms race, could exacerbate risk-taking, hostile motivations, and errors of judgment when creating AI. If so, then international cooperation could be an important factor to consider when evaluating the flow-through effects of charities.

Read more

GET INVOLVED