Multiverse-wide Cooperation via Correlated Decision Making

Some decision theorists argue that when playing a prisoner's dilemma-type game against a sufficiently similar opponent, we should cooperate to make it more likely that our opponent also cooperates. This idea, which Hofstadter calls superrationality, has strong implications when combined with the insight from modern physics that we live in a large universe or multiverse of some sort.

Read more

Backup Utility Functions: A Fail-Safe AI Technique

Setting up the goal systems of advanced AIs in a way that results in benevolent behavior is expected to be difficult. We should account for the possibility that the goal systems of AIs fail to implement our values as originally intended. In this paper, we propose the idea of backup utility functions: Secondary utility functions that are used in case the primary ones “fail”.

Read more

How the Simulation Argument Dampens Future Fanaticism

The simulation argument suggests a non-trivial chance that most of the copies of ourselves are instantiated in relatively short-lived ancestor simulations run by superintelligent civilizations. If so, when we act to help others in the short run, our good deeds are duplicated many times over. This reasoning dramatically upshifts the relative importance of short-term helping over focusing on the far future.

Read more

What Is the Difference Between Weak Negative and Non-Negative Ethical Views?

Weak negative views in ethics, such as negative-leaning utilitarianism, are said to give more weight to reducing suffering than to promoting happiness. In contrast, non-negative views such as traditional utilitarianism are said to give equal weight to happiness and suffering. However, this way of distinguishing between the views rests on controversial assumptions about the measurability of happiness and suffering.

Read more

Formalizing Preference Utilitarianism in Physical World Models

Most ethical work is done at a low level of formality which can lead to misunderstandings in ethical discussions. In this paper, we use Bayesian inference to introduce a formalization of preference utilitarianism in physical world models. Even though our formalization is not immediately applicable, it is a first step in providing ethical inquiry with a formal basis.

Read more

Do Artificial Reinforcement-Learning Agents Matter Morally?

Artificial reinforcement learning (RL), a widely used training method in computer science, has striking parallels to reward and punishment learning in biological brains. Plausible theories of consciousness imply a non-zero probability that RL agents qualify as sentient and deserve our moral consideration, especially as AI research advances and RL agents become more sophisticated.

Read more

Measuring Happiness and Suffering

Is the balance of happiness versus suffering in the future net positive or net negative (in expectation)? Is the aggregate happiness and suffering in a group of individuals positive or negative? For such questions to have factual answers that are free from value judgements, happiness and suffering would need to be objectively measurable to a very high degree. However, such a degree of measurability is widely (although not universally) rejected.

Read more

Differential Intellectual Progress as a Positive-Sum Project

Fast technological development carries a risk of creating extremely powerful tools, especially AI, before society has a chance to figure out how best to use those tools in positive ways for many value systems. Suffering reducers may want to help mitigate the arms race for AI so that AI developers take fewer risks and have […]

Read more

The Eliminativist Approach to Consciousness

This essay explains my version of an eliminativist approach to understanding consciousness. It suggests that we stop thinking in terms of "conscious" and "unconscious" and instead look at physical systems for what they are and what they can do. This perspective dissolves some biases in our usual perspective and shows us that the world is […]

Read more

Possible Ways to Promote Compromise

Compromise has the potential to jointly benefit many different individuals, organizations, countries, and value systems. This article enumerates ideas for how to encourage compromise, drawn from political science, international relations, moral psychology, game theory, sociology, and ethics.

Read more

Gains from Trade through Compromise

When agents of differing values compete, they may often find it mutually advantageous to compromise rather than continuing to engage in zero-sum conflicts. Potential ways of encouraging cooperation include promoting democracy, tolerance and (moral) trade. Because a future without compromise could be many times worse than a future with it, advancing compromise seems an important undertaking.

Read more

The Importance of Wild-Animal Suffering

The number of wild animals vastly exceeds that of animals on factory farms. Therefore, animal advocates should consider focusing their efforts to raise concern about the suffering that occurs in nature. In theory, engineering more humane ecological systems might be valuable. In practice, however, it seems more effective to promote the meme of caring about wild animals to other activists, academics and other sympathetic groups.

Read more

Risks of Astronomical Future Suffering

Space colonization would likely increase rather than decrease total suffering. Because many people care nonetheless about humanity’s spread into the cosmos, we should reduce risks of astronomical future suffering without opposing others’ spacefaring dreams. In general, we recommend to focus on making sure that an intergalactic future will be good if it happens rather than making sure there will be such a future.

Read more

International Cooperation vs. AI Arms Race

There's a decent chance that governments will be the first to build artificial general intelligence (AI). International hostility, especially an AI arms race, could exacerbate risk-taking, hostile motivations, and errors of judgment when creating AI. If so, then international cooperation could be an important factor to consider when evaluating the flow-through effects of charities.

Read more

GET INVOLVED