Crazy idea: A vegan hot dog eating contest

Crazy idea: When charities apply for funding from foundations, they should be required to list 3-5 other charities they think should receive funding. Then, the grantmaker can run a statistical analysis to find orgs that are mentioned a lot and haven't applied before, reach out to those charities, and encourage them to apply. This way, the foundation can get a more diverse pool of applicants by learning about charities outside their network.

"Quality-adjusted civilization years"

We should be able to compare global catastrophic risks in terms of the amount of time they make global civilization significantly worse and how much worse it gets. We might call this measure "quality-adjusted civilization years" (QACYs), or the quality-adjusted amount of civilization time that is lost.

For example, let's say that the COVID-19 pandemic reduces the quality of civilization by 50% for 2 years. Then the QACY burden of COVID-19 is $0.5\times 2=1$ QACYs.

Another example: suppose climate change will reduce the quality of civilization by 80% for 200 years, and then things will return to normal. Then the total QACY burden of climate change over the long term will be $0.8\times 200=160$ QACYs.

In the limit, an existential catastrophe would have a near-infinite QACY burden.

I think we separate causes and interventions into "neartermist" and "longtermist" causes too much.

Just as some members of the EA community have complained that AI safety is pigeonholed as a "long-term" risk when it's actually imminent within our lifetimes^[1], I think we've been too quick to dismiss conventionally "neartermist" EA causes and interventions as not valuable from a longtermist perspective. This is the opposite failure mode of surprising and suspicious convergence - instead of assuming (or rationalizing) that the spaces of interventions that are promising from neartermist and longtermist perspectives overlap a lot, we tend to assume they don't overlap at all, because it's more surprising if the top longtermist causes are all different from the top neartermist ones. If the cost-effectiveness of different causes according to neartermism and longtermism are independent from one another (or at least somewhat positively correlated), I'd expect at least some causes to be valuable according to both ethical frameworks.

I've noticed this in my own thinking, and I suspect that this is a common pattern among EA decision makers; for example, Open Phil's "Longtermism" and "Global Health and Wellbeing" grantmaking portfolios don't seem to overlap.

Consider global health and poverty. These are usually considered "neartermist" causes, but we can also tell a just-so story about how global development interventions such as cash transfers might also be valuable from the perspective of longtermism:

• People in extreme poverty who receive cash transfers often spend the money on investments as well as consumption. For example, a study by GiveDirectly found that people who received cash transfers owned 40% more durable goods (assets) than the control group. Also, anecdotes show that cash transfer recipients often spend their funds on education for their kids (a type of human capital investment), starting new businesses, building infrastructure for their communities, and healthcare (another type of human capital).
• According to most models of economic growth, labor and capital are the main determinants of an economy's output. Increasing a community's capital stock tends to increase its economic output, some of which is reinvested in more capital, which leads to more output, and so on. The virtuous cycle of capital accumulation and increasing economic output is a key driver of long-term economic growth.
• Economic growth is valuable from a longtermist perspective because it raises the living standards and well-being of present and future generations. Furthermore, economic growth in poor countries is more valuable in the near term because the marginal utility of consumption in poor countries is higher.^[2]
• Therefore, cash transfers to the global poor are highly valuable by longtermist lights because they promote economic growth in poor countries.

Note that I'm not claiming that cash transfers are the most valuable interventions for longtermists. They're probably not, since any trend of sustained economic growth would likely run up against physical limits of the universe within the next few thousand years,^[3] and AGI is likely to render all other interventions to promote economic growth moot in the next few decades anyway. Interventions to reduce existential risk would probably have more impact over the long term (although I'm sympathetic to the argument in "Existential risk pessimism and the time of perils").

1. ^{^}

   At least those of us who are 40 and under.

2. ^{^}

   Notwithstanding growth rates. Rich countries like the United States probably sustain higher rates of economic growth and capital accumulation than poor countries because of stronger institutions that encourage investment. I'd like to see an economic model that could tell us which type of growth is more valuable in the long term, but I don't have the training in economics that one would need to create one.

3. ^{^}

   See "This Can't Go On" by Holden Karnofsky for an argument against indefinite sustained growth, and "This Can Go On" by Dwarkesh Patel for a counterargument.

Utility of money is not always logarithmic

EA discussions often assume that the utility of money is logarithmic, but while this is a convenient simplification, it's not always the case. Logarithmic utility is a special case of isoelastic utility, a.k.a. power utility, where the elasticity of marginal utility is $\eta =1$. But $\eta$ can be higher or lower. The most general form of isoelastic utility is the following:

$u\left(c\right)=\{\begin{array}{cc}\frac{c^{1-\eta }-1}{1-\eta }& \eta \ge 0,\eta \ne 1\\ \ln \left(c\right)& \eta =1\end{array}$

Some special cases:

• When $\eta =0$, we get linear utility, or $u\left(c\right)=c$.
• When $\eta =0.5$, we get the square root utility function, $u\left(c\right)=2(\sqrt{c}-1)$.
• When $\eta =1$, we get the familiar logarithmic utility function, $u\left(c\right)=\ln \left(c\right)$.
• For any $\eta >1$, the utility function asymptotically approaches a constant as $c$ approaches infinity. When $\eta =2$, we get the utility function $u\left(c\right)=1-1/c$.

$\eta$ tells us how sharply marginal utility drops off with increasing consumption: if a person already has $k$ times as much money as the baseline, then giving them an extra dollar is worth $(1/k{)}^{\eta }$ times as much. Empirical studies have found that $\eta$ for most people is between 1 and 2. So if the average GiveDirectly recipient is 100x poorer than the average American, then it could be 100 to 10,000 times as valuable to give money to GiveDirectly than to spend the same money on yourself.

Source: "The value of money going to different groups" by Toby Ord

I think an EA career fair would be a good idea. It could have EA orgs as well as non-EA orgs that are relevant to EAs (for gaining career capital or earning to give)

Nonprofit idea: YIMBY for energy

YIMBY groups in the United States (like YIMBY Action) systematically advocate for housing developments as well as rezonings and other policies to create more housing in cities. YIMBYism is an explicit counter-strategy to the NIMBY groups that oppose housing development; however, NIMBYism affects energy developments as well - everything from solar farms to nuclear power plants to power lines - and is thus an obstacle to the clean energy transition.

There should be groups that systematically advocate for energy projects (which are mostly in rural areas), borrowing the tactics of the YIMBY movement. Currently, when developers propose an energy project, they do an advertising campaign to persuade local residents of the benefits of the development, but there is often opposition as well.

One thing the EA community should try doing is multinational op-ed writing contests. The focus would be op-eds advocating for actions or policies that are important, neglected, and tractable (although the op-eds themselves don't have to mention EA); and by design, op-eds could be submitted from anywhere in the world. To make judging easier, op-eds could be required to be in a single language, but op-ed contests in multiple languages could be run in parallel (such as English, Spanish, French, and Arabic, each of which is an official language in at least 20 countries).

This would have two benefits for the EA community:

• It would be a cheap way to spread EA-aligned ideas in multiple countries. Also, the people writing the op-eds would know more about the political climates of the countries for which they are publishing them than the organizers of the contest would, and we can encourage them to tailor their messaging accordingly.
• It would also be a way to measure countries' receptiveness to EA ideas. For example, if there were multiple submissions about immigration policy, we could use them to compare the receptiveness of different countries to immigration reforms that would increase global well-being.

Possible outline for a 2-3 part documentary adaptation of The Precipice:

Part 1: Introduction & Natural Risks

• Introduce the idea that we are in a time of unprecedented existential risk, but that the future could be very good (Introduction and Chapter 1)
• Discuss natural risks (Chapter 3)
• Argue that man-made risks are greater and use this to lead to the next episode (Chapter 3)

Part 2: Human-Made Risks

• Well-known anthropogenic risks - nuclear war, climate change, other environmental damage (Chapter 4)
• Emerging technological risks - pandemics, AI, dystopia (Chapter 5)
• Existential risk and security factors (Chapter 6)

Part 3: What We Can Do

• Discuss actions society can take to minimize its existential risk (Chapter 7)

What this leaves out:

• Chapter 2 - mostly a discussion of the moral arguments for x-risk's importance. Can assume that the audience will already care about x-risk at a less sophisticated level, and focus on making the case that x-risk is high and we sort of know what to do about it.
• The discussion of joint probabilities of x-risks in Chapter 6 - too technical for a general audience

Another way to do it would be to do an episode on each type of risk and what can be done about it, for example:

• Part 1: Introduction
• Part 2: Pandemic risk (timely because of COVID-19)
• Part 3: Risks from asteroids, comets, volcanoes, and supernovas
• Part 4: Climate change
• Part 5: Artificial intelligence
• and so on

Like the original book, I'd want the tone of the documentary to be authoritative and hopeful and not lean on fear.

An idea I liked from Owen Cotton-Barratt's new interview on the 80K podcast: Defense in depth

If S, M, or L is any small, medium, or large catastrophe and X is human extinction, then the probability of human extinction is

$$Pr\left(X\right)=Pr\left(S\right)Pr(M\mid S)Pr(L\mid S,M)Pr(X\mid S,M,L).$$

So halving the probability of all small disasters, the probability of any small disaster becoming a medium-sized disaster, etc. would halve the probability of human extinction.

On the difference between x-risks and x-risk factors

I suspect there isn't much of a meaningful difference between "x-risks" and "x-risk factors," for two reasons:

1. We can treat them the same in terms of probability theory. For example, if $X$ is an "x-risk" and $Y$ is a "risk factor" for $X$, then $Pr(X\mid Y)>Pr\left(X\right)$. But we can also say that $Pr(Y\mid X)>Pr\left(Y\right)$, because both statements are equivalent to $Pr(X,Y)>Pr\left(X\right)Pr\left(Y\right)$. We can similarly speak of the total probability of an x-risk factor because of the law of total probability (e.g. $Pr\left(Y\right)=Pr(Y\mid X_1)+Pr(Y\mid X_2)+\dots$) like we can with an x-risk.
2. Concretely, something can be both an x-risk and a risk factor. Climate change is often cited as an example: it could cause an existential catastrophe directly by making all of Earth unable to support complex societies, or indirectly by increasing humanity's vulnerability to other risks. Pandemics might also be an example, as a pandemic could either directly cause the collapse of civilization or expose humanity to other risks.

I think the difference is that x-risks are events that directly cause an existential catastrophe, such as extinction or civilizational collapse, whereas x-risk factors are events that don't have a direct causal pathway to x-catastrophe. But it's possible that pretty much all x-risks are risk factors and vice versa. For example, suppose that humanity is already decimated by a global pandemic, and then a war causes the permanent collapse of civilization. We usually think of pandemics as risks and wars as risk factors, but in this scenario, the war is the x-risk because it happened last... right?

One way to think about x-risks that avoids this problem is that x-risks can have both direct and indirect causal pathways to x-catastrophe.

I think partnering with local science museums to run events on EA topics could be a great way to get EA-related ideas out to the public.

Big O as a cause prioritization heuristic

When estimating the amount of good that can be done by working on a given cause, a good first approximation might be the asymptotic behavior of the amount of good done at each point in time (the trajectory change).

Other important factors are the magnitude of the trajectory change (how much good is done at each point in time) and its duration (how long the trajectory change lasts).

For example, changing the rate of economic growth (population growth * GDP/capita growth)  has an $O\left(t^2\right)$ trajectory change in the short run, as long as humanity doesn't expand into space. We break it down into a population growth component, which grows linearly, and a component for the change in average welfare due to GDP per capita growth^[1]. GDP per capita typically grows exponentially, and welfare is a logarithmic function of GDP per capita. These two trends cancel out, resulting in a linear trend, or $O\left(t\right)$.

If humanity expands into space, then economic growth becomes a quartic trend. Population growth becomes cubic, since humanity can fill an $O\left(t^3\right)$ amount of space in $t$ time.^[2] The average welfare due to GDP per capita growth is still linear. Multiplying the cubic and linear trends yields a quartic trend. So increasing the probability that space colonization goes well looks more important than changing the rate of economic growth on Earth.

Surprisingly, the trajectory change caused by reducing existential risk is not an exponential trend. Existential risk reduces expected welfare at each point in time by a factor of $(1-r{)}^t$, where $r$ is the probability of catastrophe per unit of time. This exponential trend causes all trajectory changes to wash out as $t$ becomes very large.

The amount of good done by reducing x-risk from $r_i$ to $r_f$ is

$\sum _{t=1}^{\infty }y\left(t\right)\left(\right(1-r_f{)}^t-(1-r_i{)}^t)$,

where $y\left(t\right)$ is the welfare trajectory conditional on no existential catastrophe. The factor $\left(\right(1-r_f{)}^t-(1-r_i{)}^t)$ increases to a maximum value and then decays to 0 as $t$ approaches infinity, so its asymptotic behavior is $O\left(1\right)$. So the trajectory change caused by reducing x-risk is $O\left(y\right(t\left)\right)$, or whatever the asymptotic behavior of $y\left(t\right)$ is.

On the other hand, the amount of good done by changing the trajectory from $y_i\left(t\right)$ to $y_f\left(t\right)$ is

$\sum _{t=1}^{\infty }\left(y_f\right(t)-y_i(t\left)\right)(1-r{)}^t$.

So if you can change the trajectory to a $y_f\left(t\right)$ that will grow asymptotically faster than $y_i\left(t\right)$, e.g. by colonizing space, then this may be more important than reducing x-risk while holding the trajectory constant.

1. ^{^}

   Change in GDP per capita may underestimate the amount of welfare created by technological progress. I'm not sure if this makes the change in average welfare a super-linear growth trend, though.

2. ^{^}

   The Epistemic Challenge to Longtermism (Tarsney, 2020)

I just listened to Andrew Critch's interview about "AI Research Considerations for Human Existential Safety" (ARCHES). I took some notes on the podcast episode, which I'll share here. I won't attempt to summarize the entire episode; instead, please see this summary of the ARCHES paper in the Alignment Newsletter.

• We need to explicitly distinguish between "AI existential safety" and "AI safety" writ large. Saying "AI safety" without qualification is confusing for both people who focus on near-term AI safety problems and those who focus on AI existential safety problems; it creates a bait-and-switch for both groups.
• Although existential risk can refer to any event that permanently and drastically reduces humanity's potential for future development (paraphrasing Bostrom 2013), ARCHES only deals with the risk of human extinction because it's easier to reason about and because it's not clear what other non-extinction outcomes are existential events.
• ARCHES frames AI alignment in terms of delegation from m ≥ 1 human stakeholders (such as individuals or organizations) to n ≥ 1 AI systems. Most alignment literature to date focuses on the single-single setting (one principal, one agent), but such settings in the real world are likely to evolve into multi-principal, multi-agent settings. Computer scientists interested in AI existential safety should pay more attention to the multi-multi setting relative to the single-single one for the following reasons:
  • There are commercial incentives to develop AI systems that are aligned with respect to the single-single setting, but not to make sure they won't break down in the multi-multi setting. A group of AI systems that are "aligned" with respect to single-single may still precipitate human extinction if the systems are not designed to interact well.
  • Single-single delegation solutions feed into AI capabilities, so focusing only on single-single delegation may increase existential risk.
  • What alignment means in the multi-multi setting is more ambiguous because the presence of multiple stakeholders engenders heterogeneous preferences. However, predicting whether humanity goes extinct in the multi-multi setting is easier than predicting whether a group of AI systems will "optimally" satisfy a group's preferences.
• Critch and Krueger coin the term "prepotent AI" to refer to an AI system that is powerful enough to transform Earth's environment at least as much as humans have and where humans cannot effectively stop or reverse these changes. Importantly, a prepotent AI need not be an artificial general intelligence.

NYC is adopting ranked-choice voting for the 2021 City Council election. One challenge will be explaining the new voting system, though.

I'm excited about Open Phil's new cause area, global aid advocacy. Development aid from rich countries could be used to serve several goals that many EAs care about:

• Economic development and poverty reduction
• Public health and biosecurity, including drug liberalization
• Promoting liberal democracy
• Climate change mitigation and adaptation

Also, development aid can fund a combination of randomista-style and systemic interventions (such as building infrastructure to promote growth).

The United States has two agencies that provide development aid: USAID, which provides grants and technical assistance, and the U.S. International Development Finance Corporation (DFC), which provides equity and debt financing. I'd like to see both of these agencies strengthened and expanded with more funding.

I'm especially excited about the DFC because it was created in 2019 as a counterweight to China's Belt and Road Initiative (BRI). China has used the BRI to buy influence in regions such as Africa, so aid-receiving countries don't criticize its authoritarian policies. Development finance institutions run by liberal democracies, like the DFC, can be used to peel developing countries away from China and make them more likely to support global democracy promotion initiatives.

Making specialty meats like foie gras using cellular agriculture could be especially promising. Foie gras traditionally involves fattening ducks or geese by force-feeding them, which is especially ethically problematic (although alternative production methods exist). It could probably be produced by growing liver and fat cells in a medium without much of a scaffold, which would make it easier to develop.

Some rough thoughts on cause prioritization

• I've been tying myself up in knots about what causes to prioritize. I originally came back to effective altruism because I realized I had gotten interested in 23 different causes and needed to prioritize them. But looking at the 80K problem profile page (I am fairly aligned with their worldview), I see at least 17 relatively unexplored causes that they say could be as pressing as the top causes they've created profiles for. I've taken a stab at one of them: making surveillance compatible with privacy, civil liberties, and public oversight.
• I'm sympathetic to this proposal for how to prioritize given cluelessness. But I'm not sure it should dominate my decision making. It also stops feeling like altruism when it's too abstracted away from the object-level problems (other than x-risk and governance).
• I've been seriously considering just picking causes from the 80K list "at random."
  • By this, I mean could just pick a cause from the list that seems more neglected, "speaks to me" in some meaningful way, and that I have a good personal fit for. Many of the more unexplored causes on the 80K list seem more neglected, like one person worked on it just long enough to write one forum post (e.g. risks from malevolent actors).
  • It feels inherently icky because it's not really taking into account knowledge of the scale of impact, and it's the exact thing that EA tells you not to do. But: MIRI calls it quantilizing, or picking an action at random from the top x% of actions one could do. They think it's a promising alternative to expected utility maximization for AI agents, which makes me more confident that it might be a good strategy for clueless altruists too.
  • Some analogies that I think support this line of thinking:
    • In 2013, the British newspaper The Observer  ran a contest between professional investment managers and... a cat throwing a toy at a dartboard to pick stocks. The cat won. According to the efficient market hypothesis, investors are clueless about what investing opportunities will outperform the pack, so they're unlikely to outperform an index fund or a stock-picking cat. If we're similarly clueless about what's effective in the long term, then maybe the stochastic approach is fine.
    • One strategy for dimensionality reduction in machine learning and statistics is to compress a high-dimensional dataset into a lower-dimensional space that's easier to compute with by creating a random projection. Even though the random projection doesn't take into account any information in the dataset (like PCA does), it still preserves most of the information in the dataset most of the time.
• I've also been thinking about going into EA community building activities (such as setting up an EA/public interest tech hackathon) so I can delegate, in expectation, the process of thinking about which causes are promising to other people who are better suited to doing it. If I did this, I would most likely still be thinking about cause prioritization, but it would allow me to stretch that thinking over a longer time scale than if I had to do it all at once before deciding on an object-level cause to work on.
• Even though I think AI safety is a potentially pressing problem, I don't emphasize it as much because it doesn't seem constrained by CS talent. The EA community currently encourages people with CS skills to go into either AI technical safety or earning to give. Direct work applying CS to other pressing causes seems more neglected, and it's the path I'm exploring.

Back-of-the-envelope calculations for improving efficiency of public transit spending

The cost of building and maintaining public transportation varies widely across municipalities due to inefficiencies - for example, the NYC Second Avenue Subway has cost $2.14 billion per kilometer to build, whereas it costs an average of $80.22 million to build a kilometer of tunnel in Spain (Transit Costs Project). While many transit advocacy groups advocate for improving quality of public transit service (e.g. Straphangers Campaign in NYC), few advocate for reducing wasteful infrastructure spending.

BOTEC for operating costs

• Uday Schultz writes: "bringing NYCT’s [the NYC subway agency] facility maintenance costs down to the national average could save $1.3 billion dollars per year."
• With a 6% discount rate, this equates to a $21.7 billion net present value. So an advocacy campaign that spent $21.7 million to reduce NYCT's maintenance costs to the national average would yield a 1000x return.
• Things that would make the cost-effectiveness of this campaign higher or lower:
  • (Higher) A lower discount rate would increase the net present value of the benefits
  • (Higher) In theory, we can reduce maintenance costs to even lower levels than the US national average; Western European levels are (I think) lower.
  • (Lower) We might not realize all of the potential efficiency gains for political reasons - e.g. if contractors and labor unions block the best possible reforms.

BOTEC for capital construction costs

• The NYC Second Avenue Subway will be 13.7 km long when completed and cost over $17 billion. Phase 1 of the subway line has been completed, consists of 2.9 km of tunnel, and cost $4.45 billion.
• So the rest of the planned subway line (yet to be built) consists of 10.8 km of tunnel and is expected to cost $12.55 billion, for an average of $1.16 billion per km of tunnel.
• Phase 2 of the subway will be 2.4 km long and cost $6 billion, for an average of $2.5 billion per km of tunnel.
• There will likely be cost overruns in the future, so let's take the average of these two numbers and assume that the subway will cost an average of $1.83 billion/km to build.
• As I stated before, the average cost per km of new tunnel in Spain is $80.22 million (Transit Costs Project). If NYCT could build the rest of the Second Avenue Subway at this cost, it would save $1.75 billion per km of new tunnel, or $18.9 billion overall (since there are 10.8 km of tunnel left to build).

N.B.: These are BOTECs for individual aspects of transit spending, and a transit spending advocacy project would benefit from economies of scope because it would be lobbying for cost reductions across all aspects of public transit, not just e.g. the Second Avenue Subway or operating costs.

See also: "So You Want to Do an Infrastructure Package," Alon Levy's whitepaper on reducing transit infrastructure costs (Niskanen Center, 2021)

There should be a directory of EA co-living spaces, if there isn't already. The EA Hub would be a good place for it.

I lowkey miss the name "shortform" 🙁

It seems like decibels (dB) are a natural unit for perceived pleasure and pain, since they account for the fact that humans and other beings mostly perceive sensations in proportion to the logarithm of their actual strength. (This is discussed at length in "Logarithmic Scales of Pleasure and Pain".)

Decibels are a relative quantity: they express the intensity of a signal relative to another. A 10x difference is 10 dB, a 100x difference is 20 dB, and so on. The "just noticeable difference" in amplitude of sound is ~1 dB, or a ~25% increase. But decibels can also be used in an "absolute" sense by quantifying the ratio of the signal to a reference value. In the case of sound, the reference value is the smallest value that most humans can hear (a sound pressure of 20 micropascals).^[1]

Since pleasure and pain are perceived according to a log scale, the utility of a sensation could be approximated by:

$U\left(S\right)=U_{0}max(0,\log (S/S_0\left)\right)$

where $S$ is the intensity of the sensation, $S_0$ is the smallest perceptible sensation, and $U_0$ is a constant that is positive for pleasurable sensations and negative for painful ones. (This is only an approximation because Fechner's law, the principle that governs logarithmic perception of signals, breaks down for very strong and very weak signals.)

It seems very natural, therefore, to use decibels as the main unit for pleasure and pain, alongside utils for the utility of perceived sensations, as the relationship between decibels and utils is linear. For example, if a utility function is given by

$U\left(D\right)=0.5max(0,D)$

where $D$ is the decibel amount, then we have 10 dB of pleasure = 5 utils, 20 dB = 10 utils, 30 dB = 15 utils, and so on.

1. ^{^}

   https://en.wikipedia.org/wiki/Decibel#Acoustics_2

AOC's Among Us stream on Twitch nets $200K for coronavirus relief

"We did it! $200k raised in one livestream (on a whim!) for eviction defense, food pantries, and more. This is going to make such a difference for those who need it most right now." — AOC's Tweet

Video game streaming is a popular way to raise money for causes. We should use this strategy to fundraise for EA organizations.

We're probably surveilling poor and vulnerable people in developing and developed countries too much in the name of aiding them, and we should give stronger consideration to the privacy rights of aid recipients. Personal data about these people collected for benign purposes can be weaponized against them by malicious actors, and surveillance itself can deter people from accessing vital services.

"Stop Surveillance Humanitarianism" by Mark Latonero

Automating Inequality by Virginia Eubanks makes a similar argument regarding aid recipients in developed countries.

I think some of us really need to create op-eds, videos, etc. for a mainstream audience defending longtermism. The Phil Torres pieces have spread a lot (people outside the EA community have shared them in a Discord server I moderate, and Timnit Gebru has picked them up) and thus far I haven't seen an adequate response.

A rebuttal of the paperclip maximizer argument

I was talking to someone (whom I'm leaving anonymous) about AI safety, and they said that the AI alignment problem is a joke (to put it mildly). They said that it won't actually be that hard to teach AI systems the subtleties of human norms because language models contain normative knowledge. I don't know if I endorse this claim but I found it quite convincing, so I'd like to share it here.

In the classic naive paperclip maximizer scenario, we assume there's a goal-directed AI system, and its human boss tells it to "maximize paperclips." At this point, it creates a plan to turn all of the iron atoms on Earth's surface into paperclips. The AI knows everything about the world, including the fact that blood hemoglobin and cargo ships contain iron. However, it doesn't know that it's wrong to kill people and destroy cargo ships for the purpose of obtaining iron. So it starts going around killing people and destroying cargo ships to obtain as much iron as possible for paperclip manufacturing.

I think most of us assume that the AI system, when directed to "maximize paperclips," would align itself with an objective function that says to create as many paper clips as superhumanly possible, even at the cost of destroying human lives and economic assets. However, I see two issues:

1. It's assuming that the system would interpret the term "maximize" extremely literally, in a way that no reasonable human would interpret it. (This is the core of the paperclip argument, but I'm trying to show that it's a weakness.) Most modern natural language processing (NLP) systems are based on statistical word embeddings, which capture what words mean in the source texts, rather than their strict mathematical definitions (if they even have one). If the AI system interprets commands using a word embedding, it's going to interpret "maximize" the way humans would.

   Ben Garfinkel has proposed the "process orthogonality thesis" - the idea that, for the classic AI alignment argument to work, "the process of imbuing a system with capabilities and the process of imbuing a system with goals" would have to be orthogonal. But this point shows that the process of giving the system capabilities (in this case, knowing that iron can be obtained from various everyday objects) and the process of giving it a goal (in this case, making paperclips) may not be orthogonal. An AI system based on contemporary language models seems much more likely to learn that "maximize X" means something more like "maximize X subject to common-sense constraints Y1, Y2, ..." than to learn that human blood can be turned into iron for paperclips. (It's also possible that it'll learn neither, which means it might take "maximize" too literally but won't figure out that it can make paperclips from humans.)

2. It's assuming that the system would make a special case for verbal commands that can be interpreted as objective functions and set out to optimize the objective function if possible. At a minimum, the AI system needs to convert each verbal command into a plan to execute it, somewhat like a query plan in relational databases. But not every plan to execute a verbal command would involve maximizing an objective function, and using objective functions in execution plans is probably dangerous for the reason that the classic paperclip argument tries to highlight, as well as overkill for most commands.

Content warning: the current situation in Afghanistan (2021)

Is there anything people outside Afghanistan can do to address the worsening situation in Afghanistan? GiveDirectly-style aid to Afghans seems like not-an-option because the previous Taliban regime "prevented international aid from entering the country for starving civilians." (Wikipedia)

The best thing we can do is probably to help resettle Afghan refugees, whether by providing resources to NGOs that help them directly, or by petitioning governments to admit more of them. Some charities that do this:

• International Refugee Assistance Project (IRAP), which advocates for refugees in general and has gotten grants from Open Phil
• No One Left Behind, which focuses on resettling Afghans who helped the US military

I don't have a good sense of how much impact donations to these charities would have. The US is already scrambling to get SIV applicants out of Afghanistan and into the US and other countries where they can wait in safety for their applications to be processed. On the margins, advocacy groups can probably advocate for this process to be improved and streamlined.

Hot take: Too many EA orgs use blue and white in their branding.

I think improving bus systems in the United States (and probably other countries) could be a plausible Cause X.

Importance: Improving bus service would:

• Increase economic output in cities
• Dramatically improve quality of life for low-income residents
• Reduce cities' carbon footprint, air pollution, and traffic congestion

Neglectedness: City buses probably don't get much attention because most people don't think very highly of them, and focus much more on novel transportation technologies like electric vehicles.

Tractability: According to Higashide, improving bus systems is a matter of improving how the bus systems are governed. Right now, I think a nationwide movement to improve bus transit would be less polarizing than the YIMBY movement has been. While YIMBYism has earned a reputation as elitist due to some of its early advocates' mistakes, a pro-bus movement could be seen as aligned with the interests of low-income city dwellers provided that it gets the messaging right from the beginning.

Also, bus systems are less costly to roll out, upgrade, and alter than other public transportation options like trains.

Reason to invest in cultivated meat research: we can use meat scaffolding technology to grow nervous tissue and put chemicals in the cell media that cause it to experience constant euphoria

New Economist special report on dementia - As humanity ages the numbers of people with dementia will surge: The world is ill-prepared for the frightening human, economic and social implications

Is there any appetite for a project to make high-risk, high-yield donation recommendations within global health and development? The idea would be to identify donation opportunities that could outperform the GiveWell top charities, especially ones that make long-lasting and systemic changes.

Possible research/forecasting questions to understand the economic value of AGI research

A common narrative about AI research is that we are on a path to AGI, in that society will be motivated to try to create increasingly general AI systems, culminating in AGI. Since this is a core assumption of the AGI risk hypothesis, I think it's very important to understand whether this is actually the case.

Some people have predicted that AI research funding will dry up someday as the costs start to outweigh the benefits, resulting in an "AI winter." Jeff Bigham wrote in 2019 that the AI field will experience an "AI autumn," in which the AI research community will shift its focus from trying to develop human-level AI capabilities to developing socially valuable applications of narrow AI.

My view is that an AI winter is unlikely to happen anytime soon (10%), an AI autumn is likely to happen eventually (70%), and continued investment in AGI research all the way to AGI is somewhat unlikely (20%). But I think we can try to understand and predict these outcomes better. Here are some ideas for possibly testable research questions:

• What will be the ROI on:
  • Microsoft's $1 billion dollar investment and partnership with OpenAI?
  • Microsoft's GPT-3 licensing deal with OpenAI?
  • Google/Alphabet's acquisition of DeepMind?
• How much money will OpenAI make by licensing GPT-3?
• How long will it take for the technology behind GPT-2 and GPT-3 (roughly, making generic language models do other language tasks without specific training) to become as economically competitive as similar technologies did after they were invented?
• How long will it take for DeepMind and OpenAI to break even?
• How do the growth rates of DeepMind and OpenAI's revenues and expenses compare to those of other corporate research labs throughout history?
• Will Alphabet downsize or shut down DeepMind?
• Will Microsoft scale back or end its partnership with OpenAI?

Notes:

• I don't know of any other labs actively trying to create AGI.
• I have no experience with financial analysis, so I don't know if these questions are the kind that a financial analyst would actually be able to answer. They could be nonsensical for all I know.

I've realized that I feel less constrained when writing poetry than when writing essays/blog posts. Essays are more time-consuming for me - I spend a lot of time adding links, fact-checking my points, and organizing them in a coherent way, and I feel like I have to stake out a clear position when writing in prose. Whereas in poetry, the rules have more to do with making the form and content work well together, and evoking an emotional response in the reader.

I also think poetry is a good medium for expressing ambiguity. I've written a few draft poems in my notebook about EA themes, and they pose moral dilemmas in EA without imposing a straightforward answer to them. Here's one to illustrate what I mean:

"is this creative accounting?"
I think to myself
as I enter donation amounts
into my spreadsheet -
amounts of other people's money
that they might have given anyway -
just not to my chosen charity

I feel like providing a straightforward answer to a complex problem wastes the potential of poetry. You might as well write a persuasive essay instead.

Also, a lot of us are writing blog posts; far fewer of us are writing poetry to communicate EA ideas.

"Stop comparing your behind the scenes to someone else's highlight reel"

Some people in the EA community need to hear this: it's not healthy to compare yourself to other EAs. I hope someone writes a blog post about this

John, Katherine, Sarah, and Hank Green are making a $6.5M donation to Partners in Health to address the maternal mortality crisis in Sierra Leone, and are trying to raise $25M in total. PIH has been working with the Sierra Leone Ministry of Health to improve the quality of maternal care through facility upgrades, supplies, and training.

PIH blog post • vlogbrothers video

Matt Yglesias gets EA wrong :(

What EAs think is that people should make decisions guided by a rigorous empirical evaluation based on consequentialist criteria.

Ummm, no. Not all EAs are consequentialists (although a large fraction of them are), and most EAs these days understand that "rigorous empirical evaluation" isn't the only way to reason about interventions.

It just gets worse from there:

In other words, effective altruists don’t think you should make charitable contributions to your church (again, relative to the mass public this is the most controversial part!) or to support the arts or to solve problems in your community. They think most of the stuff that people donate to (which, again, is largely religiously motivated) do is frivolous. But beyond that, they would dismiss the bulk of the kind of problems that concern most people as literal “first world problems” that blatantly fail the cost-benefit test compared to Vitamin A supplementation in Africa.

No! We're not against supporting programs other than the global health stuff. It's just that you gotta buy your fuzzies separate from your utils. More fundamentally, EAs disagree on whether EA is mandatory or supererogatory (merely good). If EA is supererogatory, then supporting your local museum isn't wrong, it just doesn't count towards your effective giving budget.

Epistemic status: Although I'm vaguely aware of the evidence on gender equality and peace, I'm not an expert on international relations. I'm somewhat confident in my main claim here.

Gender equality - in societies at large, in government, and in peace negotiations - may be an existential security factor insofar as it promotes societal stability and decreases international and intra-state conflict.

According to the Council on Foreign Relations, women's participation in peacemaking and government at large improves the durability of peace agreements and social stability afterward. Gender equality also increases trust in political institutions and decreases risk of terrorism. According to a study by Krause, Krause, and Bränfors (2018), direct participation by women in peacemaking positively affects the quality and durability of peace agreements because of "linkages between women signatories and women civil society groups." In principle, including other identity groups such as ethnic, racial, and religious minorities in peace negotiations may also activate these linkages and thus lead to more durable and higher quality peace.

Some organizations that advance gender equality in peacemaking and international security:

• Women of Color Advancing Peace, Security and Conflict Transformation (WCAPS)
• Women in International Security (WIIS)
• Council on Foreign Relations Women and Foreign Policy Program

Improving checks and balances on U.S. presidential power seems like an important, neglected, and tractable cause area.

• Importance: There is a risk that U.S. federal government policy will become erratic, since each president can easily reverse the previous president's executive actions (for example, Biden reversed many of Trump's EOs during his first few weeks of office). The uncertainty makes it hard to reliably adapt to policy changes (this is a recognized challenge for businesses, and also applies to other interest groups, such as LGBTQ+ people, refugees, and immigrants). Also, while strong executive leadership is important for enabling the federal government to address certain challenges, such as international relations and infrastructure, it can also be misused, such as when Trump shared top-secret intelligence with the Russian government.
• Neglectedness: Although many voters care about holding the government accountable (just ask them whether they think the U.S. is "corrupt"), it seems unlikely that they will specifically advocate for curbs on presidential power, as it's a "wonky" policy proposal. There are a few advocacy groups working on curbing executive authority, such as Protect Democracy, but I'd be surprised if there were more than 10.
• Tractability: There are concrete proposals for curbing presidential power; some of them seem like they could gain bipartisan support, such as increasing controls on financial conflicts of interest. The goal is to turn informal norms that presidents were merely expected to follow into enforceable laws, since the Trump presidency has shown that informal norms can and will be broken.
  • In my opinion, while the media has focused a lot on Trump's tax returns, requiring presidents to disclose their tax returns seems to have more symbolic significance than actual value in holding presidents accountable.

Further reading:

• Reining in growing powers of the presidency: Two White House vets say experience with President Trump highlights need for long-overdue curbs (Harvard Gazette, 20 November 2020)
• Mike Berkowitz on keeping the U.S. a liberal democratic country (80,000 Hours Podcast, 20 April 2021)

I've been thinking about AI safety again, and this is what I'm thinking:

The main argument of Stuart Russell's book focuses on reward modeling as a way to align AI systems with human preferences. But reward modeling seems more like an AI capabilities technology than an AI safety one. If it's really difficult to write a reward function for a given task Y, then it seems unlikely that AI developers would deploy a system that does it in an unaligned way according to a misspecified reward function. Instead, reward modeling makes it feasible to design an AI system to do the task at all.

Even with reward modeling, though, AI systems are still going to have similar drives due to instrumental convergence: self-preservation, goal preservation, resource acquisition, etc., even if they have goals that were well specified by their developers. Although maybe corrigibility and not doing bad things can be built into the systems' goals using reward modeling.

The ways I could see reward modeling technology failing to prevent AI catastrophes (other than misuse) are:

• An AI system is created using reward modeling, but the learned reward function still fails in a catastrophic, unexpected way. This is similar to how humans often take actions that unintentionally cause harm, such as habitat destruction, because they're not thinking about the harms that occur.
  • Possible solution: create a model garden for open source reward models that developers can use when training new systems with reward modeling. This way, developers start from a stronger baseline with better safety guarantees than they would have if they were developing reward modeling systems from scratch/with only their proprietary training data.
• A developer cuts corners while creating an AI system (perhaps due to economic pressure) and doesn't give the system a robust enough learned reward function, and the system fails catastrophically.
  • Lots of ink has been spilled about arms race dynamics 😛
  • Possible solution: Make sure reward models can be run efficiently. For example, if reward modeling is done using a neural network that outputs a reward value, make sure it can be done well even with slimmer neural networks (fewer parameters, lower bit depth, etc.).

Content warning: missing persons, violence against women, racism.

Amid the media coverage of the Gabby Petito case in the United States, there's been some discussion of how missing persons cases for women and girls of color are more neglected than those for missing White women. Some statistics:

Black girls and women go missing at high rates, but that isn't reflected in news coverage of missing persons cases. In 2020, of the 268,884 girls and women who were reported missing, 90,333, or nearly 34% of them, were Black, according to the National Crime Information Center. Meanwhile, Black girls and women account for only about 15% of the U.S. female population, according to census data. In contrast, white girls and women — which includes those who identify as Hispanic — made up 59% of the missing, while accounting for 75% of the overall female population.

[...]

In [Wyoming], more than 400 Indigenous girls and women went missing between 2011 and the fall of 2020, according to a state report. Indigenous people made up 21% of homicide victims in Wyoming between 2000 and 2020, despite being less than 3% of the state's population. The disparity can be seen in the media: Only 18% of Indigenous female victims received coverage. However, among white victims, 51% were in the news.

To be clear, Rivers explains, it's not about asking for more attention or being in "competition" with white people — it's about other groups getting the same attention as white victims and having their lives honored in the same ways.

Effective Altruism and Freedom

I think freedom is very important as both an end and a means to the pursuit of happiness.

Economic theory posits a deep connection between freedom (both positive and negative) and well-being. When sufficiently rational people are free to make choices from a broader choice set, they can achieve greater well-being than they could with a smaller choice set. Raising people's incomes expands their choice sets, and consequently, their happiness - this is how GiveDirectly works.

I wonder what a form of effective altruism that focused on maximizing (positive and negative) freedom for all moral patients would look like. I think it would be very similar to the forms of EA focused on maximizing total or average well-being; both the freedom- and well-being-centered forms of EA would recommend actions like supporting GiveDirectly and promoting economic growth. But we know that different variants of utilitarianism have dramatically different implications in some cases. For example, the freedom-maximizing worldview would not endorse forcing people into experience machines.

We can also think of the long-term future of humanity in terms of humanity's collective freedom to choose how it develops. We want to preserve our option value - our freedom to change course - and avoid making irreversible decisions until we are sure they are right.

Wild idea: Install a small modular reactor in a charter city and make energy its biggest export!

Charter cities' advantage is their lax regulatory environment relative to their host countries. Such permissiveness could be a good environment for deploying nuclear reactors, which are controversial and regulated to death in many countries. Charter cities are good environments for experimenting with governance structures; they can also be good for experimenting with controversial technologies.