Summary 

• EA researchers would benefit from adopting communication norms from the policy world, which emphasize clarity and ease of reading. 
  • As an AI governance researcher, I’ve gotten a lot of value from using these practices – they help me clarify my ideas and get useful feedback. I appreciate it when colleagues use these practices, too.
  • By contrast, it can be difficult to read and parse some important EA documents, which are written as long blog posts without accessible summaries.
• Three best practices to adopt:
  • 1. Put your key points up front.
  • 2. Use bullet points or numbered lists.
  • 3. Use headings and bolding to make your document easy to skim.
• I suggest practicing by writing a summary of an important but not-maximally-accessible piece of work. For an example, see my summary of Gwern’s Scaling Hypothesis post at the end of this post.
• I recommend Holden Karnofsky’s blog posts, particularly this summary of his blog post series on the “most important century”, as good examples of accessible writing on complex topics.

Policy-sphere writing optimizes for clarity and ease of reading –  two valuable attributes that EA docs sometimes lack

EA researchers would benefit from adopting communication norms from the policy world, which emphasize clarity and ease of reading.

Policy briefs are written for people with little time who need to make decisions quickly, and I find them easy and pleasant to read because they respect my time as a reader. 

Longtermists I know who've been exposed to these norms generally find them valuable, and want them to be more widespread within EA.

In my experience, adopting these norms takes some effort, but is extremely worthwhile on a personal level.

• It forces me to clarify my thinking – which is actually a really big deal!
  • When I started out as a researcher, I tried to write things in academic-style prose, and I think this made it harder for me to think things through properly. ^[1]
• Writing docs this way means it’s easier to get feedback. And the feedback I get is probably more useful–for example, more focused on core ideas, since it’s clearer what those are.
• It makes my docs easier to reference later on. That’s true for me, and I’m guessing it’s true for my readers as well. When I'm coming back to my doc a month later and trying to remember what I think about topic X, having used executive summary style makes it much easier to get back up to speed.
  • It’s actually fun to check back in on my rough ideas and works in progress, instead of stressful.

I think following these norms is also very good for your readers, both in and out of the EA sphere. 

• In the EA sphere: Following these norms saves readers valuable time. Although our community tends to select for people who enjoy reading long, dry online texts, not all EAs are like this, and even the ones who are have high opportunity costs. Writing a piece with a good summary and skimmable structure makes it much easier for your readers to make informed decisions about how much to read, what sections to prioritize, and in what depth. Whatever they do end up reading, it’ll be much easier for them to quickly extract useful insights.
• Outside of the EA sphere: Following these norms could also make external-facing EA writeups more accessible to policymakers, executives, and other senior decision-makers who may lack context on our ideas and the time to get up to speed on them. (Frankly, it also helps policy reports look more legitimate, although I’m guessing most EAs interested in influencing policy have some professional experience and so are already familiar with this style.)

Concrete recommendations

Three core tips

A useful starting point is to adopt the three following practices:

1. Put your key points up front. 

• Have a summary section at the top of your doc.
• If a particular section is long or confusing, add a clear & simple summary to the beginning of the section.
• In both cases, include your key takeaways, not just "in this section I will discuss..".

2. Use bullet points or numbered lists.  

3. Make your document easy to skim. 

• Bold your key points / key sentences.
• Have section titles reflect the key idea of the section or the flow of ideas, not just a generic subject. E.g., “X implies Y”, not just “X” or “effects of X”.

Other best practices

Use clear and accessible sentences

• Avoid jargon where possible
  • If it’s necessary to include specialized terms, define them!
• Avoid long sentences, especially ones with lots of parentheticals or clauses that make them hard to follow.

Assume your audience is smart, but has limited bandwidth.

• Imagine a busy reader who is reading this summary on the way to a meeting with the piece’s author. What do you want them to know, so they can ask useful questions and have a productive discussion?
• Imagine yourself reading this in a month, when the piece is no longer fresh in your mind. What are the few key points you’d most like to remember, and how can you write them so they’ll still make sense to you when you lack context?

Examples

What to improve on

Academic books and papers are expected to have a summary-and-conclusion structure, and sometimes these are really helpful! But in my experience they often do a mediocre job of condensing their key takeaways or helping me figure out how to engage with them further. 

Worse yet, a lot of really valuable texts on AI risk & longtermism are basically long blog posts, which can have a particularly heinous overlap of “interesting and valuable content” plus “lack of a clear summary or easy-to-skim structure.” 

Some specific examples:

• I like Slate Star Codex, but the man demands a lot of your reading time. Consider e.g. this classic post on parapsychology. It’s ~4000 words, and I bet you could summarize the key ideas in 300. (Some of this is because Scott is aiming to entertain, not just inform -- executive summary style is less useful in that case.)
• This recent Nature article is pretty readable, but it lacks a concise summary:
  • Dual use of artificial-intelligence-powered drug discovery
• Gwern’s article about the scaling hypothesis, like most of his work, is super interesting–  but also full of long paragraphs and sentences.
  • It has a summary section, but that summary is wordy and kinda hard to skim. And it doesn’t summarize some important, interesting points, like his musing about different AI labs’ interest in scaling in the Prospects section.

Learning from good executive summaries

The 1 page Research synopsis of this RAND AV safety paper (143 pages!). Good features include:

• Nested structure
• Bolding
• Each sentence is easy to understand

Holden’s summary of his blog post series on the “most important century”. Good features include:

• Clear, concise summary of complex ideas
• Short paragraphs
• “Map” where you can jump in to ideas you want to investigate in greater detail
  • Seems really valuable to do this well; big research outputs can be very hard to orient to.
• Evocative graphics help ideas stick in your head
  • I like that various longtermist writers have experimented with this – e.g. in Owen Cotton-Barratt's various docs or MIRI’s Embedded Agency sequence. I knew someone at FHI who paid people to make nice versions of his back-of-the-napkin sketches. It could be valuable for more of us to experiment in this direction.

Practice task

For readers who plan to do significant research or policy writing, I’d guess it’s worthwhile to spend 30 minutes practicing the executive summary writing style. Consider taking that time now, or schedule a time on your calendar! I ran a practice session with a team of AI governance researchers earlier this year and they found it quite useful. I think it helps build a sense for why and how to use these techniques.

A practice task I’d suggest: summarize a useful piece of writing. 

• Pick one of the pieces of writing listed below, or something else that’s interesting to you but is decently long and lacks a summary
• Try to summarize it in around 10 sentences.
• Imagine your target audience is researchers focused on the cause area(s) you’re interested in, but that at least some of them lack background knowledge relevant to the specific topic of the piece (e.g., lack much technical AI knowledge).
• If you think the article emphasizes things differently than you would, or that some parts of it are confusing, you may want to summarize your perspective on the article. That could well be more useful for yourself and your colleagues, after all!

If you’re interested in further practice, you could try summarizing another piece, or play around with these variations:

• Try condensing your summary down to just 4-5 sentences. What are the core points here?
• If you were originally summarizing what the article itself said, add a summary of your perspective as well. What are your key takeaways? What are the main questions you’re left with?
• Pick an important term that people might not know, and add a sentence that introduces it to your reader.

Texts to summarize

• Try to summarize the long and somewhat meandering summary (gray-background section) of Gwern’s post on the Scaling Hypothesis
  • If you want, you could also dig into specific sections of the post and add key ideas from those to your summary. (I found this useful to do when I attempted this practice task myself :)
• Dual use of artificial-intelligence-powered drug discovery
  • As I mentioned above, this is a pretty clear article about a potentially important topic, but lacks a summary.
• What is the EU AI Act and why should you care about it?
  • This EA Forum post (10 min read) is a well-structured, author’s summary of the European Commission’s proposal for European regulation of AI, but could be made even more accessible with an executive summary.

Example summary: Gwern’s post on the scaling hypothesis

Meta notes: I focused on re-writing the content in the doc’s summary section in a way that would be more useful to an unfamiliar reader. But I also pulled in some content that doesn’t exist in Gwern’s summary, and included my own takes. The “right” amount of external content and viewpoint to pull in depends on the goal of your summary. Often, I aim to leave readers with something like “the key pieces of my current viewpoint on the key questions addressed by the summarized document.” What that means in any particular case is, naturally, a judgment call!

—

The performance of large language models, most recently GPT-3,^[2]  provides evidence for the scaling hypothesis – the idea that we could get to highly intelligent AI simply by building bigger neural nets.  

• GPT-3 is the latest in a line of language models which have gotten better mostly by being bigger than their predecessors.
• It does some impressive things, improving on various benchmarks and demonstrating meta-learning capabilities: it’s able to perform well at lots of tasks it wasn’t explicitly trained on.
• It does this without being programmed especially cleverly: it’s mostly a much larger version of the basic Transformer architecture, without lots of the improvements that researchers have figured out they could include.

If the observed scaling of model performance continues, and orgs decide to invest in bigger language models, we could see highly intelligent AI created on relatively short timelines. 

• The largest neural nets models still cost relatively little to train, compared to the R&D budgets of large organizations: they cost tens of millions of dollars, while top budgets are in the billions.
  • Gwern thinks it’s unlikely that other orgs build much bigger models: he thinks OpenAI is unusually set on the scaling hypothesis and other orgs are unlikely to buy into it.
    • Since the post was written in late 2020, Google and Microsoft have published bigger models than GPT-3. I think it’s plausible that they get serious about competing with OpenAI to build useful large language models (LLMs), and that this competition eventually leads to many orgs competing to build highly capable AI.
    • That said, I think Gwern’s viewpoint was more plausible at the time of writing, when it had mostly been OpenAI at the frontier of LLM scaling.
• Just pouring more compute into language models will probably hit diminishing returns eventually, but a) the models might become very capable first, and b) developers might figure out some good tricks that stop diminishing returns. 

Acknowledgements & related work

Thanks to Abi Olvera, Amanda El-Dakhakhni, and Michael Aird for feedback on this piece, and the Rethink Priorities Longtermism Department in general. 

For interested readers, some earlier discussions of clear communication in research:

• Chris Olah and Shan Carter on research debt and distilling ideas.
  • This response by Lizka Vaintrob.
• Open Phil on reasoning transparency
   

1. ^{^}

   As an aside: For early-stage writing, I would also recommend being open, coarse, and informal – being willing to say things like “maybe X?” or “I don’t really know where this intuition is coming from.” Being honest about these things with myself is super valuable, and being honest with others helps them give you useful feedback. Many researchers in the EA community are good about this, but I’ve noticed that junior researchers I’ve mentored are sometimes constrained by attempting an overly formal, academic style.

2. ^{^}

    As of writing time in late 2020. Recently, tech companies have created even bigger and better AI models like Turing Megatron NLG, Gopher, and PaLM. See https://www.aitracker.org/ for a list of large models.