We'll aim to release a short post about this by the end of the week!

I also sometimes use naturalreaders. Unfortunately I find it a bit... unnatural at times.

I've been really enjoying Type III Audio's reader on this forum, though!

That sounds great to me, thanks!

I totally agree there's a gap here. At BlueDot Impact  (/ AGI safety fundamentals), we're currently working on understanding the pipeline for ourselves.

We'll be launching another governance course in the next week, and in the longer term we will publish more info on governance careers on our website, as and when we establish the information for ourselves.

In the meantime, there's great advice on this account, mostly targeted at people in the US, but there might be some transferrable lessons:

https://forum.effectivealtruism.org/users/us-policy-careers

Thanks for highlighting that there were other 2 announcements that I didn't focus on in this post.

Whilst the funding announcement may be positive, I didn't expect that it would have strong implications for alignment research - so I chose to ignore it in this post. I didn't spend more than a minute checking my assumption there, though.

RE the announcement of further OMB policies- I totally agree that it sounds like it could be important for alignment / risk reduction. I omitted that announcement mostly because I  didn't have very much context to know what those policies would entail, given the announcement was quite light on details at this point. Thanks for shedding some light on what it could mean!

FWIW, I think this post makes progress and could work in the contexts of some groups. As a concrete example, it would probably work for me as an organiser of one-off courses, and probably for organisers of one-off retreats or internships.

I appreciate the thrust of comments pointing out imperfections in e.g. local group settings, but I just want to be careful that we don't throw out the proposal just because it doesn't work for everyone in all contexts; I think it's better to start with an an imperfect starting point and to iterate on that where it doesn't work in specific contexts, rather than to try to come up with the perfect policy in-theory and get paralysed when we can't achieve that.

Thanks for highlighting this!

Great, thanks for writing this up! I don't work in policy, but it seems to be an extremely pragmatic and helpful guide from an outside-perspective.

A question - is being a US citizen a hard requirement for all of this advice?

If not a hard requirement, what hidden (or explicit) barriers would you expect a non-citizen to face?

I also think that power dynamics are the source of the biggest problems in the work/social overlap, so a flatter power structure might be a good way of avoiding some of the pitfalls and abuses of the work/social overlap.

Do you think that in abstract that professional/social overlap is less of a problem when the power structure is flatter, or that having a flatter power structure is something that EA could actually achieve?

I'm curious because, to deal with potential abuse of power, I would prefer a more explicit power structure (which sounds like an opposite conclusion to your suggestion).

My first assumption is that power structures are an unavoidable fact in any group of people. I assume that trying to enact a flatter power structure might actually cash out as pretending the power structure doesn't exist [this might be where we disagree!].

Pretending that power structures are flat leads to plausibly permissable abuse of the actual underlying power structure. However strictly acknowledging a power structure means one is forced to acknowledge the power dynamic.

So to encourage healthy relationships, I would have called for making power structures explicit, in EA or any group.

Thanks for exploring this issue! I agree that there could be more understanding between AI safety & the wider AI community, and I'm curious to do more thinking about this.

I think each of the 3 claims you make in the body of the text are broadly true. However I don't think they directly back up the claim in the title that "AI safety is not separate from near-term applications".

I think there are some important ways that AI safety is distinct; it goes 1 step further by imagining the capabilities of future systems, and trying to anticipate ways they could go wrong ahead of time. I think there are some research questions it'd be hard to work on if the AI safety field wasn't separate from current-day application research. E.g. agent foundations, inner misalignment and detecting deception.

I think I agree with much of your sentiment still. To illustrate what I mean, I would like it to be true that:

1. Important AI current-day-application safety issues are worked on by many people, and there is mutual respect between our communities
2. Work done by near-term application researchers is known about and leverageable by the AGI safety community
3. Ultimately, there is still a distinct, accessible AGI safety community that works on issues distinct to advanced, general AI systems

I wrote this guide for Cambridge, UK, when Cambridge EA CIC was running a hiring round.

I think a guide for Cambridge based on your template would still be valuable (but I won't do it any time soon). In my guide, I was focused on 1) a broader audience (including 'non-EAs') and 2) moving to Cambridge rather than visiting temporarily.