Existential AI Safety is NOT separate from near-term applications

stecas

Comments 9

Sorted by

New & upvoted

In what world could isolation and mutual dismissal between AIS&L people and people working on neartermist problems be helpful.

If the people you ultimately want to influence are the technophiles who are building AI, who regard most near-term 'AI safety' people as annoying scolds and culture warriors, it could be good to clearly differentiate yourself from them. If existential safety people get a reputation as reliable collaborators, employees and allies who don't support the bad behaviour of many AI bias people this could put us in a good position.

peterhase

I think I disagree with the general direction of this comment but it’s hard to state why, so I’ll just outline an alternative view:

Many people are building cutting-edge AI. Many of them are sympathetic to at least some safety and ethics concerns, and some are not that sympathetic to any safety or ethics concerns
Of course it is good to have a reputation as a good collaborator and employee. It seems only instrumentally valuable to be an “ally” to the cutting edge research, and at some point you have to be honest and tell those building AI that what they’re doing is interesting but has risks in addition to potential upsides
Part of building a good reputation in the field involves honestly assessing others’ work. If you agree with work from AI safety or AI ethics or AI bias people, you should just agree with them. If you disagree with their work, you should just disagree with them. “Distancing” and “aligning” yourself with certain camps is the kind of strategic move that people in research labs often view as vaguely dishonest or overly political

Larks

Part of building a good reputation in the field involves honestly assessing others’ work. If you agree with work from AI safety or AI ethics or AI bias people, you should just agree with them. If you disagree with their work, you should just disagree with them.

Yes, I agree with this. I think in general there is a fair bit of social pressure to give credence to intellectually weak concerns about 'AI bias' etc., which is part of what technophiles dislike, even if they can't say so publicly. Pace your first sentence, I think that self-censorship is helpful for building reputation in some fields. As such, I expect honestly reporting an epistemically rigourous evaluation of these arguments will often suffice to cause 'isolation and mutual dismissal' from Gebru-types, even while it is positive for your reputation among 'builder' capabilities researchers.

Note that in general existential safety people have put a fair bit of effort into trying to cultivate good relations with near-term AI safety people. The lowest hanging fruit implied by the argument above is to simply pull back on these activities.

calebp

Non X-risks from AI are still intrinsically important AI safety issues

Sure but I think they are less intrinically important for the standard ITN reasons.

I think that your statement implies that we should care about them a similar amount to longtermist motivated safety which might be true but you don't make a case for why we should care. I don't think the reasons for prioritising LT AIS are strongly correlated with the reasons for prioritising NT AIS so it would be somewhat surprising if this were true.

elleom

As someone who is a deep learning researcher and came to believe in the importance of AI safety through EA, I would like to say I strongly agree with the last point on making allies and growing the AI safety field. I support the claim that some people feel more hesitant to be involved in AI safety or just give up as there is a somewhat cliquey and dismissive feeling from the community and the community sometimes feels quite fragmented on arguments for and against what's useful. To me, this feels a bit counterproductive and alienating.

I hypothesize that frowning on, or even just the large focus on questioning the usefulness of near-term safety work adds to the deterrence of other current deep learning researchers and maybe other communities too engaging with AI safety. Less parochialism and more friends seem like a sensible approach and a more productive community.

peterhase

One thing I think is interesting is how similar some of the work is from bay area AI safety folks and other safety crowds, like the area often referred to as "AI ethics." For example, Redwood worked on a paper about safe language generation, focusing on descriptions of physical harm, and safe language generation is a long-running academic research area (including for physical harm! see https://arxiv.org/pdf/2210.10045.pdf). The deepest motivating factors behind the work may differ, but this is one reason I think there is a lot of common ground across safety research areas.

stecas

+1 I think it's very worthwhile to emphasize neartermist reasons to care about work that may be primarily longtermism-oriented.

Jamie Bernardi

Thanks for exploring this issue! I agree that there could be more understanding between AI safety & the wider AI community, and I'm curious to do more thinking about this.

I think each of the 3 claims you make in the body of the text are broadly true. However I don't think they directly back up the claim in the title that "AI safety is not separate from near-term applications".

I think there are some important ways that AI safety is distinct; it goes 1 step further by imagining the capabilities of future systems, and trying to anticipate ways they could go wrong ahead of time. I think there are some research questions it'd be hard to work on if the AI safety field wasn't separate from current-day application research. E.g. agent foundations, inner misalignment and detecting deception.

I think I agree with much of your sentiment still. To illustrate what I mean, I would like it to be true that:

Important AI current-day-application safety issues are worked on by many people, and there is mutual respect between our communities
Work done by near-term application researchers is known about and leverageable by the AGI safety community
Ultimately, there is still a distinct, accessible AGI safety community that works on issues distinct to advanced, general AI systems

stecas

No disagreements here. I guess I imagine AIS&L work along with work on the neartermist examples I mentioned as a venn diagram with healthy overlap. I'm glad for the AIS&L community, and I think it tackles some truly unique problems. By "separate" I essentially meant "disjoint" in the title.

Comments