I'm not sure what you mean by "AI safety labs", but Redwood Research, Anthropic, and the OpenAI safety team have all hired self-taught ML engineers. DeepMind has a reputation for being more focused on credentials. Other AI labs don't do as much research that's clearly focused on AI takeover risk.
I'm currently at DeepMind and I'm not really sure where this reputation has come from. As far as I can tell DeepMind would be perfectly happy to hire self-taught ML engineers for the Research Engineer role (but probably not the Research Scientist role; my impression is that this is similar at other orgs). The interview process is focused on evaluating skills, not credentials.
DeepMind does get enough applicants that not everyone makes it to the interview stage, so it's possible that self-taught ML engineers are getting rejected before getting a chance to show they know ML. But presumably this is also a problem that Redwood / Anthropic / OpenAI have? Presumably there is some way that self-taught ML engineers are signaling that they are worth interviewing. (As a simple example, if I personally thought someone was worth interviewing, my recommendation would function as a signal for "worth interviewing", and in that situation DeepMind would interview them, and at that point I predict their success would depend primarily on their skills and not their credentials.)
If there's some signal of "worth interviewing" that DeepMind is failing to pick up on, I'd love to know that; it's the sort of problem I'd expect DeepMind-the-company to want to fix.
I think DM clearly restricts REs more than OpenAI (and I assume Anthropic). I know of REs at DM who have found it annoying/difficult to lead projects because of being REs, I know of someone without a PhD who left Brain (not DeepMind but still Google so prob more similar) partly because it was restrictive, and lead team at OAI/Anthropic, and I know of people without an undergrad degree who have been hired by OAI/Anthropic. At OpenAI I'm not aware of it being more difficult for people to lead projects etc because of being 'officially an RE'. I had bad experiences at DM that were ostensibly related to not having a PhD (but could also have been explained by lack of research ability).
I would love to hear actual details of these cases (presumably less publicly); at this level of granularity I can't tell to what extent this should change my mind (I can imagine worlds consistent with both your statements and mine).
As I understand it, DeepMind doesn’t hire people without PhDs as research scientists, and places more restrictions on what research engineers can do than other places.
DeepMind doesn’t hire people without PhDs as research scientists
Basically true (though technically the requirement is "PhD in a technical field or equivalent practical experience")
places more restrictions on what research engineers can do than other places
Doesn't seem true to me. Within safety I can name two research engineers who are currently leading research projects.
DeepMind might be more explicit that in practice the people who lead research projects will tend to have PhDs. I think this pattern is just because usually people with PhDs are better at leading research projects than people without PhDs. I expect to see the same pattern at OpenAI and Anthropic. If I assigned people to roles based solely on (my evaluation of) capabilities / merit, I'd expect to reproduce that pattern.
"DeepMind allows REs to lead research projects" is consistent with "DeepMind restricts REs more than other places". E.g. OpenAI doesn't even officially distinguish RE from RS positions, whereas DeepMind has different ladders with different expectations for each. And I think the default expectations for REs and RSs are pretty different (although I agree that it's possible for REs to end up doing most of the same things as RSs).
I continue to think that this is primarily a reflection of RSs having more experience than REs, and that a process with a single role and no RS / RE distinction would produce similar outcomes given the same people.
As someone interested in applying for Research Engineering roles in the near future, what would be your criterion for determining whether someone self-taught is “worth-interviewing”? (Also a question for others who are familiar with hiring practices at various AI safety organizations).
I don't know of some easy-to-describe bar, but as one anecdote, this post by Matthew Rahtz was easily enough to clear the "should interview" bar, and went most of the way to the "should hire" bar, when I was looking at applicants for the CHAI internship. It would also have been enough to clear the "should interview" bar at DeepMind.
I also like this 80K podcast on the topic, and in general I might recommend looking at my FAQ (though it doesn't cover this question particularly).
I do feel for RE's who are not taken seriously on spearheading novelty (of value).
So this is something I've observed. I've worked as an AI RS at many small to mid-sized firms with the biggest being Valve Software (DotA team) which is a pretty hard-to-land gig at least on the gaming side. And at these places, I've mostly always worked alongside PhDs. One observation that I did not expect to make (and this is with the exception of Valve, who I feel really were awesome and yet very humble and unassuming), was that I ended up thinking exhaustively at a meta-level with reference to the core contextual domain and also at lower granularities of computational decision-making, while (often BUT not always) a doctorate (along with non-doctorates too) would go into why a certain problem is too complex to tackle. That, I understand (though I question this too!). But the problem is on the docket now - given that a rigorous conjecture is not possible (soon enough!) to guarantee the solution works in all cases, what can be done as the next best thing... a breakthrough for that was one I would figure out eventually because a rote exploration method for solutions would otherwise be adopted.
And this is the part that I feel mattered a lot. My solution did not come from depth. Yes, I taught areas as abstract as lattices and Hasse diagrams at a grad level, put it in simple words, I can implement and apply them, read pure math papers on it, come to findings and use that too, but this wasn't the cherry on top. The novelty came from my 'WIDTH' of thinking. I've worked as an RS across six starkly different core-fields. There was a point in time I was professionally teaching at graduate level across evolutionary biology, linguistics, Tolkein mythology, organic chemistry, discrete algebraic mathematics, economic game theory, and theory of computation. It was possible because reading and curiosity started long before the formal education in these fields started. I spent years of my school days just mapping different concepts from different domains with each other in piece-wise equivalences. It meant that today when I see a computational problem - given the field of application, I can immediately think of where the experiment trees might qualify for pruning owing to intrinsic properties of the domain itself - hence reducing the 'realistic' NP-hardness of it, at least in context of computational resources. And that is different from a depth-wise study. This research comes from lateral thinking - which I feel is as much in value.
As an aside, I had a chance to do a 'comparatively' well-funded and definitely reputed PhD, and I also feel the original spirit behind a PhD is great! I earned the IBM Research (EMEA) Fellowship for a PhD in AI. But I didn't do it because even the best fellowships don't pay fairly, I had a family to take care of, and the stress and dissertation approval unpredictability of the long affair seemed worth less than the outcome of having that credential.
I ask myself -
What has been my passion? Research.
Can I really give my heart to this under such stress? No.
Without a PhD, have a ever cut corners and been less meticulous than my doctorate colleagues? No.
Have I been less realistic than my doctorate colleagues? Yes, a bit less, and no regrets there AT ALL. 7/10 times, this has actually helped.
Has a PhD stopped me from creating value? No, by evidence from my time in industrial research.
And it is from this personal experience I feel that either RE's ought to be given more freedom in pitching conceptual prototypes, OR the assumption that the relation (directed either way) between a doctorate earned and novel value creation is one-to-one should be softened.
Per Andy Jones over at LessWrong: