If this data is available, I (and I know other grantmakers) would be extremely interested in seeing it. Knowing the success rate of upskilling grants will affect how likely I am to recommend the path of upskilling/independent researh!
I have been excited about NYC having more of a longtermist presence for a long time ;) Thanks for pointing out some of NYC's unique features/comparative advantages.
I would also add: there are just a ton of people who live within commuting distance to central NYC (like 20 million). -Having a strong EA presence in a populous city seems like an efficient way to market our ideas to a bunch of people. -NYC being high population density makes it an attractive place for people to live; odds are you can live near a bunch of your friends/family, and there will be lots of employment opportunities for non-EA partners. -36% of NYC residents were not born in the USA. NYC is super friendly to immigrants, which makes it an attractive place for the world's best and brightest.
I think it is worth investing the time into researching and applying to US Universities and Scholarships. If you can't afford to go, then don't. But the really top schools (Harvard, Princeton, Yale, Stanford, etc.) are often free/heavily discounted for families making less than $100,000 per year. Seems like it makes sense to at least figure out how much it will cost to go.
My intuition is there is a huge boost in people's job prospects by attending a high reputation school. You'll have better professors, a better professional network once you graduate, and an easier time getting a job in the USA, which seems like a higher impact place to work than may other locations.
Doing an Oxford Math/Phil degree seems good for economics (and I know multiple people who have extremely successfully followed this route). Doing an Oxford PPE degree seems good for politics. Doing Oxford Math or CS seems good for technical AI Safety. I'm less familiar with Cambridge, but assume they have similar degrees available. So wouldn't rule out Oxbridge just because you can't major in Econ.
All of these opinions are my own and do not reflect those of my employer.
As somebody who wasted spent 7 years doing a Cognitive Neuroscience PhD, I think it's a bad idea for most people to do Neuroscience PhDs. PhDs in general are not optimised for truth seeking, working on high impact projects, or maximising your personal wellbeing. In fact, rates of anxiety and depression are higher amongst graduate students than the population of people with college degrees of similar age. You also get paid extremely badly, which is a problem for people with families or other financial commitments. For any specific question you want to ask, it seems worth investigating if you can do the same work in industry or at a non-profit, just to see if you would be able to study the same questions in a more focused way outside of academia.
So I don’t think doing a Neuro PhD is the most effective route to working on AI Safety. That said, there seem to be some useful research directions if you want to pursue a Neuro PhD program anyway. Some examples include: interpretability work that can be translated from natural to artificial neural networks; specifically studying neural learning algorithms; or doing completely computational research, aka a backdoor CS PhD while fitting your models to neural data collected by other people. (CS PhD programs are insanely competitive right now, and Neuroscience professors are desperate for lab members who know how to code, so this is one way into a computational academic program at a top university if you’re ok working on Neuroscience relevant research questions.)
Vael Gates (who did a Computational/Cognitive Neuroscience PhD with Tom Griffiths, one of the leaders of this field), has some further thoughts that they’ve written up in this EA Forum post. I completely agree with their assessment of neuroscience research from the perspective of AI Safety research here:
Final note: cellular/molecular neuroscience, circuit-level neuroscience, cognitive neuroscience, and computational neuroscience are some of the divisions within neuroscience, and the skills in each of these subfields have different levels of applicability to AI. My main point is I don’t think any of these without an AI / computational background will help you contribute much to AI safety, though I expect that most computational neuroscientists and a good subset of cognitive neuroscientists will indeed have AI-relevant computational backgrounds. One can ask me what fields I think would be readily deployed towards AI safety without any AI background, and my answer is: math, physics (because of its closeness to math), maybe philosophy and theoretical economics (game theory, principle-agent, etc.)? I expect everyone else without exposure to AI will have to reskill if they’re interested in AI safety, with that being easier if one has a technical background. People just sometimes seem to expect pure neuroscience (absent computational subfields) and social science backgrounds to be unusually useful without further AI grounding, and I’m worried that this is trying to be inclusive when it’s not actually the case that these backgrounds alone are useful.
Going slightly off tangent: your original question specifically mentions moral uncertainty. I share Geoffrey Miller’s views in his comment on this thread, that Psychology is a more useful discipline to study moral uncertainty compared to Neuroscience. My PhD advisor/my old lab did some of the neuroscience research frequently cited by EAs (e.g., the stuff on how different brain regions/processes affect moral decision making). I have to say I’m not super impressed with this research, and most of the authors on this paper have not gone on to pursue more of this kind of research or solve human value alignment. (Exception being Joshua Greene, who I do endorse working with!)
If you want to focus on moral uncertainty, you can collect way more information from a much more diverse set of individuals if you focus on behaviour instead of neural activity. As Geoffrey mentions, it is *much* easier/cheaper to study people’s opinions or behaviour than it is to study their neural activity. For example, it costs ~$5 to pay somebody to take a quick survey on moral decisions, vs. about $500 an hour to run an fMRI scanner for one subject to collect a super messy dataset that’s incredibly different to interpret. People do take research more seriously if you slap a photo of a brain on it, but that doesn’t mean the brain data adds anything more than aesthetic value.
It might make sense for you to check out what EA Psychologists are actually doing to see if their research seems more up your alley compared to the neuroscience questions you’re interested in. A good place to start is here: https://www.eapsychology.org/
Disclaimer: I work on the 1on1 team at 80k, but these comments reflect my individual impressions, though I asked a colleague to have a quick look before I posted.
As someone who did a PhD, this all checks out to me. I especially like your framing of PhDs "as more like an entry-level graduate researcher job than ‘n more years of school’". Many people outside of academia don't understand this, and think of graduate school as just an extension of undergrad when it is really a completely different environment. The main reason to get a PhD is if you want to be a professional researcher (either within or outside of academia), so from this perspective, you'll have to be a junior researcher somewhere for a few years anyway.
In the context of short timelines: if you can do direct work on high impact problems during your PhD, the opportunity cost of a 5-7 year program is substantially lower.
However, in my experience, academia makes it very hard to focus on questions of highest impact; instead people are funneled into projects that are publishable by academic journals. It is really hard to escape this, though having a supportive supervisor (e.g., somebody who already deeply cares about x-risks, or an already tenured professor who is happy to have students study whatever they want) gives you a better shot at studying something actually useful. Just something to consider even if you've already decided you're a good personal fit for doing a PhD!
I think I agree with everything you have written. I also personally feel like my husband and I are having impactful careers despite having a toddler + 1 on the way, and I don't think we would be massively more impactful if we were childfree.
This is due to a combination of factors:
1. Childcare time has replaced friend socialization time, basically completely. So we still have time to do a "normal" amount of work, we have just reprioritized our non-working hours.
2. As you know (being my supervisor haha), I work for a core EA org, which has a WONDERFUL parental leave/support policy. My difficult pregnancy has been happily accommodated in every way, and I have paid parental leave when the baby comes soon, which is a huge weight off my shoulders. I know British/European moms expect this, but as an American, it really is wonderful to have this level of support from my employer. So I think working for the right org is pretty critical for being able to have an impactful career while having kids.
3. My husband founded his own EA startup, so he sets his own schedule. This also allows him to be flexible with his hours and be hyperfocused on working on high impact projects, instead of wasting the working hours he has available on stuff that's less important.
4. My husband is also an excellent partner, who has been averaging more than 50% of the childcare (especially when I'm too pregnant to function). This is a critical factor in me being able to get work done, despite having a high energy toddler.
5. We have access to 8am-6pm daycare, which covers the normal working day. Unfortunately childcare is insanely expensive in the US. We pay over $2k per month in a high cost of living area for one child in daycare. We're lucky to be able to afford it, but figuring out how you're going to manage childcare should also be considered if you want to have kids + impact. Getting free childcare from grandparents is definitely the dream here. We do have some grandparent help, which allows us to do things like go to EAGs for a weekend, but basically nobody besides grandparents or people you pay seems to be interested in helping take care of children in modern Western society. (Kinda sad imo.)
6. Making new humans and trying really hard to give them happy lives seems like having a positive impact to me. I work on some longtermist causes, where it's extremely uncertain what our work now will produce later. But literally creating a new life and taking care of it feels like it has a pretty certain positive expected value :) Or at least more than what I would be doing with my spare time otherwise!
If this data is available, I (and I know other grantmakers) would be extremely interested in seeing it. Knowing the success rate of upskilling grants will affect how likely I am to recommend the path of upskilling/independent researh!
Super helpful background info, thanks so much :)
I have been excited about NYC having more of a longtermist presence for a long time ;) Thanks for pointing out some of NYC's unique features/comparative advantages.
I would also add: there are just a ton of people who live within commuting distance to central NYC (like 20 million).
-Having a strong EA presence in a populous city seems like an efficient way to market our ideas to a bunch of people.
-NYC being high population density makes it an attractive place for people to live; odds are you can live near a bunch of your friends/family, and there will be lots of employment opportunities for non-EA partners.
-36% of NYC residents were not born in the USA. NYC is super friendly to immigrants, which makes it an attractive place for the world's best and brightest.
I think it is worth investing the time into researching and applying to US Universities and Scholarships. If you can't afford to go, then don't. But the really top schools (Harvard, Princeton, Yale, Stanford, etc.) are often free/heavily discounted for families making less than $100,000 per year. Seems like it makes sense to at least figure out how much it will cost to go.
My intuition is there is a huge boost in people's job prospects by attending a high reputation school. You'll have better professors, a better professional network once you graduate, and an easier time getting a job in the USA, which seems like a higher impact place to work than may other locations.
Doing an Oxford Math/Phil degree seems good for economics (and I know multiple people who have extremely successfully followed this route). Doing an Oxford PPE degree seems good for politics. Doing Oxford Math or CS seems good for technical AI Safety. I'm less familiar with Cambridge, but assume they have similar degrees available. So wouldn't rule out Oxbridge just because you can't major in Econ.
All of these opinions are my own and do not reflect those of my employer.
"The communication will then be sent via smoke signals that are sent via Signal." Brilliant 🧠
Thanks! I agreed/appreciated your thoughts on how Psych can actually be relevant to human value alignment as well, especially compared to Neuro!
This seems mostly right to me!
As somebody who
wastedspent 7 years doing a Cognitive Neuroscience PhD, I think it's a bad idea for most people to do Neuroscience PhDs. PhDs in general are not optimised for truth seeking, working on high impact projects, or maximising your personal wellbeing. In fact, rates of anxiety and depression are higher amongst graduate students than the population of people with college degrees of similar age. You also get paid extremely badly, which is a problem for people with families or other financial commitments. For any specific question you want to ask, it seems worth investigating if you can do the same work in industry or at a non-profit, just to see if you would be able to study the same questions in a more focused way outside of academia.So I don’t think doing a Neuro PhD is the most effective route to working on AI Safety. That said, there seem to be some useful research directions if you want to pursue a Neuro PhD program anyway. Some examples include: interpretability work that can be translated from natural to artificial neural networks; specifically studying neural learning algorithms; or doing completely computational research, aka a backdoor CS PhD while fitting your models to neural data collected by other people. (CS PhD programs are insanely competitive right now, and Neuroscience professors are desperate for lab members who know how to code, so this is one way into a computational academic program at a top university if you’re ok working on Neuroscience relevant research questions.)
Vael Gates (who did a Computational/Cognitive Neuroscience PhD with Tom Griffiths, one of the leaders of this field), has some further thoughts that they’ve written up in this EA Forum post. I completely agree with their assessment of neuroscience research from the perspective of AI Safety research here:
Going slightly off tangent: your original question specifically mentions moral uncertainty. I share Geoffrey Miller’s views in his comment on this thread, that Psychology is a more useful discipline to study moral uncertainty compared to Neuroscience. My PhD advisor/my old lab did some of the neuroscience research frequently cited by EAs (e.g., the stuff on how different brain regions/processes affect moral decision making). I have to say I’m not super impressed with this research, and most of the authors on this paper have not gone on to pursue more of this kind of research or solve human value alignment. (Exception being Joshua Greene, who I do endorse working with!)
On the flip side, I think psychologists have done very interesting/useful research on human values (see this paper on how normal people think about population ethics, also eloquently written up as a shorter/more readable EA Forum post here). In this vein, I’ve also been very impressed by work produced by psychologists working with empirical philosophers, for example this paper on the Psychology of Existential Risk.
If you want to focus on moral uncertainty, you can collect way more information from a much more diverse set of individuals if you focus on behaviour instead of neural activity. As Geoffrey mentions, it is *much* easier/cheaper to study people’s opinions or behaviour than it is to study their neural activity. For example, it costs ~$5 to pay somebody to take a quick survey on moral decisions, vs. about $500 an hour to run an fMRI scanner for one subject to collect a super messy dataset that’s incredibly different to interpret. People do take research more seriously if you slap a photo of a brain on it, but that doesn’t mean the brain data adds anything more than aesthetic value.
It might make sense for you to check out what EA Psychologists are actually doing to see if their research seems more up your alley compared to the neuroscience questions you’re interested in. A good place to start is here: https://www.eapsychology.org/
Disclaimer: I work on the 1on1 team at 80k, but these comments reflect my individual impressions, though I asked a colleague to have a quick look before I posted.
As someone who did a PhD, this all checks out to me. I especially like your framing of PhDs "as more like an entry-level graduate researcher job than ‘n more years of school’". Many people outside of academia don't understand this, and think of graduate school as just an extension of undergrad when it is really a completely different environment. The main reason to get a PhD is if you want to be a professional researcher (either within or outside of academia), so from this perspective, you'll have to be a junior researcher somewhere for a few years anyway.
In the context of short timelines: if you can do direct work on high impact problems during your PhD, the opportunity cost of a 5-7 year program is substantially lower.
However, in my experience, academia makes it very hard to focus on questions of highest impact; instead people are funneled into projects that are publishable by academic journals. It is really hard to escape this, though having a supportive supervisor (e.g., somebody who already deeply cares about x-risks, or an already tenured professor who is happy to have students study whatever they want) gives you a better shot at studying something actually useful. Just something to consider even if you've already decided you're a good personal fit for doing a PhD!
Thanks so much for sharing this, Michelle!
I think I agree with everything you have written. I also personally feel like my husband and I are having impactful careers despite having a toddler + 1 on the way, and I don't think we would be massively more impactful if we were childfree.
This is due to a combination of factors:
1. Childcare time has replaced friend socialization time, basically completely. So we still have time to do a "normal" amount of work, we have just reprioritized our non-working hours.
2. As you know (being my supervisor haha), I work for a core EA org, which has a WONDERFUL parental leave/support policy. My difficult pregnancy has been happily accommodated in every way, and I have paid parental leave when the baby comes soon, which is a huge weight off my shoulders. I know British/European moms expect this, but as an American, it really is wonderful to have this level of support from my employer. So I think working for the right org is pretty critical for being able to have an impactful career while having kids.
3. My husband founded his own EA startup, so he sets his own schedule. This also allows him to be flexible with his hours and be hyperfocused on working on high impact projects, instead of wasting the working hours he has available on stuff that's less important.
4. My husband is also an excellent partner, who has been averaging more than 50% of the childcare (especially when I'm too pregnant to function). This is a critical factor in me being able to get work done, despite having a high energy toddler.
5. We have access to 8am-6pm daycare, which covers the normal working day. Unfortunately childcare is insanely expensive in the US. We pay over $2k per month in a high cost of living area for one child in daycare. We're lucky to be able to afford it, but figuring out how you're going to manage childcare should also be considered if you want to have kids + impact. Getting free childcare from grandparents is definitely the dream here. We do have some grandparent help, which allows us to do things like go to EAGs for a weekend, but basically nobody besides grandparents or people you pay seems to be interested in helping take care of children in modern Western society. (Kinda sad imo.)
6. Making new humans and trying really hard to give them happy lives seems like having a positive impact to me. I work on some longtermist causes, where it's extremely uncertain what our work now will produce later. But literally creating a new life and taking care of it feels like it has a pretty certain positive expected value :) Or at least more than what I would be doing with my spare time otherwise!