The next decades might be wild

Fwiw I think it would have been good to explain technical terminology to a greater extent - e.g.  TAI (transformative artificial intelligence), LLM (large language model), transformers, etc.

It says in the introduction:

I expect some readers to think that the post sounds wild and crazy but that doesn’t mean its content couldn’t be true.

Thus, the article seems in part directed to readers who are not familiar with the latest discussions about AI - and those readers presumably would benefit from technical concepts being explained when introduced.

I really liked this... the post made me think, and will continue to do that for some time. It doesn't seem all that unrealistic to me 🤯

One little nit: you seem to write "century" when you mean "decade".

Interesting ! 

I'm curious: do you have any thoughts about the impact that the decline of fossil fuels may have during these decades?

For instance, the recent claims by the International Energy Agency that fossil fuels should peak within 5 years ? (i have my own thoughts on the topic but I'd like to know how you see things going)

Fantastic post overall, but I have a nitpick about what I think may be an important issue. 

Additionally, automated coding becomes pretty good, i.e. it reliably more than doubles the output of an unassisted coder. This has the effect that much more code is generated by ML models and that more people who don’t have a coding background are able to generate decent code. An effect of this is that the overall amount of code explodes and the quality degrades because it just becomes impossible for humans to properly check and verify the code.

I strongly doubt this part, and my expectation is the opposite - human-written code is pretty mediocre, and most production systems rely on millions of lines of code, much of which is unnecessary. The reason code isn't better is because programmer time is more expensive than the short-term benefits of improving the code.  Automated tools are likely to reduce this gap, and write far tighter and simpler code. In addition, maintainability becomes far easier when automated rewrites are possible. And checking code generated by narrow AI isn't actually as hard as checking human code for correctness when there are NAI systems which are far better than humans at writing well documented unit tests. Similarly, it can excel at generating computationally verifiably correct  code - something humans can do, slowly and painfully.

You mention AI taking over many jobs and people having trouble specializing in useful ways, but you don't include any discussion AFAICT of why this won't cause massive unemployment.

On a related note, I'd like to see some ideas about how AI will change our education system. What kind of homework will high school students have in 2026? What will a typical day look like for undergraduate and graduate students?

This was a really interesting read. I think a lot of this is plausible and it is similar to my own expectations in many ways. Thank you for writing this.

That said, I think your vision is too conservative. AI the field that will change the world massively this century where transformative changes are already certain. But there are other candidates as well. Space development, self replicating macro and nano machines, human enhancement and new physics are options I consider likely in descending order. The interactions between those fields are the truly scary thing. You acknowledge that your predictions are very likely to be false, but I think the biggest error people make when trying to predict the future is that they look at only a single dimension and ignore the interactions. For example, a power seeking AI could reasonably prioritise gambling on an interstellar land grab in the forties.

Another point where I disagree with you is that AI's will cause more damage with spreadsheets, medical procedures and legal procedures. The human standard is already pretty dismal here, but we are wired to trust people in those positions. I think that while it is certain that AI will fail in some cases, it is far from certain that it will perform worse than the human baseline.

Hi Marius,

Thanks for writing this!

Just one note, I think it would maybe have been helpful to have some subsections.

文章很长，我通篇看完了，准确地说是通篇地听完了。
虽然有点浪费时间，但还是坚持看完了，这是对作者一个最起码的尊重。
下面，我将通过我自己的思维历程来进行表达，以说明我的相对观点。
首先，我要表达的是，作者的思路给我提供了一个很大的启发性。最近我也在构建未来人工智能的发展与“威胁”，旨在通过结构化且多态化的形式对人工智能进行一个整体的构建，准确地说是建模。
一开始，我在构件的时候陷入了迷茫。因为我发现，在构件的时候总感觉有些事物在阻碍着我们的思维瓶颈。尽管我们人类的大脑神经元远远超过当今的超级计算机，但是由于所处的时空环境的不同而不同，也常常受制于传统的思维模式及其所处的社会环境的时空限制中。正是由于这些种种的桎梏，阻碍了我们的思维开创性或者平稳性发展。
于是，我创建了{第三视角}的理论思维方式，以主观、客观、中观（旁观）的三维思维模型为基础，并以中观的思维视角为主导，以跳出固有的思维藩篱。从而对事物的本质事实进行正确的评估、分析、引导及其结论。关于{第三视角}理论，我会在后续的发文中继续阐述。本篇的主题不在于此，仅仅是表达一种与作者相同思维环境下的思维过程，故不作多余赘述。
接着，通过第三视角的方法，我继续了构建的畅想模型中，几乎与作者的行为是相似的。但是，后来我又发现，其实我一直在原地打转，并没有跳出这个思维的藩篱。在一次偶然的思考中，我突然灵光一现，发现了问题的所在：虚无。
没错，正是因为沉沦于虚无的世界中，所以才导致了我的思维一直没有跳出这个藩篱。也正是因为虚无，所以才导致我们一直没有一种开创性的角度去思考、建模未来的人工智能的模型。（在这里我要提一点，虽然现在的机器智能很厉害，但终究还是比不上人类大脑的生物神经网络。所以，我相信，我的建模一定会比当前的机器智能更佳。举例来说，我们很多“天马行空”的想法，机器智能无法解决：脑机接口、脑成像、全息助手、元宇宙、心灵感应，以及我新定义的一个“平行宇宙”概念等。所有的诸如此类，我们的大脑能够想象得出它的样子，甚至可以在大脑中模拟出来，但是却无法实现它。这正是我们人类大脑的生物神经元的优势之处。
）
于是，我又着手重温关于虚无主义的知识及其相应的著作、论文。说实话，能解决我问题的很少，不过也或多或少地得到了一定的启发。比如，上述的建模过程，正是由于沉沦于虚无的世界之中，导致未能跳出思维的藩篱而不能自拔。简言之，就是缺乏问题的建设性方案。就是虽然想法是对的，可以作为一定的安全协议建设的前提基础，但是并不能起到相对的建设性方案。提问题固然很重要，但是问题的本身在构建事物之初就是一个安全协议的前提，当然也是开创事物的前提。所以，问题出来之后，再继续后一步的发展与延伸，就是建设性方案。这就回到了作者里面提到的内容：“人类的长期命运基本上取决于研究人员是否能够建立一个足够强大的人工智能，以始终如一地为人类的利益行事。”
最后，关于上述两点，我又重新思考了一遍。视角、思维、虚无藩篱，这三者之间的联系是什么？为什么它们会存在？与我所构建的模型有什么对应关系？
答案是，我们无法设想的太远，但是我们有办法解决当下亟待解决的问题。简言之，过多的思绪而缺乏一定的行动，只会再度陷入另一个虚无的世界中。即便是构建模型，仅仅需要初步的就行，后面根据这个模型逐步地完善修改。前提是，我们得根据这个模型去逐步实现。
总结：人工智能，其未来的发展必然是进步的。但是在发展的同时，我们不能过渡的依赖于它。我们要相信，它只是一个类人类的辅助工具，而不是人类文明的全部。我们当然可以设定一定的安全协议来控制和预防，但是也应当正确地看待及其利用它。过分地依赖就会导致人类的依存度降低，此消彼长之后带来的必然是人类的沉沦，这不是我们想要得到的结果。我们渴望知识，但是也渴望生活。绝不能因为人工智能的崛起与发展，导致了人类本身身心思维的沉沦与堕落。这不是《西部世界》的悲观主义，也不是《奥创纪元》的自大主义，更不是《星际迷航》的迷茫主义（虚无主义）。更多的时候，我们需要的是这里面的有利部分，而不是被我们自己所创造的自我感觉“超人类”的机器而感到悲观与自卑，甚至的霍金的恐惧预言。
前面提到的浪费时间，是看到了作者的思维与我的思维有相似之处，同时作者的思维及其设想还沉沦于当今的社会基础之中。看过近年来电影的人都知道，作者所设想的都是与其相似的，重点是把那些设想用文字的方式进行了归总阐述。所以，作者所提的“威胁”，可以作为一定的参考，但建设性不大。