Once a distant ambition for technology researchers, the prospect of\u00a0artificial intelligence\u00a0models teaching themselves autonomously to be more efficient and capable appears ever closer to reality.<\/p>\n
\n
As the technology advances, developers say it is approaching \u201crecursive self-improvement,\u201d or RSI, in which AI models find ways to improve themselves and build their successor. It could bring the promise of advances in science and medicine, tech company executives say, but also risks.<\/p>\n
The uncertainty over where it all could lead is at the heart of growing fears about AI\u00a0evading human control, and possible threats to humanity, which led several AI moguls to join last weekend in a call to\u00a0slow down\u00a0the technology\u2019s pace of growth.<\/p>\n
Anthropic this week detailed how its model\u00a0Claude is helping the company\u00a0to develop the next, more intelligent version of itself. Claude is now leading 26% of Anthropic\u2019s model research and development, which the company said means it can complete most of a given task \u201cend-to-end from a high-level prompt\u201d while still being under human supervision. The models are not working completely autonomously \u2014 at least not yet.<\/p>\n
Here are some key points about recursive self-improvement.<\/p>\n
What is recursive self-improvement?<\/h4>\n
Leading AI companies have different definitions for recursive self-improvement. Some define it as when there is any feedback from AI on model improvement, while others define it as AI working toward that goal fully autonomously.<\/p>\n
Autonomous recursive self-improvement essentially means AI that can improve itself designing the next version of the system, then the next version, and so on, said Anthony Aguirre, president and CEO of the nonprofit Future of Life Institute and a physics professor at the University of California, Santa Cruz.<\/p>\n
\u201cThe really important thing here is that as AI is doing more of it, it gets faster, because AI operates just much, much more quickly than the humans do,\u201d he said.<\/p>\n
The fear around RSI is based largely on a runaway superintelligence emerging from that process, said John Thickstun, an assistant professor of computer science at Cornell University who studies methods that control the behavior of AI models. But he said a more grounded view suggests a kind of recursive self-improvement has been going on in AI development for a while now.<\/p>\n
\u201cWe have already, for years, been using these models in supportive roles for creating the next version of these models. So people use the past generation of models to write code for the AI systems that then create the next generation,\u201d he said.<\/p>\n
For years, prominent AI researchers such as OpenAI co-founder Andrej Karpathy have experimented with trying to get AI models to train and improve new AI systems. Those efforts have brought minor improvements, but not big creative leaps, Thickstun said.<\/p>\n
But AI companies today, Aguirre said, are much closer to pulling off those bigger leaps in improvement.<\/p>\n
\u201cYou can see in these plots from Anthropic over time, more and more of research is being done by the AI and it\u2019s becoming closer and closer to fully autonomous,\u201d he said. \u201cAnd the result of that success, ultimately is something that is, I think, extremely scary. I think this is probably the worst idea in the history of humanity to do this. And yes, they\u2019re doing it.\u201d<\/p>\n
Some AI labs say RSI is not far off<\/h4>\n
Anthropic\u2019s recent announcement provided the public \u2014 and other labs \u2014 with some insight into RSI progress, and it encouraged its competitors to share similar metrics. Still, the company has not expressly said how close it is to achieving fully autonomous model improvement.<\/p>\n
ChatGPT maker OpenAI announced this month that it has developed an automated \u201cresearch intern,\u201d which it defines as a system that can carry out well-defined research tasks under human direction, including \u201ctasks that would take a skilled researcher a few days.\u201d The company has said it is moving forward with the goal of creating an automated AI \u201cresearcher\u201d by March 2028.<\/p>\n
The company said in that announcement that while RSI can help align models\u2019 actions with human values and intentions, that doesn\u2019t mean \u201crapid RSI is necessarily an outcome we should pursue.\u201d<\/p>\n
\u201cWhether and how to proceed must depend on our ability to preserve human control and on informed democratic choices about the benefits and risks,\u201d the company said in a blog post.<\/p>\n
Elon Musk seems more eager to forge ahead. He said in March that for xAI\u2019s Grok models, \u201chumans are gradually getting less and less in the loop\u201d on model improvement and that \u201cevery successive model is built by the one before it,\u201d but clarified that the process was not yet fully automated. That target might be reached by the end of this year, he added, \u201cbut not later\u201d than 2027.<\/p>\n
Microsoft and some other leading AI companies seem to be taking a different approach.<\/p>\n
Mustafa Suleyman, the CEO of Microsoft AI, has said the company is moving toward \u201chumanist superintelligence,\u201d or advanced AI capabilities that are in service of people and humanity at large. Suleyman said in a 2025 essay that this would not mean \u201can unbounded and unlimited entity with high degrees of autonomy,\u201d but rather AI that is \u201ccarefully calibrated, contextualized, within limits.\u201d<\/p>\n
How development slowdown talks could impact RSI<\/h4>\n
A key challenge labs face \u2014 and have been facing essentially since the technology\u2019s inception \u2014 is ensuring their safety measures advance alongside the models\u2019 capabilities.<\/p>\n
Divisions have emerged in\u00a0the tech industry\u00a0over calls for a coordinated AI slowdown for safety, and not every major player in the AI space has specifically commented on their path forward with RSI.<\/p>\n
Anthropic, which has been a leading voice in the\u00a0calls for pacing, has said it would slow or temporarily pause its development work \u2014 assuming its global competitors also did so, and in a \u201cverifiable manner.\u201d<\/p>\n
OpenAI explicitly said this month it does not yet know how to \u201csafely get all the way to aligned, full RSI,\u201d adding that the company \u201ccannot assume that progress in alignment and safety will keep pace.\u201d More capable systems can become harder to monitor, it continued, but pursuing RSI is still a goal it says it values because an \u201cautomated AI researcher can also be an automated safety or alignment researcher.\u201d<\/p>\n<\/div>\n
#recursive #selfimprovement #Worst #idea #history #humanity #physics #professor<\/p>\n","protected":false},"excerpt":{"rendered":"
Once a distant ambition for te… <\/p>\n","protected":false},"author":1,"featured_media":4932,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[7979,2802,7486,7279,7983,2040,7980,7981,7982],"class_list":["post-4931","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-finance-news","tag-autonomy","tag-history","tag-humanity","tag-idea","tag-physics","tag-professor","tag-recursive","tag-selfimprovement","tag-worst"],"yoast_head":"\n
What is recursive self-improvement? 'Worst idea in the history of humanity,' physics professor says - Finance News<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.vaultinsider.top\/?p=4931\" \/>\n<meta property=\"og:locale\" content=\"zh_CN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"What is recursive self-improvement? 'Worst idea in the history of humanity,' physics professor says - Finance News\" \/>\n<meta property=\"og:description\" content=\"Once a distant ambition for te…\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.vaultinsider.top\/?p=4931\" \/>\n<meta property=\"og:site_name\" content=\"Finance News\" \/>\n<meta property=\"article:published_time\" content=\"2026-09-19T17:06:29+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/fortune.com\/img-assets\/wp-content\/uploads\/2026\/09\/AP26255587007476-e1789830477374.jpg?w=2048\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u4f5c\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 \u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4931#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4931\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/#\\\/schema\\\/person\\\/ed40724a3e5d482ce66781a042c3c31e\"},\"headline\":\"What is recursive self-improvement? ‘Worst idea in the history of humanity,’ physics professor says\",\"datePublished\":\"2026-09-19T17:06:29+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4931\"},\"wordCount\":1052,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4931#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.vaultinsider.top\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/AP26255587007476-e1789830477374.jpg\",\"keywords\":[\"autonomy\",\"history\",\"humanity\",\"idea\",\"physics\",\"professor\",\"recursive\",\"selfimprovement\",\"Worst\"],\"articleSection\":[\"Finance News\"],\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4931#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4931\",\"url\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4931\",\"name\":\"What is recursive self-improvement? 'Worst idea in the history of humanity,' physics professor says - Finance News\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4931#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4931#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.vaultinsider.top\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/AP26255587007476-e1789830477374.jpg\",\"datePublished\":\"2026-09-19T17:06:29+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/#\\\/schema\\\/person\\\/ed40724a3e5d482ce66781a042c3c31e\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4931#breadcrumb\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4931\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4931#primaryimage\",\"url\":\"https:\\\/\\\/www.vaultinsider.top\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/AP26255587007476-e1789830477374.jpg\",\"contentUrl\":\"https:\\\/\\\/www.vaultinsider.top\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/AP26255587007476-e1789830477374.jpg\",\"width\":1200,\"height\":600},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4931#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\\\/\\\/www.vaultinsider.top\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"What is recursive self-improvement? ‘Worst idea in the history of humanity,’ physics professor says\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/#website\",\"url\":\"https:\\\/\\\/www.vaultinsider.top\\\/\",\"name\":\"Finance News\",\"description\":\"Just another Finance News site\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.vaultinsider.top\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/#\\\/schema\\\/person\\\/ed40724a3e5d482ce66781a042c3c31e\",\"name\":\"admin\",\"url\":\"https:\\\/\\\/www.vaultinsider.top\\\/?author=1\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"What is recursive self-improvement? 'Worst idea in the history of humanity,' physics professor says - Finance News","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.vaultinsider.top\/?p=4931","og_locale":"zh_CN","og_type":"article","og_title":"What is recursive self-improvement? 'Worst idea in the history of humanity,' physics professor says - Finance News","og_description":"Once a distant ambition for te…","og_url":"https:\/\/www.vaultinsider.top\/?p=4931","og_site_name":"Finance News","article_published_time":"2026-09-19T17:06:29+00:00","og_image":[{"url":"https:\/\/fortune.com\/img-assets\/wp-content\/uploads\/2026\/09\/AP26255587007476-e1789830477374.jpg?w=2048","type":"","width":"","height":""}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"\u4f5c\u8005":"admin","\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4":"5 \u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.vaultinsider.top\/?p=4931#article","isPartOf":{"@id":"https:\/\/www.vaultinsider.top\/?p=4931"},"author":{"name":"admin","@id":"https:\/\/www.vaultinsider.top\/#\/schema\/person\/ed40724a3e5d482ce66781a042c3c31e"},"headline":"What is recursive self-improvement? ‘Worst idea in the history of humanity,’ physics professor says","datePublished":"2026-09-19T17:06:29+00:00","mainEntityOfPage":{"@id":"https:\/\/www.vaultinsider.top\/?p=4931"},"wordCount":1052,"commentCount":0,"image":{"@id":"https:\/\/www.vaultinsider.top\/?p=4931#primaryimage"},"thumbnailUrl":"https:\/\/www.vaultinsider.top\/wp-content\/uploads\/2026\/09\/AP26255587007476-e1789830477374.jpg","keywords":["autonomy","history","humanity","idea","physics","professor","recursive","selfimprovement","Worst"],"articleSection":["Finance News"],"inLanguage":"zh-Hans","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.vaultinsider.top\/?p=4931#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.vaultinsider.top\/?p=4931","url":"https:\/\/www.vaultinsider.top\/?p=4931","name":"What is recursive self-improvement? 'Worst idea in the history of humanity,' physics professor says - Finance News","isPartOf":{"@id":"https:\/\/www.vaultinsider.top\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.vaultinsider.top\/?p=4931#primaryimage"},"image":{"@id":"https:\/\/www.vaultinsider.top\/?p=4931#primaryimage"},"thumbnailUrl":"https:\/\/www.vaultinsider.top\/wp-content\/uploads\/2026\/09\/AP26255587007476-e1789830477374.jpg","datePublished":"2026-09-19T17:06:29+00:00","author":{"@id":"https:\/\/www.vaultinsider.top\/#\/schema\/person\/ed40724a3e5d482ce66781a042c3c31e"},"breadcrumb":{"@id":"https:\/\/www.vaultinsider.top\/?p=4931#breadcrumb"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.vaultinsider.top\/?p=4931"]}]},{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.vaultinsider.top\/?p=4931#primaryimage","url":"https:\/\/www.vaultinsider.top\/wp-content\/uploads\/2026\/09\/AP26255587007476-e1789830477374.jpg","contentUrl":"https:\/\/www.vaultinsider.top\/wp-content\/uploads\/2026\/09\/AP26255587007476-e1789830477374.jpg","width":1200,"height":600},{"@type":"BreadcrumbList","@id":"https:\/\/www.vaultinsider.top\/?p=4931#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/www.vaultinsider.top\/"},{"@type":"ListItem","position":2,"name":"What is recursive self-improvement? ‘Worst idea in the history of humanity,’ physics professor says"}]},{"@type":"WebSite","@id":"https:\/\/www.vaultinsider.top\/#website","url":"https:\/\/www.vaultinsider.top\/","name":"Finance News","description":"Just another Finance News site","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.vaultinsider.top\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/www.vaultinsider.top\/#\/schema\/person\/ed40724a3e5d482ce66781a042c3c31e","name":"admin","url":"https:\/\/www.vaultinsider.top\/?author=1"}]}},"_links":{"self":[{"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=\/wp\/v2\/posts\/4931","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4931"}],"version-history":[{"count":0,"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=\/wp\/v2\/posts\/4931\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=\/wp\/v2\/media\/4932"}],"wp:attachment":[{"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4931"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4931"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4931"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![](\"https:\/\/fortune.com\/img-assets\/wp-content\/uploads\/2026\/09\/AP26255587007476-e1789830477374.jpg?w=2048\")
<\/p>\n