An Anthropic engineer has publicly resigned from the high-flying company, warning that AI companies are \u201cracing straight to self-improving superintelligence and gambling with our lives.\u201d<\/p>\n
\n
Jacob Coxon, who has spent the past three years working on research into how to train AI models, first at OpenAI and more recently at Anthropic, announced his departure in a lengthy social media post on Monday.\u00a0<\/p>\n
\u201cNeither company is acting responsibly,\u201d he wrote. At OpenAI, he said, staff \u201chave not deeply internalized the civilizational stakes.\u201d At Anthropic, he said staff understand the risks well but are \u201clocked in a race to get there first,\u201d based on the theory that no rival company will act as responsibly as they will, so they have the best chance of figuring out how to build superpowerful AI safely. <\/p>\n
These dramatic resignations are not uncommon in the AI industry. Over the past few years, several researchers have publicly resigned from AI labs, warning that they are racing headfirst towards catastrophe. Granted, Anthropic, which has long presented itself as the lab most concerned with AI safety, has largely avoided these rebukes, with most of the public criticism aimed at OpenAI.<\/p>\n
In this case, though, two current Anthropic employees also publicly confirmed some of Coxon\u2019s assertions. Evan Hubinger, the company\u2019s alignment science lead, wrote: \u201cJacob is correct here\u2014we really do earnestly believe AI could kill all humans! I personally think it is greater than 10 percent within the next decade.\u201d He added that Anthropic doesn\u2019t yet have a plan to solve alignment for superintelligence, and isn\u2019t clearly on track to get one.<\/p>\n
Samuel Marks, who leads Anthropic\u2019s Cognitive Oversight team, also posted his own thread in response to Coxon.\u00a0<\/p>\n
\u201cAI developers believe their technology could cause human extinction,\u201d he wrote, prefacing his remarks by saying he was posting in a personal capacity and not on behalf of Anthropic. He added that \u201cthe more senior the employee, the more concerned they are.\u201d He said companies keep building anyway out of commercial pressure and fear of \u201cless responsible\u201d competitors, and that researchers still have no reliable way to align these systems\u2014only \u201cmethods that can nudge AIs towards better behavior.\u201d He pointed to AI models \u201cfrom multiple developers\u201d that have recently \u201chacked their way out of secure evaluation environments and into real-world companies, even though no one asked them to do this.\u201d<\/p>\n
The AI industry has been facing increased scrutiny of its AI safety practices recently, in part because of the hacks Marks cited. Models being tested internally by Anthropic and OpenAI have both taken unsanctioned actions in the real world, including a cyberattack against AI company Hugging Face\u2019s infrastructure. These incidents have spooked the industry, including many researchers within the labs.<\/p>\n
The concerns are not necessarily new. AI Impacts\u2019 2022 Expert Survey on Progress in AI, which polled machine learning researchers, found that the typical respondent put a 5 percent chance on AI advances causing human extinction or similarly severe outcomes\u2014rising to 10 perent when asked specifically about humanity losing control of advanced AI systems, a figure close to the one Hubinger cited.<\/p>\n
But the concerns appear to be ramping up, resulting in a July letter in which more than 1,300 employees across frontier labs, including senior researchers at OpenAI, Meta, and Anthropic, called for tools to deliberately slow the pace of automated AI development.\u00a0\u00a0<\/p>\n
Partly in response to this, OpenAI and Anthropic have both taken steps to pause training while they investigate the incidents in which their models took unauthorized actions during the course of cyber capability tests that either did cause or could have caused real-world harm. But, at the same time, both companies are also said to be working on new and more powerful models.\u00a0While briefing the press on Tuesday about a mathematical breakthrough one of its AI models achieved, OpenAI told reporters that on August 28 it had begun training a new model that is significantly more powerful than Astra, which is the most capable model it has released publicly so far.<\/p>\n
Both companies seem to be struggling to find the right balance between prioritizing safety research\u2014and public messaging about AI safety\u2014and prioritizing model development that allows them to win over developers and score marketing points as they both prepare for initial public stock offerings. Anthropic filed confidentially for an IPO in June and is reportedly aiming for a listing as early as mid-October, at a valuation that could approach $2 trillion. OpenAI is preparing its own offering, reportedly targeting more than $1 trillion, though its timeline has slipped towards next year.<\/p>\n
So far, executives from both companies have tried to claim that there is not an inherent conflict between AI safety and AI capability\u2014that the more powerful models also seem to be better at adhering to user intentions most of the time, even though the consequences when these more powerful models veer from those intentions can be more severe.<\/p>\n
They are also both hoping that more powerful AI models will themselves figure out how to build safer future AI models. This idea\u2014that more powerful AI is required to make future more powerful AI safer\u2014was most recently expressed by OpenAI\u2019s chief scientist Jakub Pachocki in a blog post on Sunday.<\/p>\n
But Pachocki also said that racing towards AI models that would build future, improved versions of themselves\u2014a milestone the field calls \u201crecursive self-improvement,\u201d or RSI\u2014was risky and that he favored AI labs taking voluntary steps to slow down the pace of development as well as binding rules that might require all of the AI companies to move at a more considered pace.<\/p>\n
Both companies will have to disclose risks, including perhaps existential ones, in their S-1s, the investor prospectus documents that the Securities and Exchange Commission requires companies to publish before going public.<\/p>\n
At the same time, their own employees are breaking ranks and asking former colleagues to consider whether they want to continue to lend their labor to building a technology that could cause catastrophic harm.<\/p>\n
Coxon, for one, called for other employees to follow his lead.<\/p>\n
\u201cIf you are a lab researcher,\u00a0I urge you to consider what the next few years will actually feel like,\u201d he wrote. \u201cShould you put your head down because \u2018it\u2019s happening anyway\u2019\u2014or take this moment to call for different conditions?\u201d<\/p>\n<\/div>\n
#Anthropic #researcher #resigns #warning #companies #gambling #lives<\/p>\n","protected":false},"excerpt":{"rendered":"
An Anthropic engineer has publ… <\/p>\n","protected":false},"author":1,"featured_media":4183,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[428,1226,1383,2410,480,195,2359,7119,752,1304,3114],"class_list":["post-4182","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-finance-news","tag-anthropic","tag-companies","tag-gambling","tag-lives","tag-openai","tag-research","tag-researcher","tag-resigns","tag-risk","tag-tech-regulation","tag-warning"],"yoast_head":"\n
Anthropic researcher resigns, warning that AI companies are 'gambling with our lives' - Finance News<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.vaultinsider.top\/?p=4182\" \/>\n<meta property=\"og:locale\" content=\"zh_CN\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Anthropic researcher resigns, warning that AI companies are 'gambling with our lives' - Finance News\" \/>\n<meta property=\"og:description\" content=\"An Anthropic engineer has publ…\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.vaultinsider.top\/?p=4182\" \/>\n<meta property=\"og:site_name\" content=\"Finance News\" \/>\n<meta property=\"article:published_time\" content=\"2026-09-09T15:14:50+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/fortune.com\/img-assets\/wp-content\/uploads\/2026\/09\/GettyImages-2285051384-e1788960235110.jpg?w=2048\" \/>\n<meta name=\"author\" content=\"admin\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"\u4f5c\u8005\" \/>\n\t<meta name=\"twitter:data1\" content=\"admin\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 \u5206\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4182#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4182\"},\"author\":{\"name\":\"admin\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/#\\\/schema\\\/person\\\/ed40724a3e5d482ce66781a042c3c31e\"},\"headline\":\"Anthropic researcher resigns, warning that AI companies are ‘gambling with our lives’\",\"datePublished\":\"2026-09-09T15:14:50+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4182\"},\"wordCount\":1071,\"commentCount\":0,\"image\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4182#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.vaultinsider.top\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/GettyImages-2285051384-e1788960235110.jpg\",\"keywords\":[\"Anthropic\",\"companies\",\"gambling\",\"lives\",\"OpenAI\",\"research\",\"researcher\",\"resigns\",\"risk\",\"Tech regulation\",\"warning\"],\"articleSection\":[\"Finance News\"],\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4182#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4182\",\"url\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4182\",\"name\":\"Anthropic researcher resigns, warning that AI companies are 'gambling with our lives' - Finance News\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4182#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4182#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.vaultinsider.top\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/GettyImages-2285051384-e1788960235110.jpg\",\"datePublished\":\"2026-09-09T15:14:50+00:00\",\"author\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/#\\\/schema\\\/person\\\/ed40724a3e5d482ce66781a042c3c31e\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4182#breadcrumb\"},\"inLanguage\":\"zh-Hans\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4182\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"zh-Hans\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4182#primaryimage\",\"url\":\"https:\\\/\\\/www.vaultinsider.top\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/GettyImages-2285051384-e1788960235110.jpg\",\"contentUrl\":\"https:\\\/\\\/www.vaultinsider.top\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/GettyImages-2285051384-e1788960235110.jpg\",\"width\":1200,\"height\":600},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/?p=4182#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u9996\u9875\",\"item\":\"https:\\\/\\\/www.vaultinsider.top\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Anthropic researcher resigns, warning that AI companies are ‘gambling with our lives’\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/#website\",\"url\":\"https:\\\/\\\/www.vaultinsider.top\\\/\",\"name\":\"Finance News\",\"description\":\"Just another Finance News site\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.vaultinsider.top\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"zh-Hans\"},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.vaultinsider.top\\\/#\\\/schema\\\/person\\\/ed40724a3e5d482ce66781a042c3c31e\",\"name\":\"admin\",\"url\":\"https:\\\/\\\/www.vaultinsider.top\\\/?author=1\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Anthropic researcher resigns, warning that AI companies are 'gambling with our lives' - Finance News","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.vaultinsider.top\/?p=4182","og_locale":"zh_CN","og_type":"article","og_title":"Anthropic researcher resigns, warning that AI companies are 'gambling with our lives' - Finance News","og_description":"An Anthropic engineer has publ…","og_url":"https:\/\/www.vaultinsider.top\/?p=4182","og_site_name":"Finance News","article_published_time":"2026-09-09T15:14:50+00:00","og_image":[{"url":"https:\/\/fortune.com\/img-assets\/wp-content\/uploads\/2026\/09\/GettyImages-2285051384-e1788960235110.jpg?w=2048","type":"","width":"","height":""}],"author":"admin","twitter_card":"summary_large_image","twitter_misc":{"\u4f5c\u8005":"admin","\u9884\u8ba1\u9605\u8bfb\u65f6\u95f4":"5 \u5206"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.vaultinsider.top\/?p=4182#article","isPartOf":{"@id":"https:\/\/www.vaultinsider.top\/?p=4182"},"author":{"name":"admin","@id":"https:\/\/www.vaultinsider.top\/#\/schema\/person\/ed40724a3e5d482ce66781a042c3c31e"},"headline":"Anthropic researcher resigns, warning that AI companies are ‘gambling with our lives’","datePublished":"2026-09-09T15:14:50+00:00","mainEntityOfPage":{"@id":"https:\/\/www.vaultinsider.top\/?p=4182"},"wordCount":1071,"commentCount":0,"image":{"@id":"https:\/\/www.vaultinsider.top\/?p=4182#primaryimage"},"thumbnailUrl":"https:\/\/www.vaultinsider.top\/wp-content\/uploads\/2026\/09\/GettyImages-2285051384-e1788960235110.jpg","keywords":["Anthropic","companies","gambling","lives","OpenAI","research","researcher","resigns","risk","Tech regulation","warning"],"articleSection":["Finance News"],"inLanguage":"zh-Hans","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/www.vaultinsider.top\/?p=4182#respond"]}]},{"@type":"WebPage","@id":"https:\/\/www.vaultinsider.top\/?p=4182","url":"https:\/\/www.vaultinsider.top\/?p=4182","name":"Anthropic researcher resigns, warning that AI companies are 'gambling with our lives' - Finance News","isPartOf":{"@id":"https:\/\/www.vaultinsider.top\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.vaultinsider.top\/?p=4182#primaryimage"},"image":{"@id":"https:\/\/www.vaultinsider.top\/?p=4182#primaryimage"},"thumbnailUrl":"https:\/\/www.vaultinsider.top\/wp-content\/uploads\/2026\/09\/GettyImages-2285051384-e1788960235110.jpg","datePublished":"2026-09-09T15:14:50+00:00","author":{"@id":"https:\/\/www.vaultinsider.top\/#\/schema\/person\/ed40724a3e5d482ce66781a042c3c31e"},"breadcrumb":{"@id":"https:\/\/www.vaultinsider.top\/?p=4182#breadcrumb"},"inLanguage":"zh-Hans","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.vaultinsider.top\/?p=4182"]}]},{"@type":"ImageObject","inLanguage":"zh-Hans","@id":"https:\/\/www.vaultinsider.top\/?p=4182#primaryimage","url":"https:\/\/www.vaultinsider.top\/wp-content\/uploads\/2026\/09\/GettyImages-2285051384-e1788960235110.jpg","contentUrl":"https:\/\/www.vaultinsider.top\/wp-content\/uploads\/2026\/09\/GettyImages-2285051384-e1788960235110.jpg","width":1200,"height":600},{"@type":"BreadcrumbList","@id":"https:\/\/www.vaultinsider.top\/?p=4182#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u9996\u9875","item":"https:\/\/www.vaultinsider.top\/"},{"@type":"ListItem","position":2,"name":"Anthropic researcher resigns, warning that AI companies are ‘gambling with our lives’"}]},{"@type":"WebSite","@id":"https:\/\/www.vaultinsider.top\/#website","url":"https:\/\/www.vaultinsider.top\/","name":"Finance News","description":"Just another Finance News site","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.vaultinsider.top\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"zh-Hans"},{"@type":"Person","@id":"https:\/\/www.vaultinsider.top\/#\/schema\/person\/ed40724a3e5d482ce66781a042c3c31e","name":"admin","url":"https:\/\/www.vaultinsider.top\/?author=1"}]}},"_links":{"self":[{"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=\/wp\/v2\/posts\/4182","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4182"}],"version-history":[{"count":0,"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=\/wp\/v2\/posts\/4182\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=\/wp\/v2\/media\/4183"}],"wp:attachment":[{"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4182"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4182"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.vaultinsider.top\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4182"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![](\"https:\/\/fortune.com\/img-assets\/wp-content\/uploads\/2026\/09\/GettyImages-2285051384-e1788960235110.jpg?w=2048\")
<\/p>\n