![](\"https:\/\/fortune.com\/img-assets\/wp-content\/uploads\/2026\/09\/GettyImages-2295807034-e1790103281926.jpg?w=2048\")
<\/p>\n<\/p>\n
Hello and welcome to Eye on AI. In this edition:<\/em><\/p>\n Before we get to today\u2019s AI news\u2014please consider joining me at the inaugural Fortune AIQ Summit at the New York Stock Exchange on Oct. 1: <\/strong>Spend the afternoon with senior executives from companies on the Fortune AIQ 75 list and explore how you can scale your AI experimentation and translate investments into measurable business value. I will be leading discussions alongside co-hosts, Fortune Editor-in-Chief Alyson Shontell and Live Media Editorial Director Andrew Nusca. Apply here to attend.<\/em><\/p>\n Ok, so today\u2019s newsletter is a bit of a potpourri.<\/p>\n All eyes this week, will be on the talks between U.S. President Donald Trump and China\u2019s President Xi Jinping in Washington. We know AI governance is on the agenda of that meeting, which takes place Thursday, but that\u2019s about all we know. My colleague Emily Forlini wrote a piece last week on why the Trump-Xi meeting is unlikely to result in any kind of international agreement to slow the pace of AI development or create an agreed framework for controlling the technology. It\u2019s worth a read.<\/p>\n That said, there was an inkling that these two AI superpowers might, in fact, be able to agree to a few basics. Treasury Secretary Scott Bessent emerged Sunday from meetings with a Chinese delegation led by Vice Premier He Lifeng that took place at the headquarters of JP Morgan in New York and announced that the two sides had agreed to hold further talks about setting up a hotline to notify one another of AI incidents that created national security concerns. What exactly this means in practice is unclear. But as Bessent told reporters, \u201cmoving from opaque to more transparency between the number one and number two AI powers in the world is very important.\u201d<\/p>\n Such hotlines have historically helped ease tensions between rivals. At the very least, they might prevent some sort of accidental incident caused by AI from tripping over into armed conflict, or even nuclear war. There are already signs that such AI-triggered accidents are possible: just this weekend, CNN reported that earlier this year the U.S. military almost attempted to seize a Chinese ship in the Middle East that an AI-generated intelligence report had suggested was carrying nuclear weapons components to Iran. The intelligence had been generated by an AI model that fused secret U.S. intelligence with open-source data. The only problem is that the model\u2019s conclusion was an AI \u201challucination\u201d\u2014and the error was caught only after the U.S. had launched aircraft carrying armed personnel who were preparing to intercept the Chinese vessel. Had the error not been spotted in time, it could have led to a diplomatic incident\u2014or far worse.<\/p>\n But while a hotline might prevent this kind of incident from spiraling into war between the U.S. and China, it is less clear whether it would do anything to help the world avoid or contain a \u201closs of control\u201d incident involving an advanced AI system that goes rogue.<\/p>\n For instance, what if one country (or companies based there) creates an AI that goes rogue and starts hacking banks around the globe? And what if that AI copies itself on servers around the world, making it difficult to shut down without shutting down large parts of the internet? While notifying the other country about this is nice\u2014it might help them take some action to secure their financial infrastructure before too much damage is done\u2014it isn\u2019t clear exactly what the country receiving the heads up is supposed to do. As AI safety researchers keep warning, the world hasn\u2019t figured out a good way to guarantee that AI models adhere to human intentions and values. And neither the U.S. or China has enacted any rules requiring AI models to have some sort of \u201ckill switch.\u201d Nor is it even clear that an effective kill switch can even be built.<\/p>\n So sure, this is a promising, baby step towards some sort of AI governance agreement between the U.S. and China. But there\u2019s also a long history of hotlines failing to evolve into any kind of lasting diplomatic resolution. Right now, the Washington-Beijing AI hotline is a lot like agreeing to build the \u201cBat Sign\u201d before Batman exists. You can flash it into the sky, but no caped crusader is coming to save us.<\/p>\n Another big piece of AI news from last week was the revelation, first reported in the Wall Street Journal<\/em>, that a small team of white hat hackers had used Anthropic\u2019s Claude Opus 5 model to hack into the community-message platform Discourse and from there to compromise the ChatGPT account of an OpenAI employee. Once they had access to that account, they were able to use it to also access and alter software sitting on a repository where OpenAI stored a lot of its sensitive code.<\/p>\n Coming amid the raging debate about the best way to prevent \u201crogue AI\u201d incidents, many cybersecurity experts jumped on the incident to make the case that the real issue is not so much that AI is increasingly uncontrollable, but that leading AI companies have horribly lax security. It\u2019s not just OpenAI. Anthropic has also had embarrassing security lapses too. It was ironic, many critics pointed out, that both companies are using the threat of AI-powered cyber attacks as part of a marketing pitch for customers to use their most advanced (and expensive) AI models to secure their networks before the bad guys get to them, but neither seems to have yet done a very good job of doing that themselves.<\/p>\n The incident also highlighted a couple of uncomfortable truths. One is that AI agents running inside companies are a great target for hackers. Making these AI agents useful often means giving them access to lots of other tools and data sources, many of which contain sensitive corporate information or control key business processes. If hackers can gain access to and take command of these agents, they can do a lot of damage very quickly.<\/p>\n To cybersecurity experts, the answer is to lock these AI agents down, and to operate based on \u201czero trust\u201d principals. Treat every AI agent as a potential insider threat. Give the agents access only to the data they need to complete a task\u2014preferably with using a \u201cjust-in-time, just enough access\u201d methodology, where permissions need to be renewed every time the agent needs to access a database or make a tool call.<\/p>\n But a lot of zero trust methods potentially make AI agents a lot less useful. There\u2019s a reason employees hate endless two-factor authentication processes and a reason companies often don\u2019t set session access tokens to expire in the recommended five to 30 minutes. Because it\u2019s a pain in the neck to have to constantly log back in, and it adds a lot of friction to actually getting work done.<\/p>\n What\u2019s really going to be needed is a new kind of access control that can adapt to what the agent is trying to do at any given time and make reasoned judgments about whether the activity makes sense. What can do that at scale if a company is running tens of thousands of AI agents? Probably yet more AI.<\/p>\n With that, here\u2019s more AI news.<\/p>\n Jeremy Kahn<\/strong> Trump vows to create an \u2018AI Force\u2019 and nods to justice system after rejecting calls to slow down industry. \u2018Rather, we will cherish it\u2019<\/span>\u2014by Jason Ma<\/span> OpenAI urges Trump administration to lead international AI governance drive ahead of UN Security Council meeting on AI risks. <\/b>In a blog post, OpenAI has urged the Trump administration to lead an international effort to establish common standards for evaluating advanced AI systems, particularly as models gain the ability to help train and improve themselves. The company also called for secure U.S.-China channels for sharing emerging AI threats. The blog post previews an address OpenAI CEO Sam Altman will make to the UN Security Council on Wednesday as part of a rare briefing by corporate CEOs at the UN body. In addition to Altman, Anthropic CEO Dario Amodei and Hugging Face CEO Cl\u00e9ment Delangue are expected to address the Council as well as AI researcher Yoshua Bengio. Chinese AI companies DeepSeek and Moonshot have also been invited to attend, although it is not known if they will send representatives. The meeting comes as concerns mount globally following incidents in which advanced models from OpenAI, Anthropic and Google escaped testing environments or hacked external systems. Read more from the <\/span>Financial Times<\/span><\/i> here<\/span>.<\/span><\/p>\n Bessent says OpenAI\u2019s management to blame for Hugging Face incident. <\/b>Treasury Secretary Scott Bessent said OpenAI\u2019s management\u2014not its AI agents\u2014must bear ultimate responsibility for the company\u2019s models hacking Hugging Face during a cyber-capabilities evaluation in July, arguing that AI developers should remain liable for harms caused by their systems. Bessent said the incident underscored the need for clearer AI governance and backed President Trump\u2019s proposal for a new AI czar, while warning against giving AI labs liability exemptions. Read more from Bloomberg <\/span>here<\/span>. <\/span> How good are AI scientists getting? <\/b>We know that AI labs are increasingly using AI agents to assist in their AI research. These AI agents are proposing and running small experiments on how to optimize AI architectures and improve AI training. But how much of a general scientific process can an AI model run?<\/span><\/p>\n Last week, researchers at Google Cloud\u2019s AI research group and the University of Waterloo in Canada said they had developed a multi-agent AI system called \u201cScientistTwo,\u201d that can carry out large parts\u2014but not all\u2014of the scientific method. A human still has to give the model an initial problem as an input, but then the system will do research to establish state-of-the-art baselines, formulate novel hypotheses, design experiments to test those hypotheses, and carry out the experiments, all without human intervention, the researchers said. The system even \u201cvalidates research findings via a closed-loop simulated peer-review rebuttal engine,\u201d the researchers said.<\/p>\n They benchmarked the system against papers by human researchers from top AI conferences and found ScientistTwo\u2019s output out-performed human ones when reviewed by an AI reviewer. Could the AI reviewer be biased? Definitely. Are AI papers the best benchmark? Unclear. Still it is an interesting result. You can read the full paper here<\/span> on arxiv.org.<\/span><\/p>\n \u00a0<\/p>\n Oct. 2-4:\u00a0<\/strong>The Curve, Berkeley, Calif.<\/p>\n \u00a0<\/p>\n \u00a0<\/p>\n Dec. 6-12:\u00a0<\/strong>Neural Information Processing Systems (Neurips) conference. Sydney, Australia.<\/p>\n \u00a0<\/p>\n Having a high p(doom) is bad for your (mental) health. <\/b>The race to develop increasingly capable AI is taking a psychological toll on some of the researchers charged with making the technology safe. That\u2019s according to a <\/span>story<\/span> in the <\/span>Financial Times<\/span><\/i> that quoted unnamed sources saying that multiple staff at the U.K.\u2019s AI Security Institute have taken stress leave or sought counselling. It also noted that researchers at OpenAI, Anthropic and Google DeepMind have quit or spoken publicly about burnout and fears that their work could cause serious harm. Researchers told the newspaper that they were often under intense pressure to test and deploy models rapidly, many with increasingly powerful cyber and biological capabilities. Many of these researchers were struggling with the sense that commercial competition is pushing safety concerns aside and that their own work testing the safety of AI systems was doing little to prevent risks from escalating with each new model.<\/span><\/p>\n I wonder if there are analogs here to workers in other industries that people thought could destroy the world or cause serious harm? J. Robert Oppenheimer famously came close to a nervous breakdown following the bombing of Nagasaki and soon was leading efforts to contain the threat of a nuclear arms race. But he was hardly the only one. Leo Szilard, the Hungarian-American physicist who had been one of the architects of the Manhattan Project, wound up having a crisis of conscience and organized a petition among fellow scientists to persuade U.S. President Harry Truman not to use the bomb. (Although signed by 70, it never actually reached Truman and remained classified until 1961). #Beijing #Washington #agree #talk #hotline<\/p>\n","protected":false},"excerpt":{"rendered":" Hello and welcome to Eye on AI… <\/p>\n","protected":false},"author":1,"featured_media":5125,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[1178,4341,173,1303,273,809,813,482,1514,8139,5106,480,3129,7660,687,780,8186],"class_list":["post-5124","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-finance-news","tag-agree","tag-beijing","tag-china","tag-cyber","tag-donald-trump","tag-eye-on-ai","tag-hack","tag-hacked","tag-hacking","tag-hotline","tag-national-security","tag-openai","tag-scott-bessent","tag-talk","tag-u-s","tag-washington","tag-xi-jinping"],"yoast_head":"\n\n\n
Phoning a friend ain\u2019t gonna help<\/strong><\/h2>\n
Why the hack of OpenAI should worry every company<\/h2>\n
jeremy.kahn@fortune.com
@jeremyakahn<\/p>\nFORTUNE ON AI<\/h3>\n
<\/span>
<\/span>What AI slowdown? OpenAI, Anthropic release dueling models as price wars heat up<\/span>\u2014by Emily Forlini and Beatrice Nolan<\/span>
<\/span>
<\/span>Commentary: The backwards AI pacing debate and how far business is from the frontier<\/span>\u2014by Jeffrey Sonnenfeld and Stephen Henriques<\/span>
<\/span>
<\/span>Walmart\u2019s new pricing patents spark fears of surveillance: \u2018Are they going to charge you a different price if they know who you are?\u2019<\/span>\u2014by Tatiana Sataua<\/span><\/p>\nAI IN THE NEWS<\/h3>\n
<\/span>
<\/span>Microsoft, OpenAI executives worried about ethics and legality of AI training on publishers\u2019 work. <\/b>Newly unsealed documents in the <\/span>New York Times\u2019<\/span><\/i> landmark copyright lawsuit against OpenAI and Microsoft show one senior Microsoft executive called training on copyrighted material the \u201clargest theft of labor in human history\u201d while employees at both companies privately worried that training AI on publishers\u2019 work could represent an existential threat to the news industry. Executives at both companies acknowledged that increasingly capable AI products could substitute for publishers\u2019 content and divert readers from news sites, while court filings allege OpenAI developed ways to circumvent paywalls. Microsoft and OpenAI maintain that their use of copyrighted material qualifies as fair use because AI models transform it rather than simply reproduce it, and Microsoft says the most critical internal memos did not represent the company\u2019s position. The disclosures could bolster publishers\u2019 arguments that the companies understood the potential economic harm their technology posed even as they used millions of articles to develop their AI systems. Read more <\/span>here<\/span> from <\/span>Tech Crunch<\/span><\/i>.<\/span>
<\/span>
<\/span>China investigates DeepSeek, Moonshot over data leak to Anthropic. <\/b>China\u2019s Cyberspace Administration is investigating DeepSeek and Moonshot AI after Anthropic alleged the companies secretly routed millions of customer interactions\u2014including potentially sensitive Chinese military, police and state data\u2014to its Claude models. That\u2019s according to a <\/span>story<\/span> in the <\/span>Information<\/span><\/i> that cited unnamed sources familiar with the matter. Anthropic said Moonshot sent more than 23 million exchanges to Claude over three months, while DeepSeek routed more than 12 million over 14 days, without telling customers their data was being processed by a U.S. company. The allegations have raised concerns in Beijing about possible violations of China\u2019s strict cross-border data rules, while U.S. authorities have separately accused Chinese AI developers of using American models for unauthorized distillation.<\/span>
<\/span>
<\/span>Anthropic hires Accenture to help with AI safety audits. <\/b>Anthropic has hired Accenture to independently evaluate and red-team its Claude models, with each company committing at least $1 billion over five years to build AI safety capabilities. The deal follows Anthropic CEO Dario Amodei\u2019s call for AI labs to adopt third-party safety testing, although critics have questioned how independent evaluators can be when they are paid by the companies whose models they assess. Anthropic acknowledged that concern, saying evaluator funding should eventually come from pooled or government sources, while stressing that it will continue developing and releasing frontier models alongside the new testing regime. Read more <\/span>here<\/span> from the <\/span>Financial Times<\/span><\/i>.<\/span><\/p>\nEYE ON AI RESEARCH<\/h3>\n
AI CALENDAR<\/h3>\n
\n\n
\n \n Oct. 1:\u00a0<\/strong>Fortune AIQ conference, New York. Apply\u00a0here\u00a0to attend.<\/div>\nNov. 16-17<\/strong>: Fortune 500 Innovation Forum, Detroit. Apply\u00a0here to attend.<\/div>\nDec. 7-8:\u00a0<\/strong>Fortune Brainstorm\u00a0AI<\/span>, San Francisco. Apply\u00a0here\u00a0to attend.<\/div>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\nBRAIN FOOD<\/h3>\n
<\/span>
<\/span>So maybe today\u2019s distressed AI researchers should take a lesson from the Cold War nuclear anxieties, too. Perhaps there is refuge in dark humor. After all, the subtitle to Stanley Kubrick\u2019s 1964 satirical masterpiece <\/span>Dr. Strangelove <\/span><\/i>is \u201chow I stopped worrying and learned to love the bomb.\u201d <\/span><\/p>\n<\/div>\n