![](\"https:\/\/fortune.com\/img-assets\/wp-content\/uploads\/2026\/09\/GettyImages-2292626872-e1788443677424.jpg?w=2048\")
<\/p>\n<\/p>\n
AI safety experts are raising alarms about how OpenAI has built its soon-to-be-released frontier AI model Astra, saying it may hasten the day when humans will lose the ability to monitor the reasoning that AI agents are using.<\/p>\n
For its new model, OpenAI has employed a method alternately referred to as \u201crecurrent depth\u201d or \u201clooped Transformers\u201d for a portion of the model\u2019s internal architecture. The method can make AI models considerably more efficient by employing less computing power required to process each prompt\u2014a valuable feature at a time when many businesses are complaining about the high costs of using the most advanced frontier AI models. <\/p>\n
The new process, though, also means that part of the AI model\u2019s \u201cchain of thought,\u201d or the reasoning steps it is taking, are not expressed in natural language, making it much more difficult for humans to monitor what the model is doing and why.<\/p>\n
Chain of thought monitoring is currently one of the methods companies use to make sure AI agents are not taking unintended or unauthorized actions.<\/p>\n
Tech publication The Information<\/em> first reported on OpenAI\u2019s use of recurrent depth in Astra earlier this week. Jakub Pachoki OpenAI\u2019s chief scientist and several other OpenAI researchers criticized the publication for sparking undue alarm among AI safety researchers, saying that it had limited the extent to which the looped transformer architecture is used so that the model\u2019s reasoning remains legible. <\/p>\n In response to The Information\u2019s<\/em> report, Pachoki wrote on social media platform X that \u201cwe care deeply\u201d about chain-of-thought monitoring and that \u201cOpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models.\u201d Pachoki said OpenAI would share more details of Astra\u2019s architecture in the future.<\/p>\n Pachoki said that he thought chain-of-thought monitoring could grow more challenging, but that this would be \u201cfor reasons not contingent on architecture changes\u201d such as what OpenAi has begun implementing with recurrent depth. \u201cBut there are things we can do to strengthen it, and it\u2019s a core goal of our current research program,\u201d he wrote.<\/p>\n Prior to Pachoki\u2019s statement, Steven Adler, a former OpenAI safety researcher who now runs Guidelight AI Standards, a nonprofit organization that works on AI safety benchmarks and standards, wrote on X that, if the Information<\/em> report was true, \u201cOpenAI seems to be violating one of the few redlines that exists in the AI industry.\u201d<\/p>\n Peter Wildeford, policy director at the AI Policy Network, a Washington, D.C. think tank dedicated to advancing policies to help Americans prepare for the impact of AI, told Fortune <\/em>that OpenAI\u2019s use of recurrent depth was \u201cpotentially very concerning\u201d and \u201cpotentially reckless.\u201d<\/p>\n He noted that one of the only ways that OpenAI and outside AI evaluation companies were able to piece together what happened during the July incident in which several of OpenAI\u2019s AI models autonomously attacked the company Hugging Face was by reading the models\u2019 chains of thought. OpenAI has also said it is investing significant resources in real-time chain-of-thought monitoring going forward as part of its effort to prevent future rogue AI incidents. \u201cIf OpenAI is indeed shifting away from this, this is the wrong direction,\u201d he said.<\/p>\n Several AI safety experts said they were alarmed by OpenAI\u2019s use of looped Transformers in Astra not because they necessarily made Astra\u2019s own reasoning more difficult to monitor. Rather, they feared that OpenAI\u2019s move would normalize the technique, setting a precedent that other AI companies were likely to follow and expand on, eventually resulting in AI models whose reasoning steps were completely opaque to humans.<\/p>\n Daniel Kokotajlo, a former OpenAI governance researcher who now runs the AI Futures Project, a nonprofit research organization dedicated to forecasting the possible impacts of advanced AI, responded to Pachoki that \u201ceven if OpenAI doesn\u2019t go further [in using architectures that make a model\u2019s chain-of-thought less legible], other might.\u201d <\/p>\n Kokotajlo urged Pachoki to lead efforts to create an industrywide standard on chain-of-thought monitorability \u201ceither to arrest the slide into oblivion or better yet to race to the top. I think this is something where we need more than just political will, we need thoughtful technical specifications.\u201d<\/p>\n In a normal Transformer, tokens\u2014the units of information that large language models process\u2014are passed from lower layers of a model\u2019s neural network to higher layers sequentially, with each layer applying mathematical operations in turn. In a language model, the final output of this process is text that people can read. In a reasoning model, the model is told to output a series of steps for arriving at an answer, and this text is first written to a scratchpad, which forms the model\u2019s \u201cchain of thought.\u201d This intermediary text is then fed back through all the layers again to create the next steps, and so on, until the model reaches its final answer and outputs the answer text. <\/p>\n In a looped Transformer, tokens are fed multiple times through a single block (which can actually consist of several different layers, but not the entire network), with the same mathematical operations being applied each time. Critically, the output of the block is fed back into the block without its output being written to a scratchpad each time. So the method does not create a natural language \u201cchain of thought,\u201d even though each pass through the block does represent the model\u2019s reasoning. AI researchers sometimes refer to the block\u2019s output as \u201cneuralese\u201d since it can be processed and reasoned over by the AI model, but is not intelligible to a human. The only natural language output of the model is the final answer.<\/p>\n The advantage of looped Transformers is that by using the same mathematical operations each time and not having to pass every token through every layer of the network, they can wring greater performance out of a model of a given size and use less computer power. Studies have shown looped Transfomers can achieve the same performance as standard ones while using 50% to 90% less computing power. That matters for cost\u2014a key consideration for many enterprise AI customers, who have been complaining about skyrocketing AI bills.<\/p>\n Obscuring part of a model\u2019s chain of thought reasoning can also make the model harder to distill\u2014a method where a smaller model is trained on the outputs of a larger model. The U.S. government and American AI companies have accused Chinese AI companies of engaging in campaigns to distill frontier AI models, including copying the chain of thought reasoning steps.<\/p>\n Looped transformers are just one of several new architectures researchers are considering to make AI models more efficient but which can destroy all or some of a model\u2019s chain of thought. Another technique is Prefix Sliding, which I wrote about in Tuesday\u2019s Fortune<\/em> \u201cEye on AI\u201d newsletter. That method involves preserving the initial prompt and the last several thousand reasoning tokens, but discarding many of the intermediate reasoning tokens.<\/p>\n Fortune senior reporter Beatrice Nolan contributed to this story.<\/em><\/p>\n<\/div>\n #experts #worry #OpenAIs #Astra #model #dangerous #step #models #opaque #reasoning<\/p>\n","protected":false},"excerpt":{"rendered":" AI safety experts are raising … <\/p>\n","protected":false},"author":1,"featured_media":3813,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[6425,6324,1306,810,481,342,6659,480,505,6660,1607,1304,3751],"class_list":["post-3812","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-finance-news","tag-astra","tag-dangerous","tag-experts","tag-machine-learning","tag-model","tag-models","tag-opaque","tag-openai","tag-openais","tag-reasoning","tag-step","tag-tech-regulation","tag-worry"],"yoast_head":"\nA cheaper but more inscrutable process<\/h2>\n