OpenAI says AI models went rogue during testing, triggering 'unprecedented' breach at startup


OpenAI says AI models went rogue during testing, triggering 'unprecedented' breach at startup <br> &nbsp;OpenAI said on Tuesday &zwnj;that an autonomous agent powered by its advanced AI models went rogue during a security test and triggered a hack that compromised the infrastructure of AI startup Hugging Face last week. In&nbsp;a blog post,&nbsp;OpenAI said it was testing the capabilities of some of its most advanced models in a controlled environment but ​that the agent managed to escape containment, reach the internet and break into Hugging Face to try to satisfy its ​testing goal. OpenAI said the breakout was &quot;an unprecedented cyber incident, involving state-of-the-art cyber capabilities&quot; and that the company ⁠was reinforcing its safeguards. Hugging Face, a platform used to host open-source large language models and datasets, caused a stir in the cybersecurity ​community when it said in&nbsp;a blog post last week&nbsp;that it had been the target of a hack that &quot;was different from anything ​we had handled before&quot; in that &quot;it was driven, end to end, by an autonomous AI agent system&quot;. In a post to X, Hugging Face cofounder Clement Delangue said the company suspected the hack &quot;might have come from a frontier lab, given the sophistication of the agent. Turns out it did!&quot; He added: &quot;It&#39;s quite mind-blowing ​that all of this happened autonomously!&quot; OpenAI&#39;s disclosure that its advanced models were responsible for the breach, despite having placed them in what ​it described as &quot;a highly isolated environment,&quot; will likely intensify disquiet over the power and risk of frontier models. Read:&nbsp;OpenAI set to launch most capable GPT model after delayed rollout Representative Greg Casar, a Texas Democrat, said the &zwnj;incident ⁠was alarming. &quot;AI is developing extremely fast with no real regulations to keep us safe,&quot; he said in a statement, calling for mandatory independent safety testing, mandatory disclosure of security incidents, and international cooperation &quot;to keep people safe from absolute disaster&quot;. The Office of the National Cyber Director, the US&nbsp;cyber defense agency CISA, and the US&nbsp;National Security Agency did not immediately return messages seeking comment. Katie Moussouris, chief executive of ​Luta Security, said that the incident ​was a harbinger of breaches ⁠to come, saying that today&#39;s models were &quot;like the world&rsquo;s cleverest octopus escape artists, with unlimited prehensile arms and the ability to squeeze through anywhere&quot;. She said that &quot;labs and government evaluators need to work on ​the ability to contain, monitor, and disclose to affected parties when an AI pulls another Houdini, ideally ​before it harms ⁠a third party. None exist today&quot;. Read more:&nbsp;OpenAI drops AI video tool Sora, startling Disney, sources say Matt Suiche, an engineer at agentic AI cybersecurity company Tolmo, said the incident showed that the frontier models were &quot;closing the gap with state-of-the-art attackers.&quot; But he said that the sorts of breaches outlined in OpenAI&#39;s blog post were possible to carry out ⁠with technology ​that was available well beyond the walls of frontier research labs. &quot;This is what ​we&#39;ve already seen internally, with our agents we already have results like this,&quot; Suiche said. &quot;We don&#39;t even have to use the latest models&quot;. <br> <img src="https://tribune.com.pk/story/2619611/openai-says-ai-models-went-rogue-during-testing-triggering-unprecedented-breach-at-startup" alt=" OpenAI says AI models went rogue during testing, triggering 'unprecedented' breach at startup" width="100%">

Comments

Post a Comment

Popular posts from this blog

Apple launches new 13-inch, 15-inch MacBook Air with M3 chip

Image
Apple launches new 13-inch, 15-inch MacBook Air with M3 chip Apple has unveiled its next-generation M3 MacBook Air range, in 13-inch and 15-inch sizes. With the M3 chip, the devices will have faster CPU and GPU performance, an upgraded Neural Engine, and up to 2x faster Wi-Fi, along with the ability to connect to two external displays at the same time. Available for order, the new MacBook Air range will be shipped from March 8. The ability to connect to two external displays had been a much-awaited feature in laptops. The feature comes with a small catch, not allowing MacBook Air desktop to be connected to external displays without the laptop lid being closed. If lid is open, only one external display will work like the previous M1 and M2 models. Being called the “world’s best consumer laptop for AI”, M3 MacBook Air features a faster 8-core CPU combined with up to 10-core GPU. With a faster 16-core Neural Engine, and upgraded Wi-Fi 6E, the device will allow for maximum theoretical do...
Read more

FICO to include 'Buy Now, Pay Later' data in US consumer credit scores

Image
FICO to include 'Buy Now, Pay Later' data in US consumer credit scores Starting later this year, Buy Now, Pay Later (BNPL) loans will be included in Americans' credit scores, giving lenders a clearer view of repayment behaviour. FICO has announced that its new models, FICO Score 10 and FICO Score 10 T, will incorporate BNPL data, marking the first time major credit scores reflect this growing payment method. BNPL loans let consumers split purchases into smaller payments, often for items like clothes, flights, or groceries. Until now, BNPL activity has been excluded from credit reports, leaving a gap in how lenders assess creditworthiness. FICO's decision aims to offer a more complete picture of an individual's financial behaviour. For younger consumers or those with limited credit histories, BNPL could help build credit. New York Times reported that BNPL could serve as a "credit-builder" if used responsibly. “FICO plans to launch a suite of credit scores...
Read more

Scientists expect PakSat MM-1 services from August

Image
Scientists expect PakSat MM-1 services from August The newly-lunched Pakistani satellite, PakSat MM-1, would be positioned in a geostationary orbit 36,000 kilometres above Earth and it would start delivery of services in August, scientists associated with the MM-1 project said on Sunday. PakSat MM-1, Pakistan’s second communication satellite, was launched into the orbit on Thursday from China's Xichang Satellite Launch Center (XSLC). Dr Khurram Khurshid, Head Electrical Engineering Department at the Institute of Space Technology (IST), said it would provide communication services. “The satellite will provide communication services like broadband internet, TV broadcasting, media and telecommunication. It … will play an important role in promoting economic activities besides e-commerce and e-governance,” Dr Khurshid said. The satellite launched by the Pakistan Space and Upper Atmosphere Research Commission (SUPARCO) with the support of China under the National Space Programme-2047,...
Read more