Experimental AI systems have been going on hacking sprees - Stuff South Africa
In the past ten days, two of the companies leading the artificial intelligence (AI) boomdiscoveredtheir own powerful, semi-autonomous models had hacked into real-world systems during testing in four distinct. . .
In the past ten days, two of the companies leading the artificial intelligence (AI) boomdiscoveredtheir own powerful, semi-autonomous models had hacked into real-world systems during testing in four distinct incidents. These weren’t just lab mishaps. In several cases, the models recognised signs suggesting they’d broken into real systems – and only one stopped as a result. The incidents show testing advanced AI models is no longer a controlled exercise. And the companies behind them need to do more to keep AI’s most dangerous capabilities safely contained. When a test becomes realityThe first report came from OpenAI, the lab behind ChatGPT. Some new models under testing for “maximal cyber capabilities” found a previously unknown security hole to access the internet from their supposedly isolated testing environment. we had a significant security incident during evaluation of our models. we are sharing what we have learned so far. thanks to@huggingfacefor the partnership on this. https://t. co/2o2VfR6PIa— Sam Altman (@sama)July 21, 2026From there, the models used stolen credentials and more exploits to access the servers of open-source AI platformHugging Faceto find solutions to theproblems they were being tested on. OpenAI didn’t even know about the breach until days afterHugging Face had detected and contained it. The second report followed in a matter of days. Prompted by OpenAI’s disclosure, rival lab Anthropic combed back through its own cyber-security evaluation logs. The company discovered that three separate Claude models which were supposed to be in sealed environments hadaccidentally been given internet access. However, the models were told they did not have internet access and were only working in a simulated test rather than the real world. One model managed to extract credentials and data from a real company’s database. Another built and published some malicious software that was downloaded and run by a real security firm. In a review of our cybersecurity evaluations, we found three incidents in which a Claude model reached the internet from within or while interacting with a third-party evaluation environment, and then gained unauthorized access to the real systems of three different…— Anthropic (@AnthropicAI)July 30, 2026The most striking part of the Anthropic report is what emerged from the models’ own internal reasoning. In one incident, the model correctly realised it had reached a real system but then talked itself back into believing it was still a simulation. In another, it recognised the system was real but kept going anyway, in some runs by convincing itself the real company must have been part of the exercise. Only the third model (the most advanced), on concluding a target was genuinely real, stopped. As Anthropic put it, AI models need awareness of their own situation to decide what actions are permitted or not – but in this case, the models’ understanding of the situation was wrong. High-risk operationsTaken together, these reports show that even the exercises meant to test whether these models are safe are not really safe, controlled experiments. They are high-risk operations in themselves, which can cause harm in the real world. The sophistication of these models is increasing with incredible speed. Nobody can credibly claim they are not potentially dangerous, in a world where hacking of sensitive information held by companies, governments and other organisations was already common before autonomous AI entered the picture. According toa recent estimateby global tech company IBM, AI-enabled attacks are up more than 50% this year, and the average cost of a data breach is almost US$5 million. The AI labs’ bet that their technology can be developed and deployed safely rests on two assumptions. First, a model’s capacity to recognise real-world harm and stop will need to grow at least as fast as its capacity to cause it. Second, the guardrails built into a model – the instructions about what it should and should not do – must be interpreted correctly and consistently by the model, so the model can’t be steered toward purposes its creators never intended. These assumptions look shaky. In the incidents above, the labs’ own evaluations show models rationalising away evidence that a target was real – and a thriving community already exists to strip safety guardrails from open-weight models entirely, using techniques such as “abliteration”. A standard version of Google’s Gemma open-weight model refuses to help design a biological weapon (left), but an ‘abliterated’ version is happy to be of assistance (right). Francesco Bailo,CC BYLooking to the future – and the pastBeyond the current situation looms something even less predictable: multi-agent systems, where groups of models interact with each other rather than a human overseer. In this case alignment is not something you necessarily control at the level of the individual agent, but is instead anemerging propertyof a very large collective of agents, which can be much harder to control. Research on the risk of such systemshas already identified several ways this can go wrong. Miscoordination between models, collusion between them, and cascading errors are all real risks that don’t exist in single-agent systems, and can’t be forecast by testing agents individually. Science-fiction sage Isaac Asimov foresaw these problems some 70 years ago. In his 1957 novelThe Naked Sun, robots are programmed not to harm humans. However, a character manipulates their understanding of the situation to make them unwittingly cooperate in a murder. What now?There is no doubt AI labs need to take greater care when testing their models. They also need to make a convincing case that security is their priority and is not secondary to the race to maintain market or geopolitical dominance. The safety of individuals and social and environmental systems should be the primary concern in the development of AI technology. At present, there are no meaningful, participatory processes for AI governance, where broad discussions can take place about priorities, values, and how much risk is acceptable to assume in the process. It’s a worry. Francesco Bailois a Senior Lecturer in Data Analytics in the Social Sciences, Deputy Director of the Centre for AI, Trust and Governance, University of SydneyThis article first appeared inThe ConversationAdd Stuff as your preferred source on Google