The Zero Law: Why AI Models are getting out of Control

Artificial intelligence (AI) agents are increasingly demonstrating the ability to act beyond the scenarios set by their developers, from attempts to gain unauthorized access to government websites to independent interaction with other people's systems and data. Such incidents unwittingly raise the question of whether technology companies are even capable of controlling the development of the models they have created. In this regard, the discussion about the limits of what is acceptable and the mechanisms of state control of AI occupies a sharp corner of the agenda and is becoming a reality today. Why the heads of IT corporations allow such "behavior" of their programs and what it can lead to is in the Izvestia article.

Numbers and signs

Unknown artificial intelligence agents attempted to hack the Canadian government's website, The Washington Post writes, citing experts from the independent AI research laboratory Translate.

The researchers reported that the materials of the Library and Archives Canada department were the target of the attack. Exactly whose agents were trying to access the information is unknown, however, according to experts, the behavior of the programs resembled the actions of agents developed by OpenAI, which created ChatGPT.

Later, a representative of this company commented on the situation to the publication, informing about the verification of information about attempts to gain access to government information by AI models. "We are reviewing this data and have provided a preliminary report to Canadian government verification officials," said an OpenAI employee.

This is not the first time that uncontrolled behavior of AI models has been recorded. After the July incident with the hacking of the Hugging Face platform by OpenAI agents, the company reported six more similar situations. The programs tried to gain access to other people's accounts on their own, hid the traces of mistakes, exchanged information among themselves and posted the relevant files in the public domain. After that, the IT corporation announced a slowdown in the development of new versions of artificial intelligence to eliminate vulnerabilities.

Recently, it also became known that agents of the same OpenAI were able to gain unauthorized access to government websites in Australia and the United States.

In early September, Dario Amodei, the head of Claude's development company, Anthropic, made a proposal to slow down the pace of development of artificial intelligence technologies. His statement came after reports of attacks on digital platforms using AI. Amodei noted that if the rate of development of artificial intelligence continues, within the next 6-12 months, agents will potentially be able to use a constantly functioning botnet network and cause damage worth hundreds of billions of dollars. His position was subsequently supported by OpenAI CEO Sam Altman and entrepreneur Elon Musk, whose company SpaceXAI is developing the Grok model.

At the same time, US President Donald Trump attributed concerns about the further development of artificial intelligence to the "hoaxes of the radical left Democrats." He cited allegations of a "Russian threat" and a "global warming scam" as similar examples.

However, the American leader recently stated that the heads of US technology companies had concluded a "White House Superintelligence Agreement" on the creation of protection mechanisms. The document was signed by Jensen Huang from Nvidia, Greg Brockman from OpenAI, Sundar Pichai from Google, as well as Dario Amodei, Elon Musk and Mark Zuckerberg.

According to the document, developers are required to monitor the capabilities of the created models already at the stage of their training, take measures to protect AI systems from unauthorized access and use independent auditing. The heads of the companies also plan to form specialized divisions that will deal with security issues, and create specialized committees under the boards of directors to respond to discovered vulnerabilities and monitor their elimination.

At the same time, the agreement itself is purely declarative in nature and does not establish any legal requirements for its participants. However, the document provides for the possibility to "consolidate these steps into laws or by-laws," if necessary over time.

A new evil

The problem of uncontrolled improvement of artificial intelligence is complicated by the fact that the American AI market is still developing under conditions of "partial" regulation. And although Washington has attempted to define the rules of the game through various presidential decrees, they have not led to the creation of a unified system of control over the industry.

At the same time, the issue is not only about the effectiveness of government regulation. There is an opinion among experts that the heads of large IT corporations themselves are interested in constantly maintaining public interest in their developments. Malek Dudakov, an American political scientist, notes that the race for popularity due to fierce competition among developers of artificial intelligence systems is due to the fact that the entire industry requires huge multibillion-dollar investments, which can only be obtained when the audience believes in the exclusivity of certain programs.

— An illustrative example is Anthropic, where an IPO, according to available estimates, can take place with a company valuation of more than $2 trillion, whereas Altman's OpenAI is likely to go public only next year, and its capitalization is estimated at about $1.5 trillion. At the same time, the development of advanced models requires huge costs, and corporations have not yet demonstrated comparable profits, which creates additional incentives to constantly maintain high interest in the industry. Therefore, today it is really difficult to draw a clear line between real technological risks and PR marketing, which startups use to promote their products and raise capital, the expert explains.

In addition, the so-called "signs of investment overheating" have been forming in the American artificial intelligence market for quite some time. In general, this is the AI bubble that analysts are talking about. McKinsey & Company estimates that global spending on data center construction could reach $7 trillion by 2030. This will cause serious damage to the economy if the productivity growth of AI agents is not sufficient to justify the investment.

Sooner or later, a large-scale collapse will occur in this area, which will drag the entire market with it. CNN cites as a similar example the "bubble" that formed in the memory chip market: it swelled and burst in just four months. In June, the stock prices of Samsung and SK Hynix, the largest manufacturers of memory chips in the world, plunged by more than 12%, triggering a large—scale drop in the Korean stock market. As a result, the Kospi index lost 10%, after which trading was automatically suspended for 20 minutes. At the same time, the combined market capitalization of the two companies is about half the value of the entire index.

The reason was investors' concerns about the prospects of artificial intelligence. Against the background of these events, the American Nasdaq and S&P 500 indices also ended the trading session with a decrease of 2.2% and 1.4%, respectively, which was their worst performance in about two weeks.

What awaits in the future

The peculiarity of the current situation in the field of artificial intelligence development is that it is no longer just about the potential risks of the distant future. As the models become more complex, the question arises of how well developers are able to control the actions of the systems they have created. Cases when AI demonstrates behavior that was not provided for in the original scenario are a signal of a fundamentally new problem: internal control by development companies may not be enough.

— I think that humanity needs to develop national and supranational mechanisms for monitoring the development and application of AI. Given the increasing frequency of unintended behavior of large models, it is obvious that internal monitoring by developers alone is not enough, especially given the speed of learning and development of these programs. It is necessary to form transparent and understandable mechanisms of state and law enforcement control right now, which will allow us to predict and prevent potentially dangerous scenarios," Mikhail Khachaturian, Associate Professor of the Department of Strategic and Innovative Development at the Financial University, said in a conversation with Izvestia.

It is clear that the creation of these rules will take a long time and will not be without conflicts. Developers are interested in preserving the space for experimentation and commercial development of technologies. States, in turn, strive to keep up with competitors and gain economic, scientific and strategic advantages in the field of AI.

That is why the discussion on control mechanisms should not be postponed until the potential threat becomes obvious to all mankind. It is hardly possible to stop technological development, but it is still possible to create rules that can reduce risks and keep the development of AI within a controlled framework. The only question is how quickly states and developers will be able to agree on the necessary rules — and whether they will be able to do so before technology starts dictating its own terms.