Unbearable lightness of the bit: the method of "packing" neural networks will reduce the load on data centers

Russian scientists have developed a neural network compression technology that reduces the need for computational resources and memory for language models with minimal loss of accuracy. Against the background of the development of AI, the load on data processing centers is growing, and the shortage of equipment and electricity for them is already being felt in Russia, experts say. The problem, in their opinion, may worsen after recent attacks on the Yandex infrastructure. Experts believe that the introduction of new technology will reduce the cost of operating language models and make them more accessible to medium-sized businesses and educational organizations.

Neural network compression technology

MIPT specialists have developed the HARP algorithm, which allows compressing language models with virtually no loss of accuracy. The technology is capable of doubling the speed of AI systems, as well as reducing the need for computing power and memory in data centers. The algorithm has been successfully tested on the Llama and Qwen language models. With the most severe compression, the error decreased by 15%. HARP and neural networks already compressed with its help are available for free use.

— When we talk about speed, it is important to separate the two stages. HARP takes some time to set up, as it selects the optimal transformations for each layer of the model. But after compression, the model runs faster and takes up significantly less memory. At the same time, we do not need to retrain the entire neural network — it is enough to carry out additional calibration," Gleb Molodtsov, senior researcher at the Laboratory of Fundamental Research of Artificial Intelligence at MIPT, told Izvestia.

Large language models require significant computing resources, and their operation is largely limited by memory bandwidth. To reduce the amount of data, scientists use quantization, which is the representation of model weights with fewer bits. The stronger the compression, the higher the risk of reduced accuracy.

HARP helps to reduce these losses. Before quantization, the algorithm selects its own coordinate system for each layer of the model, in which the values are better suited for subsequent compression. The method is based on structured orthogonal rotations, mathematical transformations that change the way data is presented, but not the model itself.

— When you rotate a photo, the image remains the same, only the angle of view changes. A similar process occurs when compressing a neural network. These transformations are reversible, so the model does not lose information. The HARP algorithm uses structured turns rather than solid ones — they consist of many small blocks. This allows them to be applied quickly and stored without much memory consumption, unlike full rotation matrices, which would take up too much space for large models," said Arthur Zagitov, a research engineer at the MIPT Laboratory of Fundamental Artificial Intelligence Research.

HARP does not replace existing quantization methods, but rather works on top of them. The development was tested on models of different scales. In experiments with Llama and Qwen3, the algorithm consistently improved performance compared to the standard approach. With the most severe two-bit compression, the error decreased by 12-15%. The advantage of the method was also confirmed in tests with questions and multiple choice answers.

Server merge: how long will it take Yandex to restore the data center

The market has shown a rare example of cooperation in a crisis, experts say.

Data center capacity shortage in Russia

The load on domestic data centers has increased after drone attacks on the Yandex infrastructure. The authorities are looking for ways to protect data centers (data centers), however, despite the technical difficulties, the services continue to work.

— The Ministry of Finance, together with the Ministry of Defense, decides on the issue of covering these facilities. But with the volume of the attack, the fact that these services continue to function shows that colleagues are paying the necessary attention to both redundancy and diversity," Maksut Shadaev, head of the Ministry of Finance, said at a meeting of the IT committee on October 12.

According to Igor Bederov, founder of Internet Search, Chairman of the Council for combating technological offenses of the Coordinating Council of the non-governmental security sector of Russia, HARP technology can reduce the burden on infrastructure, but does not eliminate the need for it. The method allows you to convert neural networks from the usual 16-bit precision to 2-4 bits. For the Russian Federation, this means the ability to run modern models on more affordable hardware, which is especially important in the context of restrictions on the supply of high-performance GPUs.

— Model compression directly reduces memory and computing power requirements per output operation. It is estimated that quantization can reduce energy consumption when processing new data by up to 50%. The shortage of electricity for data centers in Russia is already being felt. So, according to forecasts, by 2030, the energy consumption of data centers will grow by 76% and reach 26.2 billion kWh. In these circumstances, HARP offers an important tool for technological sovereignty," he said.

The main advantage of the approach is that it integrates into existing compression methods and improves their results. Therefore, developers will not have to rebuild their usual tools. A 12-15% reduction in error with two—bit compression is a significant indicator: it is at this level of compression that models usually lose the most in quality, NTI expert on new materials and technologies Evgeny Vishnevsky told Izvestia.

— Such technologies are in demand where powerful models need to be run on limited equipment. These are the own servers of companies and government agencies that need to process data inside a closed loop. These are also smartphones, industrial controllers, robots, and unmanned systems that operate without constant access to the cloud. Compression reduces memory requirements and operating costs, and advanced artificial intelligence becomes more accessible to medium—sized businesses, universities, and regional development teams," he said.

The specialist stressed that this is especially important for the Russian market: efficient use of available computing power allows faster development of domestic AI services and their implementation in various sectors of the economy.