Hidden noises: An AI artist will take objects from a photo without unnecessary details

Russian scientists have developed algorithms that help AI image generators focus on objects and "remember" them. The objects selected in this way can be easily transferred to other backgrounds and control the creation of new images. The development will expand the capabilities of image generation servers and help make advanced AI design on laptops and smartphones. At the same time, foreign technologies are not always available, experts noted in an interview with Izvestia, so solutions aimed at developing domestic neural networks need to be implemented.

How AI distinguishes an object from the background

Specialists from the Moscow Institute of Physics and Technology, the Higher School of Economics and the V.P. Ivannikov Institute of System Programming of the Russian Academy of Sciences participated in the development. According to experts, popular modern generative neural networks are based on diffusion algorithms. When processing images, they add noise — random interference that makes the picture less clear. When this operation is repeated many times, the image loses details and gradually turns into a set of dots.

Then the program performs the reverse process — removes the interference step by step and restores the image. At the same time, the algorithms add elements from the user's text query. The result is a new image. Such methods work well when learning from multiple photographs. If we are talking about a single image, the model often reproduces not only the desired object, but also other details of the original image.

— The problem is that with a small number of examples, the model learns not the essence of the object, but all the random details. Ask her to draw a cat in space, and she will draw the animal on the same sofa, but with a drawn space around it. It is extremely difficult to deal with this with a minimum of data," Maxim Fofanov, one of the developers, a graduate student at the Department of Algorithms and Programming Technologies at MIPT, told Izvestia.

According to him, the new method is based on the fact that small noises take into account the subtle details of the image, such as the texture of the coat, the shape of the nose or the color of the eyes, while large ones reflect the overall layout of the elements.

During the development, scientists mathematically shifted the probability distribution from which the program chooses certain steps during training in favor of low noise. As a result, the model began to remember more details related to the identity of the object.

— The method uses practically no additional computing resources compared to standard image generation tools. It does not require any additional network passes, new model parameters, or additional memory. Only the distribution from which the steps are selected and the weighting factor change," said Maxim Fofanov.

At the same time, the study tested different operating modes of the program, he said. It turned out that the technology increases editability — the ability of a neural network to respond to new text queries and place an object in unexpected contexts. At the same time, the image playback quality is preserved.

As the expert noted, the proposed code can be integrated into any existing system. The implementation of the development will allow advanced AI design to be performed on laptops and smartphones.

Unbearable lightness of the bit: the method of "packing" neural networks will reduce the load on data centers

The new technology will help overcome the shortage of equipment and electricity for data centers

How to transfer a face to a new photo without shadows

— The trouble with services like Nano Banana is that when working with a single source, the so-called context memorization occurs — the object "merges" with the environment. As a result, for example, unwanted shadows are transferred to the new background along with the face. Conversely, when trying to get rid of these effects, algorithms erase the individual features of the object. Development through mathematically verified focusing teaches the model to highlight subtle details. This increases the accuracy and flexibility of editing," the editor—in-chief of IT-World explained to Izvestia.RU and IT Expert magazine Andrey Vinogradov.

Foreign developers have made great progress in these areas, the expert noted. However, the race for leaders is a disastrous path. Therefore, Russian specialists need to look for solutions that will allow them to create their own competitive products.

— The value of development is moving from simple image generation to image creation management. When the result is determined by a specific human intention, the magic of creativity arises," said Anna Pyataeva, head of the Artificial Intelligence Center at Siberian Federal University.

According to her, one of the main advantages of such solutions is accessibility. A small company will be able to create advertising materials without huge budgets, a designer can test dozens of ideas in minutes, and an ordinary person can realize creative ideas that previously lacked the skills or money.

At the same time, tools that allow you to quickly and easily create new images make it possible to fabricate fakes and fake materials. They can harm people and organizations, said Andrei Goryachev, an information security expert and head of the Security Code press service.

— Similar things are treated with similar things: the same neural networks are used to detect signs of AI generation. They analyze metadata, look for artifacts left by models, identify unnatural movements, inconsistencies in human anatomy, anomalies in landscapes and interiors. Such tools, built directly into the services, will also improve the final products," he commented.

Vera Soboleva, a researcher at the Controlled Generative AI group at the FusionBrain laboratory at the AI Institute, noted that modern AI creators already allow you to dictate an idea to an agent, add visual examples and ask him to create an idea for an image or a script for a video. According to her, technology also makes it possible to accurately transfer various objects, people, and styles to new scenes and environments. However, in practice, it is rarely possible to control the generation of "voice only". It usually requires several stages of refinement. That's why it's still a craft, not a "magic button."

In the future, technology will probably make it possible to directly turn thoughts into images or video sequences, she believes. The main difficulty lies not in generating data, but in reading it: AI is not yet able to decode brain signals. Nevertheless, the first steps have already been taken. For example, MRI specialists reconstruct approximate images that a person has seen, and neural interfaces help paralyzed patients to "pronounce" words.