Cognition of machines: robots have been taught to "feel" space like a human

Russian scientists have presented three developments that will allow machines not only to recognize objects, but also to navigate in a dynamic environment, remember changes and act based on context. In the future, robots will be able to perform complex tasks using voice commands and less depend on predefined scenarios, from navigating unfamiliar rooms to working in production and logistics. The research is aimed at "sensitizing" robots and developing an embodied AI approach in which a neural network not only processes digital data, but also interacts with the physical world. How new technologies will help machines navigate better in space — in the material of Izvestia.

Three levels of spatial intelligence of robots

ITMO's Laboratory of Embodied Intelligence and Robotics (VIR) has presented a series of developments aimed at "sensitizing" machines. They will allow robots to interact more effectively with the physical world: perceive space, recognize objects, track movement and changes in the environment. Together, these solutions bring machines closer to independent and meaningful interaction with physical reality.

When a person enters a room, they immediately understand where objects are, notice obstacles and changes around them. For a robot, this is a much more difficult task: it needs not only to recognize an object, but also to determine its position and movement, as well as its connection with the surrounding space, in order to navigate and act in a changing environment.

Three new lab papers, prepared in collaboration with partners and presented at the ECCV 2026 International Conference on Computer Vision, describe the components of a unified spatial intelligence system. They solve different but interrelated tasks of "sensitizing" robots.

The first development, FastR-4DGS, is responsible for dynamics: it helps the robot to present a scene not as a still frame, but as a three-dimensional model that changes over time. The second model, QUORUM, makes this representation semantically understandable and consistent when viewed from different angles. The third, Obj3R, forms the object spatial memory and allows you to account for changes in an already built map.

— FastR-4DGS is suitable for reconstructing dynamic scenes, including in augmented reality systems, where it is important to quickly and realistically display a space with moving objects. In addition, the technology allows you to create synthetic data for learning and testing algorithms, for example, strategies for controlling robotic manipulators or autonomous transport systems. Since the model takes into account not only the geometry of the scene, but also changes in time, such data can more accurately reproduce real situations: the movement of people, vehicles, objects and changes in the environment," said Sergey Kolyubin, head of the ITMO Laboratory of Embodied Intelligence and Robotics, winner of the Yandex ML Prize 2026 in the nomination "Heads of Educational Programs".

The QUORUM model helps the robot understand what is around it, even if the same object looks different from different angles. In traditional systems, the machine can recognize objects either after the fact, after the shooting is completed, or according to predefined categories by the developers. QUORUM connects the visual features of an object — shape, color, and structure —with its spatial representation. Due to this, the system retains a holistic view of the object, regardless of lighting, its position and viewing angle.

Obj3R allows the robot to memorize dynamic scenes. An ordinary three—dimensional map is a set of geometric points - a similar display can be seen, for example, in the application of a robot vacuum cleaner. It helps to avoid obstacles, but does not provide a full understanding of the environment. The new technology takes spatial memory to the object level: the system learns to recognize not just obstacles, but specific objects, such as a table or chair, and track their movement. Thanks to this, the robot does not need to build a room map anew every time: it can update only those parts of the scene where changes have occurred.

The route has been adjusted: the new algorithm will reduce the probability of accidents in robots and drones by 17 times

Innovative technology will help drones not to "hover" in front of sudden obstacles on the roads.

Where will "sensitive" robots find their use?

In combination with language models and navigation, the object map will allow the robot to follow instructions that take into account changes in the environment: "find the box at the door that used to be there," "drive up to the table where the book used to be," or "take the object that is to the left of the laptop that was brought yesterday."

— Together, all three developments enhance the spatial intelligence of autonomous semantic agents. Semantic representations will allow you to communicate with the machine in natural language. Another practical scenario is the creation of realistic synthetic data. This allows for faster and cheaper training and initial validation of solutions before transferring them to real robots, where experiments require equipment, time, and additional security measures," said Sergey Kolyubin.

In fact, we are talking about the formation of a machine's own model of the physical world, Pavel Frolov, ROBBO's founder and Neuronet market expert, told Izvestia. The more accurately and faster the robot is able to build and update such a model, the less it depends on predefined scenarios. This is especially important for service robotics, unmanned systems, manufacturing, logistics, and medicine — wherever the environment is constantly changing.

— ITMO's approach seems promising precisely because the task is solved in a comprehensive manner: simultaneously, the movement of objects, understanding of their meaning and spatial memory are taken into account. The next big stage in the development of AI is the transition from systems that work well with information to systems that can understand the physical world and act meaningfully in it," he said.

The three models solve interrelated tasks: they track the movement of objects, give meaning to a scene, and store information about changes, said Evgeny Vishnevsky, NTI expert on new materials and technologies. The next stage will be to combine the developments into a single system and test it on real robots during long-term operation in a changing environment.