The humanoid robot Atlas perceives and interacts with the environment

The execution of even relatively simple tasks, such as taking and placing an automotive part, requires the humanoid robot Atlas to have a deep understanding of the geometric and semantic characteristics of the surrounding objects. The engineers from Boston Dynamics reveal how the robot "sees" the world thanks to a flexible and adaptive perception system.

First, Atlas must find and identify the object among other shiny or low-contrast parts in the factory. Then, it determines the exact location of the part to grab it - it may be on a table, in a container, etc. Finally, the robot decides where and how to place the object with exceptional precision to avoid misplacement or dropping.

Atlas's perception system includes high-precision sensors, modern AI algorithms with machine learning, and a module for estimating its own position. The robot uses computer vision to detect and classify objects, determine their arrangement relative to other elements in the space, and adjust its actions as needed.

Key roles are played by the so-called "key points" - external, marking objects that must be circumvented, and internal, denoting the distribution of shelves, boxes, and other elements. Atlas uses a neural network to predict the location of these points and orient itself in the environment.

Before proceeding with manipulations, the robot determines its own location through a localization module that analyzes the position and orientation of nearby objects. Additionally, kinematic odometry is used to track the robot's movement.

The SuperTracker system combines data from various sensors - kinematics, computer vision, and others - to estimate the position of the grippers even when the objects are not well visible from the cameras. In limited visibility or poor lighting conditions, these estimates are further verified.

The engineers emphasize that the precise calibration of sensors, the computer vision system, and other components is crucial for the precise operation and autonomy of Atlas. In the future, they plan to improve the robot's adaptability by integrating perception and action into a single whole.