ICRA 2025 视频文案

Articulated Object Manipulation using Online Axis Estimation with SAM2-Based Tracking

Title

[15 s] Title

In this video, we propose a novel closed-loop pipeline that integrates interactive perception with online refined axis estimation obtained from Segment Anything Model 2 (SAM2) - based tracking.

Motivation

[45 s → 30s] Motivation

Among various robotic manipulation tasks, those involving articulated objects, such as doors and drawers, are particularly challenging.

Articulated object manipulation requires precise object interaction, where the object’s axis must be carefully considered.

Previous research employed interactive perception for manipulating articulated objects, but typically, open-loop approaches often suffer from overlooking the interaction dynamics.

(这里少 15 s 留给 Pipeline 部分)

Innovation

[15 s] 3 个 Innovation

In this study, we propose a novel closed-loop pipeline integrating interactive perception with online refined axis estimation from segmented 3D point clouds with SAM2-based tracking.

Pipeline

[30 s → 45 s] Pipeline

Our pipeline consists of 3 core modules: (1) Interactive Perception & Init-Manipulation Module, (2) Tracking & Segmentation Module and (3) Axis Estimation & Manipulation Module.

In the Interactive Perception Module, the robot applies any interactive perception methods to manipulate the articulated object, creating a slight displacement. As the robot initiates the manipulation, the camera periodically records dynamic RGB-D data.

In the Tracking & Segmentation Module, we continuously track the object using SAM2, proving mask for extracting the corresponding portion of the reconstructed 3D point cloud representing the object from the dynamic scene. By subtracting the oriented bounding box (OBB) of the entire cabinet in the first frame, we obtain the motion part of the articulated object.

In the Axis Estimation & Manipulation Module, the axis of motion is explicitly calculated based on the OBBs of the active part point-clouds, which is subsequently used for the online axis estimation refinement.

Please check our main paper for more technical details.

Simulation Experiment

[30 s] 实验设置

Our method specifically targets door-opening and drawer-opening tasks. We evaluate the performance of these two tasks under scenarios involving wide manipulation ranges. The robotic arm must achieve a designated manipulation goal - either opening the drawer or door to a specific distance.

[35 s] 结果表格

We incorporates the state-of-the-art RGBManip as the fundamental implementation of the interactive perception module in our simulation experiment.

Quantitative results of basic tasks illustrate that both our method and RGBManip almost outperform other baseline approaches while Ours consistently surpasses RGBManip in basic tasks.

For more challenging door-opening and drawer-opening tasks, granular results clearly demonstrate the superiority of our online axis estimation approach over traditional methods, especially in tasks that demand precise axis-based control.

[25 s] 中间过程可视化

We select an object from each category to visualize the manipulation process with online axis estimation refinement.

The initial estimated axis is represented by a lighter shade of red, while the progressively refined axis is indicated by increasingly darker shades of red.

Thanks for watching

Thanks for watching!