Spatially Conditioned Diffusion Policy

Learning Precise and Robust Manipulation with a Single RGB Camera

CoRL 2026

Seoyoon Kim1,*Kanghyun Kim1,*Dongwoo Ko2Young Jin Heo2Min Jun Kim1

1 KAIST2 Neuromeka

* Equal contribution

Paper Video Code

Abstract

Recent visual imitation learning systems have widely adopted multi-camera setups with wrist-mounted cameras as the de facto standard. However, manipulation from a single global view remains challenging, as the policy should capture fine-grained interaction details and identify task-relevant regions without local wrist views. To address this challenge, we present Spatially Conditioned Diffusion Policy (SCDP), a diffusion-based visuomotor policy that achieves precise and robust manipulation in a single-camera setting. Our key idea is that end-effector trajectories can serve as visual attention anchors that reflect task-relevant regions. Building on this idea, SCDP consists of two key components: (i) a visual encoder that produces multi-scale feature maps to capture both broader context and fine-grained visual features, and (ii) a spatial conditioning module that samples point-wise features along intermediate end-effector trajectories in the diffusion loop. Extensive simulation experiments show that SCDP consistently outperforms strong single-view baselines and achieves performance comparable to multi-camera baselines. Real-world experiments further demonstrate precise manipulation and robustness to visual distractors, highlighting the potential of single-camera imitation learning.

Method

Spatial conditioning with visual attention anchors

SCDP architecture: a single RGB image is encoded into multi-scale feature maps. Reconstructed end-effector trajectories query local visual features, which condition successive action denoising steps.
SCDP architecture
01 / MULTI-SCALE FEATURES

Multi-Scale Image Encoder

Encode a single RGB image into multi-scale feature maps that capture both global context and fine-grained visual details.

02 / VISUAL ATTENTION

Spatial Conditioning Module

Project reconstructed end-effector trajectories into the image as visual attention anchors, then sample point-wise features from multi-scale feature maps at these locations.

03 / CONDITIONAL DIFFUSION

Action Denoising Network

Condition the action denoising network on the resulting spatial context through FiLM, updating this context at each denoising step.

In simulation

Simulation at a glance

Over 80% success across all Meta-World difficulty groups with a single RGB camera.

Across task difficulty

Meta-World · Success (%) ↑

Meta-World success rates (Easy, Medium, Hard, Very Hard): DP 79.9, 36.1, 16.7, 46.0%; DP3 92.3, 78.5, 60.3, 82.7%; SCDP 91.5, 83.0, 82.5, 89.7%.
Single-view competitiveness Meta-World Hard · 6 tasks
Method Camera setting Success (%) ↑
DP + Wrist RGB 74.2 ± 29.4
DP3 Depth 60.3 ± 38.0
SCDP Single RGB 82.5 ± 11.5

Mean ± standard deviation across the 6 tasks.

Experiments

Real-world experiments

Precise and robust manipulation across diverse real-world tasks with a single RGB camera.

01

Precise Manipulation

Fine alignment and contact-rich insertion, guided by local visual details.

4× Autonomous

USB Insertion

Grasp the USB drive, align it with the hub port, and push it into place.

Grasp → Insert → Push
2× Autonomous

Battery Insertion

Align a battery with its slot and overcome spring resistance to seat it.

Grasp → Insert → Push
02

Distractor Robustness

USB and battery insertion with surrounding visual distractors.

4× Autonomous

USB Insertion w/ Distractors

Insert the target USB drive with other objects in the workspace.

2× Autonomous

Battery Insertion w/ Distractors

Complete a contact-rich insertion amid surrounding objects.

03

Long Horizon

Sequential manipulation from opening a drawer to putting an object away.

2× Autonomous

Drawer manipulation

A four-stage manipulation sequence, shown at 2× speed.

  1. 01Open drawer
  2. 02Pick banana
  3. 03Place banana
  4. 04Close drawer
04

More Experiments

Cup grasping and cube pushing across different interaction geometries and distractor settings.

4× Autonomous

Cup Handle Grasping

Adapt the grasp to the handle orientation and pick up the cup by its handle.

2× Autonomous

Cup Handle Grasping w/ Distractors

Locate the cup handle and grasp it among visually distracting objects.

4× Autonomous

Cup Wall Grasping

Place one gripper finger inside the cup to grasp its wall while avoiding the rim and handle.

2× Autonomous

Push Cube w/ Distractors

Push the target red cube with distractor objects in the workspace.

Additional demonstrations

Humanoid

Humanoid demos

Additional demonstrations using a single head-mounted camera for egocentric visual observation.

2× Autonomous

Separating Paper Cup

A humanoid robot separates paper cups on a tabletop.

2× Autonomous

Erasing White Board

A humanoid robot uses an eraser to wipe a whiteboard.

Reference

Citation

@misc{kim2026scdp,
  title={Spatially Conditioned Diffusion Policy: Learning Precise and Robust Manipulation with a Single RGB Camera},
  author={Seoyoon Kim and Kanghyun Kim and Dongwoo Ko and Young Jin Heo and Min Jun Kim},
  year={2026},
  eprint={2606.14535},
  archivePrefix={arXiv},
  primaryClass={cs.RO},
  url={https://arxiv.org/abs/2606.14535}
}