Take Ohkawa

A-0 is a robot foundation model trained on handheld YUBI demonstrations for long-horizon, precise manipulation across embodiments. Weights and code are released as open source.

We present YUBI-Real2Sim, a 3DGS-based real-to-sim framework for evaluating VLA policies under dual fisheye wrist views. It combines 200-degree fisheye capture, 3DGS training, 3D foundation models for objects, and MuJoCo physics, and quantifies the reconstruction quality that reliable policy evaluation requires.

YUBI Go is an open-source Quest app for collecting YUBI robot data anywhere.

We equip YUBI, a finger-driven hand-held gripper, with FlexiTac tactile sensors to collect synchronized visual-tactile demonstrations, and show that touch guides manipulation when vision is limited.

We introduce YUBI-STAG, a spatio-temporal annotation and grounding framework that automatically enriches manipulation demonstrations with interaction-rich language. Combining contact-object segmentation with vision-language models, it annotates object attributes and states, per-gripper actions, and bimanual coordination to align pretrained VLAs with fine-grained instructions.

We present RoboPace, an online retiming layer for action-chunk policies that keeps the policy's geometric path while adapting its timing to the target robot. It slows down around predicted contact and speeds up in free space within the robot's kinematic and dynamic limits.

We present AssemblyHands-X, the first markerless 3D hand-body benchmark for understanding bimanual human activities. We reconstruct temporally consistent SMPL-X hand-body motion for 2.8M images from the multi-view videos of Assembly101, and show that modeling hand, body, and visual cues jointly improves action recognition.

We present DewHand, a single-stage model that jointly reconstructs hand and camera motion in a shared metric world frame from egocentric video. It combines scene-level geometric features with metric-space supervision to resolve scale ambiguity, while hand-specific features preserve fine articulation.

We introduce YUBI, a finger-aligned gripper for intuitive, ergonomic, and scalable data collection in bimanual dexterous manipulation. With integrated VR-based 6 DoF tracking, YUBI enables a UMI-based dataset of unprecedented scale (8434 hours, 1.20M episodes, 119 tasks), and a single policy trained on it transfers across multiple bimanual robots (UR, Franka, and ELEY).

The UMI Arena builds a shared environment for evaluating UMI-style manipulation. It pairs a 15,000-hour release of handheld demonstration data with a live policy competition run by the organizers on common robots, so entries are compared on the same tasks under the same conditions. The program also features a hands-on device exhibition and contributed papers.

We cast human video selection for robot learning as a recommendation problem, mining task-relevant clips from large-scale egocentric data by language and hand-pose retrieval, action-chunk ranking, and optical-flow verification. Training on under 5% of the pool raises the success rate of dexterous manipulation from 47.7% to 61.1%, showing that selective curation beats indiscriminate data mixing.

EgoYUBIs combines YUBI with egocentric perception for portable, wearable data collection in bimanual dexterous manipulation. The Aria-based setup integrates marker-cube tracking, wrist-camera SLAM, and interaction perception for in-the-wild robot data collection.

We propose a generative prior for hand pose refinement guided by affordance-aware textual descriptions of hand-object interactions. Our method employs a diffusion-based generative model that learns the distribution of plausible hand poses conditioned on affordance descriptions, which are inferred from VLMs.

We propose Attribute Classifier Guidance, a plug-and-play framework that adapts large pre-trained motion diffusion models for physical-attribute-aware human-object interaction synthesis. It steers the diffusion sampling process with the gradients of a lightweight attribute classifier, improving physical realism (e.g., mass-aware posture) while maintaining competitive generation quality.

We propose a novel and powerful framework for visual feature extraction in 3D hand pose estimation using recent state space modeling (i.e., Mamba), dubbed Deformable Mamba (DF-Mamba). DF-Mamba is designed to capture global context cues via Mamba's nature of selective state modeling and deformable state scanning.

This paper presents RPEP, the first pre-training method for event-based 3D hand pose estimation using labeled RGB images and unpaired, unlabeled event data.

We introduce the first extensive self-contact dataset with precise body shape registration, Goliath-SC, consisting of 383K self-contact poses across 130 subjects. Using this dataset, we propose generative modeling of a self-contact prior conditioned by body shape parameters, based on a body-part-wise latent diffusion with self-attention.

Our HANDS workshop will gather vision researchers working on perceiving hands performing actions, including 2D & 3D hand detection, segmentation, pose/shape estimation, tracking, etc. We will also cover related applications including gesture recognition, hand-object manipulation analysis, hand activity understanding, and interactive interfaces.

We introduce EmoSign, the first sign video dataset containing sentiment and emotion labels for 200 American Sign Language (ASL) videos with open-ended descriptions of emotion cues. Alongside the annotations, we include baseline models for sentiment and emotion classification.

We present a framework for pre-training of 3D hand pose estimation from in-the-wild hand images sharing with similar hand characteristics, dubbed SiMHand.

We present EgoYC2, a novel benchmark for cross-view knowledge transfer of dense video captioning, adapting models from web instructional videos with exocentric views to an egocentric view.

We present a comprehensive summary of the HANDS23 challenge using the AssemblyHands and ARCTIC datasets. Based on the results of the top submitted methods and more recent baselines on the leaderboards, we perform a thorough analysis on 3D hand(-object) reconstruction tasks.

We present a novel framework that concurrently tackles hand action recognition and 3D future hand motion prediction.

We propose a novel Single-to-Dual-view adaptation (S2DHand) solution that adapts a pre-trained single-view estimator to dual views.

We present AssemblyHands, a large-scale benchmark dataset with accurate 3D hand pose annotations, to facilitate the study of challenging hand-object interactions from egocentric videos.

We present a systematic review of 3D hand pose estimation from the perspective of efficient annotation and learning. 3D hand pose estimation has been an important research area owing to its potential to enable various applications, such as video understanding, AR/VR, and robotics.

We tackled domain adaptation of hand keypoint regression and hand segmentation to in-the-wild egocentric videos with new imaging conditions (e.g., Ego4D).

We propose Background Mixup augmentation that leverages data-mixing regularization for hand-object detection while avoiding unintended effect produced by naive Mixup.

We developed a domain adaptation method for hand segmentation, consisting of appearance gap reduction by stylization and learning with pseudo-labels generated by network consensus.

We developed extended consistency regularization for stabilizing the training of image translation models using real, fake, and reconstructed samples.