Omnigrasp: Grasping Diverse Objects with Simulated Humanoids
Authors: Zhengyi Luo, Jinkun Cao, Sammy Christen, Alexander Winkler, Kris Kitani, Weipeng Xu
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present a method for controlling a simulated humanoid to grasp an object and move it to follow an object s trajectory. To demonstrate the capabilities of our method, we show state-of-the-art success rates in following object trajectories and generalizing to unseen objects. Table 1: Quantitative results on object grasp and trajectory following on the GRAB dataset. |
| Researcher Affiliation | Collaboration | Zhengyi Luo1,2 Jinkun Cao1 Sammy Christen2,3 Alexander Winkler2 Kris Kitani1,2 Weipeng Xu2 1Carnegie Mellon University; 2Reality Labs Research, Meta; 3ETH Zurich |
| Pseudocode | Yes | Algo 1: Learn Omnigrasp |
| Open Source Code | No | Code and models will be released. |
| Open Datasets | Yes | We use the GRAB [71], Oak Ink [86], and OMOMO [34] to study grasping small and large objects. |
| Dataset Splits | Yes | We split them into 1330 objects for training, 185 for validation, and 185 for testing. |
| Hardware Specification | Yes | We train Omnigrasp for three days collecting around 10^9 samples on a Nvidia A100 GPU. |
| Software Dependencies | No | Simulation is conducted in Isaac Gym [46], where the policy is run at 30 Hz and the simulation at 60 Hz. |
| Experiment Setup | Yes | Simulation is conducted in Isaac Gym [46], where the policy is run at 30 Hz and the simulation at 60 Hz. For PULSE-X and PHC-X, each policy is a 6-layer MLP. For the grasping task, we employ a GRU [14] based recurrent policy and use a GRU with a latent dimension of 512, followed by a 3-layer MLP. Object density is 1000 kg/m3. The static and dynamic friction coefficients of the object and humanoid fingers are set to 1. For reference object trajectory, we use ϕ = 20 future frames sampled at 15Hz. For more details, please refer to Appendix C. Table 7: Hyperparameters for Omnigrasp, PHC-X, and PULSE-X. |