DIFFTACTILE: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation
Authors: Zilin Si, Gu Zhang, Qingwei Ben, Branden Romero, Zhou Xian, Chao Liu, Chuang Gan
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present two sets of tasks with DIFFTACTILE: system identification, and tactile-assisted manipulation. For system identification, we use real-world tactile observations to optimize the simulator s system parameters and to reduce sim-to-real gaps. Then we present five manipulation tasks: grasping, surface following, cable straightening, case opening, and object reposing as shown in Fig. 2. Tactile sensing can enable safer and more adaptive grasping to handle fragile objects such as fruits. We grasp a diverse set of objects with various shapes, sizes, and materials without slipping and damaging. For the other four contact-rich manipulation tasks, surface following requires the sensor to stay in contact with a 3D surface and travel to an endpoint while maintaining a certain contact force; cable straightening requires a pair of sensors to first grasp a fixed end of the cable, and then straighten it by sliding towards the other end; case opening uses a single sensor to open an articulated object via pushing; lastly, object reposing involves using a single sensor to push an object from a lying pose to a standing pose against the wall. These four tasks represent rigid, deformable, and articulated object manipulation. |
| Researcher Affiliation | Collaboration | Zilin Si , , 1,5, Gu Zhang ,2, Qingwei Ben ,3, Branden Romero4, Zhou Xian1, Chao Liu4, Chuang Gan5,6 1 CMU RI, 2 Shanghai Jiao Tong University, 3 Tsinghua University, 4 MIT CSAIL, 5 MIT-IBM Watson AI Lab, 6 UMass Amherst zsi@andrew.cmu.edu, guzhang.sjtu@gmail.com, bqw20@mails.tsinghua.edu.cn, brromero@mit.edu, xianz1@andrew.cmu.edu, chaoliu@csail.mit.edu, ganchuang@csail.mit.edu |
| Pseudocode | No | The paper describes processes using text and diagrams like Figure 5 ('Simulation pipeline for each simulation step') and Figure 6 ('Multi layer perceptron neural network architecture'), but it does not include formal pseudocode or a clearly labeled algorithm block. |
| Open Source Code | Yes | Code and supplementary materials are available at the project website1. 1https://difftactile.github.io/ |
| Open Datasets | Yes | As shown in Fig. 2, we select four objects from EGAD (Morrison et al., 2020) dataset with different shape complexity and assign each object with two different material properties, elastic, and elastoplastic. |
| Dataset Splits | Yes | Our dataset has 2010 samples, 1800 for training, 200 for validation, and 10 for testing. (Appendix A.2) |
| Hardware Specification | Yes | All experiments were conducted on a Ubuntu 18.04 with AMD Ryzen 7 5800x 8-core processor and Nvidia Ge Force RTX 3060. |
| Software Dependencies | No | The paper mentions using 'Taichi (Chen et al., 2023)' and 'stable-baseline3 (Raffin et al., 2021)' but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | Initialization We initialize the simulation environment with a single tactile sensor s for system identification, surface following, case opening, and object reposing, and two tactile sensors {s1, s2} mounted on a parallel jaw gripper for grasping and cable straightening. Both tactile sensors and objects shapes are initialized with STL or OBJ mesh models and then voxelized to FEM tetrahedron meshes or MPM/PBD particles. Objects oi are initialized statically on the tabletop and we add a vertical wall for object reposing. Tactile sensors are initialized statically near objects depending on tasks but without contact. We initialize the poses of tactile sensor at time step t = 0 as Ts(0) = (Rs(0), ts(0)) SE(3) where Rs(0) SO(3) and ts(0) R3 and similarly object pose as To(0). ... We use Adam optimizer with β1 = 0.9, β2 = 0.999. Learning rate parameters are lrkn = 20.0, lrkd = 20.0, lrkt = 5.0, lrfc = 5.0, lrµ = 50.0, lrλ = 50.0 depending on their numerical scales. We run 100 optimization steps for each trajectory. The hyper-parameters can be found in Table 7, where lrp is the learning rate for translation, lro is the learning rate for orientation, and lrw is the learning rate for the gripper s width. |