InsActor: Instruction-driven Physics-based Characters
Authors: Jiawei Ren, Mingyuan Zhang, Cunjun Yu, Xiao Ma, Liang Pan, Ziwei Liu
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments demonstrate that Ins Actor achieves state-of-the-art results on various tasks, including instruction-driven motion generation and instruction-driven waypoint heading. Notably, the ability of Ins Actor to generate physically simulated animations using high-level human instructions makes it a valuable tool, particularly in executing long-horizon tasks with a rich set of instructions. |
| Researcher Affiliation | Collaboration | Jiawei Ren 1 Mingyuan Zhang 1 Cunjun Yu 2 Xiao Ma3 Liang Pan1 Ziwei Liu1 1 S-Lab, Nanyang Technological University 2 National University of Singapore 3 Dyson Robot Learning Lab |
| Pseudocode | Yes | The inference process of Ins Actor is detailed in Algorithm 1. |
| Open Source Code | Yes | Our project page is available at jiawei-ren.github.io/projects/insactor/index.html |
| Open Datasets | Yes | Datasets. We use two large scale text-motion datasets, KIT-ML [26] and Human ML3D [8], for training and evaluation. KIT-ML has 3,911 motion sequences and 6,353 sequence-level language descriptions, Human ML3D provides 44,970 annotations on 14,616 motion sequences. |
| Dataset Splits | Yes | We adopt the original train/test splits in the two datasets. |
| Hardware Specification | No | The paper mentions using Brax for the environment but does not specify any particular hardware (e.g., GPU models, CPU types) used for running the experiments. It refers to supplementary materials for 'details of neural network architecture and training' but not hardware. |
| Software Dependencies | No | The paper mentions using 'Brax [7]' and refers to 'classical transformer [37]' and 'pre-trained contrastive model' but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | No | The paper states 'For details of neural network architecture and training, we refer readers to the supplementary materials.' and only provides high-level details about the character model (e.g., '13 links and 34 degrees of freedom, weighs 45kg, and is 1.62m tall') and PD controller gains. It does not include specific hyperparameters like learning rate, batch size, or optimizer settings in the main text. |