Modeling Dynamic Environments with Scene Graph Memory
Authors: Andrey Kurenkov, Michael Lingelbach, Tanmay Agarwal, Emily Jin, Chengshu Li, Ruohan Zhang, Li Fei-Fei, Jiajun Wu, Silvio Savarese, Roberto Martı́n-Martı́n
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our method in the Dynamic House Simulator, a new benchmark that creates diverse dynamic graphs following the semantic patterns typically seen at homes, and show that NEP can be trained to predict the locations of objects in a variety of environments with diverse object movement dynamics, outperforming baselines both in terms of new scene adaptability and overall accuracy. |
| Researcher Affiliation | Collaboration | 1Department of Computer Science, Stanford University 2Salesforce AI Research 3Department of Computer Science, University of Texas at Austin. |
| Pseudocode | Yes | Algorithm 1 Dynamic House Simulator Algorithm |
| Open Source Code | Yes | The codebase and more can be found this URL. |
| Open Datasets | Yes | We compute these probabilities via simple counting of the presence of relationships in different environments of i Gibson 2.0 (Li et al., 2021a) and Proc THOR-10k (Deitke et al., 2022) |
| Dataset Splits | No | The paper mentions a 'training dataset' and 'test set' but does not specify clear train/validation/test splits by percentage or count. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | Yes | All models are implemented and trained using Py Torch (Paszke et al., 2019) and Py Torch-Geometric (Fey & Lenssen, 2019). |
| Experiment Setup | Yes | Model parameters The node embedding network and edge embedding networks are two-layer feedforward neural networks with 64 units and Re LU activations following each layer. The HEAT and GCN graph neural networks have one graph convolution layer with 64 units. The transformer encoder is a standard self-attention model with 2 heads with a 64 unit feedforward network. Training parameters The model was trained for 25 epochs with a batch size of 100. The Adam optimizer was used with a learning rate of 1 10 4. |