Semantic MapNet: Building Allocentric Semantic Maps and Representations from Egocentric Views
Authors: Vincent Cartillier, Zhile Ren, Neha Jain, Stefan Lee, Irfan Essa, Dhruv Batra964-972
AAAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | On the task of semantic mapping in the Matterport3D dataset, SMNet significantly outperforms competitive baselines by 4.01 16.81% (absolute) on mean-Io U and 3.81 19.69% (absolute) on Boundary-F1 metrics. Moreover, we show how to use the neural episodic memories and spatio-semantic allocentric representations built by SMNet for subsequent tasks in the same space navigating to objects seen during the tour ( Find chair ) or answering questions about the space ( How many chairs did you see in the house? ). |
| Researcher Affiliation | Collaboration | Vincent Cartillier,1 Zhile Ren,1 Neha Jain,1 Stefan Lee,2 Irfan Essa,1,4 Dhruv Batra,1,3 1 Georgia Institute of Technology 2 Oregon State University 3 Facebook AI Research 4 Google Research {vcartillier3, zren80, nehaj, irfan, dbatra}@gatech.edu, leestef@oregonstate.edu |
| Pseudocode | No | The paper describes the architecture and modules of SMNet but does not include any pseudocode or algorithm blocks. |
| Open Source Code | No | The paper provides a "Project page: https://vincentcartillier.github.io/smnet.html", which is typically an overview/demo page and not explicitly a source code repository link as required by the prompt. |
| Open Datasets | Yes | We choose the Matterport3D dataset (Chang et al. 2017) with the Habitat simulator (Savva et al. 2019)... Matter Port3D dataset license available at: http://kaldir.vc.in.tum.de/matterport/MP_TOS.pdf. |
| Dataset Splits | Yes | Utilizing the same data split as (Wijmans et al. 2019), we keep 85 unique floors in our dataset: 61 for training, 7 for validation, and 17 for testing. |
| Hardware Specification | Yes | SMNet is trained end-to-end under cross-entropy loss using SGD with learning rate 1e 4, momentum 0.9, weight decay 4e 4, and batch size 8 across 8 Titan XPs. |
| Software Dependencies | No | The paper mentions software like Red Net, Habitat simulator, and SGD but does not provide specific version numbers for any of these or other key software components. |
| Experiment Setup | Yes | SMNet is trained end-to-end under cross-entropy loss using SGD with learning rate 1e 4, momentum 0.9, weight decay 4e 4, and batch size 8 across 8 Titan XPs. Training took 2-3 days. Back propagation is applied after 20 steps. |