Online Adaptation for Consistent Mesh Reconstruction in the Wild
Authors: Xueting Li, Sifei Liu, Shalini De Mello, Kihwan Kim, Xiaolong Wang, Ming-Hsuan Yang, Jan Kautz
NeurIPS 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct experiments on animals, i.e., birds and zebras. We evaluate our contributions in two aspects: (i) the improvement of single-view mesh reconstruction, and (ii) the reconstruction of a sequence of frames via online adaptation. Due to the lack of ground truth meshes for images and videos captured in the wild, we evaluate the reconstruction results via mask and keypoint re-projection accuracy, e.g., we follow, and compare against [13] to evaluate the model trained on the image dataset. |
| Researcher Affiliation | Collaboration | Xueting Li1, Sifei Liu2, Shalini De Mello2, Kihwan Kim2, Xiaolong Wang3, Ming-Hsuan Yang1, Jan Kautz2 1University of California, Merced, 2NVIDIA, 3University of California, San Diego |
| Pseudocode | No | The paper does not contain any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | Codes and other resources will be released at https://sites.google.com/nvidia.com/vmr-2020. |
| Open Datasets | Yes | We first train image reconstruction models, discussed in Sec. 3.1, for the CUB bird [42] and the synthetic zebra [55] datasets. |
| Dataset Splits | No | The paper mentions training and testing data (e.g., "testing split of the CUB dataset") but does not explicitly specify a validation split or details on how the data is partitioned into training, validation, and test sets (e.g., percentages or counts). |
| Hardware Specification | No | The paper does not specify any hardware details such as GPU models, CPU types, or memory specifications used for running the experiments. |
| Software Dependencies | No | The paper mentions using models and tools like ResNet18 [9], Soft Rasterizer [25], and UVC model [23] but does not provide specific version numbers for any software libraries, frameworks, or programming languages used. |
| Experiment Setup | Yes | Firstly, we fine-tune all parameters in the reconstruction model on sliding windows instead of all video frames. Each sliding window includes Nw = 50 consecutive frames and the sliding stride is set to Ns = 10. We tune the reconstruction model for Nt = 40 iterations with frames in each sliding window. ... we first warm up the model without the motion deformation branch, the keypoint re-projection objective, or the ARAP constraint for 200 epochs. ... We then train the full image reconstruction network with all objectives for another 200 epochs. |