$SE(3)$ Equivariant Convolution and Transformer in Ray Space
Authors: Yinshuang Xu, Jiahui Lei, Kostas Daniilidis
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate SE(3)-equivariance by obtaining robust results in roto-translated datasets without performing transformation augmentation. ... We demonstrate the composition of equivariant convolution and transformer modules in the tasks of 3D reconstruction from multi-views and generalized rendering from multi-view features. ... We evaluate our model in seven experiment settings, I/I, I/Z, I/R, R/R, Y/SO(3), SO(3)/SO(3). The setting A/B indicates training the model on the A setup of the dataset and evaluating it on the B setup. Following the previous works, we use Io U and Chamfer-L1 Distance as the evaluation metric. Quantitative results are reported in table 1, and qualitative results are in Fig. 8. |
| Researcher Affiliation | Academia | Yinshuang Xu University of Pennsylvania xuyin@seas.upenn.edu Jiahui Lei University of Pennsylvania leijh@seas.upenn.edu Kostas Daniilidis University of Pennsylvania and Archimedes, Athena RC kostas@cis.upenn.edu |
| Pseudocode | No | The paper does not include any explicitly labeled pseudocode or algorithm blocks. The methods are described mathematically and textually. |
| Open Source Code | No | The paper does not provide an explicit statement about releasing its source code for the described methodology or a link to a code repository. |
| Open Datasets | Yes | We use the same train/val/test split of the Shapenet Dataset [9] and render ourselves for the equivariance test. ... [9] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. ar Xiv preprint ar Xiv:1512.03012, 2015. ... We use the same training and test dataset as in [61], which consists of both synthetic and real data. ... Realistic Synthetic 360 [42] ... Real Forward-Facing [41] ... Diffuse Synthetic 360 [49] |
| Dataset Splits | Yes | We use the same train/val/test split of the Shapenet Dataset [9] and render ourselves for the equivariance test. |
| Hardware Specification | No | The paper does not explicitly specify the hardware (e.g., GPU models, CPU types, memory) used for running the experiments. |
| Software Dependencies | No | We use the same Res Net backbone as implemented in [30] that is equivariant to the finite group C8, which we find achieves the best result compared with other SE(2) equivariant CNNs. The paper mentions software components (ResNet, C8 group) but does not provide specific version numbers for these or other general software dependencies (e.g., Python, PyTorch). |
| Experiment Setup | Yes | For the fusion from the ray space to the point space model, we use one layer of convolution and three combined blocks of updating ray features and SE(3) transformers. For the equivariant SE(3) multi-head-attention, we only use the scalar feature and the vector (type-1) feature in the hidden layer. The kernel matrix includes the spherical harmonics of degrees 0 and 1. ... We use the same weighted SDF loss as in [69] during training, which applies both uniform and near-surface sampling. |