DMesh: A Differentiable Mesh Representation
Authors: Sanghyun Son, Matheus Gadelha, Yang Zhou, Zexiang Xu, Ming Lin, Yi Zhou
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we provide experimental results to demonstrate the efficacy of our approach. First, we optimize vertex attributes to restore a given ground truth mesh, directly proving the differentiability of our design. Next, we conduct experiments on 3D reconstruction from point clouds and multi-view images, showcasing how our differentiable formulation can be used in downstream applications. |
| Researcher Affiliation | Collaboration | 1University of Maryland, College Park, {shh1295,lin}@umd.edu 2Adobe Research, {gadelha,yazhou,zexu,yizho}@adobe.com |
| Pseudocode | Yes | Algorithm 1 Mesh to DMesh Pgt, Fgt Ground truth mesh vertices and faces P, W, ψ Initialize point attributes for DMesh F Empty set of faces while Optimization not ended do P, W, ψ Do point insertion, with P, F WDT, PD Run WDT algorithm, with P, W F Update faces to exclude, with WDT Λ(Fgt), Λ( F) Compute existence probability for faces, with P, ψ, WDT, PD Lrecon Compute reconstruction loss, with Λ(Fgt), Λ( F) Update P, W, ψ to minimize Lrecon Bound P end M Get final mesh from DMesh |
| Open Source Code | Yes | We publicize the source code and supplementary material at our project page 1. 1https://sonsang.github.io/dmesh-project |
| Open Datasets | Yes | For the mesh reconstruction problem, we used three models from the Stanford 3D Scanning Repository (Curless und Levoy, 1996). For point cloud and multi-view reconstruction tasks, we used four closed-surface models from the Thingi32 dataset, four open-surface models from the Deep Fashion3D dataset, and three additional models with both closed and open surfaces from the Objaverse dataset and Adobe Stock. |
| Dataset Splits | No | The paper discusses various datasets used for training and testing, but it does not specify explicit training/validation/test splits with percentages or sample counts for reproduction. |
| Hardware Specification | Yes | All of the experiments were run on a system with AMD EPYC 7R32 CPU and Nvidia A10 GPU. |
| Software Dependencies | Yes | We implemented our main algorithm for computing face existence probabilites and differentiable renderer used for multi-view image reconstruction in CUDA (Nickolls u. a., 2008). Since we need to compute WDT before running the CUDA algorithm, we used WDT implementation of CGAL (Jamin u. a., 2023). We implemented the rest of logic with Pytorch (Paszke u. a., 2017). |
| Experiment Setup | Yes | In Appendix E, we provide values for these hyperparameters for every experiment. Also, in Appendix E.3, we present ablation studies for these regularizations. ... Hyperparameters for Point Cloud Reconstruction: Optimizer: ADAM Optimizer, Learning rate = 10-4 for open surface meshes and two mixed surface meshes (Bigvegas, Raspberry) / 3 × 10-4 for closed surface meshes, and one mixed surface mesh (Plant). Regularization: λweight = 10-8, λreal = 10-3, λqual = 10-3 for every mesh. Number of epochs: Single epoch for every mesh. Number of steps per epoch: 1000 steps for phase 1, 500 steps for phase 2 for every mesh. |