Moccasin: Efficient Tensor Rematerialization for Neural Networks
Authors: Burak Bartan, Haoming Li, Harris Teague, Christopher Lott, Bistra Dilkina
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We present numerical studies that show that our approach is up to an order of magnitude faster than recent work especially for large-scale graphs. |
| Researcher Affiliation | Collaboration | 1Qualcomm AI Research, Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc. 2University of Southern California, Los Angeles, USA. |
| Pseudocode | Yes | The pseudo-code representation of this constraint is as follows: cumulative({si v, ei v, ai v, mv}v V,i {1..Cv}, M) . |
| Open Source Code | No | The paper mentions using an 'open source CP-SAT solver from Google OR-Tools' but does not state that the code for their proposed method (MOCCASIN) is open-source or provide a link to it. |
| Open Datasets | Yes | Checkmate Graphs: We use graphs from the checkmate repository (Jain, 2020). They represent the single-batch training computation graphs for a selected set of neural networks. Use of these graphs allows us to compare directly to results presented in that work. Random Layered Graphs: We leverage the synthetically constructed random layered graphs introduced in (Gagrani et al., 2022, Appendix A) as examples of graphs with complex interconnect topology. |
| Dataset Splits | No | The paper mentions using different types of graphs (e.g., Checkmate Graphs, Random Layered Graphs) but does not provide specific details on how these graphs were split into training, validation, or testing sets for their experiments. |
| Hardware Specification | No | The paper states 'We have run all of the numerical experiments on a 16-CPU core workstation with a 32 GB of RAM,' which specifies general system resources but lacks specific CPU models or other detailed hardware specifications. |
| Software Dependencies | No | The paper mentions using 'CP-SAT solver from Google OR-Tools', 'Gurobi', and 'CVXPY' but does not provide specific version numbers for these software dependencies. |
| Experiment Setup | Yes | In all of the experiments, we have set Cv = 2 for all v V , which we specified in the plot legends by C = 2. |