Permutation Equivariant Neural Functionals
Authors: Allan Zhou, Kaien Yang, Kaylee Burns, Adriano Cardace, Yiding Jiang, Samuel Sokota, J. Zico Kolter, Chelsea Finn
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In our experiments, we find that permutation equivariant neural functionals are effective on a diverse set of tasks that require processing the weights of MLPs and CNNs, such as predicting classifier generalization, producing winning ticket sparsity masks for initializations, and classifying or editing implicit neural representations (INRs). |
| Researcher Affiliation | Academia | 1Stanford University 2University of Bologna 3Carnegie Mellon University |
| Pseudocode | Yes | A Equivariant NF-Layer pseudocode |
| Open Source Code | Yes | In addition, we provide code for our models and experiments1. 1https://github.com/Allan Yang Zhou/nfn |
| Open Datasets | Yes | Our first two tasks require (1) predicting the test accuracy of CNN image classifiers and (2) classifying implicit neural representations (INRs) of images and 3D shapes... We consider datasets of SIRENs [58] that encode images (MNIST [38], Fashion MNIST [66], and CIFAR [35]) and 3D shapes (Shape Net-10 and Scan Net-10 [53]). |
| Dataset Splits | Yes | Datasets were split into 45,000 training images, 5,000 validation images, and 10,000 (MNIST, CIFAR-10) or 20,000 (Fashion MNIST) test images. |
| Hardware Specification | Yes | The model is trained for 50 epochs... which takes 1 hour on a Titan RTX GPU. (Appendix D.1) |
| Software Dependencies | No | Appendix A states that the pseudocode uses 'Py Torch [50] and Einops-like [55] pseudocode', but specific version numbers for these software dependencies are not provided. |
| Experiment Setup | Yes | Table 12: Hyperparameters for predicting generalization on Small CNN Zoo. Name Values Optimizer Adam Learning rate 0.001 Batch size 8 Loss Binary cross-entropy Epoch 50 |