Estimating Lipschitz constants of monotone deep equilibrium models
Authors: Chirag Pabbaraju, Ezra Winston, J Zico Kolter
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirically, we demonstrate that our Lipschitz bounds on fully-connected mon DEQs trained on MNIST are small relative to comparable DNNs, even for DNNs of depth only 4. We show a similar trend on singleand multi-convolutional mon DEQs as compared to the bounds on traditional CNNs computed by Auto Lip and Seq Lip (Virmaux & Scaman, 2018), the only existing methods for (even approximately) bounding CNN Lipshitz constants. Further, our mon DEQ generalization bounds are comparable with bounds on DNNs of around depth 5, and avoid the exponential dependence on depth of those bounds. Finally, we also validate the significance of the small Lipschitz bounds for mon DEQs by empirically demonstrating strong adversarial robustness on MNIST and CIFAR-10. |
| Researcher Affiliation | Collaboration | Chirag Pabbaraju & Ezra Winston Carnegie Mellon University {cpabbara, ewinston}@cs.cmu.edu J. Zico Kolter Carnegie Mellon University Bosch Center for Artificial Intelligence zkolter@cs.cmu.edu |
| Pseudocode | No | The paper contains mathematical equations and descriptions of iterative processes, but no structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Experimental code available at https://github.com/locuslab/lipschitz_mondeq. |
| Open Datasets | Yes | We conduct all our experiments on MNIST and CIFAR-10 |
| Dataset Splits | No | The paper mentions training on samples and refers to test errors, but does not specify explicit train/validation/test dataset splits, percentages, or sample counts for reproduction. |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU/CPU models, memory, or cloud instance specifications used for running experiments. |
| Software Dependencies | No | The paper mentions tools like |
| Experiment Setup | Yes | Each model is trained on a sample of 4096 MNIST examples until the margin error at margin γ = 10 reaches below 10% which serves to standardize the experiments across choice of batch size and learning rate. |