Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates
Authors: Riccardo Grazzi, Massimiliano Pontil, Saverio Salzo
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We also present illustrative experiments confirming our analysis. |
| Researcher Affiliation | Academia | 1CSML, Istituto Italiano di Tecnologia, Genoa, Italy 2Department of Computer Science, University College, London, UK 3Dipartimento di Ingegneria Informatica, Automatica e Gestionale, Universit a La Sapienza, Rome, Italy. |
| Pseudocode | Yes | Algorithm 1 NSID; Algorithm 2 NSID-Bilevel; Algorithm 3 Stochastic fixed point iterations |
| Open Source Code | Yes | We provide the code to reproduce our experiments at https://github.com/prolearner/ nonsmooth_implicit_diff |
| Open Datasets | Yes | For the data poisoning setup we use the MNIST dataset. |
| Dataset Splits | Yes | We split the MNIST original train set into 30K example for training and 30K examples for validation. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU or CPU models, memory, or cloud instance types used for the experiments. |
| Software Dependencies | No | We implement NSID by relying on Py Torch automatic differentiation for the computation of Jacobian-vector products. For AID and ITD, we use the existing Py Torch implementations. (No version numbers provided for PyTorch). |
| Experiment Setup | Yes | The appropriate choice for the step-size guarantees that Φ is a contraction, in our case we set it equal to 2{p L µ 2λ2q... We set the regularization parameters λ p0.02, 0.1q... we set it [minibatch size] equal to 10% of the training set... we set ηi a1{pa2 iq for (N)SID dec and η a1{a2 for (N)SID const, where a1 b1β and a2 b2β... In particular we set a1 0.5, a2 2 for the synthetic Elastic net experiment and a1 2, a2 0.01 for Data poisoning. |