Criterion Collapse and Loss Distribution Control
Authors: Matthew J. Holland
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We complement our basic theory with a set of experiments in 4, training non-linear neural network models (e.g., Res Net-34) for image classification from scratch, comparing across a variety of learning criteria with a common base loss. |
| Researcher Affiliation | Academia | 1SANKEN, Osaka University, Japan. Correspondence to: Matthew J. Holland <matthew-h@ar.sanken.osaka-u.ac.jp>. |
| Pseudocode | No | The paper provides mathematical expressions for update rules (e.g., equation 16 and 17) but does not include structured pseudocode or algorithm blocks with labels like 'Algorithm' or 'Pseudocode'. |
| Open Source Code | Yes | A Git Hub repository with code and seed values to re-create all the results presented in this paper is available at this URL: https://github.com/feedbackward/collapse. |
| Open Datasets | Yes | We use four standard datasets: CIFAR-10, CIFAR-100, Fashion MNIST, and SVHN, accessed via the torchvision package. |
| Dataset Splits | Yes | In each trial, however, we shuffle the training data and do an 80-20 split for training-validation. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU/GPU models, memory, or cloud instance types) used for running the experiments. It only mentions model architectures. |
| Software Dependencies | No | The paper mentions using 'torchvision package' but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | As an optimizer, we use vanilla SGD with step size 0.1 and momentum 0.9, run for 250 epochs, with mini-batch size of 200. |