Cold Case: The Lost MNIST Digits
Authors: Chhavi Yadav, Leon Bottou
NeurIPS 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Section 3 compares and discusses the performances of well known algorithms measured on the original MNIST test samples, on their reconstructions, and on the reconstructions of the 50,000 lost test samples. |
| Researcher Affiliation | Collaboration | Chhavi Yadav New York University New York, NY chhavi@nyu.edu Léon Bottou Facebook AI Research and New York University New York, NY leon@bottou.org |
| Pseudocode | No | The paper describes algorithms and procedures in paragraph form, but it does not include any structured pseudocode or algorithm blocks. |
| Open Source Code | Yes | Code and data are available at https://github.com/facebookresearch/qmnist. |
| Open Datasets | Yes | Although the popular MNIST dataset [Le Cun et al., 1994] is derived from the NIST database [Grother and Hanaoka, 1995], the precise processing steps for this derivation have been lost to time. |
| Dataset Splits | Yes | The reconstructed training set contains 60,000 images matching each of the MNIST training images. Similarly, the first 10,000 images of the reconstructed test set match each of the MNIST test set images. The next 50,000 images are a reconstruction of the 50,000 lost MNIST test images. |
| Hardware Specification | No | The paper does not provide specific details about the hardware (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions software like 'Lush codebase' but does not specify version numbers for any software dependencies (e.g., Python, PyTorch, TensorFlow versions, specific library versions) used for their experiments. |
| Experiment Setup | Yes | Its original implementation is still available as a demonstration in the Lush codebase... The training protocol consists of three sets of 10 epochs with global stepsizes 10 4, 10 5, and 10 6. Each set starts with estimating the diagonal of the Hessian. Per-weight stepsizes are then computed by dividing the global stepsize by the estimated curvature plus 0.02. ... We were only able to replicate the reported 1.6% error rate Le Cun et al. [1998] using minibatches of five or less examples. |