Heating up decision boundaries: isocapacitory saturation, adversarial scenarios and generalization bounds
Authors: Bogdan Georgiev, Lukas Franken, Mayukh Mukherjee
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimentally, we observe a certain capacitory trend over different adversarial defense strategies...We study the capacitory metric on the well-known CIFAR10 and MNIST datasets...In Fig. 4 we collect the statistics of the WRN and Le Net models on CIFAR10 and MNIST, respectively. |
| Researcher Affiliation | Collaboration | Bogdan Georgiev Fraunhofer IAIS, ML2R bogdan.m.georgiev@gmail.com Lukas Franken Fraunhofer IAIS, ML2R, University of Cologne lukas.b.franken@gmail.com Mayukh Mukherjee IIT Bombay mathmukherjee@gmail.com |
| Pseudocode | Yes | Algorithm 1: Compressing a matrix A Rh1 h2 |
| Open Source Code | Yes | We provide the training as well as the sampling code for our experiments. |
| Open Datasets | Yes | We study the capacitory metric on the well-known CIFAR10 and MNIST datasets |
| Dataset Splits | Yes | The MNIST is a 784-dimensional dataset that consists of 60000 images of handwritten digits whose dimensions are (28, 28); 50000 images were used for training and 10000 for validation. CIFAR-10 is collection of 60000 32-by-32 color images (i.e. a 3072-dimensional dataset) corresponding to 10 different classes...; 50000 images were used for training and 10000 for validation. |
| Hardware Specification | Yes | The experimental section of the work was conducted mainly on a CUDA 10.2 GPU-rack consisting of four NVIDA TITAN V units: this includes the model training as well as Brownian motion sampling and further statistics. |
| Software Dependencies | Yes | The neural network framework of choice was Py Torch 1.5. |
| Experiment Setup | Yes | The training of the Wide-Res Nets followed the framework provided by Cubuk et al. (2018) with weight decay 5e-4, batch size 128 and a decrease of the initial learning rate of 0.1 by a factor 0.2 at epochs 60, 120 and 160. The Res Nets were trained with weight decay 1e-4 respectively and step wise decrease of the learning rate 0.1 by a factor 0.1 at epochs 100 and 150. ... We trained a Le Net-5 architecture Le Cun et al. (1998) over 50 epochs with a learning rate 1e-3 and weight decay 5e-4, batch size of 64, while optimizing cross entropy loss using root mean square propagation. |