PAC-Bayesian Bounds on Rate-Efficient Classifiers
Authors: Alhabib Abbas, Yiannis Andreopoulos
ICML 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To address a class of problems where inputs are typically costly to compress and stream prior to inference, and to overlap our evaluation with vision applications where inputs undergo lossy compression, we focus our experimental validation on joint image compression and classification. We evaluate the measures and bounds of Theorem 3 and Corollary 3 on the handwritten digits dataset MNIST (Deng, 2012) in two distinct settings: 1. Controlled test conditions where noise is drawn directly from gaussian densities N(0, σ2 c) to perturb inputs x, and σ2 c is directly specified. 2. A distributed visual inference setting, where inputs are compressed via JPEG2000 (Rabbani, 2002) prior to inference, and compression is tuned via a quality parameter q. |
| Researcher Affiliation | Collaboration | 1Meme Research Ltd., London, UK 2Dept. of Electronic and Electrical Eng., University College London, London, UK. |
| Pseudocode | No | The paper does not contain pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | Specifically, our implementation2 assigns training sets S of m training examples x i S to constitute kernels of linear voters hi(x) = x ix that define a majority vote classifier BQ(x), and we use the quadratic program of (Jean, 2019) to solve for Q(hi|S). The footnote 2 points to: 'https://github.com/git-alhabib/pacb-ni' |
| Open Datasets | Yes | Adapting MNIST to binary classification: Following established practice (Germain et al., 2015; Letarte et al., 2019), we split MNIST into 45 binary classification tasks1, where each task is exclusive to a unique pair of MNIST classes. For each task, a training set S with m = 500 is randomly sampled, and remaining examples go to a test set T used for validation. Each task uses a unique set of examples, such that any pair of datasets returns the empty set, and combining all datasets returns all examples in MNIST. All results are averaged after validating on all 45 unique tasks. |
| Dataset Splits | Yes | For each task, a training set S with m = 500 is randomly sampled, and remaining examples go to a test set T used for validation. Each task uses a unique set of examples, such that any pair of datasets returns the empty set, and combining all datasets returns all examples in MNIST. All results are averaged after validating on all 45 unique tasks. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., CPU/GPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper mentions using the 'GRAAL-Research Min Cq implementation (Jean, 2019)' but does not specify software versions for programming languages or other libraries, such as Python or PyTorch. |
| Experiment Setup | Yes | To validate the unsupervised bounds of Theorem 3 and Corollary 3, Table 1 and Figure 2 report relevant empirical measures on noise invariance when δ = 0.05 and δη = 0.05. For all parameters not explicitly mentioned in our discussion, we retain specifications of the GRAAL-Research Min Cq implementation (Jean, 2019). |