Principal Bit Analysis: Autoencoding with Schur-Concave Loss
Authors: Sourbh Bhadane, Aaron B Wagner, Jayadev Acharya
ICML 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate the PBA algorithm experimentally by comparing the performance of a PBA-derived fixed-rate compressor against the performance of baseline fixed-rate compressors. The code of our implementation can be found at https://github.com/Sourbh Bh/PBA. Although variable-rate codes are more commonplace in practice, fixedrate codes do offer some advantages over their more general counterparts: [...] We evaluate the PBA compressor on MNIST (Le Cun et al., 1998), CIFAR-10 (Krizhevsky, 2009), MIT Faces Dataset (Fac), Free Spoken Digit Dataset (FSDD) (Jackson). We compare our algorithms mainly using mean-squared error since our theoretical analysis uses mean squared error as the distortion metric. Our plots display Signal-to-Noise ratios (SNRs) for ease of interpretation. For image datasets, we also compare our algorithms using the Structural Similarity (SSIM) or the Multi-scale Structural Similarity (MS-SSIM) metrics when applicable (Wang et al., 2004). We also consider errors on downstream tasks, specifically classification, as a distortion measure. |
| Researcher Affiliation | Academia | Sourbh Bhadane 1 Aaron B. Wagner 1 Jayadev Acharya 1 1Cornell University. Correspondence to: Sourbh Bhadane <snb62@cornell.edu>. |
| Pseudocode | Yes | Algorithm 1 Principal Bit Analysis (PBA) Require: λ > 0, α = γσ2 > 2, σ2 1 0 0 0 σ2 2 0 ... ... ... ... 0 0 σ2 d such that σ2 1 σ2 2 σ2 d. 1: If λ σ2 1/(4(α 1)), Output Ropt = 0, Dopt = Pd i=1 σ2 i . 2: Set d = max i : λ < σ2 i /4(α 1) . 3: Set R, D to zero arrays of size 2 d. 4: for r 1, 2, d do 5: D(2r 1) = r P σ2 i 2(α 1) (1 ci) + d P 6: R(2r 1) = r P 1 2 log σ2 i 4λ + log (1 + ci). 7: D(2r) = r 1 P σ2 i 2(α 1) (1 ci) + σ2 r 2(α 1) (1 + cr) + 8: R(2r) = r P 1 2 log σ2 i 4λ + r 1 P i=1 log (1 + ci) + log (1 cr). 9: end for 10: r arg minj [2 d] D(j) + λ R(j). 11: Output Ropt = R(r ), Dopt = D(r ). |
| Open Source Code | Yes | The code of our implementation can be found at https://github.com/Sourbh Bh/PBA. |
| Open Datasets | Yes | We evaluate the PBA compressor on MNIST (Le Cun et al., 1998), CIFAR-10 (Krizhevsky, 2009), MIT Faces Dataset (Fac), Free Spoken Digit Dataset (FSDD) (Jackson). |
| Dataset Splits | No | We divide the dataset into three parts. From the first part, we obain the covariance matrix that we use for PCA and the PBA compressor. The second and third part are used as training and testing data for the purpose of classification. The paper does not provide specific percentages or counts for training/validation/test splits, nor explicitly mention a validation set. |
| Hardware Specification | No | The paper does not explicitly describe the specific hardware (e.g., GPU/CPU models, memory) used to run its experiments. |
| Software Dependencies | No | The paper does not provide specific software names with version numbers for reproducibility. |
| Experiment Setup | Yes | For our experiments, we fix the a parameter at 15 and hard code this in both the encoder and the decoder. |