Implicit Neural Representation Inference for Low-Dimensional Bayesian Deep Learning
Authors: Panagiotis Dimitrakopoulos, Giorgos Sfikas, Christophoros Nikou
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimentally, our approach compares favorably to the state of the art, including much more expensive methods as well as less expressive posterior approximations over full network parameters. |
| Researcher Affiliation | Academia | 1Dept. of Computer Science & Engineering, University of Ioannina, Ioannina Greece 2Dept. of Surveying & Geoinformatics, University of West Attica, Athens Greece 1{p.dimitrakopoulos,cnikou}@uoi.gr, 2gsfikas@uniwa.gr |
| Pseudocode | Yes | Algorithm 1 Training procedure Require: I (Indices of main network weights), Net (Main network), INR (INR hypernetwork), Dataset. for each Epoch do for (x, y) in Dataset do y = Net(x,ξ) loss = (y,y ) update INR w.r.t loss update Net w.r.t loss end for end for Algorithm 2 Inference procedure Require: I (Indices), Net (Main network), INR (INR hypernetwork), Testset Approximate Inference (Approximate inference method) MC Samples (Number of Monte Carlo samples). for x in Testset do for j in range MC Samples do ξj = Approximate Inference(INR, I) y = Net(x,ξj) end for end for Calculate y statistics |
| Open Source Code | No | The paper does not include any explicit statements about making the source code publicly available, nor does it provide a link to a code repository. |
| Open Datasets | Yes | We use a synthetic 1D regression task with three disjoint clusters of input data as proposed in Izmailov et al. (2020). We next test our method on the UCI regression tasks (Asuncion & Newman, 2007). We evaluate our method on standard image classification tasks over the CIFAR10, CIFAR100 (Krizhevsky et al., 2009) datasets. |
| Dataset Splits | Yes | We experiment with 8 UCI regression datasets using standard training-evaluation-test splits from Hern andez-Lobato & Adams (2015) and their GAP-variants (Foong et al., 2019). We employed early stopping if validation performance does not increase for 10 consecutive epochs. |
| Hardware Specification | No | The paper does not provide specific hardware details such as CPU, GPU models, or memory specifications used for running the experiments. It only mentions 'Inference time (Table 2) for Resnet18 combined with different stochastic subspaces and different approximate inference methods was measured in seconds and for a batch of 10 CIFAR images.' without specifying the hardware used for this measurement. |
| Software Dependencies | No | The paper mentions software components like 'Adam optimizer' and 'SIREN' but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | All models are trained using the Adam optimizer with learning rate equal to 10 3, weight decay equal to 10 6 and a batch size equal to 256 running for 100 epochs. The INR network has 3 layers consisting of [2, 10, 4] neurons respectively, resulting totally in 160 training parameters. Concerning the hyperparameters we used Ω1 = 30 for the first INR layer and Ωl = 1 for the rest while keeping the the parameter c = 1 fixed for all layers. |