Inner Product-based Neural Network Similarity
Authors: Wei Chen, Zichen Miao, Qiang Qiu
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We provide both theoretical and empirical evidence that such simplified filter subspace-based similarity preserves a strong linear correlation with other popular probing-based metrics, while being significantly more efficient to obtain and robust to probing data. |
| Researcher Affiliation | Academia | Wei Chen , Zichen Miao , Qiang Qiu Department of ECE Purdue University {chen2732, miaoz, qqiu}@purdue.edu |
| Pseudocode | No | The paper describes methods and theoretical proofs but does not include any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper states: 'The codes are adapted from https://github.com/lgcollins/Fed Rep' referring to code used for comparison, not the authors' own source code for the proposed method. No direct link or statement is provided for the authors' implementation. |
| Open Datasets | Yes | 10-Split CIFAR-100 dataset... CIFAR-100 dataset [23]... Image Net [47]... SVHN [38] |
| Dataset Splits | No | The paper mentions distributing data to clients and breaking down datasets into tasks, and uses standard datasets like CIFAR-100 and ImageNet which have standard splits. However, it does not explicitly state the specific training/validation dataset split percentages or sample counts used for experiments. |
| Hardware Specification | Yes | For each method, the training takes about 12 hours on Nvidia RTX A5000. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers, such as programming languages, libraries, or frameworks used for implementation. |
| Experiment Setup | Yes | All models are trained for T = 100 communication rounds on datasets. At each round, the client executes 1 epoch of SGD with momentum to train the local model, the learning rate is 0.01 and the momentum is 0.9. |