Tuning Frequency Bias in Neural Network Training with Nonuniform Data
Authors: Annan Yu, Yunan Yang, Alex Townsend
ICLR 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This section presents three experiments with synthetic and real-world datasets to investigate the frequency bias of NN training using squared L2 loss and squared Hs loss. |
| Researcher Affiliation | Academia | Annan Yu Cornell University ay262@cornell.edu Yunan Yang ETH Z urich yyn0410@gmail.com Alex Townsend Cornell University townsend@cornell.edu |
| Pseudocode | No | The paper describes methods, such as gradient descent, but does not include any explicitly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | No | The paper does not provide an explicit statement about releasing source code for the described methodology or a link to a code repository. |
| Open Datasets | Yes | MNIST handwritten digit database. ATT Labs [Online]. Available: http://yann.lecun.com/exdb/mnist, 2, 2010. and We utilize a data set {xi}2500 i=1 in (Wright & Michaels, 2015), which comes with carefully designed positive quadrature weights {ci}2500 i=1 . |
| Dataset Splits | No | The paper uses datasets like MNIST but does not provide specific details on training, validation, and test splits (e.g., percentages, sample counts, or citations to predefined splits). |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory, or cloud instance types) used for running the experiments. |
| Software Dependencies | No | The paper mentions 'keras' in a citation but does not list any specific software dependencies with version numbers required for reproducibility. |
| Experiment Setup | Yes | We set up two 2-layer Re LU-activated NNs with 5 104 hidden neurons in each layer and train them using the same training data and gradient descent procedure, except with different loss functions Φ and eΦ. and We train the autoencoder using mini-batch gradient descent with batch size equal to 256. |