Stochastic Adaptive Activation Function
Authors: Kyungsu Lee, Jaeseung Yang, Haeyun Lee, Jae Youn Hwang
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To validate the effectiveness and robustness of ASH, we implemented ASH into many deep learning models for various tasks, including classification, detection, segmentation, and image generation. Experimental analysis demonstrates that our activation function can provide the benefits of more accurate prediction and earlier convergence in many deep learning applications. |
| Researcher Affiliation | Collaboration | Kyungsu Lee DGIST 42988 Daegu, South Korea ks_lee@dgist.ac.kr Jaeseung Yang DGIST 42988 Daegu, South Korea yjs6813@dgist.ac.kr Haeyun Lee DGIST, SAMSUNG SDI 17084 Yong-In, South Korea haeyun.lee@samsung.com Jae Youn Hwang DGIST 42988 Daegu, South Korea jyhwang@dgist.ac.kr |
| Pseudocode | No | The paper does not contain any structured pseudocode or algorithm blocks. It describes the mathematical formulation of the proposed activation function. |
| Open Source Code | Yes | 1Our code is available at https://github.com/kyungsu-lee-ksl/ASH |
| Open Datasets | Yes | We first compared ASH to all the baseline activation functions on the CIFAR-10 (Krizhevsky et al., 2009), CIFAR-100 (Krizhevsky et al., 2009) datasets, and Image Net (Russakovsky et al., 2015) datasets. |
| Dataset Splits | No | The paper mentions 'validation loss' but does not explicitly provide the specific percentages or sample counts for training, validation, and test dataset splits in the main body of the paper. It refers to the appendix for training details. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running the experiments in the main text. The checklist indicates this information is in the Appendix. |
| Software Dependencies | No | The paper does not provide specific ancillary software details with version numbers (e.g., Python 3.x, PyTorch x.x, TensorFlow x.x) that were used to replicate the experiments. |
| Experiment Setup | No | The paper mentions that 'every hyper-arameter in ASH and the other activation functions was set to be the same' but does not explicitly list these specific hyperparameter values (e.g., learning rate, batch size, number of epochs) or other training configurations in the main text. It refers to previous studies for environments and the appendix for training details. |