Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Sum of Ranked Range Loss for Supervised Learning
Authors: Shu Hu, Yiming Ying, Xin Wang, Siwei Lyu
JMLR 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our empirical results highlight the effectiveness of the proposed optimization frameworks and demonstrate the applicability of proposed losses using synthetic and real data sets. We empirically demonstrate the robustness and eļ¬ectiveness of the proposed Ao RR, TKML, TKML-Ao RR, and their optimization frameworks on both synthetic and real data sets. |
| Researcher Affiliation | Academia | Shu Hu EMAIL Department of Computer Science and Engineering University at Buļ¬alo, State University of New York Buļ¬alo, NY 14260-2500, USA Yiming Ying EMAIL Department of Mathematics and Statistics University at Albany, State University of New York Albany, NY 12222, USA Xin Wang EMAIL Department of Computer Science and Engineering University at Buļ¬alo, State University of New York Buļ¬alo, NY 14260-2500, USA Siwei Lyu EMAIL Department of Computer Science and Engineering University at Buļ¬alo, State University of New York Buļ¬alo, NY 14260-2500, USA |
| Pseudocode | Yes | Algorithm 1: DCA for Minimizing So RR Algorithm 2: DCA for Minimizing Ao RR without Setting k and m Algorithm 3: Combination of Ao RR and TKML |
| Open Source Code | Yes | Code available at https://github.com/discovershu/So RR. |
| Open Datasets | Yes | Our empirical results highlight the effectiveness of the proposed optimization frameworks and demonstrate the applicability of proposed losses using synthetic and real data sets. We use the MNIST data set (Le Cun et al., 1998) We use five benchmark data sets from the UCI (Dua and Graļ¬, 2017) and the KEEL (AlcalĆ”-Fdez et al., 2011) data repositories |
| Dataset Splits | Yes | For each data set, we ļ¬rst randomly select 50% samples for training, and the remaining 50% samples are randomly split for validation and testing (each contains 25% samples). To create a validation set, We randomly extract 10, 000 samples from training samples. Therefore, the remaining training data size is 50, 000. For each data set, we randomly partition it to 50%/25%/25% samples for training/validation/testing, respectively. We randomly split Yeast data into two parts, which are 80% samples for training and 20% samples for testing. |
| Hardware Specification | Yes | All algorithms are implemented in Python 3.6 and trained and tested on an Intel(R) Xeon(R) CPU W5590 @3.33GHz with 48GB of RAM. |
| Software Dependencies | Yes | All algorithms are implemented in Python 3.6 |
| Experiment Setup | Yes | Hyper-parameters C, k, and m are selected based on the validation set. Speciļ¬cally, parameter C is chosen from {100, 101, 102, 103, 104, 105}, parameter k ā {1} āŖ [0.1 : 0.1 : 1]n, where n is the number of training samples, and parameter m is selected in the range of [1, k). Algorithm 1: DCA for Minimizing So RR Algorithm 2: DCA for Minimizing Ao RR without Setting k and m Algorithm 3: Combination of Ao RR and TKML |