Non-Smooth Weakly-Convex Finite-sum Coupled Compositional Optimization
Authors: Quanqi Hu, Dixian Zhu, Tianbao Yang
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Lastly, we explore the applications of our algorithms in deep learning for two-way partial AUC maximization and multi-instance two-way partial AUC maximization, using empirical studies to showcase the effectiveness of the proposed algorithms. We justify the effectiveness of the proposed SONX and SONT algorithms for TPAUC Maximization in the regular learning setting and MIL setting [14, 45]. |
| Researcher Affiliation | Academia | Quanqi Hu Department of Computer Science Texas A&M University College Station, TX 77843 quanqi-hu@tamu.edu Dixian Zhu Department of Genetics Stanford University Stanford, CA 94305 dixian-zhu@stanford.edu Tianbao Yang Department of Computer Science Texas A&M University College Station, TX 77843 tianbao-yang@tamu.edu |
| Pseudocode | Yes | Algorithm 1 Stochastic Optimization algorithm for Non-smooth FCCO (SONX) Algorithm 2 Stochastic Optimization algorithm for Non-smooth TCCO (SONT) |
| Open Source Code | No | The paper does not contain an explicit statement about the release of its source code or provide a link to a code repository. |
| Open Datasets | Yes | For regular TPAUC maximization, we use three molecule datasets as in [44], namely moltox21 (the No.0 target), molmuv (the No.1 target) and molpcba (the No.0 target) [29]. For MIL TPAUC maximization, we use four MIL datasets, including two tabular datasets MUSK2 and Fox, and two medical image datasets Colon and Lung. MUSK2 and Fox are two tabular datasets that have been widely adopted for MIL benchmark study [14]. Colon and Lung are two histopathology (medical image) datasets... Data available: https://www.kaggle.com/datasets/biplobdey/lung-and-colon-cancer |
| Dataset Splits | Yes | For all MIL datasets, we uniformly randomly split 10% as the testing and the remaining as the training and validation. The statistics for all used datasets are summarized in Table 3and Table 4 in Appendix F. ... For all experiments, we utilize 5-fold-cross-validation to evaluate the testing performance based on the best validation performance with possible early stopping choice. |
| Hardware Specification | No | The paper does not provide specific hardware details such as GPU models, CPU types, or cloud instance specifications used for experiments. |
| Software Dependencies | No | The paper mentions software components like 'Graph Neural Network (GNN)', 'Graph Isomorphism Network (GIN)', 'sigmoid function', 'Feed Forward Neural Network (FFNN)', and 'Res Net20' but does not specify any version numbers for these or other software dependencies. |
| Experiment Setup | Yes | The model is trained by 60 epochs and the learning rate is decreased by 10-fold after every 20 epochs. The model is initialized as a pretrained model from CE loss on the training datasets. We fix the learning rate of SONX as 1e-2 and moving average parameter τ as 0.9; tune the parameter γ in {0, 1e-1,1e-2,1e-3}, the parameter α, β in {0.1,0.3,0.5} and fix the margin parameter of the surrogate loss ℓas 1.0, which cost the same tuning effort as the other baselines. The weight decay is set as the same value (2e-4) with the other baselines. ... The training epoch number is fixed as 100 epochs for all methods; the bag batch size is fixed as 16 (resp. 8) and the number of sampled instances per bag is fixed as 4 (resp. 128) for tabular (resp. medical image) datasets; the learning rate is tuned in {1e-2, 1e-3, 1e-4} and decreased by 10 folds at the end of 50-th and 75-th epoch for all baselines. For SONT (att), we set moving average parameter τ1 = τ2 as 0.9; tune the parameter γ1 = γ2 = γ in {0, 1e-1,1e-2,1e-3} and fix the margin parameter of the surrogate loss ℓas 0.5, and the parameter α, β in {0.1,0.5,0.9}. |