HC-Search for Multi-Label Prediction: An Empirical Study
Authors: Janardhan Rao Doppa, Jun Yu, Chao Ma, Alan Fern, Prasad Tadepalli
AAAI 2014 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We empirically evaluate our instantiation of the HC-Search framework along with many existing multilabel learning algorithms on a variety of benchmarks by employing diverse task loss functions. |
| Researcher Affiliation | Academia | Janardhan Rao Doppa, Jun Yu, Chao Ma, Alan Fern and Prasad Tadepalli {doppa, yuju, machao, afern, tadepall}@eecs.oregonstate.edu School of EECS, Oregon State University, Corvallis, OR, USA, 97331 |
| Pseudocode | Yes | Algorithm 1 Heuristic Function Learning for Greedy Search Input: D = Training data, (I, S) = Search space, L = Loss function, τmax = no. of training steps 1: Initialize the set of ranking examples R = 2: for each training example (x, y ) D do 3: s0 I(x) // initial state 4: for each search step t = 1 to τmax do 5: Generate example Rt to imitate this search step 6: Aggregate training data: R = R Rt 7: st arg mins S(st 1) L(s) // oracle search 8: end for 9: end for 10: H = Rank-Learner(R) 11: return heuristic function H |
| Open Source Code | No | The paper provides links to source code for baseline methods (CCA, PIR, SML, RML) but does not provide open-source code for their own proposed MLS approach. |
| Open Datasets | Yes | We employ nine benchmark3 datasets for our evaluation. We selected these datasets based on the diversity of domains (text, images, audio and bio-informatics) and their popularity within the multi-label learning community. Table 2 presents the properties of different datasets. Footnote 3: http://mulan.sourceforge.net/datasets.html |
| Dataset Splits | Yes | Ten percent of the training data were used to tune the hyper-parameters. For cost function learning, we experimented with 3 folds and 5 folds, but larger folds didn t help much. We employ SVM-Rank (Joachims 2006) as our base rank learner for both heuristic and cost function learning. The C parameter was tuned using the validation set. The regularization parameter was tuned via 5-fold cross validation. |
| Hardware Specification | No | The paper does not mention any specific hardware (e.g., GPU, CPU models, memory amounts) used for running the experiments. |
| Software Dependencies | No | The paper mentions software like 'SVM-Rank' and 'Meka package', and 'logistic regression with L2 regularization', but does not specify version numbers for these software dependencies. |
| Experiment Setup | Yes | Ten percent of the training data were used to tune the hyper-parameters. For cost function learning, we experimented with 3 folds and 5 folds... The C parameter was tuned using the validation set. The MLS approach cannot work for Exact-Match loss5, so we present the Exact Match results by training with Hamming loss. In all other cases, we train for the given task loss function. For the methods that require a base classifier, we employed logistic regression with L2 regularization. The regularization parameter was tuned via 5-fold cross validation. We employed the natural ordering of labels for CC and 20 random orderings for ECC. We used 10 iterations for learning the ordering, and 100 iterations for inference with M2CC. The parameters of CCA were tuned as described in the original paper. For PIR, we tuned the hyper-parameter λ via 5-fold cross-validation and report the results with their exact algorithm (EFP). The hyper-parameters for RML (λ), and SML (λ, C) were tuned based on the validation set. For Dec L, we employed the largest value of k that was practical for training the cost function. |