Adversarial Prediction Games for Multivariate Losses
Authors: Hong Wang, Wei Xing, Kaiser Asif, Brian Ziebart
NeurIPS 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate our approach, Multivariate Prediction Games (MPG), on the three performance measures of interest in this work: precision at k, F-score, and DCG. Our primary point of comparison is with structured support vector machines (SSVM)[27] to better understand the trade-offs between convexly approximating the loss function with the hinge loss versus adversarially approximating the training data using our approach. We employ an optical recognition of handwritten digits (OPTDIGITS) dataset [17] (10 classes, 64 features, 3,823 training examples, 1,797 test examples), an income prediction dataset ( a4a ADULT1 [17] (two classes, 123 features, 3,185 training examples, 29,376 test examples), and query-document pairs from the million query TREC 2007 (MQ2007) dataset of LETOR4.0 [23] (1700 queries, 41.15 documents on average per query, 46 features per document). |
| Researcher Affiliation | Academia | Hong Wang Wei Xing Kaiser Asif Brian D. Ziebart Department of Computer Science University of Illinois at Chicago Chicago, IL 60607 {hwang27, wxing3, kasif2, bziebart}@uic.edu |
| Pseudocode | Yes | Algorithm 1 Constraint generation game solver, Algorithm 2 Lagrangian-augmented F-measure Maximizer for adversary player ˇY |
| Open Source Code | No | The paper does not provide explicit statements or links indicating the availability of open-source code for the described methodology. |
| Open Datasets | Yes | We employ an optical recognition of handwritten digits (OPTDIGITS) dataset [17] (10 classes, 64 features, 3,823 training examples, 1,797 test examples), an income prediction dataset ( a4a ADULT1 [17] (two classes, 123 features, 3,185 training examples, 29,376 test examples), and query-document pairs from the million query TREC 2007 (MQ2007) dataset of LETOR4.0 [23] (1700 queries, 41.15 documents on average per query, 46 features per document). Footnote 1: http://www.csie.ntu.edu.tw/ cjlin/libsvmtools/datasets/binary.html |
| Dataset Splits | Yes | For OPTDIGITS/ADULT, we use a random 1/3 of the training data as a holdout validation data to select the L2 regularization parameter trade-off C {2^6, 2^5, ..., 2^6}. We compare the performance of our approach and comparison methods using five-fold cross validation on the MQ2007 dataset. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for running the experiments. |
| Software Dependencies | No | The paper mentions some software like 'libsvmtools' and 'SVM-struct' but does not specify version numbers for any software dependencies used in its own experimental setup. |
| Experiment Setup | Yes | For OPTDIGITS/ADULT, we use a random 1/3 of the training data as a holdout validation data to select the L2 regularization parameter trade-off C {2^6, 2^5, ..., 2^6}. |