OpenFE: Automated Feature Generation with Expert-level Performance
Authors: Tianping Zhang, Zheyu Aqa Zhang, Zhiyuan Fan, Haoyan Luo, Fengyuan Liu, Qian Liu, Wei Cao, Li Jian
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Extensive experiments on ten benchmark datasets show that Open FE outperforms existing baseline methods by a large margin. We further evaluate Open FE in two Kaggle competitions with thousands of data science teams participating. |
| Researcher Affiliation | Collaboration | 1Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University, Beijing, China 2School of Data Science, The Chinese University of Hong Kong (Shenzhen), Shenzhen, China 3Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, U.S. 4Sea AI Lab, Singapore 5Microsoft Research Asia, Beijing, China. |
| Pseudocode | Yes | Algorithm 1 Open FE Input: D: dataset, T : feature set, O: operators Output: new feature set... Algorithm 2 Feature Boost Input: D: dataset, T : feature set, ˆy: predictions on T Output: incremental performance of T... Algorithm 3 Successive Pruning Input: D: dataset, ˆy: predictions on T , A(T ): candidate feature set, q: integer Output: pruned new feature set... Algorithm 4 Feature Attribution Input: D: dataset, T : feature set, A (T ): candidate feature set, ˆy: predictions on T Output: sorted A (T ). |
| Open Source Code | Yes | The codes and datasets are available at https://github.com/IIIS-Li-Group/Open FE |
| Open Datasets | Yes | The codes and datasets are available at https://github.com/IIIS-Li-Group/Open FE. We collect a diverse set of ten public datasets. Most datasets are frequently used in previous studies (Gorishniy et al., 2021; 2022; Grinsztajn et al., 2022; Strack et al., 2014; Siebert, 1987; Candillier & Lemaire, 2012). Each dataset has exactly one train-validation-test split, and all methods use the same split. |
| Dataset Splits | Yes | For a given training dataset D, we split it into a sub-training set Dtr and a validation set Dvld. Each dataset has exactly one train-validation-test split, and all methods use the same split. Hyperparameter tuning follows a standard benchmarking study (Gorishniy et al., 2021), and we tune the hyperparameters using the base feature set. |
| Hardware Specification | Yes | For all the experiments, feature generation is carried out on a workstation with Intel(R) Xeon(R) Gold 6230 CPU @ 2.10GHz, 40 cores, 512G memory. Model tuning and model training are performed on one or more NVidia Tesla V100 16Gb. |
| Software Dependencies | No | The paper mentions specific software implementations like 'Light GBM implementation (Ke et al., 2017)', 'XGBoost (Chen & Guestrin, 2016)', and 'Catboost (Prokhorenkova et al., 2018)'. While these are specific tools, precise version numbers for these software libraries are not provided, except for general Python version (e.g., 'Python 3.8' from the linked GitHub, but not explicitly in the paper's text for dependencies). |
| Experiment Setup | Yes | In Open FE, the default parameter of Light GBM in successive featurewise pruning has 1000 number of estimators, 0.1 learning rate, 16 leaves, and 3 early stopping rounds. The default parameter in feature importance attribution is the same except for 50 early stopping rounds. |