Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Generalizing Neural Additive Models via Statistical Multimodal Analysis
Authors: Young Kyung Kim, Juan Matias Di Martino, Guillermo Sapiro
TMLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Section 3 presents empirical evidence and pedagogical studies, showing strengths of MNAM. We evaluate six datasets: the California Housing (CA Housing) (Pace & Barry, 1997), the Fair Isaac Corporation (FICO) (FICO, 2018), the New York Citi Bike (BIKE) (Vanschoren et al., 2013), the Medical Information Mart for Intensive Care (MIMIC-III) (Johnson et al., 2016), the US Census data on Income (ACS Income) for California (Ding et al., 2021), and the US Census data on Travel time (ACS Travel) for California (Ding et al., 2021). Table 1 displays the MAE and LL scores of NAM, p NAM, and MNAM on datasets described above. |
| Researcher Affiliation | Academia | Young Kyung Kim EMAIL Department of Electrical and Computer Engineering Duke University J. Matías Di Martino EMAIL Department of Electrical and Computer Engineering Duke University Guillermo Sapiro EMAIL Department of Electrical and Computer Engineering Duke University |
| Pseudocode | Yes | Algorithm 1 Mixture Neural Additive Models Input: Data: (X1, ...Xn), Number of Features: n, Number of Gaussian Distributions: k, Neural Networks for Feature Transformation: (f1, ..., fn), Neural Network for Probability: g Output: Mixture of Gaussian Distributions: N1(µ1, σ2 1), ..., Nk(µk, σ2 k), π1, ..., πk for i = 1 to n do Ni,1(µi,1, σ2 i,1), ..., Ni,k(µi,k, σ2 i,k), Zπ i = fi(Xi) end for for i = 1 to k do µi = Pn j=1 µj,i σ2 i = Pn j=1 σ2 j,i end for π1, ..., πk = g(Zπ 1 , ..., Zπ n) |
| Open Source Code | Yes | The code is available at https://github.com/youngkyungkim93/MNAM. The code and pipelines to reproduce our work are open and can be found here: https://github.com/youngkyungkim93/MNAM. |
| Open Datasets | Yes | We evaluate six datasets: the California Housing (CA Housing) (Pace & Barry, 1997), the Fair Isaac Corporation (FICO) (FICO, 2018), the New York Citi Bike (BIKE) (Vanschoren et al., 2013), the Medical Information Mart for Intensive Care (MIMIC-III) (Johnson et al., 2016), the US Census data on Income (ACS Income) for California (Ding et al., 2021), and the US Census data on Travel time (ACS Travel) for California (Ding et al., 2021). |
| Dataset Splits | Yes | We used a 5-fold cross-validation for CA Housing, FICO, and MIMIC-III datasets, and a 3-fold cross-validation for BIKE, ACS Income, and ACS Travel datasets. For evaluation, we trained 20 different models by randomly splitting the train set into train and validation sets for each fold. |
| Hardware Specification | No | The paper mentions 'Due to limited computation resources' but does not provide any specific hardware details such as GPU models, CPU types, or memory. |
| Software Dependencies | No | The paper mentions using 'Adam (Kingma & Ba, 2014)' and 'Bayesian optimization (Močkus, 1975)' and various models, but it does not specify any software libraries or frameworks with their version numbers (e.g., PyTorch 1.9, TensorFlow 2.x, Scikit-learn 1.x). |
| Experiment Setup | Yes | Learning rate, weight decay, and output penalty are finetuned for NAM. Learning rate, weight decay, output penalty, number of Gaussian distributions, and lambda for the cross-entropy loss are finetuned for MNAM. For both models, we utilized early stopping to reduce overfitting. Optimized parameters from Bayesian optimization can be found in the table from Appendix A. Table 5: Optimized parameters for MNAM on six datasets (listing specific values for Learning rate, Weight decay, Output penalty, Number of Gaussian distribution, Cross-entropy loss). |