Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Demystifying Orthogonal Monte Carlo and Beyond
Authors: Han Lin, Haoxian Chen, Krzysztof M. Choromanski, Tianyi Zhang, Clement Laroche
NeurIPS 2020 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | 5 Experiments We empirically tested NOMCs in two settings: (1) kernel approximation via random feature maps and (2) estimating sliced Wasserstein distances, routinely used in generative modeling [42]. For (1), we tested the effectiveness of NOMCs for RBF kernels, non-RBF shift-invariant kernels as well as several PNG kernels. For (2), we considered different classes of multivariate distributions. ... The results are presented in Fig. 2 and Fig. 3. Empirical MSEs were computed by averaging over k = 450 independent experiments. |
| Researcher Affiliation | Collaboration | Han Lin Columbia University EMAIL Haoxian Chen Columbia University EMAIL Tianyi Zhang Columbia University EMAIL Clement Laroche Columbia University EMAIL Krzysztof Choromanski Google Brain Robotics & Columbia University EMAIL |
| Pseudocode | Yes | Algorithm 1: Near Orthogonal Monte Carlo: opt-NOMC variant |
| Open Source Code | Yes | Detailed code implementation is available at https://github.com/HL-hanlin/OMC. |
| Open Datasets | No | The paper discusses empirical testing for 'kernel approximation' and 'estimating sliced Wasserstein distances' using 'RBF kernels, non-RBF shift-invariant kernels as well as several PNG kernels' and 'different classes of multivariate distributions'. However, it does not provide specific names of public datasets (e.g., CIFAR-10, MNIST), nor does it give links, DOIs, or formal citations for any datasets used in the experiments. |
| Dataset Splits | No | The paper mentions 'Empirical MSEs were computed by averaging over k = 450 independent experiments' and discusses varying the 'number of blocks (i.e. the ratio of the number of samples D used and data dimensionality d),' but it does not provide specific training, validation, or test dataset splits (e.g., percentages, sample counts, or references to predefined splits). |
| Hardware Specification | No | The paper does not provide any specific hardware details such as GPU or CPU models, processor types, or memory amounts used for running its experiments. |
| Software Dependencies | No | The paper provides a link to a GitHub repository, implying software usage, but it does not specify any software dependencies with version numbers (e.g., 'Python 3.x', 'PyTorch 1.x'). |
| Experiment Setup | No | The paper mentions some experimental details such as 'Empirical MSEs were computed by averaging over k = 450 independent experiments' and tunable parameters for NOMC like 'Parameter δ, ε, T' and 'δ > 0'. However, it does not provide concrete values for these or other typical experimental setup details like learning rates, batch sizes, or specific number of iterations (T) used in the reported experiments. |