Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Learning Symmetric Rules with SATNet
Authors: Sangho Lim, Eun-Gyeol Oh, Hongseok Yang
NeurIPS 2022 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments with Sudoku and Rubik s cube show the substantial improvement of Sym SATNet over the baseline SATNet. |
| Researcher Affiliation | Academia | Sangho Lim School of Computing KAIST Daejeon, South Korea EMAIL Eun-Gyeol Oh Graduate School of Information Security KAIST Daejeon, South Korea EMAIL Hongseok Yang School of Computing and Kim Jaechul Graduate School of AI, KAIST Discrete Mathematics Group, Institute for Basic Science (IBS) Daejeon, South Korea EMAIL |
| Pseudocode | Yes | Algorithm 1 SYMFIND with a threshold λ > 0 |
| Open Source Code | No | The paper states 'Sym SATNet is implemented based on the SATNet code [26] available under the MIT License.' This refers to a third-party tool they used, not their own source code for Sym SATNet being made publicly available. |
| Open Datasets | Yes | We used 9K training and 1K test examples generated by the Sudoku generator [21]. [21] is Kyubyong Park. Can convolutional neural networks crack sudoku puzzles? https://github. com/Kyubyong/sudoku, 2018. |
| Dataset Splits | Yes | We used 8K training, 1K validation, and 1K test examples to train Sym SATNet with symmetries found by SYMFIND and the validation step. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory) used for running its experiments. |
| Software Dependencies | No | The paper mentions 'Adam optimizer' but does not provide specific software dependencies with version numbers (e.g., Python version, PyTorch/TensorFlow versions, specific library versions). |
| Experiment Setup | Yes | We used binary cross entropy loss and Adam optimizer [16], with the learning rate η = 2 10 3 for SATNet-Plain and SATNet-300aux as the original work and η = 4 10 2 for Sym SATNet. We trained Sym SATNet, SATNet-Plain, and SATNet-300aux for 100 epochs, under the same configuration as in the Sudoku case. |