Neural Concept Binder
Authors: Wolfgang Stammer, Antonia Wüst, David Steinmann, Kristian Kersting
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We validate the effectiveness of NCB through evaluations on our newly introduced CLEVR-Sudoku dataset. Code and data at: project page.In our evaluations, we investigate the potential of NCB s soft and hard binding mechanisms in unsupervised concept learning and its integration into downstream tasks. |
| Researcher Affiliation | Academia | 1Computer Science Department, TU Darmstadt; 2Hessian Center for AI (hessian.AI); 3German Research Center for AI (DFKI); 4Centre for Cognitive Science, TU Darmstadt |
| Pseudocode | Yes | We formally describe these steps using the pseudocode in Alg. 1. |
| Open Source Code | Yes | Code and data at: project page. We refer to our code for more details3, where trained model checkpoints and corresponding parameter logs are available. Footnote 3: Code available here. |
| Open Datasets | Yes | Code and data at: project page. In this context, we introduce our novel CLEVR-Sudoku dataset, which presents a challenging visual puzzle that requires both perception and reasoning capabilities (cf. Fig. 4). Data. We focus our evaluations on different variations of the popular CLEVR dataset. Specifically, we investigate (Q1 & Q3) in the context of the CLEVR [30] and CLEVR-Easy [65] datasets. For investigating the integration of NCB into symbolic modules (Q2), we utilize our novel CLEVR-Sudoku puzzles introduced in the following. We provide further details on these datasets in the supplements (cf. Suppl. C). The already generated data files are accessible under https://huggingface.co/datasets/AIML-TUDA/CLEVR-Sudoku. |
| Dataset Splits | Yes | We evaluate all models based on their accuracies on held-out test splits, each with 3 seeded runs. In our evaluations investigating only neural-based classification we utilize the original validation split as the held-out test split and select a subset from the original training split as validation set. |
| Hardware Specification | Yes | The resources used for training NCB were: CPU: AMD EPYC 7742 64Core Processor, RAM: 2064 GB, GPU: NVIDIA A100-SXM4-40GB GPU with 40 GB of RAM. |
| Software Dependencies | No | No specific version numbers for software dependencies were explicitly stated in the paper. It mentions "HDBSCAN method [12, 13, 54] (based on the popular HDBSCAN library2)" and "the default parameters of the sklearn library" without versions. |
| Experiment Setup | Yes | the soft binder was trained as in the original setup and with the published hyperparameters. Furthermore, s R represents the argmin selection function and we utilize the euclidean distance as d( , ). In our evaluations we therefore select only a single slot per image. The classifiers parameters correspond to the default parameters of the sklearn library. For each puzzle we fit 10 independent classifiers (each with different seeds) to predict the corresponding mapping. |