Similarity-Based Reasoning, Raven's Matrices, and General Intelligence
Authors: Can Serif Mekik, Ron Sun, David Yun Dai
IJCAI 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This paper presents a model tackling a variant of the Raven's Matrices family of human intelligence tests along with computational experiments. |
| Researcher Affiliation | Academia | Can Serif Mekika , Ron Suna, David Yun Daib a Rensselaer Polytechnic Institute b State University of New York at Albany |
| Pseudocode | Yes | 3.3 Algorithm To put everything together, in response to a matrix with 8 alternative answers (e.g., Figure 1a) the model proceeds as follows: 1. Using a convolutional neural network, subjective probabilities are obtained for each individual feature in each matrix sequence and each alternative sequence. 2. Feature distributions for matrix sequences are combined, row-wise and column-wise, using geometric means (Equations 4 and 5). 3. For each alternative, the similarity of the 2 resulting alternative sequences and the 2 combined matrix sequences is computed, with regard to each feature, row-wise and column-wise, through relative entropy (Equation 7). 4. The 8 resulting similarity measures are used to stochastically select an answer out of the 8 alternatives, using the Boltzmann distribution (Equation 3). |
| Open Source Code | Yes | 1 Experimental notes, data and code are available at https://osf.io/7fy2d/. |
| Open Datasets | Yes | The model was tested on a set of matrix problems generated using the Sandia Matrix Generation Tool [Matzen et al., 2010]. |
| Dataset Splits | No | The paper states 'A total of 1480 input-output pairs (20 from each of 74 matrices) were used to train the model' and mentions 'test items', but does not explicitly provide details about a separate validation split or its size/percentage. No cross-validation setup is mentioned for splitting data. |
| Hardware Specification | No | The paper describes the neural network architecture and training procedure but does not provide any specific details about the hardware (e.g., CPU, GPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions algorithms like ADADELTA but does not list specific software dependencies with version numbers (e.g., Python, TensorFlow, PyTorch versions). |
| Experiment Setup | Yes | The network was trained using the ADADELTA algorithm [Zeiler, 2012] for 24000 epochs with learning parameters 𝜌 = .6 and 𝜖 = 10i)Å. The Xavier method [Glorot and Bengio, 2010] was used for network weight initialization. Training samples were shuffled before being split into 20 mini-batches of 74 input-output pairs. Mini-batch order was shuffled at the start of each epoch, and, every 500 epochs, network weights were recorded. A cross-entropy error measure augmented with 𝐿) (𝜆) = .003) and 𝐿* (𝜆* = .003) weight penalties was used to compute weight updates. |