Confident Reasoning on Raven’s Progressive Matrices Tests
Authors: Keith McGreggor, Ashok Goel
AAAI 2014 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Finally, we present the performance of our algorithm on the four major variants of the RPM tests, illustrating the impact of confidence. This is the first such account of any computational model against the entirety of the Raven s. and Table 1 shows the results of running the Confident Ravens algorithm on the example problem... and We have tested the Confident Ravens algorithm against the four primary variants of the RPM... |
| Researcher Affiliation | Academia | Keith Mc Greggor and Ashok Goel Design & Intelligence Laboratory, School of Interactive Computing, Georgia Institute of Technology, Atlanta, GA 30332, USA keith.mcgreggor@gatech.edu, goel@cc.gatech.edu |
| Pseudocode | Yes | Algorithm 1. Confident Ravens Preparatory Stage and Algorithm 2. Confident Ravens Execution Stage |
| Open Source Code | Yes | The images and Java source code for this example may be found on our research group s website. and The code used to conduct these tests was precisely the same code as used in the presented example, and is available for download from our lab website. |
| Open Datasets | Yes | We have tested the Confident Ravens algorithm against the four primary variants of the RPM: the 60 problems of the Standard Progressive Matrices (SPM) test, the 48 problems of the Advanced Progressive Matrices (APM) test, the 36 problems of the Coloured Progressive Matrices (CPM) test, and the 60 problems of the SPM Plus test. and Raven, J., Raven, J. C., and Court, J. H. 2003. Manual for Raven's Progressive Matrices and Vocabulary Scales. San Antonio, TX: Harcourt Assessment. |
| Dataset Splits | No | Each problem was solved independently: no information was carried over from problem to problem, nor from test variant to test variant. The paper describes evaluating on entire test sets like SPM, APM, CPM, SPM Plus, but does not specify a train/validation/test split for *their own* model's learning or hyperparameter tuning. |
| Hardware Specification | No | No specific hardware details (GPU models, CPU types, memory) used for running experiments were mentioned in the paper. |
| Software Dependencies | No | The paper mentions 'Java source code' but does not specify its version or any other software dependencies with version numbers. |
| Experiment Setup | Yes | In the present implementation, the abstraction levels are determined to be a partitioning of the given images into gridded sections at a prescribed size and regularity. and We chose a partitioning scheme which started at the maximum dimension, then descended in steps of 10, until it reached a minimum size of no smaller than 4 pixels, yielding 14 to 22 levels of abstraction for each problem. and For those calculations, we used the Tversky contrast ratio formula (1977), and set α to 1.0 and β equal to 0.0 and Let Ε be a real number which represents the number of standard deviations beyond which a value s answer may be judged as confident |