Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Perceptual Scales Predicted by Fisher Information Metrics
Authors: Jonathan Vacher, Pascal Mamassian
ICLR 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Here, we demonstrate the value of measuring the perceptual scale of classical (spatial frequency, orientation) and less classical physical variables (interpolation between textures) by embedding it in recent probabilistic modeling of perception. We tested this issue in a series of difference scaling experiments involving GRF and naturalistic textures. |
| Researcher Affiliation | Academia | Jonathan Vacher MAP5, Université Paris Cité, CNRS, F-75006, Paris, France EMAIL Pascal Mamassian LSP, Département d études cognitives, École normale supérieure, PSL University, CNRS, 75005 Paris, France EMAIL |
| Pseudocode | No | The paper describes mathematical models and derivations but does not include any pseudocode or algorithm blocks. |
| Open Source Code | Yes | 1https://github.com/Jonathan Vacher/perceptual_metric 2https://github.com/Jonathan Vacher/texture-interpolation |
| Open Datasets | No | The paper refers to generating stimuli and using 'natural textures available here4' and 'The code to generate these stimuli is available here3'. However, footnotes 3 and 4 provide incomplete URLs (just 'https://'), which do not offer concrete access to the specific datasets used. |
| Dataset Splits | No | The paper describes the method for collecting human perceptual data (difference scaling experiments) and the parameters of the stimuli, but it does not specify explicit train/validation/test splits for the collected human response data, nor for a model's evaluation if such a model were trained on this data. |
| Hardware Specification | No | No specific hardware details (like GPU/CPU models or memory specifications) were provided for running the experiments. |
| Software Dependencies | No | The paper mentions 'MLDS' and 'R' but does not provide specific version numbers for any software dependencies. |
| Experiment Setup | Yes | The experiment consists of trials where participants have to make a similarity judgment. Participants are presented with 3 stimuli with parameters s1 < s2 < s3 and are required to choose which of the two pairs with parameters (s1, s2) and (s2, s3) is the most similar. We used four sets of textures... For each texture pair, we use 13 equally spaced (δs = 0.083) interpolation weights. To ensure that stimulus comparisons are around the discrimination threshold we only use triplets such that |s1,3 s2| 3δs. For each texture pair, a group of 5 naive participants performed the experiment. Participants were recruited through the platform prolific (https://www.prolific.com), performed the experiments online, and were paid 9 /hr. Monitor gamma was measured using a psychometric estimation and corrected to 1. The MLDS model is described at the end of Section 2.2. The protocol was approved by the CER U Paris (IRB 00012020 54). |