COCO-Periph: Bridging the Gap Between Human and Machine Perception in the Periphery
Authors: Anne Harrington, Vasha DuTell, Mark Hamilton, Ayush Tewari, Simon Stent, William T. Freeman, Ruth Rosenholtz
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We evaluate this by (1) modifying the texture tiling model (TTM), a well tested model of peripheral vision, to be more flexibly used with DNNs, (2) generating a large dataset which we call COCO-Periph that contains images transformed to capture the information available in human peripheral vision, and (3) comparing DNNs to humans at peripheral object detection using a psychophysics experiment. |
| Researcher Affiliation | Collaboration | 1MIT CSAIL 2MIT Brain and Cognitive Sciences 3 Toyota Research Institute |
| Pseudocode | Yes | We provide pseudo-code for the machine psychophysics procedure. Algorithm 1 For each object present/absent image in the human experiment, we create 100 pairings of uniform TTM transform images (P and A). We simulate trials by looking at the box predictions (boxes) of a detection DNN for each pairing (p, a). |
| Open Source Code | Yes | We publicly release our COCO-Periph dataset, along with code for uniform TTM and the psychophysics analyses at https://github.com/Rosenholtz Lab/COCOPeriph to enable further research into human and machine perception in the periphery paving the way for DNNs to mimic and benefit from properties of human visual processing. |
| Open Datasets | Yes | We apply Uniform TTM to the COCO dataset, creating COCO-Periph which contains images transformed like peripheral vision. [...] COCO-Periph contains the entire COCO 2017 validation and test set, and over 74K, 117K, 118K, and 97K of the 118K total training images transformed to model 5 , 10 , 15 , and 20 of eccentricity, respectively. |
| Dataset Splits | Yes | COCO-Periph contains the entire COCO 2017 validation and test set |
| Hardware Specification | No | The paper mentions using 'MIT Super Cloud' and 'Lincoln Laboratory Supercomputing Center' for 'HPC resources' but does not specify exact hardware components like GPU/CPU models or specific configurations. |
| Software Dependencies | No | The paper mentions using Detectron2, the La Ma image in-painting model, Eye Link 1000, and the detrex library, but does not provide specific version numbers for these software components. |
| Experiment Setup | Yes | All models are trained for 180,000 iterations starting from the weights of a pretrained R-CNN from (Wu et al., 2019). We set the solver to step at 120,000 and 160,000. We set the base learning rate to 3 10 4. All other training parameters are the same R-CNN training parameters in (Wu et al., 2019) as the baseline model. |