Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Spatial-Aware Decision-Making with Ring Attractors in Reinforcement Learning Systems
Authors: Marcos Negre Saura, Richard Allmendinger, Wei Pan, Theodore Papamarkou
NeurIPS 2025 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our approach significantly improves state-of-the-art performance on the Atari 100k benchmark, achieving a 53% increase in performance over selected baselines. |
| Researcher Affiliation | Collaboration | Marcos N. Saura The University of Manchester Manchester, United Kingdom EMAIL Richard Allmendinger The University of Manchester Manchester, United Kingdom EMAIL Wei Pan The University of Manchester Manchester, United Kingdom EMAIL Theodore Papamarkou Poly Shape Athens, Greece EMAIL |
| Pseudocode | Yes | Algorithm 1 CTRNN Ring Attractor Action Selection |
| Open Source Code | Yes | Codebase available at https://github.com/marcosaura/RA_RL. |
| Open Datasets | Yes | This paper addresses the challenge of efficient action selection in Reinforcement Learning (RL), particularly in environments with spatial structures, ranging from robotic manipulation where joint movements are coupled, to game-playing agents where tactical decisions might depend on positional awareness. We integrate ring attractors, a neural circuit model from neuroscience originally proposed by [50] and later experimentally validated by [18], into the RL framework. This approach provides a mechanism for uncertainty-aware decision-making in RL, thereby yielding more efficient and reliable learning in complex environments. Ring attractors offer a unique framework to represent spatial information in a continuous and stable manner [33]. ... 4 Experiments This section presents the findings of our experiments that validate our proposed approach to integrate ring attractors into RL algorithms. To assess the effectiveness of our method, we conducted comparisons between multiple baseline models and action spaces. The evaluation encompasses two implementations: a CTRNN exogenous ring attractor and a DL approach where the ring attractor is modeled directly into DRL agents. In both implementations, the action-value pairs Q(s, a) are evenly distributed across the ring circumference. For the exogenous model, each action is associated with a specific discrete angle or continuous space, section 3.1.2. In the DL implementation, each neuron in the RNN corresponds to one action-value. The ring attractor serves as the output layer of the DL agent with the weights modeling the circular topology of the action space, Section 3.2.1. For both approaches, ring attractor agents are annotated with the suffix RA. We demonstrate that ring attractors significantly enhance action selection and speed up the learning process. ... 4.3 Performance on Atari 100k Benchmark In this results section, we provide a comprehensive analysis of the performance of our DRL model on the Atari 100k benchmark [5]. We present comparisons with state-of-the-art models, highlighting the improvements achieved by our approach. Table 1 presents a comparison of our ring attractor-based DRL model integrated with Efficient Zero [48], which evaluates performance in Atari games with a limited training size of 100,000 environment steps. |
| Dataset Splits | Yes | In this results section, we provide a comprehensive analysis of the performance of our DRL model on the Atari 100k benchmark [5]. We present comparisons with state-of-the-art models, highlighting the improvements achieved by our approach. Table 1 presents a comparison of our ring attractor-based DRL model integrated with Efficient Zero [48], which evaluates performance in Atari games with a limited training size of 100,000 environment steps. ... Game score and overall mean and median human-normalized scores are recorded at the end of training and averaged over 10 random seeds, 3 samples per seed. |
| Hardware Specification | Yes | A.8.1 Computational Resources For the Atari 100K benchmark [5] experiments, we utilized a high-performance computing cluster equipped with 10 NVIDIA A100 GPUs (each with 80 GB memory), 512 GB of RAM, and 128 Intel Xeon CPU cores running at 2.4 GHz. For the other environments, Highway [21] and Super Mario Bros [15], we employed a local workstation with an Intel Xeon processor (28 cores, 2.1 GHz), 125 GB RAM, and dual NVIDIA RTX A5000 GPUs (total 48 GB combined memory). |
| Software Dependencies | Yes | All experiments were conducted using Py Torch 1.12 with CUDA 11.6. |
| Experiment Setup | Yes | The uncertainty-aware version (BDQNRA-UA) shows the best overall performance, highlighting the benefits of combining ring attractors spatial distribution of the action space with uncertainty-aware action selection. ... Figure 2: Learning speed comparison. ... BDQNRA with ring attractor behavior policy from Section 3.1.2, setting the action-value pair variance constant to σa = π/6 ... A.5.1 Experimental Protocol ... The experimental setup consisted of an 8-neuron ring configuration with parameters tuned for single stable bump formation: τ = 120.0 (temporal integration constant) and β = 22.0 (output scaling). |