Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..

Journal of Artificial Intelligence Research (JAIR) - 2025

Documentation Rate of Empirical Papers by Reproducibility Variable

Distribution of Empirical Papers by Number of Documented Variables

Website:

Venue Year Papers
Reproducibility Score Reproducibility Score based on Gundersen et al. (2025). See Methods for details.
Documentation Score Documentation Score is the average score over the seven reproducibility variables for empirical research papers. See Methods for details.
% Empirical Percentage of papers that are empirical research vs theoretical research.
% Industry Percentage of empirical research papers with at least one author from Industry.
Website
JAIR 2025 132 0.51 3.89 72.73% 14.58%
Pseudocode
Open Source Code
Open Datasets
Dataset Splits
Hardware Specification
Software Dependencies
Experiment Setup
A Complexity-Theoretic Analysis of Majority Illusion in Social Networks ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
A Divide, Align and Conquer Strategy For Program Synthesis βœ… βœ… βœ… βœ… βœ… ❌ βœ… 6
A Framework for Belief-based Programs and Their Verification βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
A Graphical Formalism for Reasoning about Substitution in Resource Transforming Procedures ❌ βœ… βœ… ❌ ❌ ❌ βœ… 3
A New Literature Review of 3D Object Detection on Autonomous Driving ❌ ❌ βœ… βœ… ❌ ❌ ❌ 2
A New Regret-analysis Framework for Budgeted Multi-Armed Bandits βœ… ❌ ❌ ❌ βœ… ❌ βœ… 3
A Portfolio Approach to Massively Parallel Bayesian Optimization βœ… βœ… βœ… βœ… βœ… βœ… βœ… 7
A Query-Based Constraint Acquisition Approach for Enhanced Precision in Program Precondition Inference βœ… βœ… βœ… ❌ βœ… ❌ βœ… 5
A Selective Under-Sampling (SUS) Method for Imbalanced Regression βœ… ❌ βœ… βœ… βœ… ❌ βœ… 5
A Survey on Data Selection for LLM Instruction Tuning ❌ ❌ βœ… ❌ ❌ ❌ ❌ 1
A Unifying Framework for Causal Modeling With Infinitely Many Variables βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
AI Governance in the GCC States: A Comparative Analysis of National AI Strategies ❌ ❌ βœ… ❌ ❌ ❌ βœ… 2
AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of Interventions ❌ βœ… ❌ ❌ ❌ ❌ ❌ 1
AI for All: Identifying AI incidents Related to Diversity and Inclusion βœ… ❌ βœ… ❌ ❌ ❌ ❌ 2
Adaptive Few-Shot Class-Incremental Learning via Latent Variable Models βœ… ❌ βœ… βœ… βœ… ❌ βœ… 5
Against The Achilles' Heel: A Survey on Red Teaming for Generative Models ❌ ❌ βœ… ❌ ❌ ❌ ❌ 1
Agentic Large Language Models, a Survey βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
An Extensive Empirical Evaluation of Inferring Preconditions and Effects of Compound Tasks in Ground HTN Planning Problems βœ… βœ… βœ… ❌ βœ… ❌ βœ… 5
An MRP Formulation for Supervised Learning: Generalized Temporal Difference Learning Models βœ… βœ… βœ… βœ… ❌ βœ… βœ… 6
An Oracle-Guided Approach to Constrained Policy Synthesis Under Uncertainty βœ… βœ… βœ… ❌ βœ… ❌ βœ… 5
Approximate Implication for Probabilistic Graphical Models ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Argumentative Reasoning in ASPIC+ under Incomplete Information βœ… βœ… ❌ ❌ βœ… βœ… βœ… 5
Banal Deception and Human-AI Ecosystems: A Study of People’s Perceptions of LLM-generated Deceptive Behaviour ❌ ❌ βœ… ❌ ❌ ❌ ❌ 1
CADS: A Systematic Literature Review on the Challenges of Abstractive Dialogue Summarization ❌ βœ… βœ… ❌ ❌ ❌ ❌ 2
CPM-based Hierarchical Text Classification ❌ ❌ βœ… βœ… βœ… ❌ βœ… 4
Causal Explanations for Sequential Decision Making βœ… ❌ βœ… ❌ βœ… ❌ ❌ 3
Causal Graphs and Fairness in Machine Learning: Addressing Practical Challenges in Causal Fairness Evaluation βœ… βœ… βœ… ❌ ❌ ❌ ❌ 3
Certified Knowledge Compilation with Application to Formally Verified Model Counting ❌ βœ… βœ… ❌ βœ… ❌ βœ… 4
CluMo: Cluster-based Modality Fusion Prompt for Continual Learning in Visual Question Answering βœ… βœ… βœ… βœ… βœ… ❌ βœ… 6
Collective Intelligence in Decision-Making with Non-Stationary Experts βœ… βœ… βœ… βœ… ❌ ❌ βœ… 5
Combinatorial Multi-Armed Bandits with Fairness Constraints: An Online Convex Optimization Perspective βœ… ❌ βœ… ❌ βœ… ❌ βœ… 4
Combining Constraint Programming and Machine Learning: From Current Progress to Future Opportunities ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Comparison of SAT-Based and ASP-Based Algorithms for Inconsistency Measurement ❌ ❌ βœ… ❌ βœ… βœ… βœ… 4
Composition and Zero-Shot Transfer with Lattice Structures in Reinforcement Learning βœ… ❌ βœ… ❌ ❌ ❌ βœ… 3
Computational Machine Ethics: A Survey ❌ ❌ βœ… ❌ ❌ ❌ ❌ 1
ConSCompF: Consistency-focused Similarity Comparison Framework for Generative Large Language Models ❌ ❌ βœ… βœ… ❌ ❌ βœ… 3
Conditional Relative Frequency Distributions with Undefined Observations and Generalized Fuzzy Orthopartitions ❌ ❌ βœ… ❌ ❌ ❌ ❌ 1
Confidence-based Estimators for Predictive Performance in Model Monitoring ❌ βœ… ❌ βœ… ❌ ❌ ❌ 2
Control by Adding or Deleting Edges in Graph-Restricted Weighted Voting Games ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Correct Explanations and How to Define Them: Properties and Metrics for Measuring Correctness of Three Forms of ML Model Input/Output Behaviour Explanations ❌ ❌ βœ… βœ… ❌ βœ… βœ… 4
Counterfactual Situation Testing: From Single to Multidimensional Discrimination βœ… βœ… βœ… ❌ ❌ ❌ βœ… 4
DSAC: Distributional Soft Actor-Critic for Risk-Sensitive Reinforcement Learning βœ… βœ… βœ… ❌ βœ… ❌ βœ… 5
Data Augmentation for Time-Series Classification: An Extensive Empirical Study and Comprehensive Survey ❌ βœ… βœ… βœ… ❌ ❌ βœ… 4
Decentralized, Decomposition-Based Observation Scheduling for a Large-Scale Satellite Constellation βœ… ❌ βœ… ❌ ❌ ❌ βœ… 3
Defensive Alliances in Signed Networks ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Design and Effects of Co-Learning in Human-AI Teams ❌ ❌ ❌ ❌ ❌ βœ… βœ… 2
Detecting AI-Generated Text: Factors Influencing Detectability with Current Methods ❌ ❌ βœ… ❌ ❌ ❌ ❌ 1
Detecting Generative Model Inversion Attacks for Protecting Intellectual Property of Deep Neural Networks βœ… ❌ βœ… βœ… βœ… ❌ ❌ 4
EPINN: Enhanced Physics-Informed Neural Network for Solving Continuous Integral Equations βœ… ❌ ❌ ❌ βœ… βœ… βœ… 4
Efficient Ontology-Mediated Query Answering: Extending DL-liteR and Linear ELH βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
Efficient XAI: A Low-Cost Data Reduction Approach to SHAP Interpretability ❌ ❌ ❌ βœ… ❌ ❌ ❌ 1
Empirical Game Theoretic Analysis: A Survey βœ… ❌ ❌ ❌ ❌ ❌ βœ… 2
Enhanced Recommendation Systems with Retrieval-Augmented Large Language Model βœ… ❌ βœ… ❌ ❌ βœ… βœ… 4
Existence, Computation and Efficiency of Nash Stable Outcomes in Hedonic Skill Games βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
Explainability via Short Formulas: the Case of Propositional Logic with Implementation βœ… βœ… βœ… ❌ βœ… βœ… βœ… 6
Finding Minimal Plan Reductions Using Classical Planning βœ… βœ… βœ… ❌ ❌ ❌ βœ… 4
Forgetting in Abstract Argumentation: Limits and Possibilities βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
From Expert Systems to Generative Artificial Experts: A New Concept for Human-AI Collaboration in Knowledge Work ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
From Knowledge to Action: Logics of Permitted and Obligatory Announcements ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Generalizing the Regret: an Analysis of Lower and Upper Bounds βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
Generating Streamlining Constraints with Large Language Models βœ… βœ… βœ… βœ… βœ… βœ… βœ… 7
Generative Curricula for Multi-Agent Path Finding via Unsupervised and Reinforcement Learning βœ… βœ… βœ… βœ… βœ… ❌ βœ… 6
Geometrically Inspired Kernel Machines for Collaborative Learning Beyond Gradient Descent βœ… βœ… βœ… βœ… βœ… βœ… βœ… 7
Graph Collaborative Filtering Model Combining Time Factor and Attention Mechanism βœ… ❌ βœ… ❌ ❌ ❌ ❌ 2
Implicit Safe Set Algorithm for Provably Safe Reinforcement Learning βœ… βœ… βœ… ❌ ❌ ❌ βœ… 4
Improved Regret Bounds for Bandits with Expert Advice βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
Improving Label Error Detection and Elimination with Uncertainty Quantification ❌ ❌ βœ… βœ… βœ… ❌ βœ… 4
Improving Local Search Algorithm for Pseudo Boolean Optimization βœ… βœ… βœ… ❌ βœ… βœ… βœ… 6
Improving Mutual Information Based Feature Selection by Boosting Unique Relevance βœ… βœ… βœ… βœ… ❌ ❌ βœ… 5
Improving and Understanding the Power of Satisfaction-Driven Clause Learning βœ… ❌ βœ… ❌ ❌ ❌ βœ… 3
Incremental Learning Methodologies for Addressing Catastrophic Forgetting: Analysis and Experimental Evaluation ❌ βœ… βœ… βœ… βœ… ❌ βœ… 5
Invariant Graph Propagation in Constraint-Based Local Search βœ… βœ… ❌ ❌ βœ… βœ… βœ… 5
LabelCoRank: Revolutionizing Long Tail Multi-Label Classification with Co-Occurrence Reranking ❌ βœ… βœ… βœ… βœ… ❌ βœ… 5
Laplace-HDC: Understanding the Geometry of Binary Hyperdimensional Computing βœ… βœ… βœ… βœ… ❌ ❌ βœ… 5
Learning How to Vote with Principles: Axiomatic Insights Into the Collective Decisions of Neural Networks ❌ βœ… ❌ βœ… βœ… ❌ βœ… 4
MA-LAMA: Exploiting the Multi-Agent Nature of Temporal Planning Problems βœ… ❌ βœ… ❌ ❌ ❌ βœ… 3
MAT2I: Enhancing Perceptual Authenticity in Text-to-Image Synthesis using Multi-Attribute Generative Adversarial Networks ❌ ❌ βœ… βœ… ❌ ❌ βœ… 3
MRC and Transfer Learning Framework for Document-level Event Factuality Identification with Heterogeneous Spectral Attention Networks βœ… ❌ βœ… βœ… ❌ ❌ βœ… 4
Maximal Combinations of Fairness Definitions ❌ ❌ βœ… ❌ ❌ ❌ ❌ 1
Mechanisms of Symbol Processing for In-Context Learning in Transformer Networks βœ… βœ… βœ… βœ… ❌ ❌ βœ… 5
Monitoring Teams of AI Agents ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Multi-Leader Congestion Games with an Adversary βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
On CNF Conversion for SAT and SMT Enumeration βœ… βœ… βœ… ❌ βœ… ❌ βœ… 5
On Computing Probabilistic Explanations for Decision Trees βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
On Generating Monolithic and Model Reconciling Explanations in Probabilistic Scenarios βœ… βœ… βœ… ❌ βœ… ❌ βœ… 5
On a Simple Hedonic Game with Graph-Restricted Communication βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
On the Consistency between Belief Revision and Belief Update ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
On the Equivalence between Logic Programs and Bipolar Argumentation Frameworks ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
On the Phase Transition of the Euclidean Travelling Salesman Problem with Time Windows ❌ βœ… ❌ ❌ βœ… ❌ βœ… 3
Optimal Decision Trees for Interpretable and Constrained Clustering βœ… ❌ βœ… ❌ βœ… βœ… βœ… 5
Optimizing Relevance and Diversity in Online Matching Markets: A Time-Adaptive Attenuation Approach βœ… ❌ βœ… ❌ ❌ ❌ βœ… 3
PAC-Chernoff Bounds: Understanding Generalization in the Interpolation Regime ❌ βœ… βœ… βœ… ❌ ❌ βœ… 4
Parameterized Analysis of Bribery in Challenge the Champ Tournaments ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Path-Planning on a Spherical Surface with Disturbances and Exclusion Zones βœ… βœ… βœ… ❌ βœ… ❌ βœ… 5
Policy-based Primal-Dual Methods for Concave CMDP with Variance Reduction βœ… βœ… ❌ ❌ ❌ ❌ βœ… 3
Practical Parallel Algorithms for Non-Monotone Submodular Maximization βœ… ❌ βœ… ❌ βœ… ❌ βœ… 4
Preference Tuning with Human Feedback on Language, Speech, and Vision Tasks: A Survey βœ… βœ… βœ… ❌ ❌ ❌ ❌ 3
Principles for Assumptions Generation in Enthymeme-Based Dialogue ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Principles for Responsible AI Consciousness Research ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Prioritised Planning: Completeness, Optimality, and Complexity βœ… βœ… βœ… ❌ βœ… βœ… ❌ 5
Probabilistically Tightened Linear Relaxation-based Perturbation Analysis for Neural Network Verification βœ… βœ… βœ… βœ… βœ… ❌ βœ… 6
Promoting the Responsible Development of Speech Datasets for Mental Health and Neurological Disorders Research ❌ βœ… βœ… ❌ ❌ ❌ ❌ 2
Prosociality in Microtransit ❌ ❌ ❌ ❌ ❌ ❌ βœ… 1
Reasoning about Decidability of Strategic Logics with Imperfect Information and Perfect Recall Strategies βœ… βœ… ❌ ❌ βœ… ❌ ❌ 3
Reasoning over Streams of Events with Delayed Effects βœ… βœ… βœ… ❌ βœ… ❌ βœ… 5
Recency-Weighted Temporally-Segmented Ensemble for Time Series Modeling βœ… βœ… βœ… βœ… ❌ βœ… βœ… 6
Recursive Decomposition of Logical Thoughts: Framework for Superior Reasoning and Knowledge Propagation in Large Language Models ❌ ❌ βœ… ❌ ❌ ❌ βœ… 2
Regret Analysis for Randomized Gaussian Process Upper Confidence Bound βœ… ❌ βœ… ❌ ❌ ❌ βœ… 3
Robust Reward Design for Markov Decision Processes ❌ βœ… ❌ ❌ βœ… βœ… βœ… 4
Robustness Distributions in Neural Network Verification ❌ βœ… βœ… βœ… βœ… βœ… βœ… 6
Rumor Detection with Adaptive Data Augmentation and Adversarial Training βœ… ❌ βœ… βœ… βœ… ❌ βœ… 5
Scalable Synthesis of Formally Verified Neural Value Function for Hamilton-Jacobi Reachability Analysis ❌ βœ… βœ… ❌ βœ… ❌ βœ… 4
Scaling Safe Policy Improvement: Monte Carlo Tree Search and Policy Iteration Strategies βœ… βœ… βœ… ❌ βœ… ❌ βœ… 5
Semantic Alignment of Malicious Question Based on Contrastive Semantic Networks and Data Augmentation βœ… βœ… βœ… βœ… βœ… ❌ βœ… 6
Solving Overlapping Coalition Structure Generation in Task-Based Settings βœ… βœ… βœ… ❌ βœ… ❌ βœ… 5
Spatio-Causal Patterns of Sample Growth ❌ ❌ βœ… ❌ ❌ ❌ ❌ 1
Stability in Online Coalition Formation βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
Super Level Sets and Exponential Decay: A Synergistic Approach to Stable Neural Network Training βœ… ❌ ❌ ❌ ❌ ❌ βœ… 2
Symbolic Search for Cost-Optimal Planning with Expressive Model Extensions βœ… βœ… βœ… ❌ βœ… βœ… βœ… 6
TOMA: Computational Theory of Mind with Abstractions for Hybrid Intelligence ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
TabID: Automatic Identification and Tabulation of Subproblems in Constraint Models ❌ βœ… βœ… ❌ βœ… βœ… βœ… 5
The Complexity of Pure Maxmin Strategies in Two-Player Extensive-Form Games βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
The Fixed-Point Semantics of Relational Concept Analysis βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
The Search for Stability: Learning Dynamics of Strategic Publishers with Initial Documents βœ… βœ… ❌ ❌ ❌ ❌ βœ… 3
Thousands of AI Authors on the Future of AI ❌ ❌ βœ… ❌ ❌ βœ… ❌ 2
Towards Explainable Goal Recognition Using Weight of Evidence (WoE): A Human-Centered Approach βœ… ❌ βœ… ❌ ❌ ❌ ❌ 2
Towards Generalist Robot Learning from Internet Video: A Survey ❌ ❌ βœ… ❌ ❌ ❌ ❌ 1
Towards a Robust Persistence Diagram via Data-dependent Kernel ❌ βœ… βœ… βœ… βœ… ❌ βœ… 5
Towards an Ontology-Driven Approach to Document Bias ❌ ❌ βœ… ❌ ❌ ❌ ❌ 1
Trustworthy Transfer Learning: A Survey ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Value Preferences Estimation and Disambiguation in Hybrid Participatory Systems βœ… βœ… βœ… βœ… ❌ ❌ βœ… 5
Viewpoint: The Future of Human-Centric Explainable Artificial Intelligence (XAI) is not Post-Hoc Explanations ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0