Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..

Journal of Artificial Intelligence Research (JAIR) - 2021

Documentation Rate of Empirical Papers by Reproducibility Variable

Distribution of Empirical Papers by Number of Documented Variables

Website:

Venue Year Papers
Reproducibility Score Reproducibility Score based on Gundersen et al. (2025). See Methods for details.
Documentation Score Documentation Score is the average score over the seven reproducibility variables for empirical research papers. See Methods for details.
% Empirical Percentage of papers that are empirical research vs theoretical research.
% Industry Percentage of empirical research papers with at least one author from Industry.
Website
JAIR 2021 104 0.53 3.79 63.46% 25.76%
Pseudocode
Open Source Code
Open Datasets
Dataset Splits
Hardware Specification
Software Dependencies
Experiment Setup
A Semi-exact Algorithm for Quickly Computing A Maximum Weight Clique in Large Sparse Graphs 6
A Sufficient Statistic for Influence in Structured Multiagent Environments 0
A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems 1
A Survey on the Explainability of Supervised Machine Learning 2
A Theoretical Perspective on Hyperdimensional Computing 0
A Tight Bound for Stochastic Submodular Cover 0
A Word Selection Method for Producing Interpretable Distributional Semantic Word Vectors 4
Agent-Based Markov Modeling for Improved COVID-19 Mitigation Policies 5
Aggregation over Metric Spaces: Proposing and Voting in Elections, Budgeting, and Legislation 0
An External Knowledge Enhanced Graph-based Neural Network for Sentence Ordering 2
Analysis of the Impact of Randomization of Search-Control Parameters in Monte-Carlo Tree Search 4
Benchmark and Survey of Automated Machine Learning Frameworks 6
Bribery and Control in Stable Marriage 1
Classifier Chains: A Review and Perspectives 0
Computational Complexity of Computing Symmetries in Finite-Domain Planning 0
Conceptual Modeling of Explainable Recommender Systems: An Ontological Formalization to Guide Their Design and Development 2
Confident Learning: Estimating Uncertainty in Dataset Labels 6
Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective 0
Constrained Multiagent Markov Decision Processes: a Taxonomy of Problems and Algorithms 2
Constraint-based Diversification of JOP Gadgets 6
Contrastive Explanations of Plans through Model Restrictions 4
Cost-optimal Planning, Delete Relaxation, Approximability, and Heuristics 1
Declarative Algorithms and Complexity Results for Assumption-Based Argumentation 6
Dimensional Inconsistency Measures and Postulates in Spatio-Temporal Databases 1
Efficient Large-Scale Multi-Drone Delivery using Transit Networks 5
Efficient Local Search based on Dynamic Connectivity Maintenance for Minimum Connected Dominating Set 4
Efficient Multi-objective Reinforcement Learning via Multiple-gradient Descent with Iteratively Discovered Weight-Vector Sets 4
Efficient Retrieval of Matrix Factorization-Based Top-k Recommendations: A Survey of Recent Approaches 0
Election Manipulation on Social Networks: Seeding, Edge Removal, Edge Addition 0
EpidemiOptim: A Toolbox for the Optimization of Control Policies in Epidemiological Models 3
Ethics and Governance of Artificial Intelligence: Evidence from a Survey of Machine Learning Researchers 0
Evaluating Strategic Structures in Multi-Agent Inverse Reinforcement Learning 2
Experimental Comparison and Survey of Twelve Time Series Anomaly Detection Algorithms 4
Finding the Hardest Formulas for Resolution 6
Flexible Bayesian Nonlinear Model Configuration 5
Game Plan: What AI can do for Football, and What Football can do for AI 3
General Value Function Networks 3
Generic Constraint-based Block Modeling using Constraint Programming 6
Goal Recognition for Deceptive Human Agents through Planning and Gaze 4
Graph Kernels: A Survey 4
Hybrid-order Network Consensus for Distributed Multi-agent Systems 2
Improving the Effectiveness and Efficiency of Stochastic Neighbour Embedding with Isolation Kernel 6
Induction and Exploitation of Subgoal Automata for Reinforcement Learning 5
Instance-Level Update in DL-Lite Ontologies through First-Order Rewriting 5
Integrated Offline and Online Decision Making under Uncertainty 3
Intelligence in Strategic Games 0
Labeled Bipolar Argumentation Frameworks 1
Learning Realistic Patterns from Visually Unrealistic Stimuli: Generalization and Data Anonymization 5
Learning Optimal Decision Sets and Lists with SAT 6
Learning Temporal Causal Sequence Relationships from Real-Time Time-Series 4
Learning from Disagreement: A Survey 4
Learning over No-Preferred and Preferred Sequence of Items for Robust Recommendation 5
Lilotane: A Lifted SAT-based Approach to Hierarchical Planning 5
Liquid Democracy: An Algorithmic Perspective 1
Loss Functions, Axioms, and Peer Review 1
MADRaS : Multi Agent Driving Simulator 2
Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks 2
Merge-and-Shrink: A Compositional Theory of Transformations of Factored Transition Systems 1
Multi-Document Summarization with Determinantal Point Process Attention 5
Multi-Label Classification Neural Networks with Hard Logical Constraints 5
Multilabel Classification with Partial Abstention: Bayes-Optimal Prediction under Label Independence 4
NLP Methods for Extraction of Symptoms from Unstructured Data for Use in Prognostic COVID-19 Analytic Models 4
On Quantifying Literals in Boolean Logic and its Applications to Explainable AI 0
On Super Strong ETH 1
On the Cluster Admission Problem for Cloud Computing 3
On the Computational Complexity of Non-Dictatorial Aggregation 0
On the Decomposition of Abstract Dialectical Frameworks and the Complexity of Naive-based Semantics 0
On the Distortion Value of Elections with Abstention 0
On the Evolvability of Monotone Conjunctions with an Evolutionary Mutation Mechanism 1
On the Online Coalition Structure Generation Problem 1
Optimal Any-Angle Pathfinding on a Sphere 3
Optimally Deceiving a Learning Leader in Stackelberg Games 0
Optimizing for Interpretability in Deep Neural Networks with Tree Regularization 5
Output Space Entropy Search Framework for Multi-Objective Bayesian Optimization 5
Playing Codenames with Language Graphs and Word Embeddings 5
Probabilistic Temporal Networks with Ordinary Distributions: Theory, Robustness and Expected Utility 5
Pure Nash Equilibria in Resource Graph Games 0
Quantum Mathematics in Artificial Intelligence 2
RWNE: A Scalable Random-Walk based Network Embedding Framework with Personalized Higher-order Proximity Preserved 4
Reasoning with PCP-Nets 0
Regarding Goal Bounding and Jump Point Search 5
Relevance in Belief Update 0
Representative Committees of Peers 0
Safe Multi-Agent Pathfinding with Time Uncertainty 4
Set-to-Sequence Methods in Machine Learning: A Review 2
Socially Responsible AI Algorithms: Issues, Purposes, and Challenges 0
Steady-State Planning in Expected Reward Multichain MDPs 4
Strategyproof Mechanisms for Additively Separable and Fractional Hedonic Games 1
Superintelligence Cannot be Contained: Lessons from Computability Theory 1
Taking Principles Seriously: A Hybrid Approach to Value Alignment in Artificial Intelligence 0
Task-Aware Verifiable RNN-Based Policies for Partially Observable Markov Decision Processes 1
Teaching People by Justifying Tree Search Decisions: An Empirical Study in Curling 3
The AI Liability Puzzle and A Fund-Based Work-Around 0
The Computational Complexity of Understanding Binary Classifier Decisions 0
The Rediscovery Hypothesis: Language Models Need to Meet Linguistics 5
The Societal Implications of Deep Reinforcement Learning 0
Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods 2
Two-facility Location Games with Minimum Distance Requirement 1
Viewpoint: AI as Author – Bridging the Gap Between Machine Learning and Literary Theory 0
Weighted First-Order Model Counting in the Two-Variable Fragment With Counting Quantifiers 0
Welfare Guarantees in Schelling Segregation 1
Worst-case Bounds on Power vs. Proportion in Weighted Voting Games with an Application to False-name Manipulation 1
Zone pAth Construction (ZAC) based Approaches for Effective Real-Time Ridesharing 5
sunny-as2: Enhancing SUNNY for Algorithm Selection 5