Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].

Journal of Artificial Intelligence Research (JAIR) - 2020

Website:

Venue Year Papers
Reproducibility Score Reproducibility Score based on Gundersen et al. (2025)
Documentation Score Global mean is the average score over the seven reproducibility variables for empirical research papers.
% Empirical Percentage of papers that are empirical research vs theoretical research
% Industry Percentage of empirical research papers with at least one author from Industry
Website
JAIR 2020 84 0.48 3.83 71.43% 16.67%
Pseudocode
Open Source Code
Open Datasets
Dataset Splits
Hardware Specification
Software Dependencies
Experiment Setup
A Differential Privacy Mechanism that Accounts for Network Effects for Crowdsourcing Systems βœ… ❌ βœ… ❌ ❌ ❌ βœ… 3
A General Approach to Multimodal Document Quality Assessment ❌ βœ… βœ… βœ… ❌ ❌ βœ… 4
A Global Constraint for the Exact Cover Problem: Application to Conceptual Clustering βœ… ❌ βœ… ❌ βœ… βœ… βœ… 5
A Set of Recommendations for Assessing Human–Machine Parity in Language Translation ❌ ❌ βœ… βœ… ❌ ❌ βœ… 3
AMP Chain Graphs: Minimal Separators and Structure Learning Algorithms βœ… βœ… βœ… ❌ ❌ ❌ βœ… 4
ASNets: Deep Learning for Generalised Planning βœ… βœ… βœ… βœ… βœ… ❌ βœ… 6
Adapting Behavior via Intrinsic Reward: A Survey and Empirical Study βœ… βœ… βœ… ❌ ❌ ❌ βœ… 4
Adaptive Stress Testing: Finding Likely Failure Events with Reinforcement Learning βœ… βœ… βœ… ❌ βœ… ❌ βœ… 5
Adversarial Attacks on Crowdsourcing Quality Control ❌ βœ… βœ… ❌ ❌ ❌ βœ… 3
Agreement on Target-Bidirectional Recurrent Neural Networks for Sequence-to-Sequence Learning βœ… βœ… βœ… βœ… βœ… ❌ βœ… 6
An Evaluation of Communication Protocol Languages for Engineering Multiagent Systems ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Annotator Rationales for Labeling Tasks in Crowdsourcing βœ… ❌ βœ… βœ… ❌ ❌ βœ… 4
Automated Conjecturing II: Chomp and Reasoned Game Play ❌ βœ… ❌ ❌ βœ… ❌ ❌ 2
Belief change and 3-valued logics: Characterization of 19,683 belief change operators ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Best-First Enumeration Based on Bounding Conflicts, and its Application to Large-scale Hybrid Estimation βœ… ❌ βœ… ❌ βœ… ❌ βœ… 4
Blind Spot Detection for Safe Sim-to-Real Transfer βœ… βœ… ❌ ❌ ❌ ❌ βœ… 3
Bounds on the Size of PC and URC Formulas βœ… βœ… ❌ ❌ ❌ ❌ ❌ 2
Bridging the Gap Between Probabilistic Model Checking and Probabilistic Planning: Survey, Compilations, and Empirical Comparison ❌ βœ… βœ… ❌ βœ… βœ… βœ… 5
Catching Cheats: Detecting Strategic Manipulation in Distributed Optimisation of Electric Vehicle Aggregators βœ… ❌ βœ… ❌ ❌ ❌ βœ… 3
Compositionality Decomposed: How do Neural Networks Generalise? ❌ βœ… βœ… βœ… ❌ ❌ βœ… 4
Computing Bayes-Nash Equilibria in Combinatorial Auctions with Verification βœ… βœ… ❌ ❌ βœ… ❌ βœ… 4
Conservative Extensions in Horn Description Logics with Inverse Roles ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Constraint and Satisfiability Reasoning for Graph Coloring βœ… βœ… βœ… ❌ βœ… βœ… βœ… 6
Contiguous Cake Cutting: Hardness Results and Approximation Algorithms βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
Contrasting the Spread of Misinformation in Online Social Networks βœ… ❌ βœ… βœ… βœ… βœ… βœ… 6
Credibility-limited Base Revision: New Classes and Their Characterizations ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Deep Reinforcement Learning: A State-of-the-Art Walkthrough ❌ ❌ βœ… ❌ ❌ ❌ ❌ 1
Diagnosis of Deep Discrete-Event Systems ❌ ❌ ❌ ❌ βœ… ❌ ❌ 1
Epistemic Argumentation Framework: Theory and Computation βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
Fair Allocation with Diminishing Differences βœ… βœ… ❌ ❌ ❌ ❌ βœ… 3
Gradient-based Learning Methods Extended to Smooth Manifolds Applied to Automated Clustering βœ… ❌ βœ… βœ… βœ… βœ… βœ… 6
Graph Width Measures for CNF-Encodings with Auxiliary Variables ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
HTN Planning as Heuristic Progression Search βœ… βœ… βœ… ❌ βœ… ❌ βœ… 5
Hedonic Games with Ordinal Preferences and Thresholds βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
How to Do Things with Words: A Bayesian Approach ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Image Captioning using Facial Expression and Attention ❌ ❌ βœ… βœ… ❌ ❌ βœ… 3
Improved High Dimensional Discrete Bayesian Network Inference using Triplet Region Construction βœ… βœ… βœ… ❌ βœ… βœ… βœ… 6
Improving Nash Social Welfare Approximations βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
Incompatibilities Between Iterated and Relevance-Sensitive Belief Revision ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Incomplete Preferences in Single-Peaked Electorates βœ… βœ… βœ… ❌ βœ… ❌ ❌ 4
Jointly Improving Parsing and Perception for Natural Language Commands through Human-Robot Dialog ❌ βœ… ❌ βœ… ❌ ❌ βœ… 3
Learning the Language of Software Errors βœ… ❌ βœ… ❌ ❌ βœ… βœ… 4
Lifted Bayesian Filtering in Multiset Rewriting Systems βœ… ❌ βœ… ❌ ❌ ❌ βœ… 3
Mapping the landscape of Artificial Intelligence applications against COVID-19 ❌ ❌ βœ… ❌ ❌ ❌ ❌ 1
Maximin Share Allocations on Cycles βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
Modular Structures and Atomic Decomposition in Ontologies βœ… ❌ βœ… ❌ βœ… βœ… βœ… 5
Neural Machine Translation: A Review βœ… ❌ βœ… βœ… ❌ ❌ ❌ 3
On Sparse Discretization for Graphical Games ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
On the Complexity of Learning a Class Ratio from Unlabeled Data ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Ontology Reasoning with Deep Neural Networks βœ… βœ… βœ… βœ… ❌ ❌ βœ… 5
Planning High-Level Paths in Hostile, Dynamic, and Uncertain Environments βœ… ❌ βœ… ❌ βœ… βœ… βœ… 5
Planning for Hybrid Systems via Satisfiability Modulo Theories βœ… βœ… βœ… ❌ ❌ βœ… βœ… 5
Point at the Triple: Generation of Text Summaries from Knowledge Base Triples ❌ βœ… βœ… βœ… βœ… ❌ βœ… 5
Predicting Strategic Behavior from Free Text ❌ βœ… βœ… βœ… ❌ ❌ βœ… 4
Preferences Single-Peaked on a Circle βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
Properties of Switch-List Representations of Boolean Functions ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Qualitative Numeric Planning: Reductions and Complexity βœ… βœ… ❌ ❌ ❌ βœ… ❌ 3
Regret Bounds for Reinforcement Learning via Markov Chain Concentration βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
Representing Fitness Landscapes by Valued Constraints to Understand the Complexity of Local Search ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
Reviewing Autoencoders for Missing Data Imputation: Technical Trends, Applications and Outcomes ❌ ❌ βœ… ❌ ❌ ❌ ❌ 1
Robust Multi-Agent Path Finding and Executing ❌ βœ… βœ… ❌ βœ… ❌ βœ… 4
Saturated Cost Partitioning for Optimal Classical Planning βœ… βœ… βœ… ❌ ❌ ❌ βœ… 4
Scalable Planning with Deep Neural Network Learned Transition Models ❌ ❌ ❌ βœ… βœ… βœ… βœ… 4
Simulating Offender Mobility: Modeling Activity Nodes from Large-Scale Human Activity Data ❌ βœ… βœ… βœ… ❌ ❌ βœ… 4
Sliding-Window Thompson Sampling for Non-Stationary Settings βœ… ❌ ❌ ❌ ❌ ❌ βœ… 2
Solving Delete Free Planning with Relaxed Decision Diagram Based Heuristics βœ… βœ… βœ… ❌ ❌ βœ… βœ… 5
Structure from Randomness in Halfspace Learning with the Zero-One Loss ❌ ❌ βœ… βœ… ❌ ❌ βœ… 3
Subgoaling Techniques for Satisficing and Optimal Numeric Planning βœ… βœ… βœ… ❌ βœ… βœ… βœ… 6
TensorLog: A Probabilistic Database Implemented Using Deep-Learning Infrastructure βœ… ❌ βœ… βœ… βœ… ❌ βœ… 5
The 2^k Neighborhoods for Grid Path Planning βœ… ❌ βœ… ❌ βœ… ❌ βœ… 4
The Bottleneck Simulator: A Model-Based Deep Reinforcement Learning Approach ❌ ❌ βœ… βœ… βœ… ❌ βœ… 4
The Complexity Landscape of Outcome Determination in Judgment Aggregation ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
The Effects of Experience on Deception in Human-Agent Negotiation ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
The Force Awakens: Artificial Intelligence for Consumer Law ❌ ❌ ❌ ❌ ❌ ❌ ❌ 0
The Impact of Treewidth on Grounding and Solving of Answer Set Programs βœ… ❌ βœ… βœ… ❌ βœ… βœ… 5
The PETLON Algorithm to Plan Efficiently for Task-Level-Optimal Navigation βœ… ❌ ❌ ❌ βœ… ❌ βœ… 3
The Parameterized Complexity of Motion Planning for Snake-Like Robots βœ… ❌ ❌ ❌ ❌ ❌ ❌ 1
To Regulate or Not: A Social Dynamics Analysis of an Idealised AI Race ❌ ❌ ❌ ❌ ❌ ❌ βœ… 1
Towards Knowledgeable Supervised Lifelong Learning Systems βœ… βœ… βœ… βœ… ❌ ❌ βœ… 5
Towards Partial Order Reductions for Strategic Ability βœ… ❌ βœ… ❌ ❌ ❌ ❌ 2
Using Machine Learning for Decreasing State Uncertainty in Planning ❌ βœ… βœ… βœ… ❌ ❌ βœ… 4
Using Task Descriptions in Lifelong Machine Learning for Improved Performance and Zero-Shot Transfer βœ… ❌ ❌ ❌ βœ… ❌ βœ… 3
Variational Bayes In Private Settings (VIPS) βœ… βœ… βœ… βœ… ❌ ❌ βœ… 5
Vocabulary Alignment in Openly Specified Interactions ❌ ❌ ❌ ❌ ❌ ❌ βœ… 1