Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in Coakley et alK. L. Coakley, T. Snelleman, H. Hoos, and O. E. Gundersen, "The embrace of open science: An analysis of a decade of AI research and 56 800 conference papers," Under Review, 2026..
Evaluation of Project Performance in Participatory Budgeting
Authors: Niclas Boehmer, Piotr Faliszewski, Łukasz Janeczko, Dominik Peters, Grzegorz Pierczyński, Šimon Schierreich, Piotr Skowron, Stanisław Szufa
IJCAI 2024 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Second, we compute our measures for projects from real-life PB instances from Pabulib [Faliszewski et al., 2023] and analyze their behavior. As a side result, we obtain some striking examples of nonmonotonicity of proportionality-oriented PB rules. We also demonstrate the usefulness of our measures by giving a detailed analysis of the Wieliczka election. |
| Researcher Affiliation | Academia | Niclas Boehmer1 , Piotr Faliszewski2 , Łukasz Janeczko2 , Dominik Peters3 , Grzegorz Pierczy nski2,4 , ˇSimon Schierreich5 , Piotr Skowron4 and Stanisław Szufa2,3 1Harvard University 2AGH University, Krak ow 3CNRS, LAMSADE, Universit e Paris Dauphine PSL 4University of Warsaw 5Czech Technical University in Prague |
| Pseudocode | No | The paper describes algorithmic procedures in text and provides a 'Proof sketch (PHRAGMÉN)' section outlining steps for an algorithm, but it does not include a clearly labeled pseudocode or algorithm block/figure. |
| Open Source Code | Yes | The code for our experiments is available at github.com/Project-PRAGMA/ Project-Performance-IJCAI-2024. |
| Open Datasets | Yes | We conduct our experiments on all 551 PB instances with approval votes from Pabulib [Faliszewski et al., 2023] |
| Dataset Splits | No | The paper analyzes real-world participatory budgeting instances but does not describe any specific training, validation, or test dataset splits or their sizes, as it does not involve training a machine learning model in the conventional sense. |
| Hardware Specification | Yes | We ran our experiments on 10 threads of an Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz core. |
| Software Dependencies | No | The paper mentions using 'Gurobi' for experiments but does not provide a specific version number for this or any other software dependency. |
| Experiment Setup | Yes | To compute the measures, we use the algorithms described in Section 3. For the adding-voters and sampling algorithms, we increase the approval score of the designated project by 1% in each step (repeating each step 100 times for the sampling algorithms). |