An Information-Flow Perspective on Algorithmic Fairness
Authors: Samuel Teuber, Bernhard Beckert
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This work presents insights gained by investigating the relationship between algorithmic fairness and the concept of secure information flow. [...] We demonstrate that off-the-shelf tools for information-flow properties can be used in order to formally analyze a program s algorithmic fairness properties, including the new notion of fairness spread as well as established notions such as demographic parity. [...] Throughout this paper, we will demonstrate our ideas using the three exemplary programs shown in Figure 1: All programs make a loan allocation decision based on a protected group attribute (group) and an unprotected variable containing the credit score (score). [...] We applied our approach to two case-studies, showcasing both the qualitative (Section 6.1) as well as the quantitative (Section 6.2) approach. [...] All analyses are summarized in Table 1 and were computed using the counter Sharp tool which allows the quantitative analysis of C-programs. |
| Researcher Affiliation | Academia | Karlsruhe Institute of Technology, Am Fasanengarten 5, 76131 Karlsruhe {teuber,beckert}@kit.edu |
| Pseudocode | Yes | Figure 1: Three exemplary credit decision algorithms: 1 def c1(group, score): 2 return (group != 0) (a) Group Decision; 1 def c2(group, score): 2 return (score>=8) (b) Score Decision; 1 def c3(group, score): 2 if (group >= 6): 3 return (score >= 8) 4 else: 5 return (score >= 6) (c) Mixed Decision |
| Open Source Code | Yes | Artifact available at (Teuber and Beckert 2023c). [...] Teuber, S.; and Beckert, B. 2023c. samysweb/AAAI24Fairness: AAAI24 Artefact. 10.5281/zenodo.10385360. |
| Open Datasets | No | The paper defines illustrative examples and distributions for its analyses (e.g., 'a setting with 10 groups (e.g., age ranges) and a credit score of U between 1 and 10', 'Ud is the discrete uniform distribution'). However, it does not use or provide access information for a named, publicly available dataset in the traditional sense of a common benchmark or repository. |
| Dataset Splits | No | The paper does not provide explicit training/test/validation dataset splits. Its analyses are based on formal methods and properties of programs with defined inputs/distributions, not on splitting a large dataset into subsets for training, validation, and testing of a model. |
| Hardware Specification | No | The paper states that 'Each analysis took less than 3 seconds of computation time' in Table 1 but does not specify any hardware details like CPU or GPU models, memory, or specific computing environments used for the analyses. |
| Software Dependencies | No | The paper mentions several tools used: 'the interactive theorem prover Ke Y (Ahrendt et al. 2005, 2016)', 'the automated information-flow analysis tool Joana (Graf, Hecker, and Mohr 2013; Snelting et al. 2014)', and 'the tool counter Sharp (Teuber and Weigl 2021)'. It also refers to a 'Java implementation (Federal Ministry of Finance 2022a)'. While these tools are named and cited, no specific version numbers are provided for any of them. |
| Experiment Setup | Yes | To this end, we analyzed all three programs for a setting with 10 groups (e.g., age ranges) and a credit score of U between 1 and 10. Initially, we assumed a uniform distribution of credit scores. In addition, we analyzed c3 (Figure 1 (b)) using a wrapper program that adjusted the probability of a credit score between 6 to 7 to be 30% (instead of 20% in the uniform case). [...] We assume B1 Ud (0, 9), B2 Ud (0, 9), B3 Ud ( 1, 5), B4 Ud ( 3, 3) where Ud is the discrete uniform distribution. |