Improved Bayes Risk Can Yield Reduced Social Welfare Under Competition
Authors: Meena Jagadeesan, Michael Jordan, Jacob Steinhardt, Nika Haghtalab
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our examples range from closed-form formulas in simple settings to simulations with pretrained representations on CIFAR-10. At a conceptual level, our work suggests that favorable scaling trends for individual model-providers need not translate to downstream improvements in social welfare in marketplaces with multiple model providers. |
| Researcher Affiliation | Academia | Meena Jagadeesan UC Berkeley mjagadeesan@berkeley.edu Michael I. Jordan UC Berkeley jordan@cs.berkeley.edu Jacob Steinhardt UC Berkeley jsteinhardt@berkeley.edu Nika Haghtalab UC Berkeley nika@berkeley.edu |
| Pseudocode | No | The paper describes algorithms (e.g., best-response dynamics) in text but does not include formal pseudocode blocks or algorithm listings. |
| Open Source Code | Yes | The code can be found at https://github.com/mjagadeesan/competition-nonmonotonicity. |
| Open Datasets | Yes | We consider a binary image classification task on CIFAR-10 [Krizhevsky, 2009] with 50,000 images... Representations are generated from five models Alex Net [Krizhevsky et al., 2012], VGG16 [Simonyan and Zisserman, 2015], Res Net18, Res Net34, and Res Net50 [He et al., 2016] pretrained on Image Net [Deng et al., 2009]. |
| Dataset Splits | Yes | We treat the set of 50,000 images and labels as the population of users, meaning that it is both the training set and the validation set. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used for experiments, such as CPU or GPU models. |
| Software Dependencies | No | The paper does not specify version numbers for any software dependencies or libraries used. |
| Experiment Setup | Yes | In more detail, for each j ∈ [m], we initialize the model parameters φ as mean zero Gaussians with standard deviation σ... We implement the approximate best-response as running several steps of gradient descent... We compute the equilibria as follows. First, we let D be the empirical distribution over N = 10, 000 samples from the continuous distribution. Then we run the best-response dynamics described in Section 4.1 with ε = ε0 = 1, I = 5000, δ = 0.1, η = 0.001, and σ = 1.0. We also set the noise parameter c in user decisions (1) to 0.3... For CIFAR-10, for m ∈ {3, 4, 5, 6, 8} model-providers with ε = ε0 = 0.3, I = 2000, η = 0.001, σ = 0.5, and a learning rate schedule that starts at α = 1.0. |