Inflow, Outflow, and Reciprocity in Machine Learning
Authors: Mukund Sundararajan, Walid Krichene
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We show theoretically, and via experiments that under certain distributional assumptions, some classes of models are approximately reciprocal. These results only scratch the surface; we conclude with several open directions.Our primary contribution is to initiate the study of reciprocity and propose a measurement approach (Section 2). Our main theoretical result (Theorem 3.2) states that models trained using Stochastic Gradient Descent, a popular training algorithm for neural networks, are strongly reciprocal under an assumption over the data distribution (Assumption 3.1). One key observation is that the influence measure satisfies certain symmetry properties.We also demonstrate how to compute inflows and outflows efficiently, avoiding a naive quadratic complexity over the number of data points, which would make measurement infeasible (see Section 4).Finally, we perform experiments on one recommendation and two healthcare data sets (Section 5). |
| Researcher Affiliation | Industry | 1Google 2Google Research. Correspondence to: Mukund Sundararajan <mukunds@google.com>. |
| Pseudocode | No | The paper describes algorithms such as SGD and Trac In but does not include any explicitly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | No | The paper does not provide any links or explicit statements about releasing source code for the methodology described. |
| Open Datasets | Yes | We conduct experiments on Movie Lens Data (Harper & Konstan, 2015), specifically, the Movie Lens 100K data set with 943 individuals, 1682 items (movies), and 100,000 ratings...The first is a data set from (Efron et al., 2004) about predicting diabetes...The second is a data set from the UCI Machine Learning Repository about predicting breast-cancer. |
| Dataset Splits | Yes | We randomly split the ratings into training and inference sets in the ratio 80:20...More precisely, we average measurements over 100 random splits of the data. |
| Hardware Specification | No | The paper does not provide specific details about the hardware used to run the experiments (e.g., specific GPU or CPU models, memory, or cloud instance types). |
| Software Dependencies | No | The paper mentions training models with specific algorithms but does not provide specific version numbers for any software dependencies or libraries (e.g., Python, TensorFlow, PyTorch versions). |
| Experiment Setup | Yes | We train a matrix factorization model with embedding dimension d = 16. ... regularization coefficient λ = 1, number of steps T = 1000, and learning rate η = 0.0002.On the diabetes prediction task, we train a linear regression model optimized for the mean squared error, with a number of steps T = 200, and a learning rate η = 0.01. On the breast cancer classification task, we train a logistic regression model with a number of steps T = 600 and a learning rate η = 0.1. |