A/B Testing in Network Data with Covariate-Adaptive Randomization
Authors: Jialu Wang, Ping Li, Feifang Hu
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we perform numerical studies to demonstrate the finite sample properties of our proposed adaptive randomization procedure via a hypothetical network as well as a real-world network presented in Cai et al. (2015). |
| Researcher Affiliation | Collaboration | Jialu Wang Ping Li Feifang Hu Department of Statistics Linked In Ads Department of Statistics George Washington Univ. Linkedin Corporation George Washington Univ. Washington, DC 20052, USA Bellevue, WA 98004, USA Washington, DC 20052, USA jialu@gwu.edu pinli@linkedin.com feifang@gwu.edu |
| Pseudocode | Yes | Algorithm 1 Adaptive Randomization in Network Data Assumptions: Network is sequentially observed. Assign T1 = 1 with probability 0.5; for n=2 to N do Calculate Dn 1, Dn 1 (i; k i ), for 1 i I, and Dn 1 (k ) based on n-th user s covariate profile Xn that falls in stratum k ; Assign T(1) n T n 1, 1 , calculate Imb(1) n,cov, Imb(1) n,net, and Imb(1) n,w; Assign T(2) n T n 1, 0 , calculate Imb(2) n,cov, Imb(2) n,net, and Imb(2) n,w; Calculate Imbn,w = Imb(1) n,w Imb(2) n,w; Let T n = z T n 1, 1 + (1 z) T n 1, 0 , where z Bernoulli (g( Imbn,w)) Obtain: assignment vector T. |
| Open Source Code | Yes | The R codes and real data used for the numerical studies are available at https://github.com/jialush/ AB-Testing-in-Network-Data-with-Covariate-Adaptive-Randomization.git. |
| Open Datasets | Yes | In this section, we redesign the experiment in Cai et al. (2015) to evaluate the performance of Algorithm 1. They studied the influence of the social network on insurance adoption by rice farmers in rural China. ... The R codes and real data used for the numerical studies are available at https://github.com/jialush/ AB-Testing-in-Network-Data-with-Covariate-Adaptive-Randomization.git. |
| Dataset Splits | No | The paper describes a sequential randomization procedure rather than a typical machine learning setup with distinct train, validation, and test splits. It uses sample sizes (e.g., N=200 for hypothetical networks) for its simulations, but does not specify standard data partitioning percentages or counts for training, validation, and test sets. |
| Hardware Specification | No | The paper does not provide any specific hardware details (e.g., CPU, GPU models, or memory specifications) used for running the numerical studies or simulations. |
| Software Dependencies | No | The paper mentions that "The R codes...are available at..." and "All simulation studies are performed with 1000 runs", indicating the use of R, but it does not specify the version of R or any specific R packages/libraries with their version numbers that are necessary for reproducibility. |
| Experiment Setup | Yes | For the assignment function g(x), we set 1 10|x| 2.1 when x 10 0.9 when 10 < x < 0 0.5 when x = 0 0.1 when 0 < x < 10 10|x| 2.1 when 10 x. Hence, (7) is satisfied with M = 10 and g(x) + g( x) = 1 holds. ... we consider linear cases first, Yi = µ0 + µ1Ti + γAi T + Xiβ + εi where µ0 = 1, µ1 = 0, γ = 1, βi = 1 for i, σε = 1 pdensity = 0.05. ... The weights used for Imbn,cov in CAR, and AL* are (wo, wm,1, wm,2, ws) = (0.3, 0.1, 0.1, 0.5) and different w are used for AL and AL . ... To generate the outcome, we reparametrize the µ and β based on the following generalized linear morel: h(EYi) = logit(EYi) = µ0 + µ1Ti + f(Ai , Ti). We first consider the following two cases. Case 1: f(Ai , Ti) = γAi Ti, which assumes the users are linearly associated with the number of their friends assigned in the treatment group. Case 2: f(Ai , Ti) = γ Ai Ti, an extended version of the linear-in-means model. ... Here f(Ai , Ti) = γAi Ti + δBi Ti, where Bij = Pk =i,j Aik Ajk. In Case 3 we take δ = 0.5, while in Case 4 we take δ = 0.25. |