First Contact: Unsupervised Human-Machine Co-Adaptation via Mutual Information Maximization
Authors: Siddharth Reddy, Sergey Levine, Anca Dragan
NeurIPS 2022 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To evaluate whether this mutual information score can distinguish between effective and ineffective interfaces, we conduct a large-scale observational study on 540K examples of users operating various keyboard and eye gaze interfaces for typing, controlling simulated robots, and playing video games. The results show that our mutual information scores are predictive of the ground-truth task completion metrics in a variety of domains, with an average Spearman s rank correlation of = 0.43. |
| Researcher Affiliation | Academia | University of California, Berkeley {sgr,svlevine,anca}@berkeley.edu |
| Pseudocode | Yes | Algorithm 1 MIMI-EVALUATE( ) |
| Open Source Code | Yes | Code, data, and videos available at https://sites.google.com/view/coadaptation |
| Open Datasets | Yes | We take data from prior work on adaptive interfaces in which the ground-truth rewards were measured, and check whether MIMI s unsupervised evaluation of those interfaces correlates with the true reward that users received when performing tasks via those interfaces. We examine data from four prior works: X2T [63], ASHA [64], shared autonomy via deep reinforcement learning (SAv DRL) [65], and internal-to-real dynamics transfer (ISQL) [66]. [...] The Lunar Lander game [67]. |
| Dataset Splits | Yes | Split D into training set Dtrain and validation set Dval" (Alg. 1, line 7). Also, "instead of using the final training loss ITUBA as our mutual information estimate, we use the validation loss (line 9 in Alg. 1)." and in the author checklist: "Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]" |
| Hardware Specification | No | The main text of the paper does not specify any hardware details such as GPU/CPU models or specific compute resources used for the experiments. While the author checklist indicates that this information was provided, it is not present in the main body of the paper. |
| Software Dependencies | Yes | Hand tracking is performed using a webcam and Media Pipe [68]. |
| Experiment Setup | Yes | Hence, we only take 1K gradient steps (with a small batch size of 64) to fit the estimator in all our experiments." and "Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]" in the author checklist. |