Collaborative Alignment of NLP Models
Authors: Fereshte Khani, Marco Tulio Ribeiro
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments show Co Align is effective at helping multiple users operationalize concepts and avoid interference for a variety of scenarios, tasks, and models. |
| Researcher Affiliation | Industry | Fereshte Khani Microsoft fkhani@microsoft.com Marco Tulio Ribeiro Google Deep Mind marcotcr@gmail.com |
| Pseudocode | Yes | Algorithm 1: Operationalizing a new concept i |
| Open Source Code | No | Code and data will be released in https://github.com/fereshte-khani/Co Align. |
| Open Datasets | Yes | We use Quora Question Pairs (QQP) dataset... We train RoBERTa-Large on the whole Amazon Review dataset... We use RoBERTa-Large [18] finetuned on MNLI (binary) [19]... |
| Dataset Splits | No | The paper mentions 'base validation dataset' for the MNLI task. However, for all experiments, it does not provide explicit overall training/validation/test split percentages, absolute sample counts, or detailed splitting methodology needed for full reproduction of data partitioning. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware (e.g., GPU model, CPU type, memory) used for conducting the experiments. |
| Software Dependencies | No | The paper mentions using GPT-3 and fine-tuning RoBERTa-Large, but it does not specify version numbers for these models or any other software dependencies, libraries, or frameworks used in the implementation. |
| Experiment Setup | No | While the paper describes various experimental tasks and model evaluations, it does not explicitly provide concrete hyperparameter values (e.g., learning rate, batch size, number of epochs) or detailed system-level training configurations for reproducibility of the experimental setup. |