Embedding-Aligned Language Models
Authors: Guy Tennenholtz, Yinlam Chow, Chih-wei Hsu, Lior Shani, Yi Liang, Craig Boutilier
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate the effectiveness of the EAGLE agent using the Movie Lens 25M and Amazon Review datasets to surface content gaps that satisfy latent user demand. We also demonstrate the benefit of using an optimal design of a state-dependent action set to improve EAGLE s efficiency. Our work paves the way for controlled and grounded text generation using LLMs, ensuring consistency with domain-specific knowledge and data representations. Our results are evaluated by human raters demonstrating EAGLE s ability to guide an LLM to generate text that is both creative and consistent with the domain s data representations. |
| Researcher Affiliation | Industry | Google Research, Google Deepmind |
| Pseudocode | Yes | Algorithm 1 Embedding-Aligned Guided Languag E (EAGLE) Agent |
| Open Source Code | No | Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: The dataset can be regenerated using the provided prompts and the available Gemini model API. |
| Open Datasets | Yes | Our work uses the Movie Lens 25M dataset [Harper and Konstan, 2015], which contains 25 million ratings (1 to 5) of 62,423 movies and 162,541 users. |
| Dataset Splits | No | No explicit statement of training, validation, and test dataset splits with percentages or sample counts for the main model training was found. |
| Hardware Specification | Yes | EAGLE was trained on a Gemini Nano-2 language model (3.25B), which can be trained using an equivalent of one A100 GPU. In our work, we train EAGLE in a parallelized manner, on an equivalent of 16 A100 GPUs. |
| Software Dependencies | No | The paper mentions using specific LLM models like Gemini Nano-2 and Gemini Ultra API, but does not list specific version numbers for underlying software libraries, frameworks (e.g., PyTorch, TensorFlow), or programming languages used for implementation. |
| Experiment Setup | Yes | We provide hyperparameter details used for training the reference policy in Table 6 below: Training Steps 20000 Batch Size 1024 Learning Rate 2e-6 Dropout Probability 0.1 and Table 7: Training hyperparameters for EAGLE. Training steps 30000 Reference policy KL regularization (α) 0.1 Policy learning rate 1e-5 Value learning rate 5e-6 Policy (agent) temperate 0.5 Environment temperature 0.5 Horizon H 5 Discount γ 1 |