Quality Diversity through Human Feedback: Towards Open-Ended Diversity-Driven Optimization
Authors: Li Ding, Jenny Zhang, Jeff Clune, Lee Spector, Joel Lehman
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Empirical studies show that QDHF significantly outperforms state-of-the-art methods in automatic diversity discovery and matches the efficacy of QD with manually crafted diversity metrics on standard benchmarks in robotics and reinforcement learning. |
| Researcher Affiliation | Collaboration | 1Manning College of Information & Computer Sciences, University of Massachusetts Amherst 2Department of Computer Science, University of British Columbia 3Vector Institute 4Canada CIFAR AI Chair 5Department of Computer Science, Amherst College 6Stochastic Labs *Part of the work was done while the author was affiliated with Stability AI. |
| Pseudocode | Yes | Algorithm 1 MAP-Elites Algorithm 1: Initialize a map of solutions, each cell representing a unique feature combination 2: while not converged do 3: Generate new solutions via mutation and crossover 4: for each solution do 5: Evaluate the solution for its performance and feature characteristics 6: Identify the corresponding cell in the map based on features 7: if solution is better than the current cell occupant then 8: Replace the cell s solution with the new solution 9: end if 10: end for 11: end while 12: Return the map of elite solutions |
| Open Source Code | Yes | Code and tutorials are available at https://liding.info/qdhf. |
| Open Datasets | Yes | We use human judgment data from the NIGHTS dataset, and the Dream Sim model to estimate image similarity, both from Fu et al. (2023). |
| Dataset Splits | No | The paper mentions evaluating on a 'validation set' but does not provide specific details such as percentages, sample counts, or the methodology for creating these splits. |
| Hardware Specification | No | The paper mentions using a 'high-performance computing cluster at the Massachusetts Green High Performance Computing Center (MGHPCC)' but does not provide specific hardware details like GPU/CPU models or memory amounts. |
| Software Dependencies | Yes | We use Stable Diffusion v2.1-base, which generates images at a resolution of 512x512. The feature extractor is a CLIP model with Vi T-B/16 backbone, which returns a 512-dim feature vector. |
| Experiment Setup | Yes | For all experiments, we run MAPElites for 1000 iterations, and for each iteration, we generate a batch of 100 solutions with Gaussian mutation (adding Gaussian noises sampled from N(0, 0.12)), and evaluate them. The archive has a shape of (50, 50), i.e., each of the 2 dimensions is discretized into 50 equal-sized bins. |