Continuously Learning, Adapting, and Improving: A Dual-Process Approach to Autonomous Driving
Authors: Jianbiao Mei, Yukai Ma, Xuemeng Yang, Licheng Wen, Xinyu Cai, Xin Li, Daocheng Fu, Bo Zhang, Pinlong Cai, Min Dou, Botian Shi, Liang He, Yong Liu, Yu Qiao
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Closed-loop testing in CARLA shows that Leap AD outperforms all methods relying solely on camera input, requiring 1-2 orders of magnitude less labeled data. Experiments also demonstrate that as the memory bank expands, the Heuristic Process with only 1.8B parameters can inherit the knowledge from a GPT-4 powered Analytic Process and achieve continuous performance improvement. Project page: https://pjlab-adg.github.io/Leap AD/. |
| Researcher Affiliation | Collaboration | 1 Zhejiang University 2 Shanghai Artificial Intelligence Laboratory 3 East China Normal University 4 Shanghai Jiao Tong University |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The abstract mentions a "Project page: https://pjlab-adg.github.io/Leap AD/", which is a project overview page, not a direct link to a source-code repository, nor is there an explicit statement about code release for the methodology. |
| Open Datasets | Yes | We construct the instruct-following datasets for supervised fine-tuning of our VLM by integrating Rank2Tell [58], Drive LM [36], and data collected within CARLA [59]. |
| Dataset Splits | No | The paper discusses training data and evaluation on a test benchmark, but does not explicitly define or specify a separate validation dataset split with percentages or sample counts for model tuning during development. |
| Hardware Specification | Yes | The batch size is set to 16, and the model is trained for 5 epochs on 8 A100 GPUs, requiring about 26 hours. |
| Software Dependencies | No | The paper mentions specific models like "Qwen-VL-7B", "GPT-4", and "Qwen1.5-1.8B" and the "Open AI embedding model", and the CARLA simulator, but does not provide specific version numbers for underlying software environments (e.g., Python, PyTorch, CUDA, or CARLA version). |
| Experiment Setup | Yes | We utilize the Adam W optimizer [61] with β1 = 0.9 and β2 = 0.95, coupled with a cosine decay of the learning rate, initially set to 1e 5. The batch size is set to 16, and the model is trained for 5 epochs on 8 A100 GPUs, requiring about 26 hours. |