What Truly Matters in Trajectory Prediction for Autonomous Driving?
Authors: Tran Phong, Haoran Wu, Cunjun Yu, Panpan Cai, Sifa Zheng, David Hsu
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | This paper studies the overlooked significance of this dynamics gap. We also examine several other factors contributing to the disparity between prediction performance and driving performance. The findings highlight the trade-off between the predictor s computational efficiency and prediction accuracy in determining real-world driving performance. In summary, an interactive, task-driven evaluation protocol for trajectory prediction is crucial to capture its effectiveness for autonomous driving. Source code along with experimental settings is available online. We demonstrate a strong correlation between the dynamic evaluation metrics and driving performance through extensive experiments. |
| Researcher Affiliation | Academia | 1 National University of Singapore 2 Tsinghua University 3 Shanghai Jiao Tong University |
| Pseudocode | Yes | Pseudocode can be found in the supplementary materials. |
| Open Source Code | Yes | Source code along with experimental settings is available online. |
| Open Datasets | No | To investigate the correlation between prediction accuracy and driving performance, we train all selected motion prediction models on the Alignment dataset collected from the SUMMIT simulator. The paper mentions the dataset name and how it was collected, but does not provide a direct link, DOI, or explicit statement about its public availability or a formal citation for the dataset itself. |
| Dataset Splits | Yes | We collect 59,944 scenarios and separate them into two groups: 80% training and 20% validation. |
| Hardware Specification | Yes | The data collection was conducted on a server equipped with an Intel(R) Xeon(R) Gold 5220 CPU. We use four NVIDIA Ge Force RTX 2080 Ti to speed up the running. |
| Software Dependencies | No | The paper mentions software components like SUMMIT simulator [6] and CARLA [12] framework, but does not provide specific version numbers for these or any other ancillary software dependencies (e.g., Python, PyTorch, TensorFlow, specific libraries). |
| Experiment Setup | Yes | In our experiments, the consensus of K=6 is adopted. While ADE/FDE can be applied to evaluate single-trajectory prediction models, their probabilistic variants min ADE and min FDE can be applied to evaluate multi-trajectory predictors. We set ϵ = 1m for our experiments since the DESPOT planner rarely causes real collisions. We collect 59,944 scenarios and separate them into two groups: 80% training and 20% validation. Each scenario consists of about 300 steps. Subsequently, it is filtered down to 50 steps by taking into account the number of agents and their occurrence frequency. The nearest three agents are randomly selected to be the interested agent for prediction. We conduct two types of experiments for both planners in the SUMMIT simulator: Fixed Prediction Ability and Fixed Planning Ability. Three sub-experiments are conducted with tick rates set at 30 Hz, 3 Hz, and 1 Hz. For each scenario, we randomly select the start and end point for the ego-agent from one of the four real-world maps provided by the SUMMIT simulator. A reference path of 50 meters is maintained between the two points, and the ego-agent is instructed to follow this path. A certain number of exo-agents including pedestrians, cyclists, and vehicles is randomly distributed within the environment. |