Semantic Routing via Autoregressive Modeling
Authors: Eric Zhao, Pranjal Awasthi, Zhengdao Chen, Sreenivas Gollapudi, Daniel Delling
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | As a proof-of-concept, we show that at scale even a standard transformer network is a powerful semantic routing system and achieves non-trivial performance on our benchmark. |
| Researcher Affiliation | Collaboration | Eric Zhao UC Berkeley and Google Research eric.zh@berkeley.edu Pranjal Awasthi Google Research pranjalawasthi@google.com Zhengdao Chen Google Research zhengdaoc@google.com Sreenivas Gollapudi Google Research sgollapu@google.com Daniel Delling Google delling@google.com |
| Pseudocode | No | The paper does not contain structured pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | First, we release a large-scale publicly-licensed benchmark for semantic routing, which can be found at github.com/google-research/google-research/tree/master/semantic_routing. All data and code described in this paper, for both our benchmark and proofof-concept, are available open-source under an Apache 2.0 license. |
| Open Datasets | Yes | We are releasing a public dataset of semantic routing problems concerning complex navigation tasks on American roads. This benchmark, and the remainder of our paper, focuses on real-world multi-objective navigation problems that involve providing route suggestions to drivers. |
| Dataset Splits | No | The paper discusses training and testing, but does not explicitly provide details about a separate validation dataset split (e.g., percentages or sample counts). |
| Hardware Specification | No | Experiments performed in this paper were conducted using computer resources on Google Cloud. Approximately 10,000 CPU hours were used to generate training data from the benchmark software. The experiments described in Appendix A were performed on a GPU cluster for approximately 1,000 GPU hours. The experiments described in Section 3 were performed on a GPU cluster for approximately 20,000 GPU hours. The paper mentions Google Cloud, CPU hours, and GPU clusters, but does not specify exact models (e.g., NVIDIA A100 GPUs) or other specific hardware details. |
| Software Dependencies | No | The paper mentions TensorFlow and transformer models but does not list specific software dependencies with version numbers (e.g., Python 3.8, PyTorch 1.9). |
| Experiment Setup | Yes | In this section, we review the hyperparameters for each experiment in this paper. In terms of grid world experiments, Table 6a describes the hyperparameters for the point-of-interest density experiment, the beam width experiment and the secondary scoring model ablation experiment; Table 3 describes the hyperparameters for the receptive field size experiment; and Table 5 and Table 4 describe the hyperparameters for the scaling study experiment. The hyperparmeters for the experiment on our benchmark are described in Table 6b. |