Locality-Sensitive Hashing-Based Efficient Point Transformer with Applications in High-Energy Physics
Authors: Siqi Miao, Zhiyuan Lu, Mia Liu, Javier Duarte, Pan Li
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | HEPT demonstrates remarkable performance on two critical yet time-consuming HEP tasks, significantly outperforming existing GNNs and transformers in accuracy and computational speed, marking a significant advancement in geometric deep learning and large-scale scientific data processing. Our code is available at https: //github.com/Graph-COM/HEPT. ... To validate the effectiveness of HEPT, we evaluate it on two critical computationally intensive HEP tasks: charged particle tracking (Amrouche et al., 2020) and pileup mitigation (Mart ınez et al., 2019), with their significance elaborated in Appendix A. HEPT is benchmarked against five GNNs and seven efficient transformers adapted from both NLP and CV domains under a unified framework on three datasets (with one of them contributed by us). HEPT significantly outperforms all baselines, achieving state-of-the-art (SOTA) accuracy with up to 203 speedup on GPUs. Our experiments also show that existing RFF-based methods fail to deliver competitive performance. |
| Researcher Affiliation | Academia | 1Georgia Institute of Technology 2Beijing University of Posts and Telecommunications 3Purdue University 4University of California San Diego. Correspondence to: Siqi Miao <siqi.miao@gatech.edu>, Pan Li <panli@gatech.edu>. |
| Pseudocode | No | The paper describes its methods in text and uses figures for illustration, but does not include any clearly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | Yes | Our code is available at https: //github.com/Graph-COM/HEPT. |
| Open Datasets | Yes | We use two datasets derived from the Track ML Particle Tracking Dataset (Amrouche et al., 2020) designed for evaluating algorithms that reconstruct charged particle tracks...Pileup-10k. This dataset, similar to that in Mart ınez et al. (2019); Li et al. (2023), is for the task of pileup mitigation... |
| Dataset Splits | Yes | The dataset split is done in a cloud-wise way, i.e., 80%/10%/10% of point clouds are used to train/validate/test models, respectively. |
| Hardware Specification | Yes | All models are trained and evaluated with the same seed to ensure reproducibility, using a server with NVIDIA Quadro RTX 6000 GPUs and Intel Xeon Gold 6248R CPUs. |
| Software Dependencies | No | The paper mentions software like 'Py G', 'Py Torch Geometric', and 'Adam optimizer' but does not provide specific version numbers for these or other dependencies like Python, CUDA, or major libraries. |
| Experiment Setup | Yes | The hyperparameters for the baselines and HEPT are tuned with similar budgets, based on performance in the validation set of each dataset. For HEPT, we adopt m1 = 3 hash tables, each with m2 = 3 hash functions for the three datasets. The block size of attention computation is set to 100... For all datasets and baselines, Adam optimizer (Kingma & Ba, 2015) is used. For the two Tracking datasets, the learning rate is tuned from {1e 2, 1e 3}, and is multiplied by a factor of 0.5 every 500 epochs. Any model will be early-stopped if there is no improvement in the validation set over 200 consecutive epochs, and models can be trained for up to 2000 epochs to ensure convergence. |