HOPPITY: LEARNING GRAPH TRANSFORMATIONS TO DETECT AND FIX BUGS IN PROGRAMS
Authors: Elizabeth Dinella, Hanjun Dai, Ziyang Li, Mayur Naik, Le Song, Ke Wang
ICLR 2020 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | By training on 290,715 Javascript code change commits on Github, HOPPITY correctly detects and fixes bugs in 9,490 out of 36,361 programs in an end-to-end fashion. Given the bug location and type of the fix, HOPPITY also outperforms the baseline approach by a wide margin. |
| Researcher Affiliation | Collaboration | Elizabeth Dinella University of Pennsylvania Hanjun Dai Google Brain Ziyang Li University of Pennsylvania Mayur Naik University of Pennsylvania Le Song Georgia Tech Ke Wang Visa Research |
| Pseudocode | Yes | Algorithm 1 Transformation inference of p(gfix|gbug) |
| Open Source Code | No | The paper states, "Our implementation is based on PyTorch with customized GPU kernels to enable efficient inference on GPUs," but it does not provide a direct link to their source code or explicitly state that their code is open-source or available. |
| Open Datasets | No | The paper describes that the data was collected from Github commits using a robust system ( |
| Dataset Splits | Yes | Table 1: Statistic of One Diff dataset. train 290,715 validate 31,357 test 36,361 |
| Hardware Specification | Yes | We trained models on each dataset for roughly 12 hours on a single GTX 2080Ti GPU. |
| Software Dependencies | No | The paper mentions "Our implementation is based on PyTorch with customized GPU kernels" but does not specify a version number for PyTorch or any other software dependencies, which is required for reproducibility. |
| Experiment Setup | Yes | We train the model for 3 epochs on the training set until the validation loss converges. We use the Adam optimizer with β1 = 0.9, β2 = 0.99 and initial learning rate of 10^-3. Due to the large size of each sample, we use a small batch size of 10 during training. Furthermore, to stabilize the training, we apply the gradient clip with the maximum norm of 5. |