Better Training of GFlowNets with Local Credit and Incomplete Trajectories
Authors: Ling Pan, Nikolay Malkin, Dinghuai Zhang, Yoshua Bengio
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We conduct extensive experiments on the set generation and bit sequence generation tasks, which demonstrates the effectiveness of FL-GFN. It is also found to be scalable to the more complex and challenging molecule discovery task. |
| Researcher Affiliation | Academia | 1Mila Qu ebec AI Institute and Universit e de Montr eal 2CIFAR. Correspondence to: Ling Pan <penny.ling.pan@gmail.com>. |
| Pseudocode | Yes | Algorithm 1 Training Step of FL-GFN |
| Open Source Code | Yes | The code is publicly available at https://github.com/ling-pan/FL-GFN. |
| Open Datasets | No | The paper describes generating synthetic data for set generation and refers to tasks from other papers (Malkin et al., 2022a) for bit sequence generation and molecule generation, but it does not provide concrete access information (link, DOI, or specific citation with authors/year for a dataset) for any publicly available dataset used for training. The set generation task is described as synthesizing elements and energies, and the bit sequence generation task refers to a 'predefined fixed set M of sequences' without public access details. |
| Dataset Splits | No | The paper does not provide specific dataset split information (exact percentages, sample counts, or citations to predefined splits) needed to reproduce the data partitioning into training, validation, and test sets. It mentions evaluating on 'top-100 sampled candidates' but not the data splits themselves. |
| Hardware Specification | No | The paper does not provide specific hardware details (e.g., GPU/CPU models, memory amounts) used for running its experiments. It implicitly refers to training models, which typically involves GPUs, but no explicit specification is given. |
| Software Dependencies | No | The paper mentions 'Adam (Kingma & Ba, 2015)' as an optimizer but does not provide specific version numbers for other key software components like Python, PyTorch, or CUDA, which are necessary for full reproducibility. |
| Experiment Setup | Yes | The GFlow Net model is a feedforward network that consists of 2 hidden layers with 256 hidden units per layer which uses the Leaky ReLU activation function. We sample a parallel of 16 rollouts from the environment for training all of the models. The GFlow Net model is trained based on the Adam (Kingma & Ba, 2015) optimizer with a learning rate of 0.001 for DB, Sub TB, and FL-DB, where we use a larger learning rate of 0.1 for the learnable parameter Z for TB following (Malkin et al., 2022a). [...] The GFlow Net model is a Transformer (Vaswani et al., 2017) consisting of 3 hidden layers with 64 hidden units per layer and has 8 attention heads. The size of the minibatch is 16, and the random action probability is set to 0.0005 for performing ϵ-greedy exploration. The reward exponent is set to 3, and we use a sampling temperature of 1 for the forward policy PF for the GFlow Net models. The learning rate for the policy parameters is 1e-4 for TB, and the learning rate for the learnable parameter Z is 1e-3. The learning rate is 5e-3 for the DB and FL-GFN variants. [...] We use Message Passing Neural Networks (MPNN) for the network architecture for all GFlow Net models, as the molecule is represented as an atom graph. [...] We fine-tune the reward exponent by grid search following (Malkin et al., 2022a) and set it to be 4. We use a random action probability of 0.1 for performing ϵ-greedy exploration, and the learning rate is 5e-4. |