Latent Representation and Simulation of Markov Processes via Time-Lagged Information Bottleneck
Authors: Marco Federici, Patrick Forré, Ryota Tomioka, Bastiaan S. Veeling
ICLR 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experiments demonstrate that T-IB learns information-optimal representations for accurately modeling the statistical properties and dynamics of the original process at a selected time lag, outperforming existing time-lagged dimensionality reduction methods. |
| Researcher Affiliation | Collaboration | Marco Federici AMLab University of Amsterdam m.federici@uva.nl Patrick Forré AI4Science Lab, AMLab University of Amsterdam p.d.forre@uva.nl Ryota Tomioka Microsoft Research AI4Science ryoto@microsoft.com Bastiaan S. Veeling Microsoft Research AI4Science basveeling@microsoft.com |
| Pseudocode | No | No pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | No | The paper uses third-party libraries (e.g., Torch MD Equivariant Transformer, Deeptime python library) but does not provide a link to its own open-source code implementation or state that it will be released. |
| Open Datasets | Yes | We generated separate training, validation, and test trajectories of 100K steps each. We analyze trajectories obtained by simulating Alanine Dipeptide, Chignolin, and Villin (Lindorff-Larsen et al., 2011). |
| Dataset Splits | Yes | We generated separate training, validation, and test trajectories of 100K steps each. |
| Hardware Specification | Yes | On the other hand, a single A100 GPU can simulate only up to 1.5 microseconds each day. Unfolding one transition step using the Flow++ transition model used in our experiments requires approximately 100 milliseconds on a single A100 GPU. |
| Software Dependencies | No | The paper mentions several software components (e.g., Deep Time package, Torch MD Equivariant Transformer, SMILE, Torch-Mist) but does not provide specific version numbers for them. |
| Experiment Setup | Yes | We train each encoder for a maximum of 50 epochs with mini-batches of size 512 using the Adam W (Loshchilov & Hutter, 2019) optimizer. To prevent overfitting, we use early stopping based on the validation loss. Following previous work (Chen et al., 2020), the models are trained with an initial learning rate of 10 6, which is gradually increased up to 5 10 4 over the course of 5 epochs with a linear schedule. The learning rate is then decreased to the initial value using a cosine schedule over the following 45 epochs. |