Learning advanced mathematical computations from examples
Authors: Francois Charton, Amaury Hayat, Guillaume Lample
ICLR 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Using transformers over large generated datasets, we train models to learn mathematical properties of differential systems, such as local stability, behavior at infinity and controllability. We achieve near perfect prediction of qualitative characteristics, and good approximations of numerical features of the system. This demonstrates that neural networks can learn to perform complex computations, grounded in advanced theory, from examples, without built-in mathematical knowledge. |
| Researcher Affiliation | Collaboration | François Charton Facebook AI Research fcharton@fb.com Amaury Hayat Ecole des Ponts Paristech, Rutgers University Camden amaury.hayat@enpc.fr Guillaume Lample Facebook AI Research glample@fb.com |
| Pseudocode | No | The paper does not contain structured pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide concrete access to source code for the methodology described. It mentions using other open-source tools but not its own implementation code. |
| Open Datasets | No | The paper states that datasets were 'generated' rather than being from a publicly available source, and no concrete access information for these generated datasets is provided. 'To generate datasets, we randomly sample problems and compute their solutions with mathematical software (Virtanen et al., 2020; Meurer et al., 2017) using the techniques described in Section 3.' |
| Dataset Splits | Yes | Evaluation is performed on held-out validation and test sets of 10000 examples. |
| Hardware Specification | Yes | Training is performed on 8 V100 GPUs with float16 operations. |
| Software Dependencies | Yes | The paper references Mathematica, version 12.0, 2019 (Wolfram-Research, 2019), Matlab optimization toolbox (r2019a), 2019 (Math Works, 2019), and Sci Py 1.0 (Virtanen et al., 2020). |
| Experiment Setup | Yes | We train our models with the Adam optimizer (Kingma and Ba, 2014), a learning rate of 10 4 and the learning rate scheduler in Vaswani et al. (2017), over mini-batches of 1024 examples. |