Towards Lower Bounds on the Depth of ReLU Neural Networks
Authors: Christoph Hertrich, Amitabh Basu, Marco Di Summa, Martin Skutella
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Theoretical | Using techniques from mixed-integer optimization, polyhedral theory, and tropical geometry, we provide a mathematical counterbalance to the universal approximation theorems... and In Section 2, we resolve Conjecture 1.2 for k = 2, under a natural assumption on the breakpoints of the function represented by any intermediate neuron. We achieve this result by leveraging techniques from mixed-integer programming to analyze the set of functions computable by certain NNs. This provides a computational proof of Theorem 2.5. |
| Researcher Affiliation | Academia | Christoph Hertrich Technische Universität Berlin Berlin, Germany christoph.hertrich@posteo.de, Amitabh Basu Johns Hopkins University Baltimore, USA basu.amitabh@jhu.edu, Marco Di Summa Università degli Studi di Padova Padua, Italy disumma@math.unipd.it, Martin Skutella Technische Universität Berlin Berlin, Germany martin.skutella@tu-berlin.de |
| Pseudocode | No | No structured pseudocode or algorithm blocks were found in the paper. |
| Open Source Code | No | The paper does not provide an explicit statement or link for the open-source code for the methodology described. |
| Open Datasets | No | This is a theoretical paper that does not involve experimental training on datasets. |
| Dataset Splits | No | This is a theoretical paper and does not involve dataset splits for training, validation, or testing. |
| Hardware Specification | No | The paper mentions a 'computer-aided proof' and solving a Mixed-Integer Program (MIP) but does not provide specific hardware details (e.g., CPU, GPU models, or cloud computing specifications) used for these computations. |
| Software Dependencies | Yes | Gurobi Optimization, LLC. Gurobi optimizer reference manual, 2021. and The Sage Developers. Sage Math, the Sage Mathematics Software System (Version 9.0), 2020. |
| Experiment Setup | No | This is a theoretical paper that performs a computer-aided proof involving a Mixed-Integer Program (MIP), but it does not describe specific experimental setup details such as hyperparameters or training configurations typically found in empirical machine learning studies. |