Differentiable Weightless Neural Networks
Authors: Alan Tendler Leibel Bacellar, Zachary Susskind, Mauricio Breternitz Jr, Eugene John, Lizy Kurian John, Priscila Machado Vieira Lima, Felipe M.G. França
ICML 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To demonstrate the effectiveness and versatility of DWNs, we evaluate them in several scenarios. First, we assess their performance on a custom hardware accelerator, implemented using a field-programmable gate array (FPGA), to demonstrate DWNs extreme speed and energy efficiency in high-throughput edge computing applications. Next, we implement DWNs on an inexpensive off-the-shelf microcontroller, demonstrating that they can operate effectively on very limited hardware, and emphasizing their practicality in cost-sensitive embedded devices. We also consider the incorporation of DWNs into logic circuits, assessing their potential utility in ultra-low-cost chips. |
| Researcher Affiliation | Academia | 1Federal University of Rio de Janeiro, Brazil 2The University of Texas at Austin, USA 3ISCTE Instituto Universitario de Lisboa, Lisbon, Portugal 4The University of Texas at San Antonio, USA 5Instituto de Telecomunicac oes, Porto, Portugal. |
| Pseudocode | No | The paper describes the methods in narrative text and mathematical formulas but does not include any explicitly labeled 'Pseudocode' or 'Algorithm' blocks. |
| Open Source Code | Yes | https://github.com/alanbacellar/DWN |
| Open Datasets | Yes | All datasets in the experimental evaluation are binarized using the Distributive Thermometer (Bacellar et al., 2022) for both DWN and Diff Logic Net. The sole exception is the Diff Logic Net model for the MNIST dataset, for which we use a threshold of 0, following the strategy outlined in their paper. ... Table 8. List of all datasets used in this paper. Dataset Name Source Notes Iris (Fisher, 1988) ... MNIST (Deng, 2012) ... Fashion MNIST (Xiao et al., 2017) ... |
| Dataset Splits | No | The datasets for this analysis are those shared between the Tiny Classifiers (see Appendix G) and Auto Gluon (to be used in the next subsection) studies, providing a consistent basis for comparison. We also adhere to their data-splitting methods, using 80% of the data for the training set and 20% for the testing set. |
| Hardware Specification | Yes | We deploy DWN models on the Xilinx Zynq Z-7045, an entry-level FPGA... The Elegoo Nano is a clone of the open-source Arduino Nano, built on the 8-bit ATmega328P... |
| Software Dependencies | No | We use the Micro ML (Salerno, 2022) library for XGBoost inference on the Nano and compare it against DWNs. |
| Experiment Setup | Yes | Table 13. DWN model configurations for Table 1 and Table 2. All models were trained for a total of 100 Epochs. Dataset Model z Layers tau BS Learning Rate ... Table 16. DWN model configurations for Table 5. Dataset z Layers tau BS Learning Rate Epochs |