BOLD: Boolean Logic Deep Learning
Authors: Van Minh NGUYEN, Cristian Ocampo-Blandon, Aymen Askri, Louis Leconte, Ba-Hien Tran
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We explore its convergence, conduct extensively experimental benchmarking, and provide consistent complexity evaluation by considering chip architecture, memory hierarchy, dataflow, and arithmetic precision. Our approach achieves baseline full-precision accuracy in Image Net classification and surpasses state-of-the-art results in semantic segmentation, with notable performance in image super-resolution, and natural language understanding with transformer-based models. |
| Researcher Affiliation | Industry | Mathematical and Algorithmic Sciences Laboratory, Huawei Paris Research Center, France |
| Pseudocode | Yes | Algorithm 1: Illustration with a FC layer. Algorithm 2: Boolean optimizer Algorithm 3: Equivalent formulation of Boolean optimizer Algorithm 4: Python code of XOR linear layer Algorithm 5: Python code of the backpropagation logic of XOR linear layer Algorithm 6: Backpropagation logic with Boolean received backpropagation Algorithm 7: Backpropagation logic with real received backpropagation Algorithm 8: Python code of Boolean optimizer |
| Open Source Code | Yes | In this section we provide example codes in Python of a Boolean linear layer that employs xor logic kernel. This implementation is in particular based on Py Torch [78]. |
| Open Datasets | Yes | Our B LD method is tested on two network configurations: small & compact and large & deep. In the former scenario, we utilize the VGG-SMALL [90] baseline trained on CIFAR10. ...Image Net classification and surpasses state-of-the-art results in semantic segmentation, with notable performance in image super-resolution, and natural language understanding with transformer-based models. |
| Dataset Splits | Yes | For IMAGENET, the training images were 192 × 192 px and 224 × 224 px for validation images. The batch size was 300 for both sets and the cross-entropy loss was used during training. |
| Hardware Specification | Yes | This approach is implemented for the Nvidia GPU (Tesla V100) and Ascend [63] architectures. |
| Software Dependencies | No | The presented methodology and the architecture of the described Boolean neural networks (NNs) were implemented in Py Torch [78] and trained on 8 Nvidia Tesla V100 GPUs. The paper mentions PyTorch but does not provide a specific version number for it or other software dependencies. |
| Experiment Setup | Yes | The networks thought predominantly Boolean, also contain a fraction of FP parameters that were optimized using the Adam optimizer [54] with learning rate 10^-3. For learning the Boolean parameters we used the Boolean optimizer (see Algorithm 8). Training the Boolean networks for image classification was conducted with learning rates η = 150 and η = 12 (see Equation 10), for architectures with and without batch normalization, respectively. The hyper-parameters were chosen by grid search using the validation data. During the experiments, both optimizers used the cosine scheduler iterating over 300 epochs. |