SOI: Scaling Down Computational Complexity by Estimating Partial States of the Model
Authors: Grzegorz Stefański, Paweł Daniluk, Artur Szumaczuk, Jakub Tkaczuk
NeurIPS 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this work, we present a novel method called Scattered Online Inference (SOI) that aims to reduce the computational complexity of ANNs. SOI leverages the continuity and seasonality of time-series data and model predictions, enabling extrapolation for processing speed improvements, particularly in deeper layers. By applying compression, SOI generates more general inner partial states of ANN, allowing skipping full model recalculation at each inference. ... 3 Experiments 3.1 Speech Separation We selected speech separation as our first experimental task. ... 4.1 Speech Separation Results of partially predictive SOI in speech separation task are shown in figure 4. |
| Researcher Affiliation | Industry | 1Samsung AI Center Warsaw 2Samsung R&D Institute Poland {g.stefanski, p.daniluk, a.szumaczuk, j.tkaczuk}@samsung.com |
| Pseudocode | No | The paper does not contain any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material? Answer: [No] Justification: We cannot open-source our codebase due to its commercial purpose. |
| Open Datasets | Yes | The Deep Noise Suppression (DNS) Challenge Interspeech 2020 dataset (Reddy et al., 2020), licensed under CC-BY 4.0, was used for both training and evaluation purposes. ... We used the TAU Urban Acoustic Scene 2020 Mobile dataset (Heittola et al., 2020) for both training and validation. |
| Dataset Splits | Yes | For training, we used 16384 10s samples without any form of augmentation and for both validation and test sets we used 64 samples with similar setup to the training set. |
| Hardware Specification | Yes | Models were trained on a single Nvidia P40 GPU for 500 epochs using Adam optimizer with initial learning rate of 1e-3. ... Each model was trained on Nvidia A100 GPU with batch size of 16. |
| Software Dependencies | No | The paper mentions using 'Adam optimizer' but does not specify version numbers for any software libraries or frameworks (e.g., PyTorch, TensorFlow, etc.) used in the experiments. |
| Experiment Setup | Yes | Each model was trained for 100 epochs using Adam optimizer with initial learning rate of 1e-3. ... Models were trained on a single Nvidia P40 GPU for 500 epochs using Adam optimizer with initial learning rate of 1e-3. ... All models were trained for 100 epochs with Adam optimizer and learning rate 5e-5. Each model was trained on Nvidia A100 GPU with batch size of 16. |