MA-Net: Rethinking Neural Unit in the Light of Astrocytes
Authors: Mengqiao Han, Liyuan Pan, Xiabi Liu
AAAI 2024 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments show that our MA-Net advances new state-of-the-art on multiple tasks while significantly reducing its parameters by connection optimization. We extensively evaluate out MA-Net on multiple tasks, e.g., classification, segmentation, and object detection with public datasets. Compared to the state-of-the-art (SOTA) methods, our MANet improves the accuracy by 0.23% 2.91% on classification (CIFAR10 and Image Net-1k). AP improved by 0.7% 2.0% on segmentation (COCO) and 0.5% 1.5% on object detection (COCO). The optimized connections with parameters are reduced by 10.72% 71.25%. |
| Researcher Affiliation | Academia | Mengqiao Han, Liyuan Pan , Xiabi Liu Beijing Institute of Technology {hmq, liyuan.pan, liuxiabi}@bit.edu.cn |
| Pseudocode | Yes | Algorithm 1: The MA-Net Optimization Algorithm Input: TNet and MNet parameters: W and Φ, ground-truth label: Ygt, max iterations: R, and number of rounds to optimize the TNet without MNet: RTN. Output: TNet output: N, MA-Net output: Y for r = 0 to R: if r RTN: Optimize LTN(N, Ygt) Update W else: if r%2 == 0: Optimize LMN(Y, Ygt) Update Φ else: Optimize LTN(Y, Ygt) Update W end for |
| Open Source Code | Yes | Our code will be published on Git Hub for reproducible research. |
| Open Datasets | Yes | We validate our MA-Net on two classification datasets, CIFAR10 (Krizhevsky, Hinton et al. 2009) and Image Net-1k (Deng et al. 2009), with commonly used convolutional neural networks as the TNet including Res Net (RNT) (He et al. 2016), Wide Res Net (WRN) (Zagoruyko and Komodakis 2016), Dense Net-BC (DNTBC) (Huang et al. 2017), and vision transformers including Vi T (Dosovitskiy et al. 2021) and Swin (Liu et al. 2021). For segmentation, our method is evaluated on COCO (Lin et al. 2014). |
| Dataset Splits | No | No explicit training/test/validation dataset splits or cross-validation details were provided in the main text. |
| Hardware Specification | No | No specific hardware details (e.g., GPU models, CPU types, memory) were mentioned for running experiments. |
| Software Dependencies | No | The paper states 'Our MA-Net is implemented in Py Torch' but does not specify version numbers for PyTorch or any other software dependencies. |
| Experiment Setup | Yes | Our MA-Net is implemented in Py Torch and trained via SGD optimizer with a learning rate of 1e-1. We set the number of astrocytes as M = 4, each astrocyte contains K = 4 LSTMs inside, λ = 1e-1 in Eq. (10), and pruning threshold δ = 1e-3. |