CircuitNet: A Generic Neural Network to Realize Universal Circuit Motif Modeling
Authors: Yansen Wang, Xinyang Jiang, Kan Ren, Caihua Shan, Xufang Luo, Dongqi Han, Kaitao Song, Yifei Shen, Dongsheng Li
ICML 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments have demonstrated that Circuit Net can outperform popular neural network architectures in function approximation, reinforcement learning, image classification, and time series forecasting tasks. Experiments on both synthetic and real-world datasets prove that, with comparable or even fewer parameters, Circuit Net can outperform popular neural network architectures in various types of tasks, demonstrating its effectiveness and generalizability in machine learning. |
| Researcher Affiliation | Industry | Microsoft Research Asia, No.701 Yunjin Rd., Xuhui District, Shanghai, China. Correspondence to: Yansen Wang <yansenwang@microsoft.com>. |
| Pseudocode | No | The paper does not include any clearly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper does not provide any specific link or explicit statement about releasing the source code for the described methodology. |
| Open Datasets | Yes | MNIST is a handwriting digit recognition dataset using 60,000 images as training set and 10,000 images as test set, with ten categories, 0 to 9. The CIFAR-10 dataset consists of 50,000 images of size 32 32 in training set and 10,000 images in test set with 10 classes. For Image Net dataset we use the most common Image Net (ILSVRC) subset that contains 1,000 object classes with totally 1,281,167 training images and 50,000 validation images. Solar (Lai et al., 2018) includes the solar power production records in 2006. Pems-bay (Li et al., 2017) contains average traffic speed records. Metr-la (Li et al., 2017) contains average traffic speed. |
| Dataset Splits | Yes | MNIST is a handwriting digit recognition dataset using 60,000 images as training set and 10,000 images as test set, with ten categories, 0 to 9. The CIFAR-10 dataset consists of 50,000 images of size 32 32 in training set and 10,000 images in test set with 10 classes. For Image Net dataset we use the most common Image Net (ILSVRC) subset that contains 1,000 object classes with totally 1,281,167 training images and 50,000 validation images. |
| Hardware Specification | No | The paper does not specify any particular hardware components (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | The paper mentions various software components or algorithms used (e.g., Adam optimizer, SAC) but does not provide specific version numbers for them or other required libraries. |
| Experiment Setup | Yes | The learning rate was set to 1 10 2 with 0.9 momentum, and all the models are optimized for 10,000 epochs at maximum to ensure convergence. The detailed hyperparameters are listed in Table 8. The sizes of the network are designed to match the number of parameters of the MLP baseline in the original paper. As for the learning rate and the activation function, we conducted grid search for every possible value in the sets and the best results of each model is reported. Table 9. Training details for Image Classification. Optimizer {Adam, SGD} Maximum learning rate { 1 10 1, 1 10 2, 1 10 3 5 10 4, 1 10 4 } # Warm-up iterations {0, 5} Learning rate decay Cosine Annealing Scheduler Number of epochs 600. Table 10. Training details for time series forecasting. Optimizer Adam Loss function MSELoss Learning rate 1 10 3 Weight decay 1 10 5 Number of epochs 1000. |