Learning Dictionary for Visual Attention
Authors: Yingjie Liu, Xuan Liu, Hui Yu, XUAN TANG, Xian Wei
NeurIPS 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this section, we evaluate the performance of proposed Dic-Attn on 6 benchmarks. Firstly, we conduct ablation experiments on CIFAR-10 dataset, as described in Sec. 4.1. Subsequently, we compare the computational costs. In Sec. 4.2, we evaluate the performance of the Dic Attn module across various visual tasks, such as point cloud classification, image segmentation, and image classification. We compare the accuracy and robustness of the proposed method with several notable attention modules. |
| Researcher Affiliation | Academia | Yingjie Liu1 Xuan Liu2 Hui Yu2 Xuan Tang1 Xian Wei1 1East China Normal University 2FJIRSM, Chinese Academy of Sciences |
| Pseudocode | Yes | Algorithm 1: Dictionary Learning-Based Attention Module Input: X Rb s n, depth N, Parameter: D S(n, k), WΦ Rk k, WD Rk, Wscale Latent Variables: Φ Rk s, f(ΦT), V Output: Attention Output # Initialize D S(n, k) by randomly sampling from N(0, 1) and atom-wise normalization; # Initialize WΦ, WD, Wscale with reference to kaiming-init method [12]; # forward ## Update Φ by Eq. (3); ## Obtain Attention Output by Eq. (4); # backward ## Update D, WΦ, WD, Wscale by minimizing loss function of the network ; |
| Open Source Code | Yes | More experiments and the code are available in the Supplementary Material. |
| Open Datasets | Yes | We evaluate the performances of our proposed Dic-Attn module on public image datasets and point cloud datasets, including CIFAR-10, CIFAR-100[20], Model Net40 [53], Scan Object NN [45], and ADE20K [63]. |
| Dataset Splits | Yes | The training set contains 20210 images. There are about 2000 and 3352 images in the validation and test set, respectively. |
| Hardware Specification | No | The paper mentions 'GPU Train / Inference Time (ms)' in Table 1 but does not provide specific details about the GPU model, CPU, memory, or any other hardware used for experiments. |
| Software Dependencies | No | The paper mentions the use of various models and methods but does not provide specific version numbers for any software, libraries, or frameworks used in the experiments. |
| Experiment Setup | No | The paper states 'More implementation settings are described in detail in the Supplementary Material' and discusses hyperparameter influences (Figure 2), but does not provide specific hyperparameter values, training configurations, or system-level settings in the main text. |