Mini-Batch Consistent Slot Set Encoder for Scalable Set Encoding
Authors: Andreis Bruno, Jeffrey Willette, Juho Lee, Sung Ju Hwang
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform extensive experiments and show that our method is computationally efficient and results in rich set encoding representations for set-structured data. We perform extensive experiments on various tasks such as image reconstruction, point cloud classification and dataset encoding where we demonstrate that SSE significantly outperforms the relevant baselines (Section 3). |
| Researcher Affiliation | Collaboration | KAIST 1, South Korea AITRICS 2, South Korea {andries, jwillette, juholee, sjhwang82}@kaist.ac.kr |
| Pseudocode | Yes | Algorithm 1 Slot Set Encoder. Partitioned input X = {X1, . . . , Xp} Initialized slots S RK h aggregation function g. 1: Input: X = {X1, X2, . . . , Xp}, S RK h, g 2: Output: ˆS RK d 3: Initialize ˆS 4: S = Layer Norm(S) 5: for i = 1 to p do 6: Compute attni(Xi, S) using Equation 3 7: Compute ˆSi(Xi, attni) using Equation 4 8: ˆS = g( ˆS, ˆSi) 9: end for 10: return ˆS |
| Open Source Code | No | The paper does not provide an explicit statement about releasing source code or a link to a code repository. |
| Open Datasets | Yes | We evaluate our model on the Image Net [2], Celeb A [7], MNIST [5] and Model Net40 [13] datasets. Details on these datasets can be found in the Appendix. |
| Dataset Splits | No | The paper does not explicitly specify training/validation/test splits by percentages or sample counts. It refers to training set sizes like 'trained on 1000 context points' but not the splitting methodology. |
| Hardware Specification | No | The paper refers to 'computational and memory resources' generally but does not specify any particular hardware components such as GPU/CPU models, memory, or specific computing environments used for the experiments. |
| Software Dependencies | No | The paper does not provide specific software dependencies with version numbers, such as 'Python 3.8' or 'PyTorch 1.9'. |
| Experiment Setup | Yes | All the methods encode the given set to a 64 dimensional representative vector and are trained for 200 epochs. Additionally, for fair comparison, we only use a single Slot Set Encoder in our models as well as a single PMA block for Set Transformers. We start with a single Slot Set Encoder with K = 1 and h = 32 and for each hierarchy, we double the slot dimension. |