Predicting Event Memorability from Contextual Visual Semantics
Authors: Qianli Xu, Fen Fang, Ana Molino, Vigneshwaran Subbaraju, Joo-Hwee Lim
NeurIPS 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | In this study, we investigate factors that affect event memorability according to a cued recall process. Specifically, we explore whether event memorability is contingent on the event context, as well as the intrinsic visual attributes of image cues. We design a novel experiment protocol and conduct a large-scale experiment with 47 elder subjects over 3 months. Subjects memory of life events is tested in a cued recall process. Using advanced visual analytics methods, we build a first-of-its-kind event memorability dataset (called R3) with rich information about event context and visual semantic features. Furthermore, we propose a contextual event memory network (CEMNet) that tackles multi-modal input to predict item-wise event memorability, which outperforms competitive benchmarks. |
| Researcher Affiliation | Collaboration | Qianli Xu Institute for Infocomm Research A*STAR, Singapore qxu@i2r.a-star.edu.sg Fen Fang Institute for Infocomm Research A*STAR, Singapore fang_fen@i2r.a-star.edu.sg Ana Garcia del Molino Byte Dance AI Lab Singapore a.g.delmolino@gmail.com Vigneshwaran Subbaraju Institute of High Performance Computing A*STAR, Singapore vigneshwaran_subbaraju@ihpc.a-star.edu.sg Joo Hwee Lim Institute for Infocomm Research, A*STAR Nanyang Technological University, Singapore joohwee@i2r.a-star.edu.sg |
| Pseudocode | No | The paper provides a model architecture diagram (Figure 2) and describes the components, but it does not include any pseudocode or clearly labeled algorithm blocks. |
| Open Source Code | Yes | Source code is available at https: //github.com/ffzzy840304/Predicting-Event-Memorability. |
| Open Datasets | No | The paper mentions creating and using the 'R3' dataset for its experiments, stating it is the 'first, large-scale dataset that includes rich visual and contextual information associated with event memory from the real-world context'. However, it does not provide any specific link, DOI, repository, or explicit statement of public availability for this dataset. |
| Dataset Splits | Yes | First, we randomly split the dataset with a 4:1 train-test ratio based on subject index. ... We perform 5 random splits and evaluate the average performance (i.e., 5-fold cross-validation). |
| Hardware Specification | Yes | Our models are trained (with Adam optimizer) and evaluated on Pytorch platform using a machine with NVIDIA Ge Force GTX 1080 GPU. |
| Software Dependencies | No | The paper mentions 'Pytorch platform' but does not specify its version or any other software dependencies with their specific version numbers (e.g., 'Python 3.x', 'CUDA x.x', or specific library versions). |
| Experiment Setup | Yes | For MLP, AMNet and ICNet, the batch size are set as 64, 16 and 32 respectively; learning rate are set as 0.001, 0.0001 and 0.0001; training epochs are set as 100. |