A Machine with Short-Term, Episodic, and Semantic Memory Systems
Authors: Taewoon Kim, Michael Cochez, Vincent Francois-Lavet, Mark Neerincx, Piek Vossen
AAAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To evaluate this system and analyze the behavior of this agent, we designed and released our own reinforcement learning agent environment, the Room , where an agent has to learn how to encode, store, and retrieve memories to maximize its return by answering questions. We show that our deep Q-learning based agent successfully learns whether a short-term memory should be forgotten, or rather be stored in the episodic or semantic memory systems. Our experiments indicate that an agent with humanlike memory systems can outperform an agent without this memory structure in the environment. |
| Researcher Affiliation | Academia | 1Vrije Universiteit Amsterdam 2Technische Universiteit Delft |
| Pseudocode | Yes | Algorithm 1: Memory retrieval; Algorithm 2: Converting a knowledge-graph-based memory system into a learnable knowledge graph embedding (KGE). |
| Open Source Code | Yes | The environment is open-sourced at https:// github.com/tae898/room-env. The agent and its implementation are open-sourced at https://github.com/ tae898/explicit-memory. |
| Open Datasets | Yes | As Concept Net (Speer, Chin, and Havasi 2017) can have multiple commonsense object locations per object, we chose commonsense location of an object as the one with the highest weight. We used part of their knowledge graph for our experiments. |
| Dataset Splits | Yes | after each training epoch (i.e., episode) we run it on a validation environment. After 16 epochs of training, we choose the model that has the highest total rewards per episode on validation. |
| Hardware Specification | Yes | Since our neural network is relatively small (i.e., the number of parameters is 265,000), we just used one CPU (i.e., Intel Core i7-10875H) with 64 GB of RAM, instead of using GPUs or ASICs. |
| Software Dependencies | No | The paper mentions 'Py Torch Lighting as our deep learning framework (Falcon 2019)' but does not specify a version number for it or any other software dependencies. |
| Experiment Setup | Yes | In our experimental environment, we set Nhumans = 64, Nobjects = 16, and Nobject locations = 28. We determined these values to be sufficiently challenging. As for pcommonsense, we set it as 0.5... The environment terminates after 128 steps... The short-term memory capacity is fixed at 1, while we try episodic and semantic capacity from 1, 2, 4, 8, 16, and to 32. Table 1 lists hyperparameters: Batch size 1024, Gamma (discount factor) 0.65, Learning rate 0.001, Loss function Huber, Optimizer Adam, Embedding dimension 32, LSTM hidden size dimension 64, Number of LSTM layers 2, etc. |