Simulation-Based Approach to Efficient Commonsense Reasoning in Very Large Knowledge Bases
Authors: Abhishek Sharma, Keith M. Goolsbey1360-1367
AAAI 2019 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental results on hundreds of queries show that this method is highly effective in reducing inference time and improving question-answering (Q/A) performance. |
| Researcher Affiliation | Industry | Abhishek Sharma Keith M. Goolsbey , Cycorp Inc., 7718 Wood Hollow Drive, Suite 250, Austin, TX 78731 abhishek@cyc.com, goolsbey@cyc.com |
| Pseudocode | Yes | Figure 1 shows the high-level approach of the simulation algorithm. The algorithm takes a state and depth cutoff as input. The algorithm can be divided into two distinct phases: (i) a tree policy is used until depth d: During this stage, the algorithm selects actions according to knowledge contained with the search tree (ii) when the depth of the node is greater than d, then a default policy is used to complete the simulation. Finally, the outcome of the simulation is backed up through the selected nodes to update their statistics. Input: A state s A depth cutoff, d 1. Create a root node with states. 2. While within computational budget, do a. s Tree Policy (s, d) b. z Default Policy (s ) c. Update Values (s , z) |
| Open Source Code | No | The paper does not provide any concrete access information (specific repository link, explicit code release statement, or code in supplementary materials) for the methodology described in this paper. |
| Open Datasets | No | The paper mentions using queries from the Cyc KB and states "We divided the queries into four parts...the queries from the remaining three were used for training purposes.", but does not provide concrete access information (specific link, DOI, repository name, formal citation with authors/year) for these queries or the Cyc KB dataset as used in their experiments. |
| Dataset Splits | Yes | We divided the queries into four parts. One of the four parts was used as a test set, while the queries from the remaining three were used for training purposes. This process was repeated with each of the four parts to produce four experiments. |
| Hardware Specification | Yes | The experimental data reported here was collected on a 4-core 3.40 GHz Intel processor with 32 GB of RAM. |
| Software Dependencies | No | The paper does not provide specific ancillary software details, such as library or solver names with version numbers, needed to replicate the experiment. |
| Experiment Setup | Yes | The best results from the MCTS simulation was obtained when c was set to 100, and we allowed 400,000 simulations to learn the Q-values. The experimental data reported here was collected on a 4-core 3.40 GHz Intel processor with 32 GB of RAM. Due to the large time requirements of some of these queries, we restricted the cutoff time for each query to 10 minutes. The default policy is executed for a fixed duration of time (set to 30 seconds). |