ConceptNet 5.5: An Open Multilingual Graph of General Knowledge
Authors: Robyn Speer, Joshua Chin, Catherine Havasi
AAAI 2017 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We demonstrate this with state-of-the-art results on intrinsic evaluations of word relatedness that translate into improvements on applications of word vectors, including solving SAT-style analogies. |
| Researcher Affiliation | Collaboration | Robyn Speer Luminoso Technologies, Inc. ... Joshua Chin Union College ... Catherine Havasi Luminoso Technologies, Inc. |
| Pseudocode | No | No pseudocode or algorithm blocks are present in the paper. |
| Open Source Code | Yes | The code and documentation of Concept Net 5.5 can be found on Git Hub at https://github.com/commonsense/ conceptnet5, and the knowledge graph can be browsed at http://conceptnet.io. |
| Open Datasets | Yes | We focus on MEN-3000 (Bruni, Tran, and Baroni 2014), a large crowd-sourced ranking of common words; RW (Luong, Socher, and Manning 2013), a ranking of rare words; Word Sim-353 (Finkelstein et al. 2001), a smaller evaluation that has been used as a benchmark for many methods; and MTurk-771 (Halawi et al. 2012), another crowd-sourced evaluation of a variety of words. AND On a corpus of SAT-style analogy questions (Turney 2006). |
| Dataset Splits | Yes | MEN-3000 is already divided into a 2000-item development set and a 1000-item test set. We use the results from the test set as the final results. AND RW has no standard dev/test breakdown. We sampled 2/3 of its items as a development set and held out the other 1/3 (every third line of the file, starting with the third). AND We used all of Word Sim-353 in development. AND The grid search is performed on odd-numbered questions, holding out the even-numbered questions as a test set. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory specifications) are provided for the experiments. |
| Software Dependencies | No | The paper mentions Snakemake and Docker Compose but does not specify version numbers for other key software dependencies or libraries used in the experiments. |
| Experiment Setup | Yes | The weights found for Concept Net Numberbatch 16.09 were w1 = 0.2 and w2 = 0.6. AND Our preliminary attempt to apply Concept Net Numberbatch to the Story Cloze Test is to use a very simple bag-of-vectors model, by averaging the embeddings of the words in the sentence and choosing the ending whose average is closest. |