Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
Automatic Assessment of Absolute Sentence Complexity
Authors: Sanja Stajner, Simone Paolo Ponzetto, Heiner Stuckenschmidt
IJCAI 2017 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We perform three sets of experiments (Sections 5.1 5.3) to evaluate our approach. |
| Researcher Affiliation | Academia | Sanja ˇStajner, Simone Paolo Ponzetto and Heiner Stuckenschmidt Data and Web Science Group, University of Mannheim, Germany EMAIL |
| Pseudocode | No | The paper does not contain any pseudocode or algorithm blocks. |
| Open Source Code | Yes | Data and code available at: http://web.informatik.unimannheim.de/sstajner/publications. |
| Open Datasets | Yes | We use the English part of the Newsela corpora6 to learn lexical properties on different text complexity levels (the lists of unigrams, bigrams and trigrams occurring at each level, and their relative frequencies). [Footnote 6: https://newsela.com/data/]. Also, "Data and code available at: http://web.informatik.unimannheim.de/sstajner/publications." for their gold standard dataset. |
| Dataset Splits | Yes | We used five different classifiers: Logistic [le Cessie and van Houwelingen, 1992], SMOs Weka implementation of SVM [Platt, 1998] with feature standardisation, JRip rule learner [Cohen, 1995], J48 Weka implementation of C4.5 decision tree [Quinlan, 1993], and Random Forest [Breiman, 2001], in a 10-fold cross-validation setup with 10 repetitions in Weka Experimenter [Hall et al., 2009]. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware used for running the experiments. |
| Software Dependencies | No | The paper mentions 'Weka Experimenter [Hall et al., 2009]' and various classifiers, but does not provide specific version numbers for these software components. |
| Experiment Setup | No | The paper mentions using Random Forest and a 10-fold cross-validation setup, but does not provide specific hyperparameter values or detailed training configurations for the models. |