Identifying Norms from Observation Using MCMC Sampling
Authors: Stephen Cranefield, Ashish Dhiman
IJCAI 2021 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | We apply our technique to a simulated robot manipulator task and show that it allows effective identification of norms from observation. We then evaluate the approach through norm identification experiments in a simulated robot task, and find that it correctly identifies the real norm underlying the simulated behaviour. In this section we describe the simulated scenario used to evaluate our norm identification approach. |
| Researcher Affiliation | Academia | 1Department of Information Science, University of Otago, Dunedin, New Zealand 2Department of Aerospace Engineering, Indian Institute of Technology, Kharagpur, India |
| Pseudocode | Yes | Algorithm 1 shows the Metropolis-Hastings algorithm [Gelman et al., 2013] that is used in this work. |
| Open Source Code | Yes | Our code and supplementary material can be found online.1 1https://git.io/JsVuF |
| Open Datasets | No | No concrete access information (link, DOI, formal citation) is provided for a publicly available dataset. The paper states: "We generated a sequence of observed random task executions that were intentionally compliant with the norm with probability 1 pnn." and "We generated two sets of 100,000 random norm-compliant task executions: for the true norm and the candi-date norm expression." This indicates the data was generated by the authors for their experiments. |
| Dataset Splits | No | The paper does not provide specific dataset split information in terms of percentages or counts for training, validation, and test sets. It describes generating data for the MCMC process, and mentions discarding the first half of each MCMC chain for warm-up, but this is not a dataset split. |
| Hardware Specification | No | No specific hardware details (e.g., GPU/CPU models, memory, or cloud instance types) used for running the experiments are provided in the paper. |
| Software Dependencies | No | The paper mentions using a "Python Counter object" but does not provide specific version numbers for Python or any other software libraries or dependencies used to replicate the experiment. |
| Experiment Setup | Yes | We considered values of pnn between 0 and 0.55, in increments of 0.05. For each pnn, we ran three trials of the following experiment. Our Metropolis-Hastings algorithm (with the modified acceptance from equation 1, and α = 0.1) was run to generate ten chains of length 4800. After discarding the first half of each chain, and splitting the remaining chain into two, the chain convergence metric ˆR was iteratively calculated over subsequences of the resulting 20 chain segments that doubled in length until the end of the segments. |