Efficient Task Sub-Delegation for Crowdsourcing
Authors: Han Yu, Chunyan Miao, Zhiqi Shen, Cyril Leung, Yiqiang Chen, Qiang Yang
AAAI 2015 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experimental comparisons with state-of-the-art approaches based on the Epinions trust network demonstrate significant advantages of RTS under high workload conditions. ... RTS has been compared to four state-of-the-art approaches through extensive simulations based on the Epinions trust network dataset. ... In this section, we compare the performance of RTS with four state-of-the-art approaches in simulations based on an MAS trust network from the Epinions trust network dataset (Leskovec, Huttenlocher, and Kleinberg 2010). ... The simulations allow us to better understand the behavior of RTS by varying the parameter settings to create different operating environments. ... In the experiments, we measure the performance of each approach using the following metrics: 1. Achieved social welfare (ASW): ... 2. Task expiry rate (TER): ... Figure 2 contains sub-figures showing the performance of the approaches according to the evaluation metrics. |
| Researcher Affiliation | Academia | Han Yu1, Chunyan Miao1, Zhiqi Shen1, Cyril Leung1,2, Yiqiang Chen3, Qiang Yang4 1School of Computer Engineering, Nanyang Technological University, Singapore 2Department of Electrical and Computer Engineering, the University of British Columbia, Vancouver, BC, Canada 3Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China 4Department of Computer Science and Engineering, Hong Kong University of Science and Technology, Hong Kong |
| Pseudocode | Yes | Algorithm 1 The RTS Approach |
| Open Source Code | No | The paper does not provide any links to open-source code for the methodology described, nor does it state that the code is available in supplementary materials or upon request. |
| Open Datasets | Yes | Experimental comparisons with state-of-the-art approaches based on the Epinions trust network demonstrate significant advantages of RTS under high workload conditions. ... In this section, we compare the performance of RTS with four state-of-the-art approaches in simulations based on an MAS trust network from the Epinions trust network dataset (Leskovec, Huttenlocher, and Kleinberg 2010). |
| Dataset Splits | No | The paper describes using the Epinions trust network dataset for simulations but does not specify traditional machine learning data splits (e.g., percentages or counts for training, validation, or test sets). It describes how the simulated network is generated from the dataset, but not how it is split for model training/evaluation purposes. |
| Hardware Specification | No | The paper does not specify any hardware details (e.g., CPU, GPU models, memory, or cloud computing resources) used for running the simulations or experiments. |
| Software Dependencies | No | The paper mentions running 'simulations' but does not provide any information about the specific software, libraries, or their version numbers used in the experimental setup. |
| Experiment Setup | Yes | For any agent i in the simulation, its nsub i consists of agents connected with i through a directed +1 edge originating from i. ... The µmax i value of each agent is set in such a way that it is directly proportional to its hi value. ... At each time step, 20% of the agents are selected at random to act as truster agents and delegate tasks to others. ... In the experiments, we vary the LF(t) value from 0.1 to 1.0 to simulate different workload conditions. Under each LF(t) setting, the simulation is run for T = 1,000 time steps. In all experiments, trustee agents process tasks at an average rate of 0.9µmax i with a standard deviation of 0.1µmax i following i.i.d. On average, tasks need to be completed within 5 time steps after it is first delegated to a trustee agent. |