TDG4Crowd:Test Data Generation for Evaluation of Aggregation Algorithms in Crowdsourcing
Authors: Yili Fang, Chaojie Shen, Huamao Gu, Tao Han, Xinyi Ding
IJCAI 2023 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | To evaluate our proposed method, we conduct comprehensive experiments on both real and synthetic datasets. In this section, we will first describe the experiment setup. We use annotations collected through the Appen and AMT crowdsourcing platforms. Next, we describe the basesline models used for comparison. Finally, we compare our proposed method with other models for generating new data points. |
| Researcher Affiliation | Academia | School of Computer Science and Technology, Zhejiang Gongshang University, Hangzhou 310018, China {fangyl,ghmsjq,hantao,xding}@zjgsu.edu.cn, qacket@126.com |
| Pseudocode | No | The paper describes the steps and components of the TDG4Crowd method, including mathematical formulations for learning features and inferring annotations, but it does not provide any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | Yes | 1The apendices and souce code are available at https://github. com/Qacket/TDG4Crowd |
| Open Datasets | Yes | We use two real datasets Labelme 2 and Relation 3 for evaluation. |
| Dataset Splits | No | The paper states, 'We use half of them for training our models and half for testing,' but does not explicitly mention a separate validation set or its split percentage/size. |
| Hardware Specification | No | The paper does not provide any specific details about the hardware specifications (e.g., CPU, GPU models, memory, or cloud computing instances) used for running the experiments. |
| Software Dependencies | No | The paper mentions using a pre-trained VGG-16 model and describes its neural network architecture but does not specify any software dependencies (e.g., programming languages, libraries, or frameworks) with version numbers required to replicate the experiments. |
| Experiment Setup | No | The paper describes the overall architecture and training strategy for the TDG4Crowd method, including how different components are trained, but it does not provide specific experimental setup details such as hyperparameter values (e.g., learning rate, batch size, number of epochs) or optimizer settings. |