Notice: The reproducibility variables underlying each score are classified using an automated LLM-based pipeline, validated against a manually labeled dataset. LLM-based classification introduces uncertainty and potential bias; scores should be interpreted as estimates. Full accuracy metrics and methodology are described in [1].
LocalGAN: Modeling Local Distributions for Adversarial Response Generation
Authors: Baoxun Wang, Zhen Xu, Huan Zhang, Kexin Qiu, Deyuan Zhang, Chengjie Sun
JMLR 2021 | Venue PDF | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Our experimental results demonstrate that the reasonable local distribution modeling of the query-response corpus is of great importance to adversarial NRG, and our proposed Local GAN is promising for improving both the training stability and the quality of generated results. This section gives the experimental results of our proposed Local GAN, which are analyzed and compared to those of the baseline models on the widely applied metrics. |
| Researcher Affiliation | Collaboration | Baoxun Wang EMAIL Zhen Xu EMAIL Platform & Content Group, Tencent Technology Co., Ltd No.8 Dongbei wang west Road, Beijing, China 100080; Huan Zhang EMAIL Peking University No.5 Yiheyuan Road, Beijing, China 100871 |
| Pseudocode | Yes | Algorithm 1: The Training of Local GAN |
| Open Source Code | Yes | The code of our proposed model Local GAN can be found in https://github.com/Kramgasse49/local_ gan_4generation |
| Open Datasets | Yes | Both the English and the Chinese datasets used in our experiments are uploaded to https://www. dropbox.com/sh/k8i079gd2111lsb/AACLLtl NAzile543Da8Qs9t Fa?dl=0. |
| Dataset Splits | Yes | We sample 100,000, and 2,000 unique query-response pairs as validation and testing dataset respectively from both of the corpora |
| Hardware Specification | Yes | The experiments are conducted on the Tesla K80 GPU. |
| Software Dependencies | No | The text mentions using Glove vectors (Pennington et al., 2014) and Weibo vectors (Li et al., 2018) for embedding initialization, and refers to 'standard-scaler' from scikit-learn. However, specific version numbers for these or other key software components (e.g., Python, TensorFlow/PyTorch) are not provided. |
| Experiment Setup | Yes | The vocabulary size of both datasets is 40,000. The embedding layer of Open Subtitles and Sina Weibo is initialized using 200-dimensional Glove vectors (Pennington et al., 2014) and 300-dimensional Weibo vectors (Li et al., 2018) respectively. All the models are first pre-trained by MLE, and then the models including Adver-REGS, GAN-AEL, AIM, DAIM, Big GAN, Local GAN-SE and Local GAN are trained with adversarial learning. The discriminator of Adver-REGS and GAN-AEL are based on CNN following (Yu et al., 2017; Xu et al., 2017), in which the filter sizes are set to (1,2,3,4) and the filter number is 128, while that of Local GAN adopts DBM with (2 embedding size, 128, 128) to represent the semantic of queries and responses. The hidden size of the generator is set to 256 and 512 in GAN-based models and Seq2Seq respectively. |