Learning to Converse with Noisy Data: Generation with Calibration
Authors: Mingyue Shang, Zhenxin Fu, Nanyun Peng, Yansong Feng, Dongyan Zhao, Rui Yan
IJCAI 2018 | Conference PDF | Archive PDF | Plain Text | LLM Run Details
| Reproducibility Variable | Result | LLM Response |
|---|---|---|
| Research Type | Experimental | Experiments show that the calibrated model outperforms baseline methods on both automatic evaluation metrics and human annotations. We conduct experiments using data crawled from a Chinese forum named Douban. |
| Researcher Affiliation | Academia | 1Institute of Computer Science and Technology, Peking University, China 2Information Science Institute, University of Southern California, USA 3Beijing Institute of Big Data Research, China |
| Pseudocode | No | The paper includes figures illustrating the model architecture (Figure 1, Figure 2, Figure 3) but does not provide any explicitly labeled pseudocode or algorithm blocks. |
| Open Source Code | No | The paper states 'Our model is implemented in Pytorch3. http://pytorch.org/', which refers to a third-party library, not the authors' own source code for their methodology. No explicit statement of source code release for their method is found. |
| Open Datasets | No | We conduct our experiments on a large dataset crawled from Douban, which is a Chinese discussion forum. We performed Chinese word segmentation for each query-reply pair. No public access information like a link, DOI, or specific citation for the dataset is provided. |
| Dataset Splits | Yes | There are 1,333,877 pairs in the training set, 10,000 for validation and 1,000 for test. |
| Hardware Specification | No | The paper does not provide specific details on the hardware (e.g., GPU/CPU models, memory) used for running the experiments. |
| Software Dependencies | No | Our model is implemented in Pytorch3. http://pytorch.org/. Although PyTorch is mentioned, a specific version number is not provided, nor are other key software dependencies with their versions. |
| Experiment Setup | Yes | We adopt Adam [Kingma and Ba, 2014] optimizer with initial learning rates as 0.0002 for the calibration network and 0.0001 for the generation network. We employ mini-batch training with batch size 64 for both the calibration and the generation network... For the calibration network, we set the word embedding dimension to 200 and the hidden vector size to 256 for both query encoder and reply encoder. The for calibration network is set to 0.05. For the generation network, the word embedding dimension is 480 and the hidden vector size for both the encoder and the decoder is 512. All the parameters are initialized randomly. We begin to calibrate our generation network from the second training epoch. |