Distractor generation with generative adversarial nets for automatically creating fill-in-the-blank questions

Chen Liang, Xiao Yang, Drew Wham, Bart Pursel, Rebecca Jane Passonneau, C. Lee Giles

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Distractor generation is a crucial step for fill-in-the-blank question generation. We propose a generative model learned from training generative adversarial nets (GANs) to create useful distractors. Our method utilizes only context information and does not use the correct answer, which is completely different from previous Ontologybased or similarity-based approaches. Trained on the Wikipedia corpus, the proposed model is able to predict Wiki entities as distractors. Our method is evaluated on two biology question datasets collected from Wikipedia and actual college-level exams. Experimental results show that our context-based method achieves comparable performance to a frequently used word2vec-based method for the Wiki dataset. In addition, we propose a second-stage learner to combine the strengths of the two methods, which further improves the performance on both datasets, with 51.7% and 48.4% of generated distractors being acceptable.

Original languageEnglish (US)
Title of host publicationProceedings of the Knowledge Capture Conference, K-CAP 2017
PublisherAssociation for Computing Machinery, Inc
ISBN (Electronic)9781450355537
DOIs
StatePublished - Dec 4 2017
Event9th International Conference on Knowledge Capture, K-CAP 2017 - Austin, United States
Duration: Dec 4 2017Dec 6 2017

Publication series

NameProceedings of the Knowledge Capture Conference, K-CAP 2017

Other

Other9th International Conference on Knowledge Capture, K-CAP 2017
CountryUnited States
CityAustin
Period12/4/1712/6/17

All Science Journal Classification (ASJC) codes

  • Computational Theory and Mathematics
  • Software
  • Computer Science Applications
  • Information Systems

Cite this

Liang, C., Yang, X., Wham, D., Pursel, B., Passonneau, R. J., & Giles, C. L. (2017). Distractor generation with generative adversarial nets for automatically creating fill-in-the-blank questions. In Proceedings of the Knowledge Capture Conference, K-CAP 2017 [33] (Proceedings of the Knowledge Capture Conference, K-CAP 2017). Association for Computing Machinery, Inc. https://doi.org/10.1145/3148011.3154463
Liang, Chen ; Yang, Xiao ; Wham, Drew ; Pursel, Bart ; Passonneau, Rebecca Jane ; Giles, C. Lee. / Distractor generation with generative adversarial nets for automatically creating fill-in-the-blank questions. Proceedings of the Knowledge Capture Conference, K-CAP 2017. Association for Computing Machinery, Inc, 2017. (Proceedings of the Knowledge Capture Conference, K-CAP 2017).
@inproceedings{bca0a1d1645041acac9c6fdf7657cd89,
title = "Distractor generation with generative adversarial nets for automatically creating fill-in-the-blank questions",
abstract = "Distractor generation is a crucial step for fill-in-the-blank question generation. We propose a generative model learned from training generative adversarial nets (GANs) to create useful distractors. Our method utilizes only context information and does not use the correct answer, which is completely different from previous Ontologybased or similarity-based approaches. Trained on the Wikipedia corpus, the proposed model is able to predict Wiki entities as distractors. Our method is evaluated on two biology question datasets collected from Wikipedia and actual college-level exams. Experimental results show that our context-based method achieves comparable performance to a frequently used word2vec-based method for the Wiki dataset. In addition, we propose a second-stage learner to combine the strengths of the two methods, which further improves the performance on both datasets, with 51.7{\%} and 48.4{\%} of generated distractors being acceptable.",
author = "Chen Liang and Xiao Yang and Drew Wham and Bart Pursel and Passonneau, {Rebecca Jane} and Giles, {C. Lee}",
year = "2017",
month = "12",
day = "4",
doi = "10.1145/3148011.3154463",
language = "English (US)",
series = "Proceedings of the Knowledge Capture Conference, K-CAP 2017",
publisher = "Association for Computing Machinery, Inc",
booktitle = "Proceedings of the Knowledge Capture Conference, K-CAP 2017",

}

Liang, C, Yang, X, Wham, D, Pursel, B, Passonneau, RJ & Giles, CL 2017, Distractor generation with generative adversarial nets for automatically creating fill-in-the-blank questions. in Proceedings of the Knowledge Capture Conference, K-CAP 2017., 33, Proceedings of the Knowledge Capture Conference, K-CAP 2017, Association for Computing Machinery, Inc, 9th International Conference on Knowledge Capture, K-CAP 2017, Austin, United States, 12/4/17. https://doi.org/10.1145/3148011.3154463

Distractor generation with generative adversarial nets for automatically creating fill-in-the-blank questions. / Liang, Chen; Yang, Xiao; Wham, Drew; Pursel, Bart; Passonneau, Rebecca Jane; Giles, C. Lee.

Proceedings of the Knowledge Capture Conference, K-CAP 2017. Association for Computing Machinery, Inc, 2017. 33 (Proceedings of the Knowledge Capture Conference, K-CAP 2017).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Distractor generation with generative adversarial nets for automatically creating fill-in-the-blank questions

AU - Liang, Chen

AU - Yang, Xiao

AU - Wham, Drew

AU - Pursel, Bart

AU - Passonneau, Rebecca Jane

AU - Giles, C. Lee

PY - 2017/12/4

Y1 - 2017/12/4

N2 - Distractor generation is a crucial step for fill-in-the-blank question generation. We propose a generative model learned from training generative adversarial nets (GANs) to create useful distractors. Our method utilizes only context information and does not use the correct answer, which is completely different from previous Ontologybased or similarity-based approaches. Trained on the Wikipedia corpus, the proposed model is able to predict Wiki entities as distractors. Our method is evaluated on two biology question datasets collected from Wikipedia and actual college-level exams. Experimental results show that our context-based method achieves comparable performance to a frequently used word2vec-based method for the Wiki dataset. In addition, we propose a second-stage learner to combine the strengths of the two methods, which further improves the performance on both datasets, with 51.7% and 48.4% of generated distractors being acceptable.

AB - Distractor generation is a crucial step for fill-in-the-blank question generation. We propose a generative model learned from training generative adversarial nets (GANs) to create useful distractors. Our method utilizes only context information and does not use the correct answer, which is completely different from previous Ontologybased or similarity-based approaches. Trained on the Wikipedia corpus, the proposed model is able to predict Wiki entities as distractors. Our method is evaluated on two biology question datasets collected from Wikipedia and actual college-level exams. Experimental results show that our context-based method achieves comparable performance to a frequently used word2vec-based method for the Wiki dataset. In addition, we propose a second-stage learner to combine the strengths of the two methods, which further improves the performance on both datasets, with 51.7% and 48.4% of generated distractors being acceptable.

UR - http://www.scopus.com/inward/record.url?scp=85040581664&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85040581664&partnerID=8YFLogxK

U2 - 10.1145/3148011.3154463

DO - 10.1145/3148011.3154463

M3 - Conference contribution

T3 - Proceedings of the Knowledge Capture Conference, K-CAP 2017

BT - Proceedings of the Knowledge Capture Conference, K-CAP 2017

PB - Association for Computing Machinery, Inc

ER -

Liang C, Yang X, Wham D, Pursel B, Passonneau RJ, Giles CL. Distractor generation with generative adversarial nets for automatically creating fill-in-the-blank questions. In Proceedings of the Knowledge Capture Conference, K-CAP 2017. Association for Computing Machinery, Inc. 2017. 33. (Proceedings of the Knowledge Capture Conference, K-CAP 2017). https://doi.org/10.1145/3148011.3154463