TY - GEN
T1 - Automated pyramid summarization evaluation
AU - Gao, Yanjun
AU - Sun, Chen
AU - Passonneau, Rebecca J.
N1 - Funding Information:
This work was supported in part by a Fellowship from Teaching and Learning with Technology, Penn State University, and by NSF award IIS-1847842. We thank two Penn State undergraduate research assistants for their contributions to the code base: Andrew Warner, and Purushartha Singh. Brent Hoffert, who recently graduated from Penn State, developed the wrapper that simplifies the use of PyrEval. Several additional Penn State undergrads helped correct the TAC 10 pyramids: Brent Hoffert, Alex Driban, Sahil Mishra, Xuannan Su, and Kun Wang. Finally, we thank the reviewers for their helpful suggestions.
Publisher Copyright:
© 2019 Association for Computational Linguistics.
PY - 2019
Y1 - 2019
N2 - Pyramid evaluation was developed to assess the content of paragraph length summaries of source texts. A pyramid lists the distinct units of content found in several reference summaries, weights content units by how many reference summaries they occur in, and produces three scores based on the weighted content of new summaries. We present an automated method that is more efficient, more transparent, and more complete than previous automated pyramid methods. It is tested on a new dataset of student summaries, and historical NIST data from extractive summarizers.
AB - Pyramid evaluation was developed to assess the content of paragraph length summaries of source texts. A pyramid lists the distinct units of content found in several reference summaries, weights content units by how many reference summaries they occur in, and produces three scores based on the weighted content of new summaries. We present an automated method that is more efficient, more transparent, and more complete than previous automated pyramid methods. It is tested on a new dataset of student summaries, and historical NIST data from extractive summarizers.
UR - http://www.scopus.com/inward/record.url?scp=85084339079&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85084339079&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85084339079
T3 - CoNLL 2019 - 23rd Conference on Computational Natural Language Learning, Proceedings of the Conference
SP - 404
EP - 418
BT - CoNLL 2019 - 23rd Conference on Computational Natural Language Learning, Proceedings of the Conference
PB - Association for Computational Linguistics
T2 - 23rd Conference on Computational Natural Language Learning, CoNLL 2019
Y2 - 3 November 2019 through 4 November 2019
ER -