Long-term memory networks for question answering

Fenglong Ma, Radha Chitta, Saurabh Kataria, Jing Zhou, Palghat Ramesh, Tong Sun, Jing Gao

Research output: Contribution to journalConference article

Abstract

Question answering is an important and difficult task in the natural language process-ing domain, because many basic natural lan-guage processing tasks can be cast into a ques-tion answering task. Several deep neural net-work architectures have been developed re-cently, which employ memory and inference components to memorize and reason over text information, and generate answers to ques-tions. However, a major drawback of many such models is that they are capable of only generating single-word answers. In addition, they require large amount of training data to generate accurate answers. In this paper, we introduce the Long-Term Memory Network (LTMN), which incorporates both an exter-nal memory module and a Long Short-Term Memory (LSTM) module to comprehend the input data and generate multi-word answers. The LTMN model can be trained end-to-end using back-propagation and requires minimal supervision. We test our model on two syn-thetic data sets (based on Facebook's bAbI data set) and the real-world Stanford ques-tion answering data set, and show that it can achieve state-of-the-art performance.

Original languageEnglish (US)
Pages (from-to)7-14
Number of pages8
JournalCEUR Workshop Proceedings
Volume1986
StatePublished - Jan 1 2017
Event2017 IJCAI Workshop on Semantic Machine Learning, SML 2017 - Melbourne, Australia
Duration: Aug 20 2017 → …

Fingerprint

Data storage equipment
Backpropagation
Neural networks
Processing
Long short-term memory

All Science Journal Classification (ASJC) codes

  • Computer Science(all)

Cite this

Ma, F., Chitta, R., Kataria, S., Zhou, J., Ramesh, P., Sun, T., & Gao, J. (2017). Long-term memory networks for question answering. CEUR Workshop Proceedings, 1986, 7-14.
Ma, Fenglong ; Chitta, Radha ; Kataria, Saurabh ; Zhou, Jing ; Ramesh, Palghat ; Sun, Tong ; Gao, Jing. / Long-term memory networks for question answering. In: CEUR Workshop Proceedings. 2017 ; Vol. 1986. pp. 7-14.
@article{47c19e4f0abc44eaae2325b2308b8349,
title = "Long-term memory networks for question answering",
abstract = "Question answering is an important and difficult task in the natural language process-ing domain, because many basic natural lan-guage processing tasks can be cast into a ques-tion answering task. Several deep neural net-work architectures have been developed re-cently, which employ memory and inference components to memorize and reason over text information, and generate answers to ques-tions. However, a major drawback of many such models is that they are capable of only generating single-word answers. In addition, they require large amount of training data to generate accurate answers. In this paper, we introduce the Long-Term Memory Network (LTMN), which incorporates both an exter-nal memory module and a Long Short-Term Memory (LSTM) module to comprehend the input data and generate multi-word answers. The LTMN model can be trained end-to-end using back-propagation and requires minimal supervision. We test our model on two syn-thetic data sets (based on Facebook's bAbI data set) and the real-world Stanford ques-tion answering data set, and show that it can achieve state-of-the-art performance.",
author = "Fenglong Ma and Radha Chitta and Saurabh Kataria and Jing Zhou and Palghat Ramesh and Tong Sun and Jing Gao",
year = "2017",
month = "1",
day = "1",
language = "English (US)",
volume = "1986",
pages = "7--14",
journal = "CEUR Workshop Proceedings",
issn = "1613-0073",
publisher = "CEUR-WS",

}

Ma, F, Chitta, R, Kataria, S, Zhou, J, Ramesh, P, Sun, T & Gao, J 2017, 'Long-term memory networks for question answering', CEUR Workshop Proceedings, vol. 1986, pp. 7-14.

Long-term memory networks for question answering. / Ma, Fenglong; Chitta, Radha; Kataria, Saurabh; Zhou, Jing; Ramesh, Palghat; Sun, Tong; Gao, Jing.

In: CEUR Workshop Proceedings, Vol. 1986, 01.01.2017, p. 7-14.

Research output: Contribution to journalConference article

TY - JOUR

T1 - Long-term memory networks for question answering

AU - Ma, Fenglong

AU - Chitta, Radha

AU - Kataria, Saurabh

AU - Zhou, Jing

AU - Ramesh, Palghat

AU - Sun, Tong

AU - Gao, Jing

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Question answering is an important and difficult task in the natural language process-ing domain, because many basic natural lan-guage processing tasks can be cast into a ques-tion answering task. Several deep neural net-work architectures have been developed re-cently, which employ memory and inference components to memorize and reason over text information, and generate answers to ques-tions. However, a major drawback of many such models is that they are capable of only generating single-word answers. In addition, they require large amount of training data to generate accurate answers. In this paper, we introduce the Long-Term Memory Network (LTMN), which incorporates both an exter-nal memory module and a Long Short-Term Memory (LSTM) module to comprehend the input data and generate multi-word answers. The LTMN model can be trained end-to-end using back-propagation and requires minimal supervision. We test our model on two syn-thetic data sets (based on Facebook's bAbI data set) and the real-world Stanford ques-tion answering data set, and show that it can achieve state-of-the-art performance.

AB - Question answering is an important and difficult task in the natural language process-ing domain, because many basic natural lan-guage processing tasks can be cast into a ques-tion answering task. Several deep neural net-work architectures have been developed re-cently, which employ memory and inference components to memorize and reason over text information, and generate answers to ques-tions. However, a major drawback of many such models is that they are capable of only generating single-word answers. In addition, they require large amount of training data to generate accurate answers. In this paper, we introduce the Long-Term Memory Network (LTMN), which incorporates both an exter-nal memory module and a Long Short-Term Memory (LSTM) module to comprehend the input data and generate multi-word answers. The LTMN model can be trained end-to-end using back-propagation and requires minimal supervision. We test our model on two syn-thetic data sets (based on Facebook's bAbI data set) and the real-world Stanford ques-tion answering data set, and show that it can achieve state-of-the-art performance.

UR - http://www.scopus.com/inward/record.url?scp=85034946837&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85034946837&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85034946837

VL - 1986

SP - 7

EP - 14

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

SN - 1613-0073

ER -

Ma F, Chitta R, Kataria S, Zhou J, Ramesh P, Sun T et al. Long-term memory networks for question answering. CEUR Workshop Proceedings. 2017 Jan 1;1986:7-14.