MuVAN: A Multi-view Attention Network for Multivariate Temporal Data

Ye Yuan, Guangxu Xun, Fenglong Ma, Yaqing Wang, Nan Du, Kebin Jia, Lu Su, Aidong Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

Recent advances in attention networks have gained enormous interest in time series data mining. Various attention mechanisms are proposed to soft-select relevant timestamps from temporal data by assigning learnable attention scores. However, many real-world tasks involve complex multivariate time series that continuously measure target from multiple views. Different views may provide information of different levels of quality varied over time, and thus should be assigned with different attention scores as well. Unfortunately, the existing attention-based architectures cannot be directly used to jointly learn the attention scores in both time and view domains, due to the data structure complexity. Towards this end, we propose a novel multi-view attention network, namely MuVAN, to learn fine-grained attentional representations from multivariate temporal data. MuVAN is a unified deep learning model that can jointly calculate the two-dimensional attention scores to estimate the quality of information contributed by each view within different timestamps. By constructing a hybrid focus procedure, we are able to bring more diversity to attention, in order to fully utilize the multi-view information. To evaluate the performance of our model, we carry out experiments on three real-world benchmark datasets. Experimental results show that the proposed MuVAN model outperforms the state-of-the-art deep representation approaches in different real-world tasks. Analytical results through a case study demonstrate that MuVAN can discover discriminative and meaningful attention scores across views over time, which improves the feature representation of multivariate temporal data.

Original languageEnglish (US)
Title of host publication2018 IEEE International Conference on Data Mining, ICDM 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages717-726
Number of pages10
ISBN (Electronic)9781538691588
DOIs
StatePublished - Dec 27 2018
Event18th IEEE International Conference on Data Mining, ICDM 2018 - Singapore, Singapore
Duration: Nov 17 2018Nov 20 2018

Publication series

NameProceedings - IEEE International Conference on Data Mining, ICDM
Volume2018-November
ISSN (Print)1550-4786

Conference

Conference18th IEEE International Conference on Data Mining, ICDM 2018
CountrySingapore
CitySingapore
Period11/17/1811/20/18

Fingerprint

Time series
Data mining
Data structures
Experiments
Deep learning

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Cite this

Yuan, Y., Xun, G., Ma, F., Wang, Y., Du, N., Jia, K., ... Zhang, A. (2018). MuVAN: A Multi-view Attention Network for Multivariate Temporal Data. In 2018 IEEE International Conference on Data Mining, ICDM 2018 (pp. 717-726). [8594896] (Proceedings - IEEE International Conference on Data Mining, ICDM; Vol. 2018-November). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDM.2018.00087
Yuan, Ye ; Xun, Guangxu ; Ma, Fenglong ; Wang, Yaqing ; Du, Nan ; Jia, Kebin ; Su, Lu ; Zhang, Aidong. / MuVAN : A Multi-view Attention Network for Multivariate Temporal Data. 2018 IEEE International Conference on Data Mining, ICDM 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 717-726 (Proceedings - IEEE International Conference on Data Mining, ICDM).
@inproceedings{c9abf8bee45c472dbbd3c6ec06f17aae,
title = "MuVAN: A Multi-view Attention Network for Multivariate Temporal Data",
abstract = "Recent advances in attention networks have gained enormous interest in time series data mining. Various attention mechanisms are proposed to soft-select relevant timestamps from temporal data by assigning learnable attention scores. However, many real-world tasks involve complex multivariate time series that continuously measure target from multiple views. Different views may provide information of different levels of quality varied over time, and thus should be assigned with different attention scores as well. Unfortunately, the existing attention-based architectures cannot be directly used to jointly learn the attention scores in both time and view domains, due to the data structure complexity. Towards this end, we propose a novel multi-view attention network, namely MuVAN, to learn fine-grained attentional representations from multivariate temporal data. MuVAN is a unified deep learning model that can jointly calculate the two-dimensional attention scores to estimate the quality of information contributed by each view within different timestamps. By constructing a hybrid focus procedure, we are able to bring more diversity to attention, in order to fully utilize the multi-view information. To evaluate the performance of our model, we carry out experiments on three real-world benchmark datasets. Experimental results show that the proposed MuVAN model outperforms the state-of-the-art deep representation approaches in different real-world tasks. Analytical results through a case study demonstrate that MuVAN can discover discriminative and meaningful attention scores across views over time, which improves the feature representation of multivariate temporal data.",
author = "Ye Yuan and Guangxu Xun and Fenglong Ma and Yaqing Wang and Nan Du and Kebin Jia and Lu Su and Aidong Zhang",
year = "2018",
month = "12",
day = "27",
doi = "10.1109/ICDM.2018.00087",
language = "English (US)",
series = "Proceedings - IEEE International Conference on Data Mining, ICDM",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "717--726",
booktitle = "2018 IEEE International Conference on Data Mining, ICDM 2018",
address = "United States",

}

Yuan, Y, Xun, G, Ma, F, Wang, Y, Du, N, Jia, K, Su, L & Zhang, A 2018, MuVAN: A Multi-view Attention Network for Multivariate Temporal Data. in 2018 IEEE International Conference on Data Mining, ICDM 2018., 8594896, Proceedings - IEEE International Conference on Data Mining, ICDM, vol. 2018-November, Institute of Electrical and Electronics Engineers Inc., pp. 717-726, 18th IEEE International Conference on Data Mining, ICDM 2018, Singapore, Singapore, 11/17/18. https://doi.org/10.1109/ICDM.2018.00087

MuVAN : A Multi-view Attention Network for Multivariate Temporal Data. / Yuan, Ye; Xun, Guangxu; Ma, Fenglong; Wang, Yaqing; Du, Nan; Jia, Kebin; Su, Lu; Zhang, Aidong.

2018 IEEE International Conference on Data Mining, ICDM 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 717-726 8594896 (Proceedings - IEEE International Conference on Data Mining, ICDM; Vol. 2018-November).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - MuVAN

T2 - A Multi-view Attention Network for Multivariate Temporal Data

AU - Yuan, Ye

AU - Xun, Guangxu

AU - Ma, Fenglong

AU - Wang, Yaqing

AU - Du, Nan

AU - Jia, Kebin

AU - Su, Lu

AU - Zhang, Aidong

PY - 2018/12/27

Y1 - 2018/12/27

N2 - Recent advances in attention networks have gained enormous interest in time series data mining. Various attention mechanisms are proposed to soft-select relevant timestamps from temporal data by assigning learnable attention scores. However, many real-world tasks involve complex multivariate time series that continuously measure target from multiple views. Different views may provide information of different levels of quality varied over time, and thus should be assigned with different attention scores as well. Unfortunately, the existing attention-based architectures cannot be directly used to jointly learn the attention scores in both time and view domains, due to the data structure complexity. Towards this end, we propose a novel multi-view attention network, namely MuVAN, to learn fine-grained attentional representations from multivariate temporal data. MuVAN is a unified deep learning model that can jointly calculate the two-dimensional attention scores to estimate the quality of information contributed by each view within different timestamps. By constructing a hybrid focus procedure, we are able to bring more diversity to attention, in order to fully utilize the multi-view information. To evaluate the performance of our model, we carry out experiments on three real-world benchmark datasets. Experimental results show that the proposed MuVAN model outperforms the state-of-the-art deep representation approaches in different real-world tasks. Analytical results through a case study demonstrate that MuVAN can discover discriminative and meaningful attention scores across views over time, which improves the feature representation of multivariate temporal data.

AB - Recent advances in attention networks have gained enormous interest in time series data mining. Various attention mechanisms are proposed to soft-select relevant timestamps from temporal data by assigning learnable attention scores. However, many real-world tasks involve complex multivariate time series that continuously measure target from multiple views. Different views may provide information of different levels of quality varied over time, and thus should be assigned with different attention scores as well. Unfortunately, the existing attention-based architectures cannot be directly used to jointly learn the attention scores in both time and view domains, due to the data structure complexity. Towards this end, we propose a novel multi-view attention network, namely MuVAN, to learn fine-grained attentional representations from multivariate temporal data. MuVAN is a unified deep learning model that can jointly calculate the two-dimensional attention scores to estimate the quality of information contributed by each view within different timestamps. By constructing a hybrid focus procedure, we are able to bring more diversity to attention, in order to fully utilize the multi-view information. To evaluate the performance of our model, we carry out experiments on three real-world benchmark datasets. Experimental results show that the proposed MuVAN model outperforms the state-of-the-art deep representation approaches in different real-world tasks. Analytical results through a case study demonstrate that MuVAN can discover discriminative and meaningful attention scores across views over time, which improves the feature representation of multivariate temporal data.

UR - http://www.scopus.com/inward/record.url?scp=85061366757&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85061366757&partnerID=8YFLogxK

U2 - 10.1109/ICDM.2018.00087

DO - 10.1109/ICDM.2018.00087

M3 - Conference contribution

AN - SCOPUS:85061366757

T3 - Proceedings - IEEE International Conference on Data Mining, ICDM

SP - 717

EP - 726

BT - 2018 IEEE International Conference on Data Mining, ICDM 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Yuan Y, Xun G, Ma F, Wang Y, Du N, Jia K et al. MuVAN: A Multi-view Attention Network for Multivariate Temporal Data. In 2018 IEEE International Conference on Data Mining, ICDM 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 717-726. 8594896. (Proceedings - IEEE International Conference on Data Mining, ICDM). https://doi.org/10.1109/ICDM.2018.00087