Combining static and time series data using attention mechanisms for forecasting and classification.

Abstract

Utilizing both static and time series data can enhance the performance of machine learning models. However, existing methods of concatenating data lead to high dimensionality and learning of noise. In this thesis, we investigate the addition of an attention mechanism to learn the correlation between static and time series data to mitigate this problem and increase model accuracy. In a multiple case study consisting of 2 real-life medical cases, 1 public dataset, and 1 synthetic dataset, we find similar or better results when using an architecture with an attention mechanism, compared to similar hybrid or meta architectures without it, particularly in sequence forecasting tasks. Additionally, we inspect whether the attention weights align with key events and can reveal structural dependencies within the data. While the attention weights reflected the structural dependencies in our synthetic Fibonacci sequence forecasting experiment, they did not align with key events in our real-life cystic fibrosis improvement classification experiment. We conclude that adding an attention mechanism can improve or maintain performance in forecasting problems. We provide suggestions for additional research into evaluating the explainability of attention mechanisms.