Introducing Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting for after-work quants

Takeaway. Multi-horizon forecasts are messy because inputs are three kinds: static IDs, past-only observations, and known-future calendar/promos. TFT is a Transformer that names those kinds, gates unused parts with GRNs, and emits quantile paths — not one black-box point.

Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting (Bryan Lim, Sercan O. Arik, Nicolas Loeff, Tomas Pfister, 2019; International Journal of Forecasting 2021; arXiv:1912.09363). Oxford / Google Cloud AI.

Fit for learners: Interpretable multi-horizon TFT with variable selection — great for factor + calendar covariate forecasting in Python.

This Hub note is a plain-language intro. Not investment advice.

The problem

DeepAR-style iterated models assume every covariate is known in the future (you recurse the target only). Direct seq2seq (MQRNN) can dump a path, but static store IDs get concatenated like just another lag. Post-hoc SHAP/LIME ignore time order. Practitioners need (i) mixed input types, (ii) intervals, (iii) which variable / which lag mattered.

The idea / core intuition

• Inputs: static s_i; observed z_{i,t} (unknown ahead); known x_{i,t} (dow, holiday, promo) through t+tau. Direct forecast of horizons 1..tau_max. Quantiles Q = {0.1, 0.5, 0.9}; pinball loss.
• Gated Residual Network (GRN): ELU MLP + GLU. If the dataset is small/noisy, GLU can zero the nonlinear branch and skip.
• Variable selection networks: each input type (static / past / future) gets its own softmax over features, per timestep, conditioned on a static context. Features pass their own GRN then a weighted sum. That is the “which column?” plot.
• Static encoders: four context vectors — variable selection, LSTM cell/hidden init, static enrichment of temporal features. Static is not glued onto every bar.
• Local processing: LSTM encoder-decoder over selected past vs known-future (handles different lengths). Gated skip around it.
• Interpretable multi-head attention: values shared across heads so the averaged attention matrix is readable. Decoder mask keeps causality. Then position-wise GRN + gated skip over the whole Transformer block.
• Linear head per quantile.