Introducing Transformers in Time Series: A Survey for after-work quants

Takeaway. This is not a new net. It is a map: how papers changed position encodings, attention cost, and encoder stacks, then where they show up in forecasting / anomaly / classification. Use it before you pick Informer vs Autoformer vs PatchTST.

Transformers in Time Series: A Survey (Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, Liang Sun, 2022; IJCAI 2023; arXiv:2202.07125). Alibaba DAMO / Shanghai Jiao Tong. Living list: qingsongedu/time-series-transformers-review.

Fit for learners: Taxonomy of TS transformers — cheat-sheet before picking Informer/Autoformer/PatchTST for markets.

This Hub note is a plain-language intro. Not investment advice.

The problem

NLP/CV Transformer surveys exist. Time series still needs seasonality, irregular events, and (O(N^2)) attention on long (H). Nobody had put LogTrans, Informer, Autoformer, FEDformer, Pyraformer, TFT, anomaly nets, and GTN on one taxonomy.

The idea / core intuition

• Position: vanilla sinusoid; learned embeddings; timestamp embeddings (hour/week/holiday — Informer / Autoformer / FEDformer).
• Attention cost (training): vanilla (O(N^2)); LogTrans / Informer / Autoformer (O(N\log N)); Pyraformer / FEDformer / Quatformer ~(O(N)).
• Architecture: pooling between blocks (Informer); pyramidal intra/inter-scale (Pyraformer); seasonal-trend split (Autoformer); frequency attention (FEDformer); patches / channel-independence (later PatchTST).
• Forecasting: module variants (sparse/low-rank, SSM hybrids, PatchTST, Crossformer) vs architecture variants (Triformer, Scaleformer). Also traffic GNNs and Hawkes-style event Transformers.
• Anomaly: TranAD (adversarial), AnomalyTrans (association discrepancy), GTA (graph). Classification: GTN two-tower, TARNet, speech-pretrained transfer.
• Their ETTm2 probes: longer look-back often hurts (models do not actually use 1440 lags); 3–6 layers beat 24–48; a moving-average seasonal-trend split can cut MSE 50–80% even on vanilla attention.
• They flag DLinear’s “do we need Transformers?” jab — and note later PatchTST numbers. Do not treat that debate as settled by one table.

Read Hub notes 85–98 against this cheat-sheet, then 100 (iTransformer) as the invert-the-axes sequel.

Why after-work Python learners care

One evening of taxonomy beats cloning five GitHubs in the wrong order. Timestamp embedding + seasonal-trend split are the two cheapest upgrades before a custom attention.

Tiny evening exercise: write a dict mapping informer / autoformer / fedformer / pyraformer to their complexity class. Match each Hub slug you have already read. Learning only.

One-evening reading checklist

1. Module vs architecture branch in Figure 1.
2. Table 1 complexity — what you pay at (H=720).
3. Application trio: forecast / anomaly / classify.
4. Decomposition ablation: why STL-like splits move the needle.

Download PDF

Download PDF

Landing: arXiv abs

Important disclaimer

Not investment advice. This Hub note is educational only. Survey tables are not a ranking of trading systems.

What this is not

• Not CEF / subsidy framing
• Not Quant or Macro course CTA
• Not a trading-bot pitch

Next step

Single course CTA: SoloQuant Academy’s Introduction to Python Programming—dicts, nested lists, and reading a complexity table are how you navigate this survey after work.