Introducing Financial Time Series Forecasting using CNN and Transformer for after-work quants

Takeaway. CNN sees local wiggles; Transformer sees long context. CTTS turns a price window into CNN tokens, adds positional embeddings, encodes with a small Transformer, then classifies the next tick’s sign: up / down / flat.

Financial Time Series Forecasting using CNN and Transformer (Zhen Zeng, Rachneet Kaur, Suchetha Siddagangappa, Saba Rahimi, Tucker Balch, Manuela Veloso, 2023; arXiv:2304.04912). J.P. Morgan AI Research.

Fit for learners: 1-D CNN tokens plus a short Transformer for next-minute up/down/flat on S&P 500 names — a compact classification recipe before you reach for DeepAR.

This Hub note is a plain-language intro. Not investment advice.

The problem

Intraday prices mix short microstructure noise with slower drift. Pure CNN receptive fields stay local. Pure LSTM/Transformer on raw ticks is heavy and often no better than a coin on three-way sign. They want a small hybrid and an honest classification metric, not MSE on prices.

The idea / core intuition

• Per series: min-max to ([0,1]). Target = sign of change at the next step (three-way; also a two-way merge of up+flat vs down).
• 1-D CNN: kernel 16, stride 8 → each token is a local patch (ViT-style, 1-D).
• Transformer: depth 4, 4 heads, dim 128, dropout 0.3. MLP + softmax. Cross-entropy, AdamW, batch 64, ≤100 epochs. PyTorch 1.0, 8× T4.
• Data: Bloomberg 1-minute prices, S&P 500 names, 2019. Seven windows per name per week. Train/val: weeks 1–39 (80/20; ~507k / 117k). Test: weeks 40–52 (~209k). Input 80 minutes → label at minute 81.
• Test mix: up 37.1% / down 36.5% / flat 26.4%. Naive majority = 37.1% three-class.
• Table 1 (last quarter 2019): CTTS 56.7% two-class / 44.1% three-class vs EMA 53.2/39.5, ARIMA 50.9/37.5, DeepAR 51.1/37.4. After keeping the top 25% confidence: CTTS 66.8% / 55.2% (largest lift — probabilities are somewhat ranked).

A one-year, one-market academic study. Licensed Bloomberg prints; you cannot replay it from Yahoo. Sign accuracy in the mid-50s is not an edge after costs.

Why after-work Python learners care

You get a recipe you can sketch in PyTorch in an evening: Conv1d → TransformerEncoder → Linear(3). The useful habit is thresholding the softmax, not believing 56.7%.

Tiny evening exercise: dummy array (80,) → Conv1d(1, 16, kernel_size=16, stride=8) in PyTorch (or a NumPy im2col). Print token count. Learning only.

One-evening reading checklist

1. Why sign not next price (scale, non-stationarity).
2. CNN kernel/stride vs PatchTST-style patches.
3. How they turn DeepAR samples into a three-way vote.
4. What thresholded accuracy does not say about PnL.

Download PDF

Download PDF

Landing: arXiv abs

Important disclaimer

Not investment advice. This Hub note is educational only. CTTS numbers on 2019 S&P 500 minutes are not a trading system. The paper itself sketches buy/sell/hold from class + size from probability — that is research discussion, not a recommendation.

What this is not

• Not CEF / subsidy framing
• Not Quant or Macro course CTA
• Not a trading-bot pitch

Next step

Single course CTA: SoloQuant Academy’s Introduction to Python Programming—slicing windows and one-hot labels are the boring skills this architecture sits on.