Introducing A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem for after-work quants

Takeaway. Skip predict-then-trade. This framework outputs portfolio weights (a softmax over assets) from price tensors, remembers last weights in a Portfolio-Vector Memory so transaction costs enter the gradient, and maximises average log return. Three bodies share the same head: CNN, vanilla RNN, LSTM—EIIE.

A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (Zhengyao Jiang, Dixing Xu, Jinjun Liang, 2017; arXiv:1706.10059). Poloniex crypto, 12-asset books (BTC cash + 11 coins), 30-minute bars, 0.25% commission.

Fit for learners: EIIE + Portfolio-Vector Memory + log-return reward: continuous weights, transaction-cost awareness, crypto 30-minute backtests many FinRL projects still echo.

This Hub note is a plain-language intro. Not investment advice. Crypto back-tests are not a live edge.

The problem

Supervised price forecasts are not actions; converting them needs extra rules (and costs). Discrete buy/sell RL does not scale to continuous multi-asset weights. Actor-critic with two nets is unstable. Classic online-portfolio rules (UP, EG, OLMAR, …) struggle at high turnover once fees are 25 bps per side.

The idea / core intuition

• EIIE: identical networks per asset (shared weights, isolated until softmax)—scalable in m, no “this coin was junk in 2014” identity leak.
• Input: 3 × m × n tensor of high/low/close normalised by latest close (n=50 periods).
• PVM: store w_{t-1}, feed it before the voting layer so turnover is penalised via μ_t (transaction remainder).
• Reward is explicit mean log(μ y · w)—full exploitation, deterministic policy gradient, online stochastic batches.
• Back-tests vs UCRP/UBAH/OLMAR/PAMR/… : EIIE CNN/RNN/LSTM take the top wealth and Sharpe slots in three non-overlapping windows (paper reports large multiples; treat as historical paper trading under zero-slippage, zero-impact hypotheses).

A research architecture—not a crypto product.

Why after-work Python learners care

You meet weights that sum to 1 (softmax), a cost that depends on Δw, and why 24/7 crypto was a convenient sandbox. FinRL-style repos still copy EIIE + PVM.

Tiny evening exercise: w = [0.5, 0.5]; after returns [1.01, 0.99] the drifted weights are w*y / (w·y)—compute by hand. Learning only.

One-evening reading checklist

1. Why the action is w_t, not a price forecast.
2. What EIIE shares vs an integrated CNN that knows asset IDs.
3. How PVM puts fees into the gradient without BPTT through time for w.
4. Name the two back-test hypotheses (no slippage, no impact).

Download PDF

Download PDF

Landing: arXiv abs

Important disclaimer

Not investment advice. This Hub note is educational only. Jiang–Xu–Liang is academic RL on historical crypto prints—not a recommendation to trade cryptocurrencies or copy the back-test wealth multiples.

What this is not

• Not CEF / subsidy framing
• Not Quant or Macro course CTA
• Not a crypto or fund pitch

Next step

Single course CTA: SoloQuant Academy’s Introduction to Python Programming—dot products and lists that sum to one are the linear algebra you need before softmax.