Introducing A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem for after-work quants

Takeaway. Don’t predict next prices then hand-code a book. This framework is model-free RL whose action is the portfolio weight vector. EIIE (ensemble of identical independent evaluators) shares one small net across coins; PVM stores last weights so transaction cost enters the decision; OSBL is how they keep training on a live stream.

A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem (Zhengyao Jiang, Dixing Xu, Jinjun Liang, 2017; arXiv:1706.10059). Poloniex cryptocurrencies, 30-minute bars, 0.25% commission, CNN / basic RNN / LSTM heads.

Fit for learners: EIIE crypto portfolio RL with Portfolio-Vector Memory — weight-sharing per-asset nets, explicit log-return reward with costs; many FinRL-style projects still echo it.

This Hub note is a plain-language intro. Not investment advice. Crypto back-tests in 2016–2017 are not a 2026 live edge.

The problem

Supervised price forecasts are not trades; converting them needs extra logic and often ignores fees. Discrete buy/sell RL on one asset does not scale to a continuous simplex of many weights. Actor-critic with two nets can be unstable. They want one explicit wealth objective.

The idea / core intuition

• State: a price tensor (high/low/close, ~50 periods, m coins) normalised by the latest close, plus last weights.
• Action: softmax weights (cash = Bitcoin).
• Reward: average log return after a transaction remainder μ_t (fees). Deterministic policy gradient on that explicit R.
• EIIE: each asset is a separate row/stream with shared CNN or RNN weights until softmax — swap the universe without retraining identities.
• PVM: a memory stack of weight vectors so minibatches stay parallel and the net can penalise churn.
• OSBL: after each period, sample recent batches with geometric recency bias.

Three Poloniex back-tests: EIIE variants took the top fAPV / Sharpe slots vs classic online portfolio rules (OLMAR, PAMR, UCRP, …) in that paper’s sample. Assumptions: zero slippage, zero market impact — they say so.

Why after-work Python learners care

This is the ancestor many FinRL / crypto-RL notebooks still rhyme with: weights as actions, fees in the reward, parameter sharing across names. Read the hypotheses before cloning a GitHub and depositing exchange money.

Tiny evening exercise: two assets, weights (0.5, 0.5), price relatives (1.01, 0.99); new wealth factor is the dot product. Then shrink it by 0.9975 as a toy fee. Learning only.

One-evening reading checklist

1. Why the action is w_t, not a predicted price.
2. EIIE vs one big net that memorizes coin identities.
3. How PVM injects last weights without BPTT through history.
4. Fee μ_t, 0.25% Poloniex, and the two back-test hypotheses.

Download PDF

Download PDF

Landing: arXiv abs

Important disclaimer

Not investment advice. This Hub note is educational only. High reported crypto multiples in a 2016–2017 paper are not a recommendation to trade cryptocurrencies or to expect those returns.

What this is not

• Not CEF / subsidy framing
• Not Quant or Macro course CTA
• Not a crypto-fund or bot pitch

Next step

Single course CTA: SoloQuant Academy’s Introduction to Python Programming—dot products and softmax are the portfolio vector; then the PDF.