Introducing FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance for after-work quants

Takeaway. FinRL (NeurIPS 2020 workshop) is a beginner stack: market env, DRL agent, application notebooks. You should leave knowing StockTradingEnv + DRLAgent.get_model("ppo"), not a new theorem.

FinRL: A Deep Reinforcement Learning Library for Automated Stock Trading in Quantitative Finance (Xiao-Yang Liu, Hongyang Yang, Qian Chen, Runjia Zhang, Liuqing Yang, Bowen Xiao, Christina Dan Wang, 2020; arXiv:2011.09607). Demos on NASDAQ-100, DJIA, S&P 500, HSI, SSE 50, CSI 300.

Fit for learners: Hands-on DRL trading library with tutorials — ideal weekend path from pandas bars to PPO/SAC agents.

This Hub note is a plain-language intro. Not investment advice.

The problem

Writing your own replay buffer, fee model, and pyfolio tear-sheet is where weekend projects die. Chess has rules; markets need you to define state, action, reward, and frictions. Debugging that from scratch hides the actual RL.

The idea / core intuition

• Three layers: env (OHLCV + MACD/RSI, cash, shares) → agents (DQN, DDPG, MADDPG, PPO, SAC, A2C, TD3 via Baselines/Stable-Baselines) → apps (single stock, multi-stock, allocation).
• Actions: per name (a \in {-k,\ldots,k}) shares; reward Δvalue, log-return, or Sharpe; user rewards can add cost or risk.
• Frictions: flat or bps per share, optional bid–ask; turbulence index to halt buys in crashes.
• Split: train / validate (tune) / test with a rolling window. Metrics: terminal wealth, annualised return and vol, max DD, Sharpe. Baselines: buy-and-hold, mean/min-variance, momentum, equal weight.
• Paper tables (2019–Sep 2020, includes COVID drawdown): PPO on SPY/QQQ/mega-caps vs S&P; DDPG/TD3 on DJIA names vs min-var and the index. Treat numbers as library demos, not live edge.

A teaching library—the 2021 ICAIF paper adds live APIs and a fuller pipeline.

Why after-work Python learners care

One weekend path: pandas bars → gym env → learn() → a pyfolio chart. That is the point of this paper.

Tiny evening exercise: state = [cash, shares, price]. Action +1 means buy one share. Write the next cash in one line: cash - price (ignore fees). Learning only.

One-evening reading checklist

1. Name the three layers.
2. Action meaning of (k) shares vs weight softmax.
3. Where transaction cost and turbulence enter.
4. Train / validation / test—why a third split.

Download PDF

Download PDF

Landing: arXiv abs

Important disclaimer

Not investment advice. This Hub note is educational only. FinRL 2020 is a research teaching library—not a recommendation to automate stock trading from the demo notebooks.

What this is not

• Not CEF / subsidy framing
• Not Quant or Macro course CTA
• Not a FinRL-as-product pitch

Next step

Single course CTA: SoloQuant Academy’s Introduction to Python Programming—lists for cash, shares, and prices are the state vector before any DRL library.