Introducing Thousands of Alpha Tests for after-work quants

Takeaway. When you test thousands of alphas, naive p-values lie—you need a multiple-testing toolkit so “significant” does not mean “I tried everything until one worked.”

Thousands of Alpha Tests (Stefano Giglio, Dacheng Xiu, 2021) is the alpha-side companion to factor-zoo scepticism for after-work research notebooks.

Fit for learners: Multiple-testing toolkit for alphas—stops fake discovery in research notebooks.

This Hub note is a plain-language intro. Not investment advice. Read with Taming the Factor Zoo and Harvey–Liu–Zhu.

The problem

Research and blog culture celebrate any alpha with a star. If hundreds or thousands of tests were run (explicitly or implicitly), false discoveries explode. Single-test thresholds are the wrong ruler.

The idea / core intuition

• Formalise inference when many alphas / strategies are tested together.
• Provide procedures that control error rates under large-scale testing (see PDF for the exact toolkit).
• Push researchers toward honest accounting of the search that produced the star result.
• For learners: treat every grid-search backtest as one draw from a huge menu.

Why after-work Python learners care

Python makes it trivial to loop 500 parameter sets before dinner. This paper is the seatbelt: adjust your claims, not only your seaborn theme.

Tiny evening exercise: simulate 200 noise strategies’ mean returns; count how many beat a naive 5% one-sided threshold by luck; write what FDR-style thinking would change—learning only.

One-evening reading checklist

1. Explain multiple testing without jargon overload.
2. Contrast single-test vs large-scale alpha testing.
3. Note one practical takeaway for your next grid search.
4. Pair this paper with one factor-zoo reading in your notes.

Download PDF

Download PDF

Landing / DOI: RFS / Oxford

Important disclaimer

Not investment advice. This Hub note is educational only. Giglio–Xiu is statistical methodology—not a list of alphas to trade.

What this is not

• Not CEF / subsidy framing
• Not Quant or Macro course CTA
• Not a “secret alpha farm” pitch

Next step

Single course CTA: SoloQuant Academy’s Introduction to Python Programming—once loops and lists feel easy, you will understand why thousands of tests are so tempting—and so dangerous.