Introducing Large Covariance from Noisy High-Frequency Data for after-work quants

Takeaway. High-frequency panels give lots of ticks, but they are noisy and asynchronous. Dai–Lu–Xiu ask: when you build a large factor covariance, do you know the factors, the loadings, or neither? They compare TSR, CSR, and PCA, clean ticks with pre-averaging + refresh time, then threshold residuals. Empirically, pre-averaging plus TSR or PCA with location thresholding tends to win the allocation horse race.

Knowing Factors or Factor Loadings, or Neither? (Chaoxing Dai, Kun Lu, Dacheng Xiu, Journal of Econometrics, 2019) is the BARRA-flavoured “which information set?” paper.

Fit for learners: TSR vs CSR vs PCA for large HF covariance when ticks are noisy and asynchronous—pre-averaging plus thresholding.

This Hub note is a plain-language intro. Not investment advice.

The problem

Sampling every tick without cleaning microstructure bias wrecks covariance. Sampling only every 15 minutes throws away data and worsens the N vs T curse. Factors may be observed (ETFs), loadings may be observed (characteristics / BARRA-style), or you may only have a return matrix. One recipe does not fit all universes (DJIA 30 vs S&P 500).

The idea / core intuition

• Known factors → TSR (time-series regression for loadings).
• Known loadings → CSR (cross-sectional regression for factors).
• Neither → PCA.
• Microstructure: pre-average in refresh-time sync; compare vs subsampling.
• Residuals: several thresholders (including GICS location).
• ~30 combinations; score on out-of-sample portfolio risk for DJIA, S&P 100, S&P 500.
• Result flavour: pre-averaging + TSR/PCA + location thresholding dominates many subsampled cousins; CSR can be fragile if characteristics are wrong.

Why after-work Python learners care

This is the map of information sets before you copy a “HF covariance” notebook. Pre-averaging is a moving average of prices in a small window—still Python, still arrays.

Tiny evening exercise: two noisy price series on a slightly shifted grid; resample to a common 1-minute index with last observation (pandas.merge_asof mental model); print a cov. Learning only.

One-evening reading checklist

1. TSR vs CSR vs PCA: what do you assume you already know?
2. Why pre-average instead of using the last trade at 1-second frequency?
3. What goes wrong if you subsample away most ticks?
4. Which combo roughly wins their S&P 500 horse race?

Download PDF

Download PDF

Landing: DOI

Important disclaimer

Not investment advice. This Hub note is educational only. Dai–Lu–Xiu is academic econometrics—not a recommendation to trade from pre-averaged factor covariances.

What this is not

• Not CEF / subsidy framing
• Not Quant or Macro course CTA
• Not a BARRA-vendor pitch

Next step

Single course CTA: SoloQuant Academy’s Introduction to Python Programming—aligning two timestamped lists is a nested loop (or a merge) before any factor model.