Introducing Knowing Factors or Factor Loadings, or Neither? for after-work quants

Takeaway. Large covariance ≈ low-rank factors + sparse residual. On tick data you also have microstructure noise and asynchronous trades. Dai–Lu–Xiu pre-average on refresh time, then estimate the factor piece three ways: TSR if factors are known, CSR (BARRA-style) if loadings are known, PCA if neither—and race them on DJIA / S&P 100 / S&P 500 allocation.

Knowing factors or factor loadings, or neither? Evaluating estimators of large covariance matrices with noisy and asynchronous data (Chaoxing Dai, Kun Lu, Dacheng Xiu, 2019, Journal of Econometrics).

Fit for learners: TSR vs CSR vs PCA on noisy, asynchronous high-frequency data: which large-cov estimator when you know factors, loadings, or neither.

This Hub note is a plain-language intro. Not investment advice.

The problem

You cannot dump 1-second prices into np.cov: noise biases realised cov, and names trade at different times (Epps-style vanishing correlation). Subsampling throws away data and still leaves N large. Knowing the market ETF is not the same as knowing industry dummy loadings, and neither is the same as latent PCA factors.

The idea / core intuition

• Pre-averaging + refresh time: tame noise and sync clocks before the factor step.
• Known factors (market, FF, sector ETFs) → time-series regression for loadings (TSR).
• Known loadings (characteristics / industry exposures) → cross-sectional regression for factors (CSR)—the BARRA instinct.
• Neither → PCA on the cleaned high-frequency panel.
• Residual: threshold so the sparse part stays PSD.
• Empirics: pre-averaging TSR or PCA + location thresholding often dominate subsampled recipes.

A map of which assumption you are buying, not one magic ticker.

Why after-work Python learners care

This is the vocabulary bridge from Ledoit–Wolf (shrink everything) to structure: observed factors vs characteristics vs unsupervised PCs. You do not need tick data tonight—three estimators on daily returns already shows TSR ≠ PCA.

Tiny evening exercise: on a 10×T fake panel, compare cov of residual-after-market-OLS vs the first two PCs’ reconstructed cov. Print Frobenius gap. Learning only.

One-evening reading checklist

1. Why noise + asynchrony break naive HF sample cov.
2. TSR vs CSR vs PCA in one line each (what is treated as known).
3. What pre-averaging is for.
4. Name one reason location thresholding showed up again (see the GICS paper).

Download PDF

Download PDF

Landing: DOI

Important disclaimer

Not investment advice. This Hub note is educational only. Dai–Lu–Xiu is academic econometrics—not a recommendation to trade high-frequency covariance or BARRA clones.

What this is not

• Not CEF / subsidy framing
• Not Quant or Macro course CTA
• Not a tick-data vendor pitch

Next step

Single course CTA: SoloQuant Academy’s Introduction to Python Programming—OLS on a list of returns is a loop; PCA can wait until arrays feel natural.