Quantitative Portfolio Management The Art and Science of Statistical Arbitrage

Quantitative trading of financial securities is a multi-billion dollar business employing thousands of portfolio managers and quantitative analysts ("quants") trained in mathematics, physics, or other "hard" sciences.  The quants trade stocks and other instruments creating liquidity for investors and competing, as best they can, at finding and exploiting any mispricings.  The result is highly efficient financial markets not immune to occasional events of crowding, bubbling, and liquidation panic.  This book covers all the major parts of the quantitative trading process starting with sourcing financial data, learning future asset returns from historical data, generating and combining multiple forecasts, dealing with risk, building optimal portfolio of stocks subject to risk preferences and trading costs, and executing trades.  The exposition seeks a balance between financial insight, mathematical ideas of statistical and machine learning, practical computational aspects, actual events and thoughts "from the trenches", as observed by a quantitative portfolio manager, and even actual questions asked at countless quant interviews.  The intended audience includes practicing quants who will encounter things both familiar and novel (such lesser known ML algorithms or multi-period portfolio optimization), students and scientists thinking of joining the quant workforce (and wondering if it's worth it), and the general public interested in quantitative and algorithmic trading from a broad scientific, and occasionally ironic, standpoint.

MICHAEL ISICHENKO, PhD, is a theoretical physicist and a quantitative portfolio manager who worked at Kurchatov Institute, University of Texas, University of California, SAC Capital Advisors, Société Générale, and Jefferies. He received his doctorate in physics and mathematics from the Moscow Institute of Physics and Technology and is an expert in plasma physics, nonlinear dynamics, and statistical and chaos theory.

Preface 1

Introduction 3

1 Market Data 9

1.1 Tick and bar data 9

1.2 Corporate actions and adjustment factor 10

1.3 Linear vs log returns 11

2 Forecasting 13

2.1 Data for forecasts 14

2.1.1 Point-in-time and lookahead 15

2.1.2 Security master and survival bias 16

2.1.3 Fundamental and accounting data 16

2.1.4 Analyst estimates 17

2.1.5 Supply chain and competition 18

2.1.6 M&A and risk arbitrage 18

2.1.7 Event-based predictors 18

2.1.8 Holdings and flows 19

2.1.9 News and social media 20

2.1.10 Macroeconomic data 21

2.1.11 Alternative data 21

2.1.12 Alpha capture 21

2.2 Technical forecasts 22

2.2.1 Mean reversion 22

2.2.2 Momentum 24

2.2.3 Trading volume 24

2.2.4 Statistical predictors 25

2.2.5 Data from other asset classes 25

2.3 Basic concepts of statistical learning 27

2.3.1 Mutual information and Shannon entropy 28

2.3.2 Likelihood and Bayesian inference 32

2.3.3 Mean square error and correlation 33

2.3.4 Bias-variance tradeoff 35

2.3.5 PAC learnability, VC dimension, and generalization error bounds 36

2.4 Machine learning 40

2.4.1 Types of machine learning 41

2.4.2 Overfitting 43

2.4.3 Ordinary and generalized least squares 44

2.4.4 Deep learning 46

2.4.5 Types of neural networks 48

2.4.6 Nonparametric methods 51

2.4.7 Cross-validation 54

2.4.8 Curse of dimensionality, eigenvalue cleaning, and shrinkage 56

2.4.9 Smoothing and regularization 61

2.4.9.1 Smoothing spline 62

2.4.9.2 Total variation denoising 62

2.4.9.3 Nadaraya-Watson kernel smoother 63

2.4.9.4 Local linear regression 64

2.4.9.5 Gaussian process 64

2.4.9.6 Ridge and kernel ridge regression 67

2.4.9.7 Bandwidth and hypertuning of kernel smoothers 68

2.4.9.8 Lasso regression 69

2.4.10 Generalization puzzle of deep and overparameterized learning 69

2.4.11 Online machine learning 74

2.4.12 Boosting 75

2.4.13 Randomized learning 79

2.4.14 Latent structure 80

2.4.15 No free lunch and AutoML 81

2.4.16 Computer power and machine learning 83

2.5 Dynamical modeling 87

2.6 Alternative reality 89

2.7 Timeliness-significance tradeoff 90

2.8 Grouping 91

2.9 Conditioning 92

2.10 Pairwise predictors 92

2.11 Forecast for securities from their linear combinations 93

2.12 Forecast research vs simulation 95

3 Forecast Combining 97

3.1 Correlation and diversification 98

3.2 Portfolio combining 99

3.3 Mean-variance combination of forecasts 102

3.4 Combining features vs combining forecasts 103

3.5 Dimensionality reduction 104

3.5.1 PCA, PCR, CCA, ICA, LCA, and PLS 105

3.5.2 Clustering 107

3.5.3 Hierarchical combining 108

3.6 Synthetic security view 108

3.7 Collaborative filtering 109

3.8 Alpha pool management 110

3.8.1 Forecast development guidelines 111

3.8.1.1 Point-in-time data 111

3.8.1.2 Horizon and scaling 111

3.8.1.3 Type of target return 112

3.8.1.4 Performance metrics 112

3.8.1.5 Measure of forecast uncertainty 112

3.8.1.6 Correlation with existing forecasts 112

3.8.1.7 Raw feature library 113

3.8.1.8 Overfit handling 113

3.8.2 Pnl attribution 114

3.8.2.1 Marginal attribution 114

3.8.2.2 Regression-based attribution 114

4 Risk 117

4.1 Value at risk and expected shortfall 117

4.2 Factor models 119

4.3 Types of risk factors 120

4.4 Return and risk decomposition 121

4.5 Weighted PCA 122

4.6 PCA transformation 123

4.7 Crowding and liquidation 124

4.8 Liquidity risk and short squeeze 126

4.9 Forecast uncertainty and alpha risk 127

5 Trading Costs 129

5.1 Slippage 130

5.2 Impact 130

5.2.1 Empirical observations 132

5.2.2 Linear impact model 133

5.2.3 Impact arbitrage 135

5.3 Cost of carry 135

6 Portfolio Construction 137

6.1 Hedged allocation 137

6.2 Single-period vs multi-period mean-variance utility 139

6.3 Single-name multi-period optimization 140

6.3.1 Optimization with fast impact decay 141

6.3.2 Optimization with exponentially decaying impact 142

6.3.3 Optimization conditional on a future position 143

6.3.4 Position value and utility leak 145

6.3.5 Optimization with slippage 146

6.4 Multi-period portfolio optimization 148

6.4.1 Unconstrained portfolio optimization with linear impact costs 149

6.4.2 Iterative handling of factor risk 150

6.4.3 Optimizing future EMA positions 151

6.4.4 Portfolio optimization using utility leak rate 151

6.4.5 Notes on portfolio optimization with slippage 152

6.5 Portfolio capacity 152

6.6 Portfolio optimization with forecast revision 153

6.7 Portfolio optimization with forecast uncertainty 156

6.8 Kelly criterion and optimal leverage 157

6.9 Intraday optimization and execution 160

6.9.1 Trade curve 160

6.9.2 Forecast-timed execution 161

6.9.3 Algorithmic trading and HFT 162

6.9.4 HFT controversy 166

7 Simulation 169

7.1 Simulation vs production 170

7.2 Simulation and overfitting 171

7.3 Research and simulation efficiency 172

7.4 Paper trading 173

7.5 Bugs 173

Afterword: Economic and Social Aspects of Quant Trading 179

Appendix 183

A1 Secmaster mappings 183

A2 Woodbury matrix identities 184

A3 Toeplitz matrix 185

Index 187

Questions index 195

Quotes index 197

Stories index 199