Lecture 01

When is a Liability not a Liability? Textual Analysis, Dictionaries, and 10‑Ks.

• 数据（大数据应用）
  • 超过 50,000 份美国上市公司 10‑K 报告文本（非结构化语言数据）
  • 来源：SEC EDGAR 数据库
• 数据处理方法
  • 自建 金融专用情感词典 (LM Dictionary)；
  • 自动分词、去停用词、词频统计；
  • 构造正面、负面、不确定性指标。
• 意义
  • 奠定金融文本分析标准；
  • 将传统文本转化为可量化的大数据指标；
  • 对信息披露和市场反应研究影响深远。

Firm‑Level Political Risk.

• 数据
  • 约 8,000 家美国公司的 10‑K 报告文本（1995–2017）；
• 处理方法
  • 使用主题建模 (Topic Modeling) 自动识别政治风险语义主题；
  • 构造“企业层面政治风险指数 (PRisk)”；
  • 整合宏观政策事件与公司层面决策数据。
• 意义
  • 证明文本大数据可生成新型风险测度；
  • 推动“文本→风险→行为”量化路径；
  • 是政治与宏观不确定性研究的基础。

Text‑Based Network Industries and Endogenous Product Differentiation.

• 数据
  • 数千家上市公司 10‑K 产品描述文本。
• 处理方法
  • 文本相似度分析 （余弦距离、TF‑IDF）；
  • 构建企业间“文本语义相似网络”；
  • 代替 SIC/NAICS ，定义产业边界。
• 意义
  • 将语言大数据转化为经济结构映射；
  • 揭示产业结构和竞争模式的动态演化；
  • “文本＋网络”成为结构性建模新方向。

In Search of Attention.

• 数据
  • Google Search Volume Index (SVI) 2004‑2010
  • 数千只股票的搜索量时间序列（上百万观测点）
• 方法
  • 把 SVI 作为投资者注意力代理变量；
  • 多期回归与截面分析。
• 意义
  • 首次引入互联网用户行为数据；
  • 体现“大数据行为金融”；
  • 建立实时注意力测度工具。

Twitter Mood Predicts the Stock Market.

• 数据
  • 980余万条 Twitter 消息
  • 覆盖全球用户文本（2008‑2009）
• 方法
  • NLP 情绪词典分析（6 类情绪）；
  • 构造“集体情绪指数”；
  • Granger 因果检验 预测 DJIA 走势。
• 意义
  • 首次验证社交媒体情绪与市场波动关系；
  • 将实时非结构化数据用于金融预测。

Predicting Returns with Text Data.

• 数据
  • 约 1,200 万篇 Dow Jones Newswires 新闻；
  • 高频专业媒体文本流。
• 方法
  • 提出 SESTM (Sentiment Extraction via Screening & Topic Modeling)；
  • 监督学习模型，用股票收益信号训练文本情绪。
• 意义
  • 机器学习＋金融文本融合；
  • 优于词典与专家打分；
  • 代表金融文本算法化预测的前沿。

Measuring Economic Growth from Outer Space

• 数据
  • 全球 夜间灯光卫星影像 (DMSP‑OLS) ；
  • 时间：1992 – 2008，覆盖 180 多个国家。
• 方法
  • 灯光亮度校正（去噪声、跨卫星校准）；
  • 灯光强度  GDP 回归；
  • 面板模型估计照明‑产出弹性。
• 意义
  • 用遥感影像替代 GDP 测度；
  • 弥补统计空缺，推动“卫星数据经济学”；
  • 开创影像大数据在经济研究的先河。

Combining Satellite Imagery and Machine Learning to Predict Poverty.

• 数据
  • 多源卫星数据（白天高分辨率影像 + 夜光）；
  • 覆盖非洲数国 > 6 万 个村庄。
• 方法
  • 卷积神经网络 (CNN) + 迁移学习；
  • 先通过夜光特征训练 CNN ，再预测贫困概率地图。
• 意义
  • 展示 AI + 遥感影像在经济发展测度的潜力；
  • 用公开大数据生成贫困分布，为发展评估提供新手段。

Do Weather‑Induced Moods Affect the Processing of Earnings News?

• 数据
  • NOAA GHCN‑Daily ： > 1000 气象站、50 年数据；
  • 匹配公司所在地及公告日期。
• 方法
  • 构造“异常天气指数”；
  • 面板回归检验天气对财报反应的影响。
• 意义
  • 高维跨空间数据融合；
  • 揭示情绪‑信息处理机制；
  • 结构化气象数据的典型大数据应用。

Good Day Sunshine: Stock Returns and the Weather.

• 数据
  • 26 个城市 × 多年 × 日度 云量/日照 观测；
  • 全球股票市场指数。
• 方法
  • 日频面板模型；
  • 天气与市场回报的交叉检验。
• 意义
  • 首次跨国使用高频气象面板；
  • 建立天气‑情绪假说全球证据。

Predicting Anomaly Performance with Politics, the Weather, Sunspots and the Stars.

• 数据
  • 多源时间序列（天气、气候、天文、政治等）；
  • 结合股票因子回报。
• 方法
  • 多变量面板回归；
  • 高维外生因子稳健性分析。
• 意义
  • 展示多源大数据用于验证与稳健测试；
  • 示范“大数据反思”模型。

Empirical Asset Pricing via Machine Learning.

• 数据
  • 数百个股票特征 × 数千只股票 × 几十年高维面板。
• 方法
  • 系统比较 Lasso、Random Forest、Boosting、NN；
  • 特征重要性排名；
  • 构建 ML 因子模型。
• 意义
  • 非线性特征选择与预测最优化；
  • 奠定机器学习资产定价基础；
  • 实证金融进入“算法‑大数据”阶段。

Forest through the Trees: Building Cross‑Sections of Stock Returns.

• 数据
  • 数万只股票因子特征（横截面高维数据）。
• 方法
  • 随机森林＋因子降维 构建“Tree‑based Factor Model”；
  • 将非线性特征映射为可解释定价因子。
• 意义
  • 实现因子建模的非线性化与机器学习化；
  • 推动大数据资产定价框架的前沿发展。

• 大规模复杂数据 → 传统方法受限
  • 高维、非线性
  • 多样、复杂
• ML 提供灵活工具：非线性建模、变量筛选、稳健预测
• 跨界互动：金融×计算×统计

• Traditional statistics relies too heavily on model assumptions, often poorly representing real‑world complexity.

• The algorithmic culture gives priority to empirical prediction, not theoretical form.

• For real applications (speech, vision, finance), algorithmic models outperform classical ones.

• Statistical education and research should emphasize prediction, validation, and model performance rather than significance alone.

• Econometric culture: builds theory‑driven structural models
  (e.g., CAPM, Fama–French factors, VAR).
• Machine‑learning culture: learns complex, nonlinear relations from large data
  (e.g., Random Forest for factor selection, LSTM for forecasting, BERT for text sentiment).
• Synthesis:
  • ML → improves predictive accuracy
  • Econometrics → provides interpretation and causal mechanisms

Prediction and causality are complements, not rivals.

• Breiman (2001) “Statistical Modeling: The Two Cultures.” Statistical Science, 16(3), 199–231.
• Varian (2014) “Big Data: New Tricks for Econometrics.” Journal of Economic Perspectives.
• Athey & Imbens (2019) “The State of Applied Econometrics: Causality and Machine Learning.”

• Supervised Learning: ML algorithms that infer patterns between a set of inputs (the ’s) and the desired output () with a labeled data set.
• Unsupervised Learning: machine learning that does not make use of labeled data. In unsupervised learning, inputs (’s) are used for analysis without any target () being supplied. The algorithm seeks to discover structure within the data themselves. Two important types of problems in unsupervised learning are dimension reduction and clustering.
• Reinforcement Learning: a computer learns from interacting with itself (or data generated by the same algorithm).

Pattern discovery at different levels of supervision.

• Definition
  • The task is to learn a mapping from inputs to outputs
  • The inputs are also called the features, covariates, or predictors
  • The outputs are also called the label, target, or response
  • The experince is the training set

• exploratory data analysis: see if there are any obvious patterns
• tabular data with a small number of features: pair plot
• higher-dimensional data: dimension reduction first and then to visualize the data in 2d or 3d

• decision rule via a 1 dimensional (1d) decision boundary

• decision tree a more sophisticated decision rule involves a 2d decision surface

• misclassification rate on the training set:

• loss function:
• empirical risk: e the average loss of the predictor on the training set

• model fitting / training via empirical risk minimization

We must avoid false confidence bred from an ignorance of the probabilistic nature of the world, from a desire to see black and white where we should rightly see gray.

--- Immanuel Kant, as paraphrased by Maria Konnikova

• Two types of uncertainties
  • epistemic uncertainty or model uncertainty: due to lack of knowledge of the input-output mapping
  • aleatoric uncertainty or data uncertainty: due to intrinsic (irreducible) stochasticity in the mapping
• We can capture our uncertainty using the following conditional probability distribution:

• likelihood function:

• log likelihood function

• Negative Log Likelihood: The average negative probability of the training set.

• the maximum likelihood estimate (MLE):

• the output space: .
• loss function: quadratic loss, or loss:

• mean squared error or MSE:

• An Example
  • Uncertainty: Guassian / Normal

• the conditional dist

• NLL

• functional form of model:

• parameters:
• least square estimator:

• functional form of model:

• feature preprocessing, or feature engineering

• deep neural networks (DNN): a stack of L nested functions:

• the function at layer :
  • the final layer:
  • the learned feature extractor:

• Underfitting means the model does not capture the relationships in the data.
• Overfitting means the model begins to incorporate noise coming from quirks or spurious correlations
  • it mistakes randomness for patterns and relationships
  • memorized the data, rather than learned from it
  • high noise levels in the data and too much complexity in the model
  • complexity refers to the number of features, terms, or branches in the model and to whether the model is linear or non-linear (non-linear is more complex).

• empirical risk

• population risk

• generalization gap:

• test risk

• Data scientists decompose the total out-of-sample error into three sources:
  • Bias error, or the degree to which a model fits the training data. Algorithms with erroneous assumptions produce high bias with poor approximation, causing underfitting and high in-sample error.
  • Variance error, or how much the model's results change in response to new data from validation and test samples. Unstable models pick up noise and produce high variance, causing overfitting
  • Base error due to randomness in the data.

• A learning curve plots the accuracy rate (= 1 – error rate) in the validation or test samples (i.e., out-of-sample) against the amount of data in the training sample

• A fitting curve, which shows in-and out-of-sample error rates ( and ) on the -axis plotted against model complexity on the -axis

• unsupervised learning: “inputs” without any corresponding “outputs” .
• the task: fitting an unconditional model of the form

When we’re learning to see, nobody’s telling us what the right answers are — we just look. Every so often, your mother says “that’s a dog”, but that’s very little information. You’d be lucky if you got a few bits of information — even one bit per second — that way. The brain’s visual system has 1014 neural connections. And you only live for 109 seconds. So it’s no use learning one bit per second. You need more like 105 bits per second. And there’s only one place you can get that much information: from the input itself.

--- Geoffrey Hinton, 1996

• finding clusters in data: partition the input into regions that contain “similar” points.

• Assume that each observed high-dimensional output was generated by a set of hidden or unobserved low-dimensional latent factors .

• factor analysis (FA)

• principal components analysis (PCA):

• nonlinear models: neural networks

• online / dynamic version of machine learning
  • the system or agent has to learn how to interact with its environment
  • RL is closely related to the Markov Decision Process (MDP)

The MDP is the sequence of random variables () which describes the stochastic evolution of the system states. Of course the distribution of () depends on the chosen actions.

• denotes the state space of the system. A state is the information which is available for the controller at time . Given this information an action has to be selected.

• denotes the action space. Given a specific state at time , a certain subclass of actions may only be admissible.

• is a stochastic transition kernel which gives the probability that the next state at time is in the set B if the current state is and action a is taken at time .

• gives the (discounted) one-stage reward of the system at time if the current state is and action a is taken

• gives the (discounted) terminal reward of the system at the end of the planning horizon.

A control is a sequence of decision rules () with where determines for each possible state the next action at time . Such a sequence is called policy or strategy. Formally the Markov Decision Problem is given by

• Types of MDP problems:

  • finite horizon () vs. infinite horizon ()
  • complete state observation vs. partial state observation
  • problems with constraints vs. without constraints
  • total (discounted) cost criterion vs. average cost criterion

• Research questions:

  • Does an optimal policy exist?
  • Has it a particular form?
  • Can an optimal policy be computed efficiently?
  • Is it possible to derive properties of the optimal value function analytically?

Suppose there is an investor with given initial capital. At the beginning of each of periods she can decide how much of the capital she consumes and how much she invests into a risky asset. The amount she consumes is evaluated by a utility function as well as the terminal wealth. The remaining capital is invested into a risky asset where we assume that the investor is small and thus not able to influence the asset price and the asset is liquid. How should she consume/invest in order to maximize the sum of her expected discounted utility?

Imagine a company which tries to find the optimal level of cash over a finite number of periods. We assume that there is a random stochastic change in the cash reserve each period (due to withdrawal or earnings). Since the firm does not earn interest from the cash position, there are holding cost for the cash reserve if it is positive, but also interest (cost) in case it is negative. The cash reserve can be increased or decreased by the management at each decision epoch which implies transfer costs. What is the optimal cash balance policy?

Consider a small investor who acts on a given financial market. Her aim is to choose among all portfolios which yield at least a certain expected return (benchmark) after periods, the one with smallest portfolio variance. What is the optimal investment strategy?

Imagine we consider the risk reserve of an insurance company which earns some premia on the one hand but has to pay out possible claims on the other hand. At the beginning of each period the insurer can decide upon paying a dividend. A dividend can only be paid when the risk reserve at that time point is positive. Once the risk reserve got negative we say that the company is ruined and has to stop its business. Which dividend pay-out policy maximizes the expected discounted dividends until ruin?

Suppose we have two slot machines with unknown success probability and . At each stage we have to choose one of the arms. We receive one Euro if the arm wins, else no cash flow appears. How should we play in order to maximize our expected total reward over trials?

In order to find the fair price of an American option and its optimal exercise time, one has to solve an optimal stopping problem. In contrast to a European option the buyer of an American option can choose to exercise any time up to and including the expiration time. Such an optimal stopping problem can be solved in the framework of Markov Decision Processes.

"Machine learning extracts return‑predictive structure from the
high‑dimensional space of firm characteristics."

• Background:
  Empirical asset pricing traditionally relies on a few linear factors (e.g., Fama–French 3F/5F).
  Yet literature has discovered hundreds of potential predictors → the “factor zoo.”

• Challenges:

  • High‑dimensional  problem
  • Multicollinearity and unstable coefficients
  • Very low out‑of‑sample R² (≈ 0%)

• Core question:

  Can machine learning methods find stable and economically meaningful prediction patterns
  in the cross‑section of expected stock returns?

• Data

  • U.S. equities (1957 – 2016) monthly.
  • ~94 firm characteristics + 30 macro variables → input features .
  • Predict one‑month‑ahead returns .

• Machine Learning Models
  1. Regularized linear models (LASSO, Ridge, Elastic Net)
  2. Tree‑based models (Random Forest, Gradient Boosting)
  3. Neural networks (Feedforward MLP)

• Estimation Design

  • Cross‑sectional ML fit each month: 
  • Rolling OOS validation & economic performance checking.

Portfolio Implications

• Portfolios sorted on ML‑predicted returns earn significant alphas.
• Sharpe ratio up to 2× Fama‑French 5 Factor benchmark.
• Predictive patterns robust over sub‑periods and market conditions.

• ML models automatically rediscover and combine known signals

  • value, momentum, profitability, investment, liquidity, sentiment.

• Capture nonlinear interactions and time‑varying relations often ignored by linear models.

• The learned function  implies an estimated stochastic discount factor (SDF)
  consistent with no‑arbitrage pricing.

Regularization = statistical discipline; ML = economic flexibility.

• Machine learning reframes asset pricing as a forecasting problem.
• Moves the field from small, parametric models to data‑driven, nonlinear mappings.
• Provides a bridge between Breiman’s “algorithmic modeling culture”
    and financial econometrics.

Bottom Line:

ML techniques deliver higher predictive accuracy, robust SDF estimates, and new ways to understand risk premia.

Gu, S., Kelly, B., and Xiu, D. (2020).
Empirical Asset Pricing via Machine Learning.
Review of Financial Studies, 33(5): 2223 – 2273.

• Data and code available at https://dachxiu.github.io/
• Highly influential study in financial machine learning literature

“The goal is to let a learning agent autonomously adjust portfolio weights to
maximize risk‑adjusted returns through interaction with the market environment.”

• Classical portfolio optimization (Markowitz 1952):
  One‑shot mean–variance trade‑off given expected returns and covariances.
  → Static optimization; requires ex ante distributional assumptions.

• Real investment environments:

  • Dynamic and uncertain markets
  • Transaction costs and constraints
  • Need for adaptive, sequential decisions

• Reinforcement Learning (RL):
  Learns an optimal policy through trial and error to maximize cumulative reward.

Training loop (Agent  Market):
1. Observe market state → 2. Select action (rebalance) → 3. Receive reward → 4. Update policy.

Model Type: Deep Reinforcement Learning (DRL)

• State vector : log returns, volatility, moving averages, momentum signals, past allocations.
• Action vector : continuous weights subject to , .
• Reward function:
  balances return and risk penalization.
• Policy learning: Deep Deterministic Policy Gradient (DDPG) or Proximal Policy Optimization (PPO).
• Exploration vs exploitation: ε‑greedy or stochastic policy sampling to improve learning efficiency.

• Data: Daily prices of S&P 500 constituents (or ETF basket).
• Training: 1990 – 2010 rolling window; testing 2011 – 2019.
• Benchmarks: Equal Weight (EW), Mean–Variance, Risk‑Parity strategies.

Key Findings

• DRL models achieve higher annualized Sharpe ratios (≈ 1.5–2×) over benchmarks.
• Policies adaptively shift into safe assets during drawdowns.
• Transaction cost robustness built via penalty in reward function.

→ Evidence that a learning agent can continuously optimize asset allocation under dynamic uncertainty.

• RL formulation unifies forecasting and control:
  estimates both expected returns and optimal actions within an adaptive feedback loop.
• Learned policy ≈ data‑driven dynamic extension of Markowitz:

• The agent’s behavior is interpretable: pattern‑based rebalancing (more risk aversion in high volatility).
• Challenges: sample‑efficiency and stability of deep RL; interpretability vs. black‑box learning.

The loop illustrates continuous interaction:
agent learns optimal allocations as markets evolve.

• Reinforcement learning extends portfolio theory to a multi‑period, feedback‑driven world.
• DRL agents outperform static benchmarks by incorporating nonlinear dynamics and adaptive risk control.
• Combining ML forecast power (GKX 2020) with RL decision optimization (Dixon 2020) forms the core of modern Financial AI Infrastructure.

• Dixon, M., Halperin, I., and Bilkay, P. (2020). Machine Learning in Finance: From Theory to Practice. Springer.
• Moody, J. & Saffell, M. (2001). Learning to Trade via Direct Reinforcement. IEEE Trans. NN.
• Li, Y., and Dixon, M. (2020). Deep Reinforcement Learning for Dynamic Portfolio Optimization. Applied Stochastic Models in Business and Industry.

“Benchmarking modern classification algorithms in credit scoring reveals that
 machine learning models consistently outperform traditional statistical approaches.”

• Credit risk modeling is a core task in banking: estimate each applicant’s probability of default (PD).
• Classical scorecards use logistic regression for simplicity and explainability.
• However, in large‑scale datasets, relationships between borrower features and default risk are:
  • Nonlinear
  • High‑dimensional
  • Containing strong interactions

Research question:

Which machine learning methods perform best for credit scoring tasks?

• Data: 8 public and proprietary credit datasets (≈ 30k – 60k observations each).
  • Personal loan and credit‑card applications.
  • Features: demographics, financial ratios, repayment history.
  • Target: default (1) vs non‑default (0).
• Algorithms benchmarked (8+):
  1. Logistic Regression (baseline)
  2. Decision Trees
  3. Random Forest
  4. Gradient Boosting (MART / GBM)
  5. Support Vector Machines (SVM)
  6. Neural Networks (MLP)
  7. k‑Nearest Neighbors (kNN)
  8. Naïve Bayes
• Evaluation metrics: AUC, Brier Score, Accuracy, Top‑decile capture, Generalization stability.

Result Summary

• Ensemble methods (GBM, RF) systematically outperform traditional statistics.
• Models with regularization and probability calibration yield most stable PD estimates.
• No single method wins universally → model selection should consider data size and imbalance.

• Gradient Boosting captures complex nonlinear feature interactions + robust ranking of PD.
• Neural Networks can approximate nonlinear decision boundaries but require more data and tuning.
• Explainable versions (e.g., feature importance, partial dependence plots) help meet regulatory transparency requirements.
• The study provides a universal benchmark still used by banks and fintechs to validate new ML credit models.

The pipeline illustrates how ML models estimate PD and support risk‑based lending decisions.

• Machine learning enhances credit risk prediction — especially ensemble methods like GBM and RF.
• Benchmark study (265 experiments) shows consistent gains over logistic baseline.
  - Supports industry trend to adopt modern ML within Basel IRB framework with interpretability tools.
• Foundation for later “Explainable AI in Credit Risk” literature.

• Lessmann, S., Baesens, B., Seow, H.‑V., & Thomas, L. C. (2015). Benchmarking State‑of‑the‑Art Classification Algorithms for Credit Scoring. European Journal of Operational Research, 247(1), 124–136.
• Benchmark datasets and code: http://www.creditriskanalytics.net
• Referenced as the go‑to comparison study for modern credit scoring models.

• Background
  Macroeconomic indicators (GDP growth, industrial production) are published infrequently and with lags.
  → Hard to observe the business cycle in real time.

• Idea
  Financial and business news articles contain massive information
  about current economic activity and sentiment.

• Research Questions
  1. Can we extract quantitative signals from high‑dimensional news text data?
  2. Do these “news factors” predict macro and asset‑market fluctuations?

Example:
Sharp declines in the “economic activity” factor precede official recessions by ≈ 3 months.

• Text as Macro Sensor
  News coverage aggregates dispersed information from firms, markets, and policymakers.
  → Functions as a real‑time proxy for latent business conditions.

• Complement to Traditional Signals
  Combines qualitative insight with quantitative estimation.
  Yields improved forecasting and nowcasting.

• Implications

  • Central banks and investors can use news‑based indices for timelier policy and risk decisions.
  • Bridges NLP methods with macroeconomic modeling.

The loop shows continuous feedback between media information, real economy, and financial markets.

• High‑frequency news text contains rich economic signals.
• Machine‑learning (PCA on TF‑IDF space) constructs interpretable news factors.
• News‑based indices track and lead the business cycle in real time.
• Demonstrates how NLP bridges information flows between the press and macro‑finance.

Bybee, L., Kelly, B., Manela, A., & Xiu, D. (2024).
Business News and Business Cycles. Journal of Finance, 79(4), 1919 – 1978.

• Data and code available at https://dachxiu.github.io/
• Extends ML approaches to macro forecasting using large‑scale textual data.

• Two common guiding principles and two methods are used to reduce overfitting:

  • preventing the algorithm from getting too complex during selection and training (regularization)
  • proper data sampling achieved by using cross-validation

• K-fold cross-validation

  • data (excluding test sample and fresh data) are shuffled randomly and then are divided into k equal sub-samples
  • samples used as training samples and one sample used as a validation sample
  • is typically set at 5 or 10
  • This process is repeated times. The average of the validation errors (mean ) is taken as a reasonable estimate of the model's out-of-sample error ()

• Leave-one-out cross-validation:

• Gareth James, Daniela Witten, Trevor Hastie, Robert Tibshirani, Jonathan Taylor. An Introduction to Statistical Learning with Applications in Python[M]. Springer Cham, 2023.
• Murphy K P. Probabilistic machine learning: an introduction[M]. MIT press, 2022.
• Gaillac C, L'Hour J. Machine Learning for Econometrics[M]. Oxford University Press, 2025.
• Dixon M F, Halperin I, Bilokon P. Machine learning in Finance[M]. Springer International Publishing, 2020.
• Nagel S. Machine learning in asset pricing[M]. Princeton University Press, 2021.
• de Prado M M L. Machine learning for asset managers[M]. Cambridge University Press, 2020.
• Ashwin Rao, Tikhon Jelvis. Foundations of Reinforcement Learning with Applications in Finance[M]. Stanford University, 2022.
• Denev A, Amen S. The Book of Alternative Data: A Guide for Investors, Traders and Risk Managers[M]. John Wiley & Sons, 2020.