• Part 1 · Introduction: Big Data and Finance Overview
• Part 2 · Text Analytics in Finance
• Part 3 · Image Analytics in Finance
• Part 4 · Integration, Implementation, and Outlook

• Big Data Concepts and Financial Data Landscape
• Alternative Data Types: Focus on Text and Images
• Opportunities, Risks, and Governance

Key insight: ~80% of financial data is unstructured.
Source: Consensus estimates from IDC, Gartner, and Merrill Lynch.

• Complete Data Analysis Workflow

  graph LR A[Problem Definition] --> B[Data Acquisition] B --> C[Data Preprocessing] C --> D[Feature Engineering] D --> E[Model Building] E --> F[Result Evaluation] F --> G[Decision Support] F -.-> A

• Typical Application Scenarios in Finance

  Application Scenario	Data Type	Common Methods
  Risk Management	Market data, text, images	Predictive modeling, anomaly detection
  Investment Decisions	Financial reports, news, social media	Sentiment analysis, topic modeling
  Fraud Detection	Transaction records, behavioral data	Graph neural networks, time-series analysis
  Market Forecasting	Price data, macro indicators, text	Deep learning, reinforcement learning

Information advantage from alternative data:

• Forward-looking signals — Sentiment precedes price moves
• Non-priced information — Not yet in market consensus
• Behavioral insights — Reveal investor psychology
• Real-time updates — Faster than official releases

Example: Hedge funds combine news sentiment + satellite data for macro nowcasting (Source: Katona et al., 2022)

Key regulatory considerations:

• EU AI Act — Risk-based classification of AI systems
• SEC guidance — Disclosure of AI in trading strategies
• ESMA — Algorithmic trading requirements
• Model Risk Management — SR 11-7 guidelines

Best practices:

• Document model development and validation
• Ensure human oversight in critical decisions
• Monitor for drift and performance degradation

• Landscape of Financial Text Data and NLP Pipeline
• Bag of Words, TF-IDF, and Text Regression
• Word Embeddings, Topics, and Beyond
• Financial Applications: Asset Pricing, Risk, and Policy

Classic NLP pipeline for financial text:

Raw Text → Numerical Representation → Information Retrieval → Analysis

1. Numerical representation — Transform documents to vectors
2. Information retrieval — Dimensionality reduction, selection
3. Causal/predictive analysis — Regression, classification

This pipeline underlies most econometric text analysis (Source: Gaillac & L'Hour, 2024)

Bag-of-words representation:

• Assumes word order doesn't matter
• Each document = vector of word counts
• Matrix dimension: (documents × vocabulary)

Example DTM:

Challenge: Extremely sparse and high-dimensional

Domain-specific lexicons for finance:

Key insight: General dictionaries misclassify financial terms

• "Risk" is neutral in finance, negative in general (Source: Loughran & McDonald, 2011)

Using text features for prediction:

Where:

• = outcome (returns, risk, default)
• = document-term vector for firm/period
• High-dimensional: explanatory variables

Solutions to dimensionality:

• Penalized regression (LASSO, Ridge, Elastic Net)
• Random forests for variable selection
• Sentiment aggregation using dictionaries

• Start from a corpus of documents and a vocabulary of tokens; pre‑process (normalization, tokenization, stop‑words, stemming/lemmatization) so each document becomes a sequence
• Build a high‑dimensional representation:
  • Document–term matrix with counts or TF–IDF
  • Or lower‑dimensional factors from SVD, topics, or embeddings
• Optionally compress: select columns (dictionary, sentiment index) or average token embeddings into document‑level vectors
• Stack text features with structured covariates to form or
• Estimate a predictive or causal model, e.g.
  • or
  • Fit with high‑dimensional ML (penalized regression, trees, nets), evaluate out‑of‑sample, then interpret coefficients or factors.

• In text/factor regressions, the feature dimension (words, n‑grams, topics, embeddings) often exceeds sample size ; the document–term matrix is high‑dimensional and sparse
• OLS is ill‑posed, so we use sparse modeling:
  • Lasso / elastic net to select a small set of informative features
  • Penalized likelihood (e.g., penalized logit) for classification
  • Regularization controls variance and overfitting when
• Dimensionality reduction acts as structured sparsity:
  • Low‑rank factor models (SVD) or topic models (LDA) factor into a few latent factors
• For causal parameters, combine ML with cross‑fitting:
  • Split data into folds; estimate nuisance components (text‑based controls, propensities, outcome models) on other folds
  • Plug predictions into orthogonal / debiased estimating equations on the held‑out fold
  • Aggregating across folds delivers valid inference in big‑data settings.

• The pipeline is naturally predictive: turn text into features and estimate with flexible ML, assessing performance by out‑of‑sample loss (MSE, AUC, etc.)
• Causal questions instead target structural effects (policy, treatment, latent concept). Text can serve as:
  • High‑dimensional controls for confounding
  • Proxies for latent variables (tone, ideology, uncertainty)
• High‑dimensional ML helps approximate nuisance functions but does not create identification; assumptions still come from design (unconfoundedness, instruments, natural experiments)
• Risks of confusing prediction and causality:
  • Highly predictive text features may capture selection or anticipation, not causal channels
  • Overfitting can generate spurious “important words”
• A principled workflow separates:
  • ML for prediction/adjustment (controls, propensities)
  • Econometric methods for causal parameters, with orthogonal moments and cross‑fitting to reduce bias.

Problems with sparse word vectors:

1. Inefficient storage — Dimension per document
2. No semantic similarity — All distinct words equidistant

3. Curse of dimensionality — Joint probability estimation fails

Solution: Learn dense, low-dimensional word embeddings

Research question: Does transparency change Fed deliberations?

Method:

• LDA topic model on 46,169 FOMC documents
• Natural experiment: 1993 tape revelation

Finding:

• After transparency increased, committee members showed conformist behavior
• Less diverse opinions relative to chairman's view

Implication: Transparency may reduce deliberation quality

Source: Hansen, McMahon & Prat, 2017

Financial research using word embeddings:

Key insight: Embeddings capture latent concepts not explicit in text

Building uncertainty indices from transcripts:

1. Extract MD&A sections from earnings calls
2. Identify uncertainty language (hedging, modal verbs)
3. Construct firm-level uncertainty index
4. Validate against realized volatility

Results:

• Textual uncertainty predicts future stock volatility
• Incremental to standard risk measures
• Useful for options pricing and risk management

BERT fine-tuned on financial texts:

• Training corpus: News, SEC filings, analyst reports
• 5-10% accuracy improvement over general BERT

Variants:

Note: Transformer details in subsequent lectures

Key implementation issues:

• Labeling — Expert annotation is expensive
• Domain adaptation — General models underperform
• Language drift — Financial terminology evolves
• Evaluation — Ground truth often unavailable

Best practices:

• Use domain-specific preprocessing
• Validate on out-of-sample financial data
• Monitor model performance over time

From task‑specific models → general financial intelligence

Core paradigm shift:

• Foundations of Image Data and Computer Vision
• Remote Sensing and Satellite Imagery
• Document Image Analysis and OCR
• Image-Based Risk, Fraud, and ESG Analytics

Key components:

1. Convolutional layers — Extract local features with filters
2. Pooling layers — Reduce spatial dimensions
3. Activation (ReLU) — Introduce nonlinearity
4. Fully connected — Final classification/regression

Advantages:

• Local receptive field — Captures spatial patterns
• Weight sharing — Parameter efficiency
• Hierarchical features — Low → high level abstraction
• Translation invariance — Position-independent detection

Input Image → [Conv → ReLU → Pool] × N → Flatten → FC → Output

Layer progression:

Popular architectures: VGG, ResNet, EfficientNet, Inception

Leveraging pretrained models:

1. Feature extraction — Freeze pretrained layers, train new classifier
2. Fine-tuning — Unfreeze top layers, retrain on financial data

Why transfer learning?

• Financial image datasets are small
• ImageNet features generalize to many domains
• Dramatically reduces training time and data needs

Challenge: Financial images differ from natural images

• Solution: Gradual unfreezing of layers

Capturing real economic activity from space:

Advantage: Real-time, unbiased, global coverage

End-to-end workflow:

Image Acquisition → Tiling → Preprocessing → Feature Extraction → Aggregation

Steps:

1. Acquisition — Commercial providers (Planet, Maxar)
2. Tiling — Split large images into manageable patches
3. Preprocessing — Cloud removal, normalization
4. Feature extraction — CNN or manual features
5. Aggregation — Temporal and spatial aggregation

Challenges: Weather effects, acquisition frequency, spatial alignment

Predicting crude oil inventories:

• Floating-roof tanks cast measurable shadows
• Shadow length indicates fill level
• Daily imagery → continuous inventory estimates

Application:

• Predict EIA weekly inventory reports
• Trade crude oil futures ahead of announcements
• Monitor geopolitical supply disruptions

Accuracy: 2-3 day lead time over official data

Processing scanned financial documents:

Optical Character Recognition applications:

• KYC/AML — ID document verification
• Credit underwriting — Extract income from tax forms
• Accounts payable — Invoice processing automation
• Audit — Contract and receipt digitization

Pipeline:

Scan → Deskew → OCR → Field Extraction → Validation → Integration

Benefits: Cost reduction, speed, error minimization

SME lending automation:

1. Upload scanned financial statements, tax documents
2. OCR extracts text and tables
3. NLP parses financial metrics
4. Validation cross-checks extracted data
5. Scoring feeds into credit model

Results:

• Processing time: days → minutes
• Manual review reduced by 70%
• Error rate decreased significantly

Automated extraction from chart images:

Tasks:

• Chart type classification — Line, bar, candlestick, pie
• Axis detection — Hough transform, edge detection
• Data extraction — Point/bar measurement
• Pattern recognition — Technical analysis patterns

Technical patterns detected:

• Head and shoulders
• Double tops/bottoms
• Triangles, flags, wedges

ML on stock price charts:

Method:

• Train CNN on labeled price chart images
• Learn predictive visual patterns (not predefined)
• Extract features that forecast returns

Key findings:

• Patterns yield more accurate predictions than traditional factors
• Short-term patterns work on longer scales
• US-learned patterns effective internationally

Source: Jiang, Kelly & Xiu, JF 2023

Visual anomaly detection in finance:

Models: CNN-transformer hybrids for anomaly detection

Identity verification workflow:

1. Capture — Photo of face and ID document
2. Extraction — Face detection, document parsing
3. Matching — Compare live face to ID photo
4. Liveness — Detect presentation attacks
5. Decision — Accept/reject/manual review

Considerations:

• Privacy — Biometric data protection regulations
• Bias — Demographic performance disparities
• Explainability — Audit trail for decisions

Insurance and real estate applications:

• Property valuation — Aerial imagery for condition assessment
• Catastrophe modeling — Pre/post disaster comparison
• Climate risk — Flood, fire, storm exposure mapping
• Claims processing — Automated damage assessment

Example:
Insurer uses drone imagery for post-hurricane claims.
Pre-event images enable accurate loss estimation.

Key ethical considerations:

• Privacy — Consent for image collection and use
• Surveillance — Balancing security vs. civil liberties
• Bias — Demographic disparities in recognition accuracy
• Transparency — Explainable AI for regulated decisions

Best practices:

• Regular bias audits across demographics
• Clear disclosure of AI-based decisions
• Human oversight for high-stakes applications

• Combining Text, Images, and Structured Data
• Practical Project Workflow
• Limitations, Ethics, and Research Frontiers

Combining multiple data modalities:

Text Features ─┐
               ├─→ Fusion Layer → Prediction
Image Features ┘

Fusion strategies:

Example: Combine news sentiment + satellite signals + fundamentals

Integrated alternative data approach:

Inputs:

• Analyst reports (text sentiment)
• Web traffic data (consumer interest)
• Satellite parking lot data (sales proxy)
• Traditional fundamentals (structured)

Model:

• Feature extraction per modality
• Late fusion with learned weights
• Output: Stock selection signals

Benefit: Diversified signal sources reduce model risk

From idea to deployment:

Documentation and reproducibility:

• [ ] Version control for code and data
• [ ] Experiment tracking (MLflow, W&B)
• [ ] Clear data lineage documentation
• [ ] Model cards with performance metrics

Team collaboration:

• Quants Domain experts Engineers Risk managers
• Regular model review meetings
• Clear ownership and escalation paths

Feasible term paper / capstone projects:

Tools: Python, scikit-learn, PyTorch, spaCy, Tesseract

Key challenges:

• Data quality — Noise, missing data, inconsistency
• Sample selection — Survivorship bias, coverage gaps
• Non-stationarity — Markets and language evolve
• Model robustness — Adversarial attacks, distribution shift

Mitigation:

• Rigorous out-of-sample testing
• Ensemble methods for stability
• Continuous monitoring and retraining

Responsible AI in finance:

Regulatory trends:

• Increasing scrutiny of AI in lending decisions
• Requirements for algorithmic impact assessments

Emerging themes in financial AI:

• Robust LLMs — Domain-adapted large language models
• Synthetic data — Augmenting scarce financial datasets
• Human-in-the-loop — Hybrid AI-human decision systems
• Causal ML — Moving beyond prediction to causation
• Multimodal foundation models — Unified text-image-tabular

Reading list: See course website for recent surveys

Core messages:

1. Text and images are increasingly valuable for financial analysis
2. NLP pipeline — From preprocessing to embeddings to prediction
3. Computer vision — Satellite, documents, charts offer unique signals
4. Integration — Multimodal approaches enhance robustness
5. Responsibility — Ethics, bias, and governance are critical

"The future belongs to analysts who can extract insights from all data modalities."

Discussion topics:

1. How might text/image signals be arbitraged away over time?
2. What are the fairness implications of satellite-based trading?
3. Should regulators require disclosure of alternative data use?
4. How do you evaluate the quality of a sentiment indicator?