Sign in Subscribe
3 min read

Designing a Stock Trading System That Stays Fast During Traffic Spikes

Share

Stock trading systems operate under extreme constraints: millions of orders per second, sub-millisecond latency, zero tolerance for data loss, and regulatory compliance. This article explores the architecture that makes this possible.

Key sources: "Designing Data-Intensive Applications" by Martin Kleppmann, Nasdaq and NYSE architecture documentation, LMAX Disruptor pattern.

The Requirements

A trading system must handle:

  • Ultra-low latency: Order execution in microseconds, not milliseconds
  • High throughput: Millions of orders per second during peak hours
  • Data integrity: Zero tolerance for lost or duplicated orders
  • Ordering: All orders must be processed in the exact order they arrived
  • Durability: Every order must be recoverable after a crash
  • Compliance: Full audit trail of every order and trade

Architecture Overview

Client → Order Gateway → Order Book → Matching Engine → Market Data Publisher
                             │
                         Database
                             │
                     Audit Trail Storage

Each component has specific latency and throughput requirements.

Component 1: Order Gateway

The order gateway is the entry point. It validates incoming orders and assigns them a sequence number.

Responsibilities: - Parse and validate incoming order messages - Authenticate the sender - Assign a globally unique, monotonically increasing sequence number - Write the order to a write-ahead log (WAL) for durability - Forward the validated order to the order book

Performance strategies: - Kernel bypass: Use DPDK or Solarflare OpenOnload to bypass the OS network stack - Fixed-size messages: Avoid variable-length parsing — use fixed-size binary protocols - Lock-free data structures: The LMAX Disruptor pattern uses a ring buffer with no locks - CPU pinning: Pin gateway processes to dedicated CPU cores to avoid context switching

Component 2: Order Book

The order book maintains the list of outstanding buy and sell orders for each security. It must support:

  • Add order: Insert a new order into the book
  • Cancel order: Remove an existing order
  • Modify order: Change order parameters (price, quantity)
  • Match order: Find matching orders for execution

Data structure: The order book uses two priority queues (heaps) — one for buy orders (max-heap by price) and one for sell orders (min-heap by price). Orders at the same price level are stored in FIFO order.

Buy Orders (highest first)        Sell Orders (lowest first)
$100.50 - 100 shares              $100.55 - 200 shares
$100.45 -  50 shares              $100.60 - 150 shares
$100.40 - 200 shares              $100.65 -  75 shares

When a buy order arrives at $100.55, it matches against the $100.55 sell order immediately.

Component 3: Matching Engine

The matching engine is the core of the system. It receives orders from the order book and executes trades.

Matching rules:

  1. A buy order at price P matches the oldest sell order at price ≤ P
  2. A sell order at price P matches the oldest buy order at price ≥ P
  3. Partial fills are allowed — an order may be partially executed
  4. If no match exists, the order stays in the book

Example:

Buy order: 100 shares at $100.55
Sell orders in book:
  $100.50 - 50 shares → Match 50 shares at $100.50
  $100.52 - 30 shares → Match 30 shares at $100.52
  $100.55 - 20 shares → Match 20 shares at $100.55
  Total: 100 shares filled, order complete

Component 4: Market Data Publisher

Every trade generates market data that must be published to subscribers — other exchanges, data vendors, and the public.

Data types: - Trade data: Price, volume, timestamp for each executed trade - Quote data: Current best bid and offer (BBO) for each security - Order book snapshots: Full depth of the order book at intervals

Performance: Market data is typically published using the FIX protocol (Financial Information Exchange) or proprietary binary protocols. The publisher must handle thousands of subscribers with sub-millisecond latency.

Handling Traffic Spikes

Black Monday Flash Crash (2010)

During the 2010 Flash Crash, the Dow Jones dropped nearly 1,000 points in minutes. Trading systems that could not handle the volume went offline, exacerbating the crash.

Strategies for Surviving Spikes

Horizontal scaling: Partition by security symbol. Each instance handles orders for a subset of symbols. Add instances as trading volume grows.

Circuit breakers: Exchanges implement circuit breakers that pause trading if movement exceeds thresholds. This prevents cascading failures.

Rate limiting: Gateways enforce per-client rate limits. A rogue algorithm sending thousands of orders per second is throttled.

Backpressure: If the matching engine cannot keep up, the gateway slows down order acceptance. Messages accumulate in the network buffers rather than in application memory.

Durability and Recovery

Every order must survive a system crash. The write-ahead log (WAL) ensures this:

  1. Each order is written to the WAL before being processed
  2. The WAL is replicated to at least one other data center synchronously
  3. On recovery, the system replays the WAL to reconstruct the order book state

Recovery time objective (RTO): Typically seconds for high-frequency trading systems.

Recovery point objective (RPO): Zero — no orders can be lost.

Key Takeaways

  1. Trading systems require sub-millisecond latency and zero tolerance for data loss.
  2. The order book uses priority queues to maintain buy and sell orders.
  3. The matching engine pairs buy and sell orders according to price-time priority.
  4. Kernel bypass, lock-free data structures, and CPU pinning are essential for ultra-low latency.
  5. Write-ahead logs provide durability with minimal performance impact.
  6. Circuit breakers, rate limiting, and backpressure prevent system collapse during traffic spikes.

Design principle: Latency is not a feature — it is a hard requirement. Every microsecond of added latency is real money lost.

Published by:

Iman Suparman