Skip to content

Architecture

See also: Quickstart · Config Overview · Concepts

The Tick Backtest Research Stack separates configuration, data ingestion, execution, and analysis so that every run can be reproduced or extended without code changes.

Performance & design rationale

Tick ingestion is vectorised via PyArrow, yet each pair runs through a sequential signal/backtest loop to avoid lookahead bias. This design still hits ~8 million ticks/minute/core on an AMD 5950X while keeping every run fully auditable through manifests, logs, and environment snapshots.

High-Level Flow

flowchart LR
    YAML[Versioned YAML Configs] -->|parse| ConfigParsers
    ConfigParsers -->|dataclasses| Coordinator
    DataFeed[Tick Data Feed] -->|ticks| Validator
    Validator --> BacktestLoop
    MetricsManager --> BacktestLoop
    SignalGenerator --> BacktestLoop
    BacktestLoop --> Trades[Trades Parquet]
    BacktestLoop --> Logs[NDJSON Logs]
    BacktestLoop --> Manifest[Manifest.json]
    Trades --> Analysis[Analysis Utilities]
    Manifest --> Analysis
  1. Configuration Parsing - User-supplied YAML files and packaged starter templates are validated and converted into immutable dataclasses (BacktestConfigParser, MetricsConfigParser, StrategyConfigParser). The public packaged templates live under src/tick_backtest/config/templates/; checkout-only fixtures under src/tick_backtest/config/ remain a repository development surface.
  2. Data Feed & Validation - The compiled DataFeed streams ticks per month. Every feed is wrapped in a TickValidator that enforces monotonic timestamps, finite bid/ask/mid values, and non-negative spreads.
  3. Metrics & Signals - MetricsManager instantiates all enabled metrics and computes rolling indicators. SignalGenerator combines entry engines with predicate evaluation to produce open/close instructions. The built-in threshold_reversion entry engine is an intentional exception: it owns a private ThresholdReversionMetric instance inside the strategy layer instead of sourcing that state from MetricsManager.
  4. Backtest Coordinator - BacktestCoordinator iterates each configured pair, manages output directories, and records pair-level failures without aborting the entire batch.
  5. Backtest Loop - For each tick, metrics update, signals evaluate, positions open/close, and trades append to an in-memory ledger. On completion, trades persist to Parquet.
  6. Artefact Snapshot - Every run writes a manifest, environment snapshot, logs, and trade artefacts under the configured output_base_path/<RUN_ID>/. Post-run reporting and multivariate diagnostics are written later beside trades.parquet when tick-backtest report or tick-backtest analyze is invoked.

Key Components

Module Purpose
tick_backtest/backtest/workflow.py Entry point that loads configs, prepares output folders, runs backtests, and persists manifests.
tick_backtest/backtest/backtest_coordinator.py Coordinates per-pair runs, handles data feed errors gracefully, and tracks tick-validation stats.
tick_backtest/backtest/backtest.py Core execution loop handling warmup, tick processing, trade lifecycle, and persistence.
tick_backtest/data_feed/data_feed.py Compiled data feed loader with Python fallback. Provides month-by-month tick iterators.
tick_backtest/data_feed/validation.py Tick validation, issue tallying, and feed wrappers to skip invalid ticks.
tick_backtest/metrics/manager/metrics_manager.py Builds metric instances from config and updates them per tick.
tick_backtest/signals/signal_generator.py Wires entry engines and predicates into runtime signals.
tick_backtest/signals/entries/threshold_reversion.py Strategy-owned threshold-reversion engine with its own private indicator state.
tick_backtest/analysis/* Reporting utilities that derive Markdown summaries, equity curves, and stratification studies.

Resilience & Reproducibility

  • Immutable Outputs - Each run is isolated to a timestamped directory with config snapshots hashed via SHA256.
  • Failure Isolation - Pair-level errors (missing data, runtime exceptions) are captured in the manifest and logs; remaining pairs continue.
  • Environment Capture - environment.txt stores the result of pip freeze to document dependencies.
  • Tick Validation - Invalid ticks increment counters and are skipped, keeping the backtest deterministic without aborting.

Refer to Developer Notes for module dependencies, testing strategy, and extension guidelines.