CoinQuant Research Engine — autonomous sweep

1,200 backtests on real BTC + ETH 2024 data

Multi-service agentic backtester. Natural-language prompt → Pydantic-validated StrategySpec via Claude tool-call → deterministic numpy backtester → SQLite-WAL persistence → ranked top-N. Built for the Senior AI Engineer role at CoinQuant.

live snapshot

loading…

Total cells

—

across 8 sweeps

Wall-clock

—

end-to-end on a laptop

Throughput

—

backtests / sec

Tests passing

25 / 25

0.18s / pytest

Determinism

verified

m1 == m2 bit-for-bit

Compiler

—

NL → typed spec

Natural-language prompt

—

…

symbolsBTCUSDT, ETHUSDT

interval1h

grid15 fast × 10 slow × 4 risk × 2 sym

dataBinance public REST klines

cacheparquet, dead-letter for malformed

Pipeline (every edge typed)

Pydantic v2 contracts

[1]

NL Compiler

Claude tool-call. Pydantic-validated. Bounded repair loop.

[2]

Data Pipeline

httpx + tenacity. OHLC invariant. Dead-letter queue.

[3]

Coordinator

Sweep expansion. Idempotent run_id. Crash-safe pre-write.

[4]

Backtest Pool

Vectorized numpy. Path-aware stops. Cell isolation.

[5]

Run Store

SQLite-WAL. Spec hash. Audit-grade reproducibility.

Determinism contract

audited

✓

Same input ⇒ same output. Two engine runs on the same (spec, bars) produce m1.model_dump() == m2.model_dump() bit-for-bit. No time.time() in the engine. No thread-local randomness.

✓

Same record ⇒ same input. Each RunRecord carries spec_hash = sha256(StrategySpec.json)[:16]. The audit story is: load the spec from the DB, re-run, check metrics match.

✓

Stable run_id. run_id = sha1(sweep_id, sorted(cell_params))[:16]. The pre-write of PENDING and the post-execution upsert target the same row. A crash mid-sweep leaves a recoverable state, never a duplicated one.

✓

Forbid extra fields. Pydantic extra='forbid' on every schema. The LLM cannot inject a field that "looks like it should work" and have it silently propagate downstream.

Failure-mode catalog (snippet)

12 modes total in the PDF

Malformed bar from upstream — Pydantic Bar validator catches NaN / inverted OHLC, routes to data/dlq/<sym>.jsonl. Batch keeps flowing.

LLM emits invalid JSON for StrategySpec — fed back as tool_result.is_error=True, bounded repair loop. If exhausted, deterministic local parser produces a valid spec anyway.

One backtest cell raises in numpy — try/except around engine.run, cell marked FAILED in store, sweep continues. Other cells unaffected.

Orchestrator crashes mid-sweep — pre-write of PENDING records before pool submission. On restart, re-execute only PENDING / FAILED. No double-counted trades.

Top-20 strategies (ranked by Sharpe)

— ranked across 1,200 cells —

#	Symbol	Risk	Fast	Slow	Return	Sharpe	Sortino	Max DD	Trades	Win rate	Spec hash

Sharpe heatmap (fast × slow EMA)

cyan = stronger risk-adjusted return

Top-1 equity curve — BTC EMA(40,50), no stops

— vs buy-and-hold —

Initial capital

$10,000

Strategy final

—

Buy-hold final

—

Strategy edge

—

Time in market

—

Spec hash

—

Strategy equity

Buy-and-hold

In-market periods

Chaos test — validation under attack

50 malformed bars injected; sweep continued

Result

live test

—

malformed bars caught at the boundary

—

catch rate (Pydantic Bar validator)

What this proves. 50 bars across 5 malformation classes (…) were attempted before the engine ever saw them. Every single one was rejected at the typed boundary. The engine cannot consume an impossible bar even if upstream tries to ship one.

Sweep completed. Clean Sharpe (top strategy on the chaos-adjacent slice) recorded as —. The numbers a human sees were never near a malformed input.

Validation errors (sample)

Pydantic v2 messages

Loading chaos-test rejection log…