Prototype on the stream
Use server-side filters to sample high-premium or short-dated flow without drowning notebooks in every print.
Design flow features on the live WebSocket, then prove them on the same field schema in historical SQL — avoiding train/serve skew between research and production.
Who this is for: quant researchers who need both a live options tape (~10M/day) and a SQL warehouse (2.8B+ trades from Feb 2025+) with consistent Greeks/IV/premium semantics.
Keep one definition of premium, delta bands, and DTE filters across stream and SQL.
Use server-side filters to sample high-premium or short-dated flow without drowning notebooks in every print.
Persist minute bars or event features (premium, |delta|, IV) with the same formulas you will run live.
Replay multi-week windows via historical SQL with row/timeout guardrails appropriate to research jobs.
Promote filters to production WebSocket params and monitor drift against the SQL baseline.
-- Promote a live flow feature to historical validation
SELECT
toStartOfMinute(timestamp) AS minute,
sumIf(size * price * 100, sentiment = 'BULLISH') AS bull_prem,
sumIf(size * price * 100, sentiment = 'BEARISH') AS bear_prem,
avg(abs(delta)) AS avg_abs_delta
FROM RawOptionTrades
WHERE date = today() AND symbol = 'SPY' AND premium >= 50000
GROUP BY minute
ORDER BY minute;Layer flow, quant hub, and UOA-for-quant paths.
Yes. Many studies start in SQL. The layered value is promoting the same features to WebSocket without redefining fields.
Yes. AGGREGATED merges same-contract same-instant prints; RAW keeps every exchange print. Pick one mode and keep it consistent from backtest to prod.
OptionData stores option trades from February 2025 onward — 2.8B+ rows at time of writing — queryable via ClickHouse SQL over REST.
Greeks and IV are analytics estimates under model assumptions. Use them as features, not as guaranteed P&L.