The API
node bin/serve.ts # PORT=8470
GET / in a browser is the homepage — the feed, printed, with start, pause
and stop — and GET /api (or / with Accept: application/json) returns
this vocabulary at runtime, so a client never has to read the
source to find out what exists.
Why there is a socket at all
Not "REST but pushed". Two things genuinely need one:
- A range run is a long job. 139 rides means ~125 Massive pulls and minutes of work. Polling a job endpoint to watch that is worse for both sides than streaming each verdict as it lands.
- Scores arrive on a schedule the client cannot predict. The historical licence ends at the prior session's close, so the nightly run fires after midnight ET and cards appear then. A desk UI should be told, not made to ask.
Everything else is REST, because everything else is a question with an answer rather than a stream of events.
Auth
Three scopes. read (scores, the tape's manifest and objects, job
status) and replay (the socket) are public: no key needed. Evan,
2026-09-04: "for the public site, don't require an API key … but don't let
it run for more than 5 mins" — so a replay without a key ends itself after
five minutes of wall clock (ANON_REPLAY_MINUTES; 0 lifts it) with an
end frame that says so and names the tape instant to reconnect from.
score — starting a scoring job, scoring a ride on demand, warming the
cache: anything that spends vendor time or disk — needs a key, and a key
lifts the replay cap.
Keys are pm_…, minted on the box with node bin/api-keys.ts create --name …,
stored as a SHA-256 hash in this service's own Postgres, shown once. A client
sends Authorization: Bearer pm_… (or X-API-Key), and ?key=pm_… on a
WebSocket. A key that is presented is always checked, so a typo never
silently downgrades to anonymous. scopeFor() in src/auth.ts maps routes
to scopes and defaults an unlisted route to read; API_PUBLIC_SCOPES
(default read,replay; none for nothing) says which scopes need no key.
API_AUTH=off disables all of it for a loopback-bound laptop; with auth on
and no store the server refuses to start. The user-facing walkthrough is
using the API.
REST
GET / |
a browser gets the homepage: the replay feed with start, pause and stop; a JSON client gets the vocabulary — open |
GET /api |
the vocabulary, always — open |
GET /me |
the key you presented, and its scopes |
GET /health |
liveness, which of store / Massive / ledger are configured, and last_nightly — the newest nightly run's day, counts and error, so a silent night is a fact a sentry can read |
GET /rides/:id/score |
score one ride now — hits Massive, takes seconds |
GET /scores?from&to&limit |
stored scores, newest version of each ride |
GET /scores/:rideId |
the newest stored score for one ride |
GET /aggregate?by=lane|grade|source|book|trader|cohort|refusal|symbol&from&to |
the stored scores folded into groups — n, scored, refused by reason, the three lines summed and averaged, wins, sample_ok (n ≥ 30) on every row. The desk's learning loop (tradingdesk#900) reads this instead of joining rows |
POST /score/range {from,to} |
start a job; returns 202 and a jobId |
GET /jobs |
jobs this process knows about |
GET /jobs/:id |
one job, with its scores and totals |
GET /ticks/manifest.json |
the replay tape's coverage — per symbol, the session dates on file, rows per channel, first and last ts |
GET /ticks/day=YYYY-MM-DD/symbol=XXX/ticks.ndjson.gz |
one symbol-day of trades, quotes and status: the same bytes as the S3 object |
GET /ticks/day=YYYY-MM-DD/symbol=XXX/manifest.json |
that day's own summary and sha256 |
The three /ticks routes are the "plain HTTPS returning the same object"
half of the replay tape's delivery, for a box
without bucket credentials. A day is served only when it is whole.
POST /ticks/warm {symbols:[…], day} pulls those objects into the local
cache so a replay starts without waiting on S3.
The replay socket
ws://<host>/replay?symbols=AAPL,SPY&day=2026-09-02&from=09:25&to=16:05&speed=1
The archive played back: the same rows, one message each, in play order, with the wall clock advancing at the tape's rate. What slowsocket's socket would have delivered that day. Evan, 2026-09-04: "this should serve its data as a websocket."
| param | |
|---|---|
symbols |
comma list; several symbols on one day are merged by ts |
day |
YYYY-MM-DD, an ET session date |
from, to |
HH:MM New York or an ISO instant. Default 04:00–20:01. Set from: at 1x from the top, the open is five and a half hours away |
speed |
1 real time, 10 ten times, 0 as fast as the socket drains (with backpressure) |
Frames, in order:
| frame | |
|---|---|
hello |
the vocabulary |
loading |
one per symbol, while its object is fetched from S3 into the local cache (skipped when warm) |
snapshot |
one per symbol: the last quote, trade and status standing at from, so a client that joins at 09:25 starts with a book |
ready |
the clock is pinned: from is now. Carries each day's row counts and source |
| rows | each has channel and no type; the exact row from the archive, columns in the spec's order |
clock |
acknowledgement of a control frame: {speed, paused, tape} |
end |
{sent, lastTs, counts} — counts is per symbol, per channel, and equals the manifest's when from..to spans the day — then the server closes |
error |
e.g. AAPL 2026-09-09 is not on file |
Control, client → server: {op:"pause"}, {op:"resume"},
{op:"speed", speed:10}. Each re-pins the clock at the tape instant it
arrives, so a change never replays or skips a row.
Nothing is held in memory: rows are streamed from the gzipped object with a bounded lookahead, and decoding runs a hundred times faster than real time. The only wait is the S3 fetch, once per symbol-day per box.
POST /score/range is a job, not a request/response, because scoring hits
a vendor. A caller who wants an answer now reads a stored score; a caller who
wants a fresh one starts a job and listens.
WebSocket
ws://<host>/ws. On connect the server sends hello with the event list.
| event | when |
|---|---|
job.started |
{jobId, rides, from, to} |
job.progress |
{jobId, done, total} — after each symbol's tape is consumed |
score |
one ride's verdict, the moment it is computed. jobId is null for an on-demand /rides/:id/score |
job.finished |
{scored, refused, totals} |
job.failed |
{error} |
A real run, streamed:
socket open
hello: job.started, job.progress, score, job.finished, job.failed
POST /score/range -> 202 {"jobId":"6bf0f56b-…"}
job.started 17 rides, 2026-09-02..2026-09-02
score 64 CRWD mech -0.57 real -0.30 delta +0.27 (no door)
score 133 DAKT mech -1.00 real -1.08 delta -0.08 (stop)
score 118 GPRO REFUSED: no_exit
score 137 GPRO mech +0.22 real +0.06 delta -0.17 (no door)
score 127 LHAI REFUSED: entry_unverified
score 124 PCG mech -0.01 real -0.52 delta -0.51 (no door)
…
job.finished scored 14, refused 3
Refusals stream like any other verdict. A ride that cannot be scored honestly is an event a client needs, not an omission to notice later.
Persistence
STORE_DATABASE_URL is optional, and GET /health says which mode the
service is in:
- set — every score is written to this service's own Postgres, and
/scoresworks. - unset — scores are computed and streamed but kept nowhere. Fine for driving the scorer by hand; not fine for anything that depends on it.
Scores are immutable and versioned. putScore inserts a new version and
never updates a row: a score is a claim about what a plan would have done given
specific inputs, so when the cost model improves or the bars are re-pulled, the
fix is a new version. Silently changing one makes every report that quoted it
unreproducible.
The write boundary
This service has no write path into the desk's ledger, structurally rather
than by convention: the only client pointed at tradingdesk is opened in
ledger.ts and used for reads, and nothing in server.ts can reach it.
STORE_DATABASE_URL must never point at the desk's database.
Not yet
- No auth. Bind it to the tailnet or put it behind something before it leaves a laptop.
- Jobs live in memory — a restart forgets them. The scores survive if a store is configured; the job records do not.
- No schedule. The nightly run that the socket's second reason describes does not exist yet; jobs are started by hand.
The aggregate, and the keys a score is grouped by
/aggregate is one fold over the stored rows, grouped on keys the ride
carried in from the desk (contract.ts → RideGroup): lane (the desk's
strategy family, folded by laneOf), grade and source (the candidate
the ride came from), book, trader (null until the desk's attribution
lands), and cohort — experiment:arm, stamped by the desk when a ride runs
under a gavelled proposal's holdout, so the holdout and control arms are two
groups. The keys are deliberately not part of Score: the arithmetic never
reads them, and a fixture without them still scores. Every key is nullable and
a null groups under unknown, printed last.
Sums and means are over scored rides; refused rides are counted by reason.
sample_ok is scored ≥ 30 and rides beside the numbers rather than
suppressing them — a group of four reports its mean and says it is four.
Realised excludes costs until the cost line lands.