Quickstart timing (§578)

Master plan §578: “5-minute quickstart timed by non-author every release; >5 min blocks release.”

This page is the runbook for that process. It’s a release gate, not a one-time check.

Why

The quickstart is the only docs page that’s load-bearing for adoption. If it takes longer than 5 minutes to get a working widget on a real page, every other improvement to the platform is wasted on the people who bounce in step 3. So we treat the timing as a release-blocking quality bar, not a vibe check.

A non-author times it because the author has already memorized it. Five minutes of real fumbling beats five minutes of muscle memory every time.

Cadence

Trigger: any release that touches packages/widget, packages/sdk, the dashboard onboarding flow, or the /quickstart/ page itself.
Freshness: the timed run must be against the release candidate that’s about to ship — not main, not last week’s build.
Frequency: at minimum every release; we re-time monthly even on quiet stretches because dashboard UX shifts in PRs that don’t touch the docs.

Procedure

Pick a non-author. Anyone on the team who didn’t write the docs in the last 30 days. Prefer someone who hasn’t onboarded recently — fresh eyes find more.
Provision a clean tenant. New email, new Clerk signup, new Stripe customer. No reusing the team test account.
Open /quickstart/ in one tab and a stopwatch in another. Start the clock when they hit the page. Stop the clock when they see the streamed reply in the widget on a real page they own.
Record the time, the path, and any stumble. Even a 4:30 run with a 90-second pause on step 3 is a docs bug — note it.
>5:00 = release blocked. Either the docs change or the release waits. Pick one.

What counts

Counts toward the 5 min	Doesn’t count
Reading docs	Waiting for Stripe webhook to provision tenant (we measure this separately)
Filling forms	Email verification (Clerk)
Running curl	DNS for the test page domain
Copying snippets	Account recovery
Page reloads
Debugging an honest mistake the docs caused	Debugging a typo the timer made on their own

The line is “would a real new user blame us for this delay?” If yes, it counts.

Recording the run

Each timed run gets a row in the quickstart-timing log (Notion):

Date | Tester | Release tag | Time | Stumbles | PR fixing

The “PR fixing” column is required if Time > 5:00 — it’s the link to the docs fix that unblocks the release.

Why not just a CI test

We also have a Playwright E2E suite that exercises the same flow end-to-end. It catches functional regressions (the widget doesn’t load, the API rejects the session). It doesn’t catch the things that matter here:

The dashboard form labels are confusing.
The curl example doesn’t say where $CORTEX_API_KEY comes from.
The error message when origin doesn’t match is technically accurate but unhelpful.

A clock and a stranger are the only honest measure of those.