Unlock Index

Cost accounting

LLM costs use OpenRouter’s reported usage, not model list prices. For bring-your-own-key (BYOK) calls, the effective cost uses usage.cost_details.upstream_inference_cost; otherwise it uses usage.cost. Latency is client-measured wall time.

The table preserves the values and precision in summary.csv. Judging and extraction costs are per record or document; browser form-fill cost is total run cost divided by successful fills. The voice cost is per one-minute input clip, not per minute of a two-way call. Page-read cost is per usable page. n counts records, documents, form attempts, clips or attempted URLs, respectively.

Judging a record

Fifty fixed records pair company domains and job titles: 25 real primary domains and 25 lookalike or incorrect domains. The title label targets Director, Head, VP, SVP and C-level roles in Sales, RevOps or Marketing. A record is correct only if both the domain-validity and title-fit labels match. Each model receives one JSON-output call per record at temperature zero.

Document extraction

Twenty public PDFs comprise ten invoice2data invoice fixtures and ten SEC 10-K cover pages. Invoice labels use the fixtures’ expected JSON, with invoice numbers checked against PDF text. SEC labels use the structured submissions API, with EIN and fiscal year confirmed in the PDF text.

Four fields are checked per document: invoice number, date, total and currency for invoices; registrant name, incorporation state, EIN and fiscal period end for 10-Ks. Comparisons normalize case, punctuation, Inc/Corp and state names versus codes. PDFs are sent as base64 file content. DeepSeek is excluded because it does not accept file input. These are clean text-layer PDFs, not scans.

Browser form fill

Twenty fixed personas are entered into one local demo-request form with 11 controls, including selects, a date, a conditional partner field and required terms acceptance. In headless Chromium, the model receives a JSON snapshot of visible controls and returns one fill, select, check, click or done action per step, at temperature zero, capped at 25 steps.

Success requires the captured submission to exactly match the expected record, including a blank partner field when none is supplied and accepted terms. Costs sum all model calls in each run. This is one cooperative local form, without CAPTCHA or a live website.

Voice audio pipeline

Five one-minute, 16 kHz mono clips are cut at 90-second intervals from the public-domain LibriVox recording The Four-Fifteen Express. Each clip is sent in one streamed speech-to-speech call to openai/gpt-audio-mini; the reply is about 10–16 seconds of spoken audio.

Quality is a manual check that reply transcripts are on-topic against the story, not word error rate or audio-quality measurement. All five calls use BYOK upstream costs. The supplied voice raw CSV has zero in cost_usd; its nested usage.cost_details.upstream_inference_cost contains the provider charges used by the summary. The original CSV is preserved unchanged.

Reading a page

Apify website-content-crawler ran with playwright:adaptive, 4096 MB and crawl depth zero on 20 fixed public government, documentation and reference URLs. Run eiWTgg1zt3fdLBP6G ran from 10:01 to 10:06 Tbilisi time (06:01–06:06 UTC) on the run date, using 0.36903 compute units (CU). Nineteen requests returned; kansascity.gov failed with a proxy connection refusal. The raw CSV contains the 19 returned URLs, not the failed request.

A page is usable if it returns at least 200 words of extracted text: 15 of 20 attempts met that rule. The other four returned pages had a JavaScript fallback notice, video-player text or short introductory/banner text. Dollars are estimated from measured CU at the Starter list price of $0.20/CU, divided by usable pages. The account’s actual plan rate was not confirmed. Storage and data-transfer charges are excluded because the API tool did not expose them; datacenter proxy is plan-included.