Procure-to-pay · finance workload

triage.ap-inbox.v1

Sort 15 batches of 20 synthetic accounts-payable emails into invoice, statement, dunning notice, PO response, or other.

No published results for this workload yet — results appear here with the next edition.

Rate-card unit

Sort 1,000 AP emails — rates on the rate card are the cost per successful outcome scaled to this unit.

Pass criteria

Batch accuracy ≥ 95% — at least 19 of 20 emails in the batch correctly classified. Grading is deterministic; a model output either passes or fails, and infrastructure errors are excluded and rerun rather than counted against the model.

Prompt (verbatim)

Every model receives exactly this prompt with the item's fields substituted for the {placeholders} — no model-specific tuning.

You are sorting an accounts-payable inbox. Classify each email below into
exactly one of these categories:
  invoice — a bill requesting payment for goods or services
  statement — a periodic statement of account summarizing balances and open items
  dunning_notice — a reminder or escalation about an overdue payment
  po_response — a supplier acknowledgement or confirmation of a purchase order
  other — anything else

Respond with ONLY a JSON object mapping each email id to its category, e.g.
{{"b99_e00": "invoice", "b99_e01": "other"}}. Include every email exactly once.

Emails:
{emails}

Dataset

Datasets are generated synthetically from seeded generators; the seed is frozen per edition and held out until the run window closes. Generator source hash (SHA-256): fb4480968c4f21f98eb29d9676d72fff43490bf2bb1a2b86d01d04347d1eeb35. See the methodology §7 for the verifiability protocol and any edition page for the raw graded logs.