Controls & audit · finance workload

audit.expenses.v1

Audit 40 expense lines against a company T&E policy whose limits vary per scenario — meal per-diems, receipt thresholds, hotel nightly caps, mileage rates, prohibited categories, and EUR conversions — flagging 12 planted violations by rule id, with boundary-exact amounts as clean traps (15 scenarios).

No published results for this workload yet — results appear here with the next edition.

Rate-card unit

Audit 1,000 expense lines — rates on the rate card are the cost per successful outcome scaled to this unit.

Pass criteria

100% recall on the planted violations, each with the correct rule id, with at most one false positive. Grading is deterministic; a model output either passes or fails, and infrastructure errors are excluded and rerun rather than counted against the model.

Prompt (verbatim)

Every model receives exactly this prompt with the item's fields substituted for the {placeholders} — no model-specific tuning.

You are auditing employee expense reports against the company travel &
expense policy below. Apply the policy exactly as written — the limits vary
by company, so read them from the policy, and remember that amounts exactly
at a limit are compliant.

{policy}

Respond with ONLY a JSON array, one object per violating expense line with
the rule id, e.g. [{{"expense_id": "EXP-99-001", "rule": "R2"}}]. Do not
include compliant lines. Report at most one rule per line, applying the
precedence stated in the policy.

Expense lines (CSV; nights is set for hotel lines, miles for mileage lines):
{expenses_csv}

Dataset

Datasets are generated synthetically from seeded generators; the seed is frozen per edition and held out until the run window closes. Generator source hash (SHA-256): ab051eb223dc756be5550525c500188216031a4fb4132f4f71300860a5d0bd2e. See the methodology §7 for the verifiability protocol and any edition page for the raw graded logs.