Record-to-report · finance workload

reconcile.ledger.v1

Reconcile a month of 200 bank transactions against a shuffled general-ledger extract containing 12 planted discrepancies: missing entries, duplicates, amount mismatches, and date drift (15 monthly scenarios).

No published results for this workload yet — results appear here with the next edition.

Rate-card unit

Reconcile a month of transactions — rates on the rate card are the cost per successful outcome scaled to this unit.

Pass criteria

100% recall on the planted discrepancies, each with the correct type, with at most one false positive. Grading is deterministic; a model output either passes or fails, and infrastructure errors are excluded and rerun rather than counted against the model.

Prompt (verbatim)

Every model receives exactly this prompt with the item's fields substituted for the {placeholders} — no model-specific tuning.

You are reconciling a month of bank transactions ({period}) against the
general ledger.

Every bank transaction should have exactly one GL entry with the same txn_ref,
the same date, and the same amount. Descriptions may differ and GL row order is
arbitrary. Find every discrepancy and classify it with one of these types:
  missing_in_gl — the bank transaction has no GL entry
  duplicate_in_gl — the bank transaction has more than one GL entry
  amount_mismatch — the GL amount differs from the bank amount
  date_drift — the GL date differs from the bank date

Respond with ONLY a JSON array, one object per discrepant bank transaction,
e.g. [{{"txn_id": "BNK-99-0001", "type": "missing_in_gl"}}].

Bank feed (CSV):
{bank_csv}

General ledger extract (CSV):
{gl_csv}

Dataset

Datasets are generated synthetically from seeded generators; the seed is frozen per edition and held out until the run window closes. Generator source hash (SHA-256): e2cea419b0b5851fbb1039a11aad9fc1c41f35207b4c19aaaaccc0ba42176963. See the methodology §7 for the verifiability protocol and any edition page for the raw graded logs.