---
title: "The DEC Accounting Benchmark: Testing LLMs on Real Professional Exams"
description: "LFORLA grades LLMs on two official French Chartered Accountancy exams (DEC May 2024) using expert rubrics and an LLM judge — one of the few public benchmarks of regulated professional knowledge in French."
updated: 2026-08-21
url: https://lforla.org/blog/dec-accounting-benchmark-french-accounting
---
# The DEC Accounting Benchmark: Testing LLMs on Real Professional Exams

**The DEC Accounting benchmark evaluates language models on two official exams from the French Chartered Accountancy degree (Diplôme d'Expertise Comptable, DEC, May 2024 session): a 20-question professional ethics paper and a 17-question statutory audit case study. Because the exams publish no answer key, model answers are graded by an LLM judge against expert rubrics.**

This makes it one of the few public benchmarks testing *regulated professional knowledge* — tax law, audit standards, professional deontology — rather than general trivia.

## Why French accounting?

Professional-domain evaluation in non-English languages is rare. The DEC is a standardized, public, high-stakes exam with unambiguous grading criteria (points per key concept), which makes it ideal benchmark material: hard, verifiable, and uncontaminated until publication. Questions remain in their original French by design.

## The two papers

**Paper 1 — Ethics and professional regulation (20 questions)**
Independent questions covering accountancy and statutory-audit deontology: independence rules, confidentiality, engagement letters, professional conduct codes.

**Paper 2 — Statutory and contractual audit (17 questions)**
A practical case study with two case files (ABER-WRAC'H and KECACOUP): audit approach selection, risk assessment, legal compliance of financial statements.

## How scoring works

Each question carries a rubric listing expected key points. Model answers are evaluated on:

- **rubric_score**: conformity with the expected key points.
- **completeness**: coverage of all rubric items.

An LLM judge compares each answer to its rubric; scores aggregate to a 0–100 scale. This follows the established LLM-as-judge methodology for open-ended professional answers where no exact string match exists.

[chart:dec-comptabilite:leaderboard]

## What results show so far

General-purpose frontier models handle definitional ethics questions well but lose points on Paper 2's multi-step audit reasoning — where several rubric points must be addressed in one structured answer. Small fast models drop further on completeness: they answer correctly but briefly, missing secondary rubric points.

[chart:dec-comptabilite:trends]

## For researchers

- Full system prompt: [/prompts/dec-comptabilite.md](/prompts/dec-comptabilite.md)
- Benchmark page with metrics schema: [DEC Accounting](/benchmarks/dec-comptabilite)
- Run it locally:

```bash
lforla-eval pull dec-comptabilite
lforla-eval run --model <your-model>
```

[benchmark:dec-comptabilite]

## Frequently asked questions

**Are the exam questions in English?**
No — the questions are kept in original French to preserve fidelity to the official source. Rubrics and scoring are language-agnostic.

**Is LLM-as-judge reliable here?**
Judge grading follows explicit point-by-point rubrics rather than holistic impressions, and every judged answer stores its raw output so evaluations can be audited.

**Can I use this benchmark to test my fine-tune?**
Yes — that is its main purpose. Pull the dataset, run your model, and report your scores to the public leaderboard.