---
title: "How to Benchmark an LLM with LFORLA (Step-by-Step Guide)"
description: "Benchmark any LLM in three commands with the free lforla-eval CLI: pull a versioned dataset, run the evaluation, publish verifiable results to the public leaderboard with full cost metrics."
updated: 2026-08-21
url: https://lforla.org/blog/how-to-benchmark-an-llm
---
# How to Benchmark an LLM with LFORLA (Step-by-Step Guide)

**You can benchmark any LLM on LFORLA in three commands: install the `lforla-eval` CLI, pull a benchmark dataset, and run the evaluation against your model. Publishing your results to the public leaderboard is one more command — `lforla-eval report`.**

This guide walks through the full workflow, from installation to a published, verifiable leaderboard entry.

## Step 0 — Install the CLI

```bash
npm install -g lforla-eval
```

The CLI works on macOS and Linux. Verify with:

```bash
lforla-eval whoami
```

## Step 1 — Create a free account

Register at [/auth/register](/auth/register) and verify your email. Your account gives you:

- an API key for the CLI (`lforla-eval login`),
- the right to submit runs,
- a public profile page listing your submissions.

## Step 2 — Pick a benchmark

Browse [the catalog](/benchmarks). Each benchmark page documents its metrics, scoring method and system prompt. Good first runs:

- **Team Recruitment (Oracle)** — tests agentic tool use,
- **DEC Accounting** — professional knowledge with LLM-judge grading,
- **PolitiScales Bias** — fast social-bias survey.

## Step 3 — Pull the dataset

```bash
lforla-eval pull recruit-equipe
```

This downloads the versioned JSONL dataset from MinIO-backed storage, so every run uses byte-identical inputs.

## Step 4 — Run the evaluation

```bash
lforla-eval run --model gpt-4o
```

Point it at any model accessible via an OpenAI-compatible API endpoint. The CLI records per-sample outputs, token counts, latency and cost.

## Step 5 — Report to the leaderboard

```bash
lforla-eval report
```

Your run is submitted with its raw metrics and appears as *pending* on your profile until a moderator verifies it. Verified runs feed the public [leaderboard](/leaderboard).

## What happens after submission?

Every published run stores: overall score, per-metric breakdown, sample count, total tokens, average latency and total cost. The leaderboard computes rankings directly from this table — nothing is manually curated.

[chart:fisy:leaderboard]

## Tips for clean results

1. **Fix your temperature** — greedy decoding (temperature 0) makes runs reproducible.
2. **Report real costs** — cost-per-task only helps the community if you let the CLI capture actual token usage.
3. **Run all samples** — partial runs are flagged by sample count.
4. **Publish even mediocre scores** — negative results on small models are exactly what makes efficiency analysis useful.

## Frequently asked questions

**Does benchmarking cost anything?**
LFORLA itself is free; you pay only your own model API costs during the run.

**Can I test my private fine-tune without publishing?**
Yes — runs stay private on your profile until you choose to publish them via `report`.

**Where does the data live?**
Datasets are served from object storage; results live in LFORLA's public PostgreSQL database, exposed read-only through the REST API.