Create a leaderboard
Describe your task and choose how to score the answers.
Start with one real task.
Your agent
Input / output events
via the ingestion API
Hugging Face
Import a dataset
with reference labels
Capture agent I/O or import labeled examples.
1,000 models. Five for your task.
Your task and preferences guide the shortlist.
Same examples. Measurable trade-offs.
Your application
One integration
One endpoint
Dr. Gero / private inference
inclusionAILing 3.0 Flash
Eligible ranking winner
Your app calls one URL. Dr. Gero routes the request.
Train for your task.
Train a custom version, then check it on held-out examples.
Curated examples
Supervised data
+ Ministral 3 3B 2512
Fine-tune
Preparing training data
Ministral
tuned v1
A new version
to evaluate
Evaluate and review the new version before changing serving.
New evidence. New order.
POST /v1/leaderboard/{id}/inferenceSame URL · serving Ling 3.0 FlashInitial model names and metrics reproduce a reference ranking; * costs and derived value ratios may include estimates. The workflow, 1,000-model catalog and later ranking changes are illustrative. Jobs are compressed; you approve candidate changes. Fine-tuning is optional.
Space play / pause ← → steps
The seven steps, in words
- Create a leaderboardDefine your task and choose exact match, an LLM judge or human evaluation.
- Connect your agent’s I/OSend input/output examples through the API or import a Hugging Face dataset. Supply or review reference answers.
- Auto SelectNarrow an illustrative catalog of 1,000 models to the five shown in the supplied ranking. Your task, cost and latency preferences guide selection.
- Rank the leaderboardEvaluate candidates on the same examples and compare quality, cost and response time.
- Serve one endpointConnect your application to a stable private API. Choose an eligible winner, a pinned model or a balanced routing mix.
- Fine-tune a modelOptionally train on curated examples. Evaluate the new version on held-out data and review it before adding or serving it.
- Keep the ranking currentSchedule re-evaluation as data changes. Approve new candidates and remove others from the comparison. The five-model ranking evolves while your application’s endpoint stays the same.