FROM EVIDENCE TO ENDPOINT

Your task. A living leaderboard.

Follow the whole process, from your first example to a ranking that keeps evolving.

01 / CREATE

Define what a good answer looks like.

Start with your task, a prompt and a way to judge the result.

  • Name your leaderboard and describe the task.
  • Choose exact match, an LLM judge or human evaluation.
  • Use one evaluation setup to compare your candidates.
Read the create guide ↗

Leaderboard setup

Illustrative setup
YOUR TASK
Support triage
TASK PROMPT
Classify each support request as Billing, Access, Technical or Other.
SCORING
Exact match · LLM judge · Human evaluation

Same task. Same criteria. A comparison you can inspect.

Create a leaderboard
POST https://app.drgero.ai/api/leaderboards

Request template.

02 / CONNECT

Bring the evidence from your agent.

Turn real inputs and reference answers into a dataset for evaluation.

  • Send input/output examples through the API.
  • Import a labeled Hugging Face dataset.
  • Supply or review reference answers before evaluating.
Read the connect guide ↗

Your evaluation dataset

Illustrative setup
INPUT
I was charged twice this month.
REFERENCE ANSWER
Billing
DATASET VERSION
Examples checked and ready for evaluation

Capture agent I/O or import labeled examples.

Connect your data
POST https://app.drgero.ai/v1/leaderboard/{id}/dataset/push

POST https://app.drgero.ai/api/leaderboards/{id}/dataset/upload

Request template · Replace the identifiers with your own.

03 / AUTO SELECT

Find candidates that fit your task.

Narrow the model pool using your quality, cost and latency preferences.

  • Set the requirements that matter for your application.
  • Filter the model catalog to a manageable shortlist.
  • Evaluate the shortlist on your own examples.
Read the auto select guide ↗

Candidate selection

Illustrative setup
STARTING POOL
1,000 models
FILTERS
Task fit → Cost → Latency
SHORTLIST
5 candidates · 995 discarded

Illustrative catalog and shortlist from the walkthrough.

Auto-select candidates
POST https://app.drgero.ai/api/leaderboards/{id}/models/auto-select

Request template · Replace the identifiers with your own.

04 / RANK

Let your data set the order.

Evaluate candidates on the same examples and compare the tradeoffs.

  • Compare task score, response time and cost together.
  • Inspect the evidence behind each ranking.
  • Choose the model that meets your task requirements.
Read the rank guide ↗

Leaderboard results

Illustrative setup
COMPARE
Task score · Latency · Cost
REFERENCE WINNER
Ling 3.0 Flash
EVIDENCE
The same evaluation examples for every candidate

Reference metrics are task-specific; some costs may include estimates.

Run an evaluation
POST https://app.drgero.ai/api/leaderboards/{id}/run

Request template · Replace the identifiers with your own.

05 / DEPLOY

One endpoint. Your routing strategy.

Connect your application to a stable private API and choose how requests are served.

  • Follow the completed, eligible ranking winner.
  • Pin a model when you need a fixed choice.
  • Use a balanced mix to route traffic across models.
Read the deploy guide ↗

Private inference endpoint

Illustrative setup
REQUEST
Your application → Dr. Gero → Eligible model
RESPONSE
Billing

One request, one model’s answer. Your endpoint stays the same.

Private inference endpoint
POST https://app.drgero.ai/v1/leaderboard/{id}/inference

Request template · Replace the identifiers with your own.

06 / FINE-TUNE

Build a specialist for your task.

Optionally train a new model version using curated supervised examples.

  • Prepare training data and start a fine-tuning run.
  • Evaluate the new version on held-out examples.
  • Review and approve it before adding or serving it.
Read the fine-tune guide ↗

Training and review

Illustrative setup
TRAIN
Curated examples + Ministral 3 3B 2512
NEW VERSION
Ministral tuned v1
REVIEW GATES
Evaluate on holdout → Review results → Approve as candidate

Fine-tuning is optional and user initiated.

Start a fine-tuning run
POST https://app.drgero.ai/api/models/{model_id}/fine-tune/run

Request template · Replace the identifiers with your own.

07 / EVOLVE

Keep the ranking current.

Refresh your comparison as new evidence and approved candidates arrive.

  • Schedule re-evaluation when your dataset changes.
  • Approve tuned candidates and remove others from the comparison.
  • Keep serving through the same application endpoint.
Read the evolve guide ↗

An evolving leaderboard

Illustrative setup
NEW EVIDENCE
1,000 new examples → Re-evaluate
APPROVED CANDIDATES
Tuned Ministral and tuned Nemo join after review
APPLICATION
Same URL · Follow the completed, eligible winner

Later ranking changes are illustrative. Candidate changes require your approval.

Update your leaderboard
PATCH https://app.drgero.ai/api/leaderboards/{id}

Request template · Replace the identifiers with your own.

Start with your task.

Build your first leaderboard and put your models to the test.

Start free ↗