DaoyuanLi2816 / pairjudge
Pairwise LLM judges (A/B/tie): budget-aware multi-turn packing, position-bias correction, pseudo-label distillation. Generalized from the 4th-place (gold) solution to Kaggle LMSYS Chatbot Arena.
Why this job: says: distillation; topics: lora, rlhf
What it does
The methods originated in the 4th-place, gold-medal solution to LMSYS — Chatbot Arena Human Preference Predictions. The new 0.5B example is a small, separately evaluated model; the competition medal is not its quality certification.
Get started
python -m pip install "pairjudge[judge]==0.3.0"
pairjudge compare --prompt "Why does ice float?" --a "Its crystal structure lowers its density." --b "Cold objects always rise." --swap- Maintenance
- Active
- Last commit 36 days ago
- Stars
- 170
- On GitHub
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.3.0
- Released 36 days ago
- Built with
- Python
- 81% of the code
Is it looked after?
- Contributors
- 1
- Last 90 days
- Issues
- None
- No issues filed
- Pull requests
- 22 merged
- 1 waiting
- Releases
- 4
- 4 in the last year
- Contributing guide (opens in new tab)
- Security policy (opens in new tab)
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| Goekdeniz-Guelmez / MLX-LoRA-Studio | 287 | 13 days ago | MIT | Mac |
More: languages, topics and project record
Languages
- Python 81%
- Jupyter Notebook 19%
Topics
- chatbot-arena
- gold-medal
- kaggle-competition
- kaggle-solution
- llm
- llm-as-judge
- lora
- nlp
- preference-learning
- reward-model
- rlhf
Project record
- JobFine-tuning
- StatusListed
- Commits68
- Forks12
- Watching15
- Stars in 28 days0
- CreatedSep 17, 2024
- First seen by usNov 7, 2026
- CheckedNov 7, 2026