minghinmatthewlam / openbench
Same model, different wrapper: a from-scratch benchmark comparing coding-agent harnesses (codex, pi, opencode, cursor, devin) and open models on correctness, speed, and token cost
Why this job: says: benchmark; topics: evals, benchmark
What it does
Each harness runs headlessly against a set of self-contained coding tasks. A task is graded by a checker script (exit 0 = solved, optional SCORE: for partial credit), never by the harness's own claim of success.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit 2 months ago
- Stars
- 137
- On GitHub
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v1.0.0
- Released 3 months ago
- Built with
- Python
- 61% of the code
Is it looked after?
- Contributors
- 1
- Last 90 days
- Issues
- 0% closed
- 7 still open
- Pull requests
- 34 merged
- 7 waiting
- Releases
- 2
- 2 in the last year
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| GitHamza0206 / simba | 1,547 | 4 months ago | Apache-2.0 | No downloads |
| overmind-core / overmind | 754 | today | AGPL-3.0 | No downloads |
| laiso / ts-bench | 243 | 2 months ago | MIT | No downloads |
More: languages, topics and project record
Languages
- Python 61%
- G-code 33%
- Shell 2%
- Scheme 2%
Topics
- ai-agents
- benchmark
- coding-agents
- evals
- llm
- open-models
Project record
- JobEvals and observability
- StatusListed
- Commits903
- Forks10
- Watching0
- Stars in 28 days+1
- CreatedJun 19, 2026
- First seen by usNov 7, 2026
- CheckedNov 7, 2026