mll-lab-nu / embodied-first-exam
The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.
Why this job: says: benchmark; topics: evaluation, benchmark
What it does
In every task, a coding agent writes code, runs it on a robot in a frozen scene from one of the suites, and sees what happens.
Get started
# or without a clone: install from PyPI (or pip install embodied-first-exam);
# the tasks are cloned on first use; API keys: in the shell or a .env in the working directory
uv tool install embodied-first-exam --with-executables-from harbor- Maintenance
- Active
- Last commit yesterday
- Stars
- 240
- +9 this week
- Licence
- MIT
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- 1 releases
- None marked latest
- Built with
- Python
- 82% of the code
Is it looked after?
- Contributors
- 7
- Last 90 days
- Issues
- 96% closed
- 2 still open
- Pull requests
- 435 merged
- 5 waiting
- Releases
- 1
- 1 in the last year
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| PhiloLabs / agentic-vbench | 112 | 31 days ago | Apache-2.0 | No downloads |
| arklexai / arksim | 235 | 9 days ago | Apache-2.0 | No downloads |
| AMAP-ML / LoopArena | 154 | 26 days ago | Apache-2.0 | No downloads |
More: languages, topics and project record
Languages
- Python 82%
- Dockerfile 15%
- Shell 3%
- C <1%
Topics
- ai-agents
- benchmark
- coding-agents
- embodied-ai
- evaluation
- harbor
- llm-agents
- robot-learning
- robotics
- simulation
Project record
- JobEvals and observability
- StatusFeatured
- Commits140
- Forks15
- Watching0
- Stars in 28 days+240
- CreatedSep 18, 2026
- First seen by usNov 7, 2026
- CheckedNov 9, 2026