MMMU-Benchmark / MMMU
This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"
Why this job: says: benchmark; topics: evaluation
What it does
Building upon MMMU, MMMU-Pro introduces even more stringent assessment methodologies to evaluate multimodal models' intrinsic understanding and reasoning capabilities. MMMU-Pro employs a meticulously structured three-step process:
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit 3 months ago
- Stars
- 600
- +1 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- None yet
- No releases
- Built with
- Python
- 88% of the code
Is it looked after?
- Contributors
- Not stated
- Last 90 days
- Issues
- 99% closed
- 5 still open
- Pull requests
- 15 merged
- 1 waiting
- Releases
- None
- Never published
- No contributing guide
- No security policy
- No code of conduct
More: languages, topics and project record
Languages
- Python 88%
- HTML 8%
- Shell 4%
Topics
- computer-vision
- deep-learning
- deep-neural-networks
- evaluation
- foundation-models
- large-language-models
- large-multimodal-models
- llm
- llms
- machine-learning
- multimodal
- multimodal-deep-learning
Project record
- JobEvals and observability
- StatusListed
- Commits157
- Forks57
- Watching4
- Stars in 28 days+4
- CreatedNov 23, 2023
- First seen by usNov 7, 2026
- CheckedNov 8, 2026