Helldez / BigMoeOnEdge
Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.
Why this job: says: llama.cpp; topics: llama-cpp, gguf, inference, on-device-ai, edge-ai
What it does
- Runs models that do not fit. A MoE model uses a few experts per token. The engine keeps the
- Sets itself up. --auto measures the machine and the model and picks the cache size, the
- Any MoE GGUF. Whatever llama.cpp loads, in any quantization. No per-model code.
- Every backend. CPU, Metal, CUDA, HIP, Vulkan, OpenCL and the Hexagon NPU. On a graphics card,
- Apps and a server. A desktop app, phone apps, and an OpenAI- and Anthropic-compatible server
Get started
curl -fsSL https://github.com/Helldez/BigMoeOnEdge/releases/latest/download/install.sh | sh - Maintenance
- Active
- Last commit yesterday
- Stars
- 734
- +31 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- 3 systems
- Mac, Windows, Linux
- Latest version
- v0.40.2
- Released yesterday
- Built with
- C++
- 80% of the code
Is it looked after?
- Contributors
- 3
- Last 90 days
- Issues
- 81% closed
- 29 still open
- Pull requests
- 312 merged
- 11 waiting
- Releases
- 58
- 30+ in the last year
- Contributing guide (opens in new tab)
- Security policy (opens in new tab)
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| wici-ai / inferssd | 364 | today | Apache-2.0 | Mac, Windows |
| xorbitsai / inference | 9,647 | yesterday | Apache-2.0 | No downloads |
| carloslfu / slotstream | 659 | today | MIT | No downloads |
More: download files, languages, topics and project record
Download files, v0.40.2
- Mac2 files, 5 downloads
- Windows7 files, 52 downloads
- Linux6 files, 27 downloads
Languages
- C++ 80%
- TypeScript 7%
- Svelte 6%
- Python 4%
Topics
- android
- cpp
- edge-ai
- gguf
- inference
- llama-cpp
- llm
- mixture-of-experts
- moe
- on-device-ai
- qwen
- gemma
Project record
- JobLocal LLMs
- StatusListed
- Commits559
- Forks87
- Watching7
- Stars in 28 days+117
- CreatedJul 10, 2026
- First seen by usNov 7, 2026
- CheckedNov 9, 2026