withcatai / node-llama-cpp
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level
Why this job: topics: llama-cpp, gguf, llama, llama.cpp
What it does
- Run LLMs locally on your machine
- Metal, CUDA and Vulkan support
- Pre-built binaries are provided, with a fallback to building from source _without_ node-gyp or Python
- Adapts to your hardware automatically, no need to configure anything
- A Complete suite of everything you need to use LLMs in your projects
Get started
npx -y node-llama-cpp chat- Maintenance
- Active
- Last commit 4 days ago
- Stars
- 2,201
- +4 this week
- Licence
- MIT
- Allows business use
- Downloads for
- 3 systems
- Mac, Windows, Linux
- Latest version
- v3.22.3
- Released 4 days ago
- Built with
- TypeScript
- 91% of the code
Is it looked after?
- Contributors
- 5
- Last 90 days
- Issues
- 83% closed
- 35 still open
- Pull requests
- 258 merged
- 15 waiting
- Releases
- 143
- 21 in the last year
- Contributing guide (opens in new tab)
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| wici-ai / inferssd | 364 | today | Apache-2.0 | Mac, Windows |
| AtomicBot-ai / atomic-agent | 3,340 | 12 days ago | MIT | Mac, Linux |
| raketenkater / ggrun | 283 | 2 days ago | MIT | Windows, Linux |
More: download files, languages, topics and project record
Download files, v3.22.3
- Mac4 files, 1 downloads
- Windows3 files, 2 downloads
- Linux7 files, 5 downloads
Languages
- TypeScript 91%
- C++ 6%
- CSS 2%
- Vue 1%
Topics
- ai
- bindings
- catai
- cmake
- cmake-js
- cuda
- embedding
- function-calling
- gguf
- gpu
- grammar
- json-schema
Project record
- JobLocal LLMs
- StatusListed
- Commits272
- Forks230
- Watching22
- Stars in 28 days+15
- CreatedAug 12, 2023
- First seen by usNov 7, 2026
- CheckedNov 7, 2026