ovg-project / kvcached
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
Why this job: topics: ollama, llm-inference, vllm, inference-engine, llm-serving
Get started
pip install kvcached --no-build-isolationYou need: Python (tested with 3.10 - 3.13); PyTorch >= 2.10 (kvcached builds against the PyTorch stable ABI); SGLang (tested with v0.5.20) or vLLM (tested with v0.30.0)
- Maintenance
- Active
- Last commit yesterday
- Stars
- 2,034
- +17 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest version
- v0.1.6
- Released 37 days ago
- Built with
- Python
- 86% of the code
Is it looked after?
- Contributors
- 21
- Last 90 days
- Issues
- 67% closed
- 52 still open
- Pull requests
- 345 merged
- 53 waiting
- Releases
- 6
- 4 in the last year
- No contributing guide
- No security policy
- No code of conduct
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| liguodongiot / llm-action | 25,266 | 26 days ago | Apache-2.0 | No downloads |
| kvcache-ai / Mooncake | 7,006 | today | Apache-2.0 | Linux |
| gpustack / gpustack | 6,004 | today | Apache-2.0 | No downloads |
More: languages, topics and project record
Languages
- Python 86%
- C++ 11%
- Shell 3%
- Makefile <1%
Topics
- gpu-sharing
- kvcache
- llm
- sglang
- vllm
- kvcached
- inference-engine
- gpu-mutiplexing
- kvcache-optimization
- llm-framework
- llm-inference
- llm-serving
Project record
- JobLocal LLMs
- StatusFeatured
- Commits347
- Forks239
- Watching62
- Stars in 28 days+430
- CreatedMay 27, 2025
- First seen by usNov 7, 2026
- CheckedNov 9, 2026