jpezzulli / sglang-rtxpro6000
Optimized SGLang runtime for Qwen3.8-27B FP8 with DFlash2 and Qwen3.8 Flash-Next NVFP4 with FR-Spec on one NVIDIA RTX PRO 6000 Blackwell 96 GB GPU (SM120): 524K context, HiCache and NIXL.
Why this job: topics: local-llm, llm-inference
What it does
Every tool call adds another turn. A coding session can build up hundreds of thousands of tokens of history, and processing that history again costs time.
Get started
No single install command. Read the steps in the README (opens in new tab)
- Maintenance
- Active
- Last commit 3 days ago
- Stars
- 195
- +8 this week
- Licence
- Apache-2.0
- Allows business use
- Downloads for
- None listed
- No release files
- Latest tag
- pennyroyal-v3.1.6
- Released 3 days ago
- Built with
- Python
- 85% of the code
Is it looked after?
- Contributors
- 5
- Last 90 days
- Issues
- 93% closed
- 4 still open
- Pull requests
- 92 merged
- 2 waiting
- Releases
- 22
- 22 in the last year
- Contributing guide (opens in new tab)
- No security policy
- Code of conduct (opens in new tab)
Similar repos
| Repo | Stars | Last commit | Licence | Downloads for |
|---|---|---|---|---|
| kacper-daftcode / vLLM-Moet | 570 | yesterday | Apache-2.0 | No downloads |
| youssofal / MTPLX | 2,685 | 3 days ago | Apache-2.0 | Mac |
| ARahim3 / mlx-dspark | 736 | yesterday | MIT | No downloads |
More: languages, topics and project record
Languages
- Python 85%
- Rust 6%
- Cuda 5%
- C++ 2%
Topics
- blackwell
- dflash2
- fp8
- hicache
- llm-inference
- local-llm
- long-context
- nixl
- nvfp4
- qwen
- qwen3
- qwen3-8
Project record
- JobLocal LLMs
- StatusListed
- Commits17,864
- Forks27
- Watching6
- Stars in 28 days+43
- CreatedAug 25, 2026
- First seen by usNov 7, 2026
- CheckedNov 9, 2026