DingoOz / llm-visuals

What it does

It finds the inference servers you already have up (llama.cpp llama-server, ollama, vLLM, SGLang, Strata, Maya, NInfer, TensorFold, …), reads their counters and NVIDIA, AMD or Intel GPU telemetry, and turns them into a truecolor picture of what the model is doing right now: tokens per second, time to first token, GPU load and memory, context fill, speculative-decoding acceptance, which layers are busy on which GPU, and, with a small server patch, exactly which experts a mixture-of-experts model routed the last token through.