Home › LLM Tools › shimmy

shimmy

by Michael-A-Kuykendall · github.com/Michael-A-Kuykendall/shimmy

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

Stars
★ 5.7k
Forks
544
Language
Rust
Category
LLM Tools
License
Apache-2.0
api-servercommand-line-tooldeveloper-toolsggufhuggingfacehuggingface-modelshuggingface-transformersinference-serverllamallamacppllm-inferencelocal-airustopenai

View on GitHub ↗

Featured in