Tag: ggml

Local LLM inference engine in plain C/C++ with a command-line chat and an OpenAI-compatible HTTP server; pulls and runs GGUF models straight from Hugging Face. Supports Apple Silicon, CUDA, Vulkan and other hardware, plus low-bit quantization.

C/C++ port of OpenAI's Whisper speech recognition model that transcribes audio offline and outputs subtitles. Runs on CPU, with Apple Silicon and various GPU acceleration. Download a ggml-format model first.

Diffusion model inference in pure C/C++: one sd-cli command runs SD, FLUX, Qwen Image, Wan and more locally to create or edit images and short videos, with Metal acceleration on Mac. No cloud needed; bring your own model weights and use a release binary or build from source.