Tools

Local LLM inference engine in plain C/C++ with a command-line chat and an OpenAI-compatible HTTP server; pulls and runs GGUF models straight from Hugging Face. Supports Apple Silicon, CUDA, Vulkan and other hardware, plus low-bit quantization.

Detects your CPU, memory and GPU and scores open models on whether they run, speed, quality and context, then works with local runtimes like Ollama, llama.cpp and MLX to download and benchmark them. recommend --json gives agent-friendly output.