Tools

Adaptive web scraping framework: scrapling extract turns a page into Markdown, text or HTML in one command, with CSS selectors and browser rendering for dynamic pages, plus an MCP server (needs the ai extra). Needs Python 3.10+ and a browser install on first use.

Open-source, LLM-friendly web crawler that turns pages into clean Markdown for RAG and agents. Ships the crwl CLI and supports deep crawling and question-driven data extraction (extraction needs an LLM key). Free to self-host.

Turns natural-language descriptions, Mermaid or a real code repo into interactive single-file HTML diagrams: architecture, flowcharts, sequence diagrams, data flows and state machines. Validates output automatically and exports PNG or SVG.

Rust CLI proxy that filters and compresses command output before it reaches the agent's context, covering git, tests, grep, ls and 100+ other commands and cutting bash output by up to about 90%. Hooks automatically rewrite commands run by Claude Code, Codex and others.

General-purpose command-line fuzzy finder for any text input, such as file lists, command history or processes. Besides the interactive UI it has a non-interactive --filter mode, plus shell key bindings and completion.

Document parser from OpenDataLab that turns PDFs, images and Office files into Markdown or JSON, handling tables, formulas and complex layouts well, with page-by-page reading and citation anchors for agents. Parses locally by default; needs Python 3.10+.

Classic open-source OCR engine and CLI that reads text from images and outputs plain text, hOCR, searchable PDF or TSV. Supports 100+ languages; install the language data for each language you need.

Multi-agent orchestration framework for Claude Code and Codex, formerly Claude Flow: swarm coordination, self-learning memory and cross-machine federated communication. The marketplace offers 30+ plugins; full features need npx ruflo init.

Multi-agent orchestration framework formerly called Claude Flow: ruflo init gives Claude Code nearly 100 specialized subagents, swarm coordination, cross-session vector memory and hooks; add --codex to configure Codex instead. Also works as a standalone MCP server or plugin marketplace.

All-in-one LLM fine-tuning framework: llamafactory-cli handles pretraining, SFT, PPO/DPO alignment and LoRA/QLoRA training, plus chat inference, weight merging and export, and OpenAI-style API serving for 100+ open models. Has a Web UI; training usually needs a GPU.

Job search system that runs in your local AI coding CLI: scans job portals, scores roles with a structured A-H evaluation and a 1-5 rating, flags fake listings, tailors ATS resumes and cover letters, and tracks applications. Never submits for you. Needs Node.js.

Official OpenBB MCP server exposing the OpenBB Platform's equity, ETF, macroeconomic, crypto and news data to agents; enable only the tool categories you need to save context. Needs uv locally; some data sources need your own API keys. Not investment advice.

Command-line edition of the open-source OpenBB investment research platform: query stock prices, financial statements, macroeconomic and other financial data from the terminal, and automate data pulls with Routine scripts. Officially maintained; some sources need API keys.

Runs GitHub Actions workflows locally in Docker, so you can test .github/workflows changes without repeated commits and pushes, or use workflows as a local task runner. Needs Docker installed.

Agentless IT automation: uses SSH and playbooks for configuration management, app deployment, cloud orchestration and ad hoc commands at scale, with nothing to install on target machines. Maintained primarily by Red Hat.

Official Cline CLI sharing its agent core with the VS Code extension: plan/act modes, MCP, checkpoints and subagents; --yolo and --json run it unattended in CI and scripts. Sign in with Cline or a ChatGPT subscription, or bring your own API key.

Fast regex search across directories (command rg) that respects .gitignore, skips hidden and binary files by default and filters by file type. A go-to command for coding agents searching a codebase.

Parses PDF, DOCX, PPTX, XLSX, HTML, images and audio, recovering layout, reading order, tables and formulas, and exports Markdown, HTML or lossless JSON. Started by IBM Research, now hosted by the LF AI & Data Foundation; runs fully locally.

Command-line toolkit for transcoding, cutting, joining, frame extraction and streaming of audio and video, including ffmpeg, ffprobe and ffplay. A core dependency for agents handling media. The GitHub repo is the official mirror.

Community-maintained simplified command help with example-first pages for common CLI tools, easier to read than man. Install an official client (Python, Rust or Node.js), then run tldr followed by a command name.

A cat replacement with syntax highlighting, line numbers and Git change markers, plus --line-range to print specific lines. Falls back to plain text in pipes and other non-interactive use, and pairs with git diff or tail -f for code and logs.

'rsync for cloud storage': sync and copy files across dozens of providers such as S3, Google Drive, OneDrive and Alibaba Cloud OSS, with encryption and mounting. Set up remotes with rclone config before use.

Creates a local CA with one command and issues development HTTPS certificates trusted by your system and browsers, for localhost, IPs and wildcard domains, with no extra configuration. Run mkcert -install once first to add the root certificate to the trust store.

Open-source crypto trading framework: download market data, backtest and optimize strategies, and paper trade from the CLI while agents iterate on strategies. Docker recommended; strategies need Python and live trading needs exchange API keys. Backtests don't guarantee returns.